Problems using java.nio in Matlab, underlying goal is to get all contents of folder with isDirectory attribute

My goal is to check if a file with a particular (part of the name) is found in a folder on the network, also taking into account all folders below it. To do so I need a way to efficiently get a list of all files and folders in and below a given folder. My recursive function does ~2500 items/s on a local drive, but only several/sec on a network drive. I need something faster.

The core question is: what is the fastest way to get a list of items in a folder including the attribute isDirectory or something similar?

I put my hope on the walkFileTree functionality of java.nio, but I am unable to use it. (version: 8.4.0.150421 (R2014b) with Java 1.7.0_11-b21 with Oracle Corporation Java HotSpot™ 64-Bit Server VM mixed mode)

Current problem: I am unable to use any functionality from java.nio

java.io works, e.g. create a file object:

jFile = java.io.File('C:\')  % then use jFile.list or jFile.isDirectory or jFile.toPath, it all works!

Naively calling nio fails:

java.nio.file.Files('C:\') % -> No constructor 'java.nio.file.Files' with matching signature found.

I realize java.nio.file works a bit differently, to use the methods in Files a path is needed, which can be constructed with java.nio.file.Path.get. This thing eats a string. But this also fails:

 java.nio.file.Paths.get('C:\')  % -> No method 'get' with matching signature found for class 'java.nio.file.Paths'.

However the method exists:

methods java.nio.file.Paths % ->  Methods for class java.nio.file.Paths: 
equals     getClass   notify     toString   
get        hashCode   notifyAll  wait

So what is going wrong here? I am not allowed to feed a matlab string? Should I use a Java string? This too fails:

jString = java.lang.String('C:\');
java.nio.file.Paths.get(jString)  % -> No method 'get' with matching signature found for class 'java.nio.file.Paths'.

An oracle workaround is to create the path in java.io, but feeding that to java.nio also fails..

path = java.io.File('C:\').toPath;
java.nio.file.Files.isDirectory(path) % -> No method 'isDirectory' with matching signature found for class 'java.nio.file.Files'.

So I am not getting any closer to even trying the walkFileTree. I can not get java.nio to do anything in Matlab.

Help: so does anybody have any idea on how to call the java.nio.file functions or answer my core question?

ps: examples of straightforward methods so far without java.nio, examples do no include the recursive part but show the horrible performance

strategy 1: recursively use Matlab's 'dir' function. It is a nice function, as it also gives attributes, but it is a bit slow. In my starting network folder (contains 150 files/folders, path stored as string Sdir) the following command takes 34.088842 sec :

tic;d=dir(Sdir);toc

strategy 2: use java.io.File to speed things up, this hardly helps, because isDirectory needs calling.. Using a heuristic on the names of the items is too dangerous, I am forced to use folders with dots in them. Example in same dir, 31.315587 sec:

tic;jFiles = java.io.File(Sdir).listFiles;LCVdir = arrayfun(@isDirectory, jFiles, 'UniformOutput',0);toc

[Jim Hokanson]

Additional details can be found at this SO page:

https://stackoverflow.com/questions/41428612/how-to-use-java-nio-in-matlab

Answer 1

[Walter Roberson]

I was trying to dig into this layer for another Question, and noticed that your Question was one of the very few that mentioned the java nio package. Your questions helped me to get further. I looked at them, and dug through the few MATLAB source code references to java.nio, and looked at undocumentedmatlab, and experimented a bunch, and managed to come up with a working example (well, at least for MacOS)

At the moment, the example does not handle the possibility of file already existing: I have not started looking into how to handle java exceptions in MATLAB.

The below example shows three different ways of converting a file name string into the kind of path that java.nio needs, and carries that through to testing the directory attribute.

It shows the construction of non-default file attributes for createFile purposes, because nio createFile insists that file attributes be passed in.

It shows the creation of non-default link options for isDiresctory purposes, because nio isDirectory insists that link attributes be passed in.

It was not until after I had it working with real permissions and link option that I had the idea to test with empty arrays

The most complete Mathworks-provided code I found that uses java.nio is at fullfile(matlabroot, './toolbox/shared/bigdata/+matlab/+bigdata/+internal/+spark/+worker/MappedIpcBuffer.m')

----

directory_to_create_in = pwd;
filename_to_create = 'testjunk.txt';

perms = java.nio.file.attribute.PosixFilePermissions.fromString('rw-rw----') %#ok<*NOPTS>
attrs = java.nio.file.attribute.PosixFilePermissions.asFileAttribute(perms)
jattrs_array = javaArray('java.nio.file.attribute.FileAttribute', 1)
jattrs_array(1) = attrs

jlinkoption = java.nio.file.LinkOption.NOFOLLOW_LINKS
jlinkoption_array = javaArray('java.nio.file.LinkOption', 1)
jlinkoption_array(1) = jlinkoption

js_array = javaArray('java.lang.String', 1)
js_filename_to_create = java.lang.String(filename_to_create)
js_array(1) = js_filename_to_create

jpath = java.nio.file.Paths.get(directory_to_create_in, js_array)

java.nio.file.Files.createFile(jpath, jattrs_array)

is_file_directory = java.nio.file.Files.isDirectory( jpath, jlinkoption_array )

is_parent_directory = java.nio.file.Files.isDirectory( jpath.getParent, jlinkoption_array )
filename_to_create2 = fullfile(directory_to_create_in, 'testjunk2.txt' );

jattrs_array2 = javaArray('java.nio.file.attribute.FileAttribute', 0)
jlinkoption_array2 = javaArray('java.nio.file.LinkOption', 0)

jpath2 = toPath(java.io.File(filename_to_create2))

java.nio.file.Files.createFile(jpath2, jattrs_array2)

is_parent_directory2 = java.nio.file.Files.isDirectory( jpath2.getParent, jlinkoption_array2 )
filename_to_create3 = fullfile(directory_to_create_in, 'testjunk3.txt' );

jpath3 = java.nio.file.Paths.get(filename_to_create3, javaArray('java.lang.String', 0))