how to modify read function of imageDataStore in MATLAB to read Large Images block wise?

I have very large RGB images of size as high as 20000 x 18000 pixels. I want to fit in the whole image for SEGNET training rather than breaking the image into blocks. I would loose global structure of the pattern if I train the image as blocks. Therefore, I tried to use blockproc function in MATLAB to read my image from imageDataStore funtion and process it blockwise yet together (since blockproc will consider it as one whole image).

imds.ReadFcn=@(loc)blocproc(loc,[256,256],@blockwise);

function blockwise(bs)
imread(bs.data);
end

I am getting this error :

function blockwise(bs)
 ↑
Error: Function definition not supported in this context. Create functions in code file

Then I modified again like this :

blockproc(imread(imds.Files{1}),[256,256],@blockwise);

function blockwise(bs)
imds.ReadFcn=@(loc)imread(bs.data);
end

to which I am receiving this error :

BLOCKPROC encountered an error while evaluating the user-supplied function handle, FUN.

The cause of the error was:

Undefined function 'blockwise' for input arguments of type 'struct'.

Error in blockprocFunDispatcher (line 13)
        output_block = fun(block_struct);

Error in blockprocInMemory (line 80)
[ul_output fun_nargout] = blockprocFunDispatcher(fun,block_struct,...

Error in blockproc (line 243)
    result_image = blockprocInMemory(source,fun,options);

I want to know if it is even possible to read images in datastore blockwise using blockproc and, if it is, then how should I proceed?

[Saumya Goel]

I would like to know, what is the exact issue that you are facing if you pass it as a single image, rather than trying to break it into blocks or using blockproc. Also, is it going to impact if you resize the image which would help you to retain global structure but will lead to compression of the image. If you would like to try resize option, you can have a look at this link .

[Suvidha Tripathi]

image is too big to pass, I get an out of memory error if i try to pass even a single image. Also, resizing is not an option since these are multi resolution images, If i resize them, I would loose lots of information.

Answer 1

[Saumya Goel]

As per my understanding, use of datastore is normally done in the cases where one needs to work on a collection of image files, where each individual image fits in memory, but the entire collection of images does not necessarily fit.

The issue here seems to be that the image is too large to fit into the installed RAM of the machine. One way to resolve this might be to increase the installed RAM to allow the image to fit into memory.

Apart from the above approach, these are few of the examples that might help you read large images:

Block Processing Large images

Image adapters for Big Images

Large Image Viewing