Data extraction from a research article (an electronic pdf with highly unstructured data)
7 views (last 30 days)
Show older comments
My research area is applied machine learning in material science. I am struggling for an algorithm which could retrieve the desired data values (categorical and numerical both) from the research article (an electronic pdf document which is highly unstructured). There are thousands of such documents (pdfs) from which the data needs to be extracted which is a time intensive task. Some pdf may carry data in graphs while others in table or text. Kindly guide me through the process using which I can efficiently extract the data.
1 Comment
dpb
on 2 Nov 2022
Essentially an impossible task -- the data/figures in a pdf file are not stored in a retrievable format other than by interpreting/rendering the pdf document itself.
Answers (2)
Image Analyst
on 3 Nov 2022
2 Comments
dpb
on 3 Nov 2022
@Image Analyst is pointing out that the "converters" actually use OCR to recognize and convert pdf content to text. Then, you'll have to have code to find the particular code words of ineterst.
That's only the text portion; tables and images in my experience weren't converted to raw data but simply embedded into the document as objects. That might get you at least part of the way, but it's not going to be anything simple to do for a generic collection of papers.
See <acrobat/online/convert-pdf.html> and Google is your friend to find alternates outside Adobe...altho your uni probably has a site license.
This really isn't much a MATLAB Q?
Image Analyst
on 3 Nov 2022
I doubt thousands of articles would all be in this format/style. You might be able to get the text and numbers out but it could be tough to automatically figure out which numbers are in a table and what they mean. For the plots, you might just need to convert them to images and then use one of the File Exchange submissions. But even then I imagine it's going to require a lot of manual processing.
Maybe you could just use Amazon Mechanical Turk to hire a bunch of cheap global workers to do it for you.
0 Comments
See Also
Categories
Find more on Image Data Workflows in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!