How to stop table headers being truncated ?

I have a code that filters through data in csv files and outputs a new table with only the columns that I want. It has been working fine but when I try to use it on a new set of data it will work up until one file where it throws up 'Table variable names were truncated to the length namelengthmax. The original names are saved in the VariableDescriptions property'. This file has the same headings as all the others so I don't understand why it is throwing up the error. I have checked all the headings and they aren't any longer than the other files. I am getting the table headers using 'opts = detectImportOptions(fullFileName,'NumHeaderLines',0,'VariableNamingRule','preserve');

test = readtable(fullFileName,opts);'

Any idea how to avoid or fix this?

Thanks

[Stephen23]

"This file has the same headings as all the others so I don't understand why it is throwing up the error. I have checked all the headings and they aren't any longer than the other files."

It is quite possible that the automagical detection algorithms is detecting e.g. the columns differently for that file. We often get questions similar to this where the user does not notice that their data is formatted differently after several hundred lines of data, or non-numeric characters in supposedly "numeric" data, or something of that ilk. It does not require that the heading themselves must be different.

You should start debugging by looking at the imported data (not just the headers).

If you want help with this please upload two sample data files: one which imports correctly, one which does not.

Answer 1

[Image Analyst]

From the error it sounds like there is a maximum number of characters allowed for a table header name, just like there is for variable names, file names, etc. It's just now setup to have table headers that are a bazillion characters long. I'm not sure what the max is but it might be something like 32 characters long or 256 characters long or something? Try to pick shorter names.

[Steven Lord]

namelengthmax is currently 63, though we have stated in the Release Notes that this limit will be increasing in a future release.

namelengthmax

ans = 63

If I had to guess I'd guess that the file has variable names that are supposed to be a fixed width (and have one or more spaces between the names) but one of the names is long enough that it lacks a space between it and the next variable name. If those variable names were 32 characters long, two of those variable names being treated as one would try to make a variable with a 64 character long name and that would lead to the warning.