How to do one hot encoding of unusual letters in matlab?

4 views (last 30 days)
I have a table of german characters. I would like to one hot encode them so that i can input it to a neural network. How should i go about?
The problem is some of the characters are not accepted by matlab as characters. For example, 'ä' 'ö' 'ü' 'ß'
Regardless, I would like to know how to one hot encode any character from a TABLE in matlab.
Thanks in advance!

Accepted Answer

Walter Roberson
Walter Roberson on 19 Jul 2019
Edited: Walter Roberson on 19 Jul 2019
https://machinelearningmastery.com/why-one-hot-encode-data-in-machine-learning/ describes One-Hot Encoding (a term I was not aware of)
You might want to first construct a list of permitted characters, and map the input into an offset in that list. That will potentially save you from wasting bits on characters such as Œœ that you are not using.ŒŒ
  5 Comments
Walter Roberson
Walter Roberson on 30 Jul 2019
permitted = ['A' : 'Z', 'a' : 'z', 'ä', 'ö', 'ü', 'ß', ' ', '.' ] ;
[found, idx] = ismember(YourText, permitted);
assert(all(found), 'unpermitted character detected')
OneHot = ind2vec(idx) ;
Sanjana Sankar
Sanjana Sankar on 31 Jul 2019
OneHot = full(ind2vec(idx,max(idx)));
This is exactly what I was looking for and I got it from your code. Thank You! This works :)

Sign in to comment.

More Answers (1)

Guillaume
Guillaume on 30 Jul 2019
file containing the characters that I want to one hot encode. Do you know how I can go about now
%phoneme_set: A cell array of phonemes to one-hot encode
assert(numel(phoneme_set) > 64, 'Cannot one-hot encode more than 64 phonemes with a 64-bit integer')
phoneme_set(:, 2) = num2cell(2 .^ uint64(0:size(phoneme_set, 1)-1))
  3 Comments
Guillaume
Guillaume on 30 Jul 2019
number binary pattern (64 bits)
2^0 0000000000000000000000000000000000000000000000000000000000000001
2^1 0000000000000000000000000000000000000000000000000000000000000010
2^2 0000000000000000000000000000000000000000000000000000000000000100
...
2^63 1000000000000000000000000000000000000000000000000000000000000000
This is what one-hot encoding is. As I commented, this is useful for FPGAs and similar which operate at the bit level. On the generic processor of a computer, it's a complete waste of space but my answer does what you asked.
You could encode the one-hot encoded numbers that I generate as a vector of 0 and 1 (double) for even more waste of space:
phoneme_set(:, 3) = num2cell(fliplr(eye(size(phoneme_set, 1), 64)), 2)
Note that the binary pattern of the 0s in that encoding is 0000000000000000000000000000000000000000000000000000000000000000b and of the 1s is 0011111111110000000000000000000000000000000000000000000000000000b.
However, I suspect that what we have here is a XY problem. You have some unspecified problem doing something and you think you can solve it by using one-hot encoding (without really understanding what it means) so ask about one-hot encoding instead of your actual problem.
Sanjana Sankar
Sanjana Sankar on 30 Jul 2019
I have to feed characters to a neural network. And the most common method is one hot encoding. But I haven't found any examples on how it is done in MATLAB.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!