There is currently a GitHub repository (https://github.com/matlab-deep-learning/transformer-models) which implements some deep learning transformer models for NLP tasks in MATLAB. This repository includes popular models, such as BERT, FinBERT, and GPT-2. However, other models will need to be manually imported into MATLAB.
Unfortunately, there is no simple function or example detailing how to do this as of right now. However, this is a workflow our development team is very much interested in developing and is actively investigating.
In the meantime, the currently recommended workflow is to export the model and weights to ONNX format. Hugging Face provides a detailed explanation as to how this can be done in their documentation linked below.
Once you have the model and weights in ONNX format, you can use either the "importONNXNetwork" or the "importONNXFunction", depending on whether you want a layer graph or custom function (the documentation for both of these functions are linked below) to import the model into MATLAB.
Unfortunately, due to the wide variety of transformers and self-attention layers found in Hugging Face models, there may be a number of custom layers that are not cleanly imported into the corresponding MATLAB layer. This may lead to custom layers in the imported network/function which contain multiple single layers. It may then be necessary to break these conglomerated layers into their individual parts manually.
Furthermore, when using "importONNXFunction", it is likely that some of the layers will be replaced by placeholder layers. You will need to manually replace these placeholder layers with their correct layers in order to use this imported model for downstream tasks.
