An autoencoder is a type of deep learning network that is trained to replicate its input data. Autoencoders have surpassed traditional engineering techniques in accuracy and performance on many applications, including anomaly detection, text generation, image generation, image denoising, and digital communications.
How Do Autoencoders Work?
Autoencoders output a reconstruction of the input. The autoencoder consists of two smaller networks: an encoder and a decoder. During training, the encoder learns a set of features, known as a latent representation, from input data. At the same time, the decoder is trained to reconstruct the data based on these features. The autoencoder can then be applied to predict inputs not previously seen. Autoencoders are very generalizable and can be used on different data types, including images, time series, and text.
What Applications Use Autoencoders?
Autoencoders will naturally ignore any input noise as the encoder is trained. This feature is ideal for removing noise or detecting anomalies when the inputs and outputs are compared (see Figures 2 and 3).
The latent representation can also be used to generate synthetic data. For example, you can automatically create realistic looking handwriting or phrases of text (Figure 4).
Time series-based autoencoders can also be used to detect anomalies in signal data. For example, in predictive maintenance, an autoencoder can be trained on normal operating data from an industrial machine (Figure 5).
The trained autoencoder is then tested on new incoming data. A large variation from the autoencoder’s output indicates an abnormal operation, which could require investigation (Figure 6).
- Autoencoders do not require labeled input data for training: they are unsupervised
- There are several varieties of autoencoders built for different engineering tasks, including:
- Convolution autoencoders – The decoder output attempts to mirror the encoder input, which is useful for denoising
- Variational autoencoders – These create a generative model, useful for anomaly detection
- LSTM autoencoders – These create a generative model for time series applications