T-distributed stochastic neighbor embedding

T-SNE_visualisation_of_word_embeddings_generated_using_19th_century_literature

T-SNE_Embedding_of_MNIST

T-distributed stochastic neighbor embedding (t-SNE) is a machine learning algorithm for dimensionality reduction developed by Geoffrey Hinton and Laurens van der Maaten. It is particularly well-suited for the visualization of high-dimensional datasets.

Overview

t-SNE is a non-linear technique primarily used for data visualization. It converts similarities between data points to joint probabilities and tries to minimize the Kullback-Leibler divergence between the joint probabilities of the low-dimensional embedding and the high-dimensional data. The algorithm is an extension of Stochastic Neighbor Embedding (SNE) and aims to address the crowding problem inherent in SNE.

Algorithm

The t-SNE algorithm consists of two main stages: 1. **Computing pairwise similarities**: In the high-dimensional space, the similarity between two points is measured using a Gaussian distribution. In the low-dimensional space, a Student's t-distribution is used to measure similarity. 2. **Minimizing the Kullback-Leibler divergence**: The algorithm iteratively adjusts the positions of points in the low-dimensional space to minimize the divergence between the high-dimensional and low-dimensional similarity distributions.

Applications

t-SNE is widely used in various fields such as:

• Bioinformatics for visualizing gene expression data.
• Natural language processing for visualizing word embeddings.
• Computer vision for visualizing image features.

Advantages and Disadvantages

Advantages

• Effective in capturing the local structure of the data.
• Produces visually interpretable results.

Disadvantages

• Computationally intensive, especially for large datasets.
• The results can be sensitive to the choice of hyperparameters such as perplexity.

Related Techniques

• Principal Component Analysis (PCA)
• Multidimensional scaling (MDS)
• Isomap
• Locally Linear Embedding (LLE)

See Also

• Dimensionality reduction
• Data visualization
• Machine learning

External Links

```

This template is designed for use in marking articles related to machine learning as stubs, which are articles that are too short to provide more than rudimentary information about a subject. When this template is placed on a page, it automatically adds the page to the "Machine learning stubs" category, making it easier for contributors to find and expand short articles in this subject area.