OpenAI Docs

OpenAI’s text embeddings are used to measure the relatedness of text strings, facilitating applications like search, clustering, recommendations, anomaly detection, diversity measurement, and classification. The Assistants API now includes retrieval and message history management. An embedding is a numerical vector where distances between vectors indicate the level of relatedness.

To obtain embeddings, you send text to the embeddings API endpoint using a specific model ID, such as text-embedding-ada-002, and receive a vector in response. OpenAI offers a second-generation and several first-generation embedding models, with text-embedding-ada-002 recommended for most uses.

Embeddings can be applied in several use cases, such as data visualization, machine learning models for regression and classification, zero-shot classification, user and product embeddings for recommendations, clustering, and text and code search.

However, the models have limitations, including encoding social biases and lacking knowledge of events after August 2020. Users should be aware of these limitations and the social risks involved in their applications. OpenAI also provides guidance on counting tokens, retrieving nearest embedding vectors using vector databases, and recommends cosine similarity as the preferred distance function. Customers own their input and output data, but must ensure compliance with applicable laws and OpenAI’s Terms of Use.