Embedding
A numeric fingerprint of meaning — turns text into coordinates so similar things land near each other. This is how Roost finds the right policy document when you ask a technical question about coverage.
An embedding is a numeric representation of text as a vector in a high-dimensional space. By mapping semantic meaning to geometric proximity, it allows Roost to perform similarity searches that identify relevant policy documents based on context rather than just matching keywords.
An embedding is a dense vector representation of discrete data, such as text, projected into a continuous high-dimensional latent space. By preserving semantic relationships through vector distance metrics like cosine similarity, it enables the system to retrieve policy documents by identifying the nearest neighbors to a query vector, effectively transcending the limitations of lexical keyword matching.
evolution
- 2003 · historyNeural Probabilistic Language Models
Yoshua Bengio et al. introduced the concept of learning distributed representations for words to capture semantic relationships.
- 2013 · historyWord2Vec
Tomas Mikolov and his team at Google popularized efficient vector space models, enabling the mapping of words into high-dimensional semantic space.
- 2014 · historyGloVe
Stanford researchers introduced Global Vectors for Word Representation, combining global matrix factorization with local context window methods.
- 2018 · historyBERT
Google introduced Bidirectional Encoder Representations from Transformers, shifting embeddings from static word vectors to context-aware, dynamic representations.
- 2019 · historySentence-BERT (SBERT)
Nils Reimers and Iryna Gurevych modified BERT to derive semantically meaningful sentence embeddings, significantly improving performance for semantic search and clustering.