Databricks 上的矢量搜索简介.pdf

1、2024 Databricks Inc.All rights reservedIntroduction Introduction to Mosaic AI to Mosaic AI Vector Search Vector Search Akhil Gupta,VP of Engineering,AI SystemsAkhil Gupta,VP of Engineering,AI SystemsJun 13,2024Jun 13,202412024 Databricks Inc.All rights reservedEmbeddings A numerical representation o

2、f data as a point in N-dimensional space Vector of two data objects similar to each other will be close to each other.Generated using models2Vector Search Vector Search Basic ConceptsBasic Concepts2024 Databricks Inc.All rights reservedRetrieval Algorithm Nearest NeighborApproximate Nearest Neighbor

3、(ANN)vs.Top-K Nearest Neighbor(KNN)Hybrid Search Trade-off between latency vs.recall Indexing techniques matter.3Vector Search Vector Search Basic ConceptsBasic Concepts2024 Databricks Inc.All rights reservedDatabase Indexing Scalability Performance Durability Governance4Vector SearchVector SearchBa

4、sic ConceptsBasic Concepts2024 Databricks Inc.All rights reservedEmbeddingsRetrieval Algorithm5Vector SearchVector SearchBasic ConceptsBasic ConceptsImpacts Quality2024 Databricks Inc.All rights reservedDatabase impacts performance,security and ease of use6Vector SearchVector SearchBasic ConceptsBas

5、ic Concepts2024 Databricks Inc.All rights reserved Use it for semantic search over unstructured dataText,Video,AudioNot good for typical database SQL-style queriesAggregation,JoinsA must-have component when building GenAI applicationsCritical to reduces hallucinations and provide better context to L

6、LMsExample GenAI Applications-RAG,Sentiment Analysis/Classification7Vector SearchVector SearchWhat it is good for and what it is not good forWhat it is good for and what it is not good for2024 Databricks Inc.All rights reserved8Embedding modelText chunksVector SearchindexUserEnterprisedataVector emb