利用vLLM部署多模式RAG系统.pdf

1、1|Copyright 8/16/23 Zilliz1|Copyright 8/16/23 ZillizStephen Batifol MilvusQCon LondonMultimodal RAG using vLLM and Pixtral2|Copyright 8/16/23 Zilliz2|Copyright 8/16/23 ZillizStephen BatifolDeveloper Advocate,Zilliz/MilvusAbout M 8/16/23 Zilliz3|Copyright 8/16/23 Zilliz3|Copyright 8/16/23 Zilliz3|Cop

2、yright 8/16/23 ZillizWhat is Vector Search?4|Copyright 8/16/23 Zilliz4|Copyright 8/16/23 ZillizVectors unlock Unstructured Data5|Copyright 8/16/23 Zilliz5|Copyright 8/16/23 ZillizVector Space6|Copyright 8/16/23 Zilliz6|Copyright 8/16/23 ZillizBrute-force:measure the distance to all points and choose

3、 the shortest!Computing Nearest Neighbors7|Copyright 8/16/23 Zilliz7|Copyright 8/16/23 ZillizML Models that predict how to approximate close neighbors.These models are called Indexes and take parameters to trade off model complexity,quality and search speed.Approximate Nearest Neighbors8|Copyright 8

4、/16/23 Zilliz8|Copyright 8/16/23 ZillizApproximate Nearest Neighbors Trade-offs Search SpeedModel ComplexityModel Quality9|Copyright 8/16/23 Zilliz9|Copyright 8/16/23 ZillizFLATBrute ForceIndexes IVFInverted File IndexHNSWHierarchical Navigable Small World Graphs10|Copyright 8/16/23 Zilliz10|Copyrig

5、ht 8/16/23 ZillizPicking,Evaluating,&Tuning the Index IVF QuantizationDiskANNCAGRA GPUHNSWScANNIVF_FLATNOTE Points farther from center indicate better performance Higher Accuracy,Lower Latency,Higher Throughput,Faster Index Time,Lower Cost)11|Copyright 8/16/23 Zilliz11|Copyright 8/16/23 ZillizSummar

6、yIndexTrade-offsFLAT Simple&accurate Slow at scaleIVF Fast enough for most tasks Needs tuning,accuracy can varyHNSW High QPS,great accuracy Slow to build,costly to updateDISKANN Scales beyond RAM,cheaper infra Latency from disk access12|Copyright 8/16/23 Zilliz12|Copyright 8/16/23 ZillizSummaryIndex