高吞吐量机器学习：掌握企业级高效模型服务.pdf

1、High-Throughput MLMastering Efficient Model+Feature Serving at Enterprise ScaleMingyang Ge,Yucheng QianJune 10,2025I have a classification model.I want to achieve 15K QPS.How?Fraud Detection Example3A common ML use case in productionClient/AppModel ServingOnline Feature StoreReq user_idRes is_fraudR

2、eq user_idResavg_amount_1davg_amount_7d15K QPSWhat we are going to doDeploy Model+Feature on DatabricksScale testing Databricks Model ServingProductionize Serving EndpointModels+FeaturesSame features for both training and servingStep 1 Define Feature lookup in training_setStep 2 Log model with train

3、ing_setStep 3 Publish tables to Databricks Online Feature StoreDemoRecapHow to achieve high throughput?Select Capacity Unit based on data sizeSelect Readable Secondaries based on trafficEnable route optimizationSelect concurrency based on trafficQPS model execution timeTake anticipated traffic into

4、accountDatabricks Online feature storeDatabricks Model serving endpoint8Provision the right resourcesKey takeawaysDeploy Model+Feature endpoint in 3 stepsCreate training setLog model with the training setPublish to Online Feature StoreDeploy with confidenceOut-of-box load testing environmentScalable

5、 serving infrastructure Easy to scaleSupport 250k+qps1000+concurrency9Whats next for Online Feature Store?See you tomorrow at Keynote!Questions?AI DAIS 2025AI DAIS 202511Building Intelligent AI Agents With Claude Models and Building Intelligent AI Agents With Claude Models and Databricks Mosaic AI F

6、rameworksDatabricks Mosaic AI Frameworks11:30 AM 12:10 PM PDTWest,Level 2,Room 2001How Skyscanner Runs RealHow Skyscanner Runs Real-Time AI at Scale with Time AI at Scale with DatabricksDatabricks11:30 AM 12:10 PM PDTSouth,Level 3,Room 305Talking to All Your Data:Building MultiTalking to All Your Da