2762 - 在本地部署 Frontier 大型语言模型.pdf

1、WatsonWatsonx x.ai.aiLeveraging Frontier Large Language Models on PremisesLast update:September 29th,2025IBM and Business Partner IBM ConfidentialProblemsVendor Lock-InToday there are numerous models with different capabilities offered across many platformsand not all are openthe core requirement fo

2、r accelerated development.Customers do not want to be do not want to be dependent on a single model dependent on a single model provider provider for their enterprise workloads.Security&GovernanceEnterprises need security by design with the ability to manage access and usage across many different us

3、er groups.Without fine-grained management,models are prone to be used in ways they are not meant to be.Deriving More ValueFoundation model development is a rapid field with closed and open-source innovations.Accessing the latest model capabilities is paramount in maintaining an edge in experimentati

4、on to production for building AI applications.2Customers want access to the latest LLMs,yet are struggling with managing the scope of models across numerous platformsIBM Software/2025 IBM Corporation/IBM ConfidentialIntroducingModel GatewayStandard InterfaceSimply by providing your URL&credentials,w

5、e can route your route your query to the provider of your query to the provider of your choice through fully OpenAI choice through fully OpenAI API compatible endpointsAPI compatible endpoints allowing for:Seamless switching between most model providersReduced operational overheadCross-cluster&cross

6、-region inferencingControlsBuild added security and governance into your inferencing workloads by:Managing user accessSetting rate limits to prevent outages and maintain fine-grained control*Models accessed through the gateway can take advantage of the integrations with watsonx.governance and watson