当前位置:首页 > 报告详情

在 Oracle 云上使用 AMD Instinct GPU 大规模提供内存密集型 LLM 服务 [LRN1371](2).pdf

上传人: Fl****zo 编号:971016 2025-11-08 40页 3.18MB

1、 LLM Inference on OCI with AMD InstinctGPUDeploy LLM using AMD MI300X on OCIGautam KarmakarMaster Principal Solution ArchitectNA Cloud Engineering,Oracle Cloud InfrastructureOctober 14,2025 LLM Inference on OCI with AMD Instinct GPUsDeploy LLM using AMD MI300X on OCIBinh ChuSr.Product ManagerDatacen

2、ter GPUsOctober 14,2025The following is intended to outline our general product direction.It is intended for information purposes only,and may not be incorporated into any contract.It is not a commitment to deliver any material,code,or functionality,and should not be relied upon in making purchasing

3、 decisions.The development,release,timing,and pricing of any features or functionality described for Oracles products may change and remains at the sole discretion of Oracle Corporation.Safe harbor statement3Copyright 2025,Oracle and/or its affiliates|Confidential:Internal/Restricted/Highly Restrict

4、edForward-Looking statementsThis presentation is intended to outline our general product direction.It is intended for information purposes only,and may not be incorporated into any contract.It is not a commitment to deliver any material,code,or functionality,and should not be relied upon in making p

5、urchasing decisions.The development,release,timing,and pricing of any features or functionality described for Oracles products may change and remains at the sole discretion of Oracle Corporation.Statements in this presentation relating to Oracles future plans,expectations,beliefs,intentions,and pros

6、pects are“forward-looking statements”and are subject to material risks and uncertainties.A detailed discussion of these factors and other risks that affect our business is contained in Oracles Securities and Exchange Commission(SEC)filings,including our most recent reports on Form 10-K and Form 10-Q

word格式文档无特别注明外均可编辑修改,预览文件经过压缩,下载原文更清晰!
三个皮匠报告文库所有资源均是客户上传分享,仅供网友学习交流,未经上传用户书面授权,请勿作商用。
根据报告的内容,以下是全文关键点的概括: 1. **主题**:在Oracle Cloud Infrastructure (OCI)上使用AMD Instinct GPU部署和运行大型语言模型(LLM)。 2. **核心数据**: - 7/10最大的AI公司使用AMD Instinct GPU。 - MI300X GPU在内存受限场景中表现最佳,适合长输出序列和严格延迟约束的任务。 - MI300X具有1.5TB HBM3内存,10.4 PF的FP16/BF16 FLOPS,1.7 TF/W的性能,以及896 GB/s的聚合双向带宽。 3. **关键点**: - OCI提供高效的AI基础设施,包括快速的内部节点带宽、裸金属GPU计算和具有RDMA的超集群。 - OCI支持80多个合规计划,包括HIPAA、FedRAMP等。 - OCI与Cerner、LHN、Enviza、NVIDIA等合作,提供独特的混合和多云能力。 - 使用vLLM库优化LLM推理,提高GPU内存利用率和处理长序列的能力。 - AMD Instinct MI300X在价格性能方面具有优势,适用于大规模推理任务。 - OCI支持多种Kubernetes引擎功能,包括私有集群、加密、容器图像扫描和签名等。 - OCI提供灵活的部署选项,包括管理节点和虚拟节点,以及与OCI服务的无缝集成。
"LLM在OCI上高效部署" "AMD MI300X助力OCI AI推理" "OCI如何简化LLM部署?"
客服
商务合作
小程序
服务号
折叠