DOJO：使用以太网传输协议(TTPoE)的Exa级有损人工智能网络.pdf

1、HOT CHIPS 2024Tesla Transport Protocol over Ethernet(TTPoE)A new lossy,Exa-Scale fabric for the Dojo AI SupercomputerEric Quinnell,Ph.D.Dojo Fabric Lead2HOT CHIPS 2024Problem StatementTCP/IP is too slow for scaled AI interconnectBound by CPU SW kernelLossless fabrics are complex and brittlePriority

2、Flow Control(PFC)affects the global networkIdeal Fabric:Lowest latency Highest bandwidth Simple SoftwareFor Tesla AI:Layer 2 onlyCollective communications and ingestLow congestion,single application3HOT CHIPS 2024TTPoETesla Transport Protocol over Ethernet(TTPoE)is a peer-to-peer ethernet Transport

3、Layer Protocol executed entirely in hardware.Why a custom transport protocol?1.Vertical Integration extend Dojo RDMA onto optical fabric2.“Lossy”ethernet network ease of scaling,cost,congestion mgmt.3.Use 3rd party hardware Ethernet II frames“Just Work”TCP got it right just do it in hardware4HOT CHI

4、PS 2024Dojo OSI LayersOSI LayerExample Protocols(TCP/IP)TCP/IP ImplementationOSI LayerExample ProtocolsDojo ImplementationLayer 7ApplicationHTTP,Telnet,FTPLayer 7ApplicationPytorch,DojotorchLayer 6PresentationJPEG,PNG,MPEGLayer 6PresentationFFMPEG,HEVC,YUVLayer 5SessionNFS,SQLLayer 5SessionDojo RDMA

5、 DescriptorsLayer 4TransportTCP,UDPLayer 4TransportTTPLayer 3NetworkIPv4/IPv6Layer 3(Optional)NetworkIPv4/IPv6(Optional)Layer 2Data LinkEthernet Frames,MAC addresses,VLANLayer 2Data LinkEthernet Frames,MAC addresses,VLANLayer 1PhysicalData Encoding,Physical SpecsLayer 1PhysicalData Encoding,Physical

6、 SpecsHardwareSoftwareHardwareSoftwareStandard StackDojo Stack5HOT CHIPS 2024TimeTTP Link CommunicationTTP Device ATTP Device BTimeTTP Link CommunicationTTP Device ATTP Device BClean TTP transfer ExampleNACK TTP transfer Example.TTP_PAYLOAD,ID=3 is either lost or out of orderTTP transaction examples