您的当前位置：首页 > 行业数据 > 搭载Llama370B模型推理场景下吞吐量比较

搭载Llama370B模型推理场景下吞吐量比较

行业数据

下载图片

原图定位

同报告图片

/ 4

PHRhYmxlPgogICAgPHRoZWFkPgogICAgICAgIDx0cj4KICAgICAgICAgICAgPHRoIHJvd3NwYW49IjIiPuiCoeelqOS7o+eggTwvdGg+CiAgICAgICAgICAgIDx0aCByb3dzcGFuPSIyIj7ogqHnpajlkI3np7A8L3RoPgogICAgICAgICAgICA8dGggcm93c3Bhbj0iMiI+5pS255uY5Lu3PGJyPu+8iOe+juWFgy/ogqHvvIk8L3RoPgogICAgICAgICAgICA8dGggcm93c3Bhbj0iMiI+5oC75biC5YC8PGJyPu+8iOS6v+e+juWFg++8iTwvdGg+CiAgICAgICAgICAgIDx0aCBjb2xzcGFuPSI0Ij7ln7rmnKzmr4/ogqHmlLbnm4ogRVBT77yI576O5YWDL+iCoe+8iTwvdGg+CiAgICAgICAgICAgIDx0aCBjb2xzcGFuPSIzIj5QRSDvvIjlgI3vvIk8L3RoPgogICAgICAgIDwvdHI+CiAgICAgICAgPHRyPgogICAgICAgICAgICA8dGQ+MjAyNDwvdGQ+CiAgICAgICAgICAgIDx0ZD4yMDI1RTwvdGQ+CiAgICAgICAgICAgIDx0ZD4yMDI2RTwvdGQ+CiAgICAgICAgICAgIDx0ZD4yMDI3RTwvdGQ+CiAgICAgICAgICAgIDx0ZD4yMDI1RTwvdGQ+CiAgICAgICAgICAgIDx0ZD4yMDI2RTwvdGQ+CiAgICAgICAgICAgIDx0ZD4yMDI3RTwvdGQ+CiAgICAgICAgPC90cj4KICAgIDwvdGhlYWQ+CiAgICA8dGJvZHk+CiAgICAgICAgPHRyPgogICAgICAgICAgICA8dGQ+V0RDLk88L3RkPgogICAgICAgICAgICA8dGQ+6KW/6YOo5pWw5o2uPC90ZD4KICAgICAgICAgICAgPHRkPjkwLjQ5PC90ZD4KICAgICAgICAgICAgPHRkPjMxMy45PC90ZD4KICAgICAgICAgICAgPHRkPi0yLjYxPC90ZD4KICAgICAgICAgICAgPHRkPjUuMTQ8L3RkPgogICAgICAgICAgICA8dGQ+NS41MzwvdGQ+CiAgICAgICAgICAgIDx0ZD41Ljc1PC90ZD4KICAgICAgICAgICAgPHRkPjE3LjY8L3RkPgogICAgICAgICAgICA8dGQ+MTYuNDwvdGQ+CiAgICAgICAgICAgIDx0ZD4xNS43PC90ZD4KICAgICAgICA8L3RyPgogICAgICAgIDx0cj4KICAgICAgICAgICAgPHRkPlNUWC5PPC90ZD4KICAgICAgICAgICAgPHRkPuW4jOaNt+enkeaKgDwvdGQ+CiAgICAgICAgICAgIDx0ZD4xODMuOTg8L3RkPgogICAgICAgICAgICA8dGQ+MzkxLjM8L3RkPgogICAgICAgICAgICA8dGQ+MS42MDwvdGQ+CiAgICAgICAgICAgIDx0ZD42Ljc3PC90ZD4KICAgICAgICAgICAgPHRkPjEwLjM2PC90ZD4KICAgICAgICAgICAgPHRkPjEyLjQ2PC90ZD4KICAgICAgICAgICAgPHRkPjI3LjI8L3RkPgogICAgICAgICAgICA8dGQ+MTcuODwvdGQ+CiAgICAgICAgICAgIDx0ZD4xNC44PC90ZD4KICAgICAgICA8L3RyPgogICAgICAgIDx0cj4KICAgICAgICAgICAgPHRkPlJNQlMuTzwvdGQ+CiAgICAgICAgICAgIDx0ZD5SYW1idXM8L3RkPgogICAgICAgICAgICA8dGQ+NzUuNDA8L3RkPgogICAgICAgICAgICA8dGQ+ODEuMTwvdGQ+CiAgICAgICAgICAgIDx0ZD4xLjY3PC90ZD4KICAgICAgICAgICAgPHRkPjIuNTI8L3RkPgogICAgICAgICAgICA8dGQ+Mi44OTwvdGQ+CiAgICAgICAgICAgIDx0ZD4zLjQ0PC90ZD4KICAgICAgICAgICAgPHRkPjI5Ljk8L3RkPgogICAgICAgICAgICA8dGQ+MjYuMTwvdGQ+CiAgICAgICAgICAgIDx0ZD4yMS45PC90ZD4KICAgICAgICA8L3RyPgogICAgICAgIDx0cj4KICAgICAgICAgICAgPHRkIGNvbHNwYW49IjciPuW5syDlnYc8L3RkPgogICAgICAgICAgICA8dGQ+MjIuNDwvdGQ+CiAgICAgICAgICAgIDx0ZD4xNy4xPC90ZD4KICAgICAgICAgICAgPHRkPjE1LjM8L3RkPgogICAgICAgIDwvdHI+CiAgICAgICAgPHRyPgogICAgICAgICAgICA8dGQ+TVUuTzwvdGQ+CiAgICAgICAgICAgIDx0ZD7nvo7lhYnnp5HmioA8L3RkPgogICAgICAgICAgICA8dGQ+MTI0LjIxPC90ZD4KICAgICAgICAgICAgPHRkPjEzOTAuMDc8L3RkPgogICAgICAgICAgICA8dGQ+MC43MDwvdGQ+CiAgICAgICAgICAgIDx0ZD44LjI5PC90ZD4KICAgICAgICAgICAgPHRkPjEyLjY4PC90ZD4KICAgICAgICAgICAgPHRkPjE0LjYwPC90ZD4KICAgICAgICAgICAgPHRkPjE0Ljk8L3RkPgogICAgICAgICAgICA8dGQ+OS44PC90ZD4KICAgICAgICAgICAgPHRkPjguNTwvdGQ+CiAgICAgICAgPC90cj4KICAgIDwvdGJvZHk+CjwvdGFibGU+CgrotYTmlpnmnaXmupDvvJpCbG9vbWJlcmfjgIHlm73ms7DmtbfpgJrnoJTnqbbmiYA=

PHRhYmxlPgogICAgPHRoZWFkPgogICAgICAgIDx0cj4KICAgICAgICAgICAgPHRoIHJvd3NwYW49IjIiPuiCoeelqOS7o+eggTwvdGg+CiAgICAgICAgICAgIDx0aCByb3dzcGFuPSIyIj7ogqHnpajlkI3np7A8L3RoPgogICAgICAgICAgICA8dGggcm93c3Bhbj0iMiI+5pS255uY5Lu3PGJyPu+8iOe+juWFgy/ogqHvvIk8L3RoPgogICAgICAgICAgICA8dGggcm93c3Bhbj0iMiI+5oC75biC5YC8PGJyPu+8iOS6v+e+juWFg++8iTwvdGg+CiAgICAgICAgICAgIDx0aCBjb2xzcGFuPSI0Ij7mgLvokKXmlLbvvIjkur/nvo7lhYPvvIk8L3RoPgogICAgICAgICAgICA8dGggY29sc3Bhbj0iMyI+UFMg77yI5YCN77yJPC90aD4KICAgICAgICA8L3RyPgogICAgICAgIDx0cj4KICAgICAgICAgICAgPHRkPjIwMjQ8L3RkPgogICAgICAgICAgICA8dGQ+MjAyNUU8L3RkPgogICAgICAgICAgICA8dGQ+MjAyNkU8L3RkPgogICAgICAgICAgICA8dGQ+MjAyN0U8L3RkPgogICAgICAgICAgICA8dGQ+MjAyNUU8L3RkPgogICAgICAgICAgICA8dGQ+MjAyNkU8L3RkPgogICAgICAgICAgICA8dGQ+MjAyN0U8L3RkPgogICAgICAgIDwvdHI+CiAgICA8L3RoZWFkPgogICAgPHRib2R5PgogICAgICAgIDx0cj4KICAgICAgICAgICAgPHRkPldEQy5PPC90ZD4KICAgICAgICAgICAgPHRkPuilv+mDqOaVsOaNrjwvdGQ+CiAgICAgICAgICAgIDx0ZD45MC40OTwvdGQ+CiAgICAgICAgICAgIDx0ZD4zMTMuOTwvdGQ+CiAgICAgICAgICAgIDx0ZD4xMzAuMDwvdGQ+CiAgICAgICAgICAgIDx0ZD45NS4yPC90ZD4KICAgICAgICAgICAgPHRkPjEwMy43PC90ZD4KICAgICAgICAgICAgPHRkPjEwNS40PC90ZD4KICAgICAgICAgICAgPHRkPjMuMzwvdGQ+CiAgICAgICAgICAgIDx0ZD4zLjA8L3RkPgogICAgICAgICAgICA8dGQ+My4wPC90ZD4KICAgICAgICA8L3RyPgogICAgICAgIDx0cj4KICAgICAgICAgICAgPHRkPlNUWC5PPC90ZD4KICAgICAgICAgICAgPHRkPuW4jOaNt+enkeaKgDwvdGQ+CiAgICAgICAgICAgIDx0ZD4xODMuOTg8L3RkPgogICAgICAgICAgICA8dGQ+MzkxLjM8L3RkPgogICAgICAgICAgICA8dGQ+NjUuNTwvdGQ+CiAgICAgICAgICAgIDx0ZD45MS4wPC90ZD4KICAgICAgICAgICAgPHRkPjEwNS42PC90ZD4KICAgICAgICAgICAgPHRkPjExNC42PC90ZD4KICAgICAgICAgICAgPHRkPjQuMzwvdGQ+CiAgICAgICAgICAgIDx0ZD4zLjc8L3RkPgogICAgICAgICAgICA8dGQ+My40PC90ZD4KICAgICAgICA8L3RyPgogICAgICAgIDx0cj4KICAgICAgICAgICAgPHRkPlJNQlMuTzwvdGQ+CiAgICAgICAgICAgIDx0ZD5SYW1idXM8L3RkPgogICAgICAgICAgICA8dGQ+NzUuNDA8L3RkPgogICAgICAgICAgICA8dGQ+ODEuMTwvdGQ+CiAgICAgICAgICAgIDx0ZD41LjY8L3RkPgogICAgICAgICAgICA8dGQ+Ny4xPC90ZD4KICAgICAgICAgICAgPHRkPjguMDwvdGQ+CiAgICAgICAgICAgIDx0ZD45LjI8L3RkPgogICAgICAgICAgICA8dGQ+MTEuNTwvdGQ+CiAgICAgICAgICAgIDx0ZD4xMC4yPC90ZD4KICAgICAgICAgICAgPHRkPjguODwvdGQ+CiAgICAgICAgPC90cj4KICAgICAgICA8dHI+CiAgICAgICAgICAgIDx0ZCBjb2xzcGFuPSI0Ij7lubPlnYc8L3RkPgogICAgICAgICAgICA8dGQ+PC90ZD4KICAgICAgICAgICAgPHRkPjwvdGQ+CiAgICAgICAgICAgIDx0ZD48L3RkPgogICAgICAgICAgICA8dGQ+PC90ZD4KICAgICAgICAgICAgPHRkPjYuNDwvdGQ+CiAgICAgICAgICAgIDx0ZD41LjY8L3RkPgogICAgICAgICAgICA8dGQ+NS4xPC90ZD4KICAgICAgICA8L3RyPgogICAgICAgIDx0cj4KICAgICAgICAgICAgPHRkPk1VLk88L3RkPgogICAgICAgICAgICA8dGQ+576O5YWJ56eR5oqAPC90ZD4KICAgICAgICAgICAgPHRkPjEyNC4yMTwvdGQ+CiAgICAgICAgICAgIDx0ZD4xMzkwLjA3PC90ZD4KICAgICAgICAgICAgPHRkPjI1MS4xMTwvdGQ+CiAgICAgICAgICAgIDx0ZD4zNzMuMTc8L3RkPgogICAgICAgICAgICA8dGQ+NDg5LjU5PC90ZD4KICAgICAgICAgICAgPHRkPjU2MC4xOTwvdGQ+CiAgICAgICAgICAgIDx0ZD4zLjc8L3RkPgogICAgICAgICAgICA8dGQ+Mi44PC90ZD4KICAgICAgICAgICAgPHRkPjIuNTwvdGQ+CiAgICAgICAgPC90cj4KICAgIDwvdGJvZHk+CjwvdGFibGU+CgrotYTmlpnmnaXmupDvvJpCbG9vbWJlcmfjgIHlm73ms7DmtbfpgJrnoJTnqbbmiYA=

fCDkuqflk4EgICB8IOmhueebriAgICAgICAgIHwgMjAyMyAgICAgICB8IDIwMjQgICAgICAgIHwgMjAyNUUgICAgICAgfCAyMDI2RSAgICAgICAgfCAyMDI3RSAgICAgICAgfAp8LS0tLS0tLS18LS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLXwKfCBEUkFNICAgfCDmlLblhaXvvIjnmb7kuIfnvo7lhYPvvIkgfCAxMDk3OC4wMCAgIHwgMTc2MDMuMDAgICAgfCAyODM2Mi4wNCAgICB8IDM4NzIyLjQ5ICAgICB8IDQ1Mjk5Ljg0ICAgICB8CnwgRFJBTSAgIHwgeW9577yIJe+8iSAgICAgIHwgLSAgICAgICAgICB8IDYwLjM1JSAgICAgIHwgNjEuMTIlICAgICAgfCAzNi41MyUgICAgICAgfCAxNi45OSUgICAgICAgfAp8IE5BTkQgICB8IOaUtuWFpe+8iOeZvuS4h+WFg++8iSAgIHwgNDIwNi4wMCAgICB8IDcyMjcuMDAgICAgIHwgODY0Ny4zNiAgICAgfCA5ODc2LjQ3ICAgICAgfCAxMDM1OS4xNSAgICAgfAp8IE5BTkQgICB8IHlvee+8iCXvvIkgICAgICB8IC0gICAgICAgICAgfCA3MS44MyUgICAgICB8IDE5LjY1JSAgICAgIHwgMTQuMjElICAgICAgIHwgNC44OSUgICAgICAgIHwKfCBPdGhlciAgfCDmlLblhaXvvIjnmb7kuIflhYPvvIkgICB8IDM1Ni4wMCAgICAgfCAyODEuMDAgICAgICB8IDMwOC4wMCAgICAgIHwgMzYwLjAwICAgICAgIHwgMzYwLjAwICAgICAgIHwKfCBPdGhlciAgfCB5b3nvvIgl77yJICAgICAgfCAtICAgICAgICAgIHwgLTIxLjA3JSAgICAgfCA5LjYxJSAgICAgICB8IDE2Ljg4JSAgICAgICB8IDAuMDAlICAgICAgICB8Cnwg5ZCI6K6hICAgfCDmlLblhaXvvIjnmb7kuIfnvo7lhYPvvIkgfCAxNTU0MC4wMCAgIHwgMjUxMTEuMDAgICAgfCAzNzMxNy40MCAgICB8IDQ4OTU4Ljk3ICAgICB8IDU2MDE4Ljk5ICAgICB8Cnwg5ZCI6K6hICAgfCB5b3nvvIgl77yJICAgICAgfCAtICAgICAgICAgIHwgNjEuNTklICAgICAgfCA0OC42MSUgICAgICB8IDMxLjIwJSAgICAgICB8IDE0LjQyJSAgICAgICB8Cnwg5ZCI6K6hICAgfCDmiJDmnKzvvIjnmb7kuIfnvo7lhYPvvIkgfCAxNjk1Ni4wMCAgIHwgMTk0OTguMDAgICAgfCAyMjQ5My4yMCAgICB8IDI3MTc4LjI3ICAgICB8IDMxMDM3LjI2ICAgICB8Cnwg5ZCI6K6hICAgfCB5b3nvvIgl77yJICAgICAgfCAtICAgICAgICAgIHwgMTQuOTklICAgICAgfCAxNS4zNiUgICAgICB8IDIwLjgzJSAgICAgICB8IDE0LjIwJSAgICAgICB8Cnwg5ZCI6K6hICAgfCDmr5vliKnnjofvvIgl77yJICAgIHwgLTkuMTElICAgICB8IDIyLjM1JSAgICAgIHwgMzkuNzIlICAgICAgfCA0NC40OSUgICAgICAgfCA0NC42MCUgICAgICAgfAoK6LWE5paZ5p2l5rqQ77yaQmxvb21iZXJn77yM5Zu95rOw5rW36YCa6K+B5Yi456CU56m2

fCDmjIfmoIflkI3np7AgICAgICAgICAgICAgICAgICAgICB8IEZZMjAyMiAgIHwgRlkyMDIzICAgfCBGWTIwMjQgICB8IEZZMjAyNUUgIHwgRlkyMDI2RSAgfCBGWTIwMjdFICB8CnwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLXwtLS0tLS0tLS0tfC0tLS0tLS0tLS18LS0tLS0tLS0tLXwtLS0tLS0tLS0tfC0tLS0tLS0tLS18Cnwg6Z2e546w6YeR6LCD5pW06aG55Yqg5oC7KOWQq+aKmOaXp+S4juaRiumUgCkgfCA3LDcyNyAgICB8IDEwLDI4OCAgIHwgOCw2MTMgICAgfCA5LDQwMSAgICB8IDEwLDc3MSAgIHwgMTIsMzI0ICAgfAp8IOiQpei/kOi1hOmHkeWPmOWKqCAgICAgICAgICAgICAgICAgfCAoMSwyNDUpICB8ICgyLDg5NikgIHwgKDg4NCkgICAgfCAoMiw5NDQpICB8ICg0MTcpICAgIHwgKDI2OCkgICAgfAp8IOe7j+iQpea0u+WKqOS6p+eUn+eahOeOsOmHkea1gemHjyAgICAgICB8IDE1LDE4MSAgIHwgMSw1NTkgICAgfCA4LDUwNyAgICB8IDE0LDgwNyAgIHwgMjMsNDY0ICAgfCAyNywxMTAgICB8Cnwg6LWE5pys5byA5pSvICAgICAgICAgICAgICAgICAgICAgfCAoMTIsMDY3KSB8ICg3LDY3NikgIHwgKDgsMzg2KSAgfCAoMTUsMjE2KSB8ICgyMSw4MjQpIHwgKDI0LDk3MSkgfAp8IOWFtuS7luaKlei1hOa0u+WKqOWKoOaAuyAgICAgICAgICAgICB8ICgzMzQpICAgIHwgMSw0ODUgICAgfCA3NyAgICAgICB8IDEsMjYyICAgIHwgKDIwOCkgICAgfCAoMjM4KSAgICB8Cnwg5oqV6LWE5rS75Yqo5Lqn55Sf55qE546w6YeR5rWB6YePICAgICAgIHwgKDExLDU4NSkgfCAoNiwxOTEpICB8ICg4LDMwOSkgIHwgKDEzLDk1MykgfCAoMjIsMDMyKSB8ICgyNSwyMDkpIHwKfCDmlK/ku5jogqHmga8gICAgICAgICAgICAgICAgICAgICB8ICg0NjEpICAgIHwgKDUwNCkgICAgfCAoNTEzKSAgICB8ICg1MjIpICAgIHwgKDUyMCkgICAgfCAoNTIwKSAgICB8Cnwg6ZW/5pyf6LSf5YC65Y+Y5YqoICAgICAgICAgICAgICAgICB8IDExICAgICAgIHwgNiw1MzIgICAgfCAoMzMyKSAgICB8IDIsNDc3ICAgIHwgLSAgICAgICAgfCAtICAgICAgICB8Cnwg6IKh56Wo5Y+R6KGMICAgICAgICAgICAgICAgICAgICAgfCAtICAgICAgICB8IC0gICAgICAgIHwgLSAgICAgICAgfCAtICAgICAgICB8IC0gICAgICAgIHwgLSAgICAgICAgfAp8IOiCoeS7veWbnui0rSAgICAgICAgICAgICAgICAgICAgIHwgKDIsNTU3KSAgfCAoNDI1KSAgICB8ICgzMDApICAgIHwgKDEwMCkgICAgfCAoNDAwKSAgICB8ICg0MDApICAgIHwKfCDlhbbku5bnrbnotYTmtLvliqjnjrDph5HmtYHlj5jliqjpobnliqDmgLsgfCAyNyAgICAgICB8ICg2MjApICAgIHwgKDY5NykgICAgfCAoMSw4NzEpICB8IC0gICAgICAgIHwgLSAgICAgICAgfAp8IOiejei1hOa0u+WKqOS6p+eUn+eahOeOsOmHkea1gemHjyAgICAgICB8ICgyLDk4MCkgIHwgNCw5ODMgICAgfCAoMSw4NDIpICB8ICgxNikgICAgIHwgKDkyMCkgICAgfCAoOTIwKSAgICB8Cnwg5rGH546H5Y+Y5YqoICAgICAgICAgICAgICAgICAgICAgfCAoMTA2KSAgICB8ICgzNCkgICAgIHwgNDAgICAgICAgfCAoMykgICAgICB8IC0gICAgICAgIHwgLSAgICAgICAgfAp8IOWPl+mZkOWItueOsOmHkeWKoOaAuyAgICAgICAgICAgICAgIHwgMjk1ICAgICAgfCAzMDYgICAgICB8IDMwNSAgICAgIHwgMzIgICAgICAgfCAyNCAgICAgICB8IDI0ICAgICAgIHwKfCDnjrDph5Hlj5jliqjvvIjlkKvlj5fpmZDliLbotYTph5HvvIkgICAgIHwgNTEwICAgICAgfCAzMTcgICAgICB8ICgxLDYwNCkgIHwgODM0ICAgICAgfCA1MTMgICAgICB8IDk4MiAgICAgIHwKfCDmnJ/liJ3njrDph5HjgIHnjrDph5HnrYnku7fnianlj4rlj5fpmZDliLbotYTph5EgfCAzNSwwNzcgIHwgMzcsMjM4ICAgfCAzMiw1NjcgICB8IDMxLDQ4MSAgIHwgMzcsMTgzICAgfCAzOSw4OTIgICB8Cnwg5pyf5pyr546w6YeR44CB546w6YeR562J5Lu354mp5Y+K5Y+X6ZmQ5Yi26LWE6YeRIHwgMzUsNTg3ICB8IDM3LDU1NSAgIHwgMzAsOTYzICAgfCAzMiwzMTUgICB8IDM3LDY5NiAgIHwgNDAsODc0ICAgfAoK6LWE5paZ5p2l5rqQ77ya5ZCM6Iqx6aG6IGlGaW5k77yM5Zu95rOw5rW36YCa6K+B5Yi456CU56m2

所属报告：