三个皮匠报告—— 严选型全行业研究报告分享下载平台,您的专属行业智库!
欢迎来到三个皮匠报告!
登录
帮助中心
首页
最新
研搜
BETA
合集
英文
会议
数据
财报库
政策库
自研报告
入驻
您的当前位置:
首页
>
行业数据
>
MLA与其他注意力机制的优劣对比
MLA与其他注意力机制的优劣对比
行业数据
下载Excel
下载图片
原图定位
图片属性
图片格式:PNG
图片大小:706KB
图片尺寸:2350*1034
同报告图片
/
4
5aW955qE77yM5Lul5LiL5piv5o+Q5Y+W55qE5Zu+54mH5Lit55qE5pWw5o2u5bm255Sf5oiQ55qE6KGo5qC877yM5bm25Zyo5LiL5pa55bGV56S65LqG6LWE5paZ5p2l5rqQ77yaCgp8IOazqOaEj+WKm+acuuWItiB8IOWtmOWCqOaWueW8jyB8IOeJueeCuSB8IOiDveWKmyB8CnwgLS0tIHwgLS0tIHwgLS0tIHwgLS0tIHwKfCBNSEEgfCDkuLrmr4/kuKrlpLTni6znq4vnvJPlrZhLZXktVmFsdWUgfCDkv6Hmga/lhajpnaLvvIzkvYbnvJPlrZjov4flpKfpmZDliLbmjqjnkIbmlYjnjocgfCDog73lipvlvLogfAp8IEdRQSB8IOWIhue7hOWFseS6q0tleS1WYWx1ZSB8IOW5s+ihoeWOi+e8qeeOh+S4juWkmuagt+aAp++8jOmcgOaJi+WKqOWIhue7hCB8IOiDveWKm+S4reetiSB8CnwgTVFBIHwg5aSa5aS05YWx5LqrS2V55ZKMVmFsdWUgfCDljovnvKnnjofpq5jvvIzkvYblj6/og73mjZ/lpLHlpJrlpLTlpJrmoLfmgKcgfCDog73lipvlvLEgfAp8IE1MQSB8IOS9juenqeiBlOWQiOWOi+e8qUtleS1WYWx1ZSB8IOmrmOaViOWOi+e8qe+8jOS9huWvueaKleW9seefqemYteiuvuiuoeaVj+aEnyB8IOiDveWKm+i+g01IQeW8uiB8CgrotYTmlpnmnaXmupDvvJpEZWVwU2Vlay1WMuiuuuaWh+OAgeaLm+WVhuivgeWIuA==
5aW955qE77yM5Lul5LiL5piv5o+Q5Y+W55qE5Zu+54mH5Lit55qE5pWw5o2u5bm255Sf5oiQ55qE6KGo5qC877yM5bm25Zyo5LiL5pa55bGV56S65LqG6LWE5paZ5p2l5rqQ77yaCgp8IOazqOaEj+WKm+acuuWItiB8IOWtmOWCqOaWueW8jyB8IOeJueeCuSB8IOiDveWKmyB8CnwgLS0tIHwgLS0tIHwgLS0tIHwgLS0tIHwKfCBNSEEgfCDkuLrmr4/kuKrlpLTni6znq4vnvJPlrZhLZXktVmFsdWUgfCDkv6Hmga/lhajpnaLvvIzkvYbnvJPlrZjov4flpKfpmZDliLbmjqjnkIbmlYjnjocgfCDog73lipvlvLogfAp8IEdRQSB8IOWIhue7hOWFseS6q0tleS1WYWx1ZSB8IOW5s+ihoeWOi+e8qeeOh+S4juWkmuagt+aAp++8jOmcgOaJi+WKqOWIhue7hCB8IOiDveWKm+S4reetiSB8CnwgTVFBIHwg5aSa5aS05YWx5LqrS2V55ZKMVmFsdWUgfCDljovnvKnnjofpq5jvvIzkvYblj6/og73mjZ/lpLHlpJrlpLTlpJrmoLfmgKcgfCDog73lipvlvLEgfAp8IE1MQSB8IOS9juenqeiBlOWQiOWOi+e8qUtleS1WYWx1ZSB8IOmrmOaViOWOi+e8qe+8jOS9huWvueaKleW9seefqemYteiuvuiuoeaVj+aEnyB8IOiDveWKm+i+g01IQeW8uiB8CgrotYTmlpnmnaXmupDvvJpEZWVwU2Vlay1WMuiuuuaWh+OAgeaLm+WVhuivgeWIuA==
所属报告:
计算机行业AIGC系列报告之DeepSeek深度解析:DeepSeek推动高性能AI普惠AI生态繁荣发展-250204(17页).pdf
打包全文图表
相关数据
对各媒体渠道广告的人均注意力集中度(%)
收藏
行业数据
2026-04-20
原图定位
查看详情
2014-2023全球媒体渠道用户注意力格局
fCDlubTku70gfCBPcGVuIEludGVybmV0ICglKSB8IFdhbGxlZCBHYXJkZW5zICglKSB8CnwtLS0tLS18LS0tLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLS0tLXwKfCAyMDE0IHwgMzggICAgICAgICAgICAgICAgfCA2MiAgICAgICAgICAgICAgICAgfAp8IDIwMTcgfCA1MSAgICAgICAgICAgICAgICB8IDQ5ICAgICAgICAgICAgICAgICB8CnwgMjAyMCB8IDU3ICAgICAgICAgICAgICAgIHwgNDMgICAgICAgICAgICAgICAgIHwKfCAyMDIzIHwgNjEgICAgICAgICAgICAgICAgfCAzOSAgICAgICAgICAgICAgICAgfAoK6LWE5paZ5p2l5rqQ77yaR2xvYmFsV2ViSW5kZXjjgIHljY7ms7DnoJTnqbY=
收藏
竞争格局
2026-03-13
原图定位
查看详情
稀疏注意力机制(DSA)
fCDnu4Tku7Yv5YWD57SgICAgICAgICAgICAgICAgIHwg5o+P6L+wICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8CnwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS18Cnwg5Yqo5oCB56iA55aP5rOo5oSP5Yqb5py65Yi277yIRFNB77yJIHwg5pW05L2T5p625p6E5ZCN56ewICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfAp8IElucHV0IEhpZGRlbiAgICAgICAgICAgIHwg6L6T5YWl6ZqQ6JeP5bGC77yM5qCH6K6w5Li6IFwoIGhfdCBcKSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfAp8IE91dHB1dCBIaWRkZW4gICAgICAgICAgIHwg6L6T5Ye66ZqQ6JeP5bGC77yM5qCH6K6w5Li6IFwoIHVfdCBcKSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfAp8IE11bHRpIC0gUXVlcnkgQXR0ZW50aW9uICB8IOaguOW/g+azqOaEj+WKm+aooeWdlyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8CnwgVG9wIC0gayBTZWxlY3RvciAgICAgICAgfCDnlKjkuo7pgInlj5YgdG9wIC0gayDlhYPntKDnmoTmqKHlnZcgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfAp8IExpZ2h0bmluZyBJbmRleGVyICAgICAgICB8IOe0ouW8leebuOWFs+aooeWdlyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwKfCBSb1BFICAgICAgICAgICAgICAgICAgICB8IOW6lOeUqOS6juebuOW6lOWQkemHj+S7peWunueOsOaXi+i9rOS9jee9rue8lueggeeahOaTjeS9nCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8CgrotYTmlpnmnaXmupDvvJrjgIpEZWVwU2VlayAtIFYzLjI6IFB1c2hpbmcgdGhlIEZyb250aWVyIG9mIE9wZW4gTGFyZ2UgTGFuZ3VhZ2UgTW9kZWxz44CL
收藏
行业数据
2025-12-10
原图定位
查看详情
通过拼接或更通用的交叉注意力机制对潜在扩散模型(LDMs)进行条件约
fCDmqKHlnZcgICAgICAgICB8IOe7hOS7ti/lip/og70gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwKfC0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tfAp8IFBpeGVsIFNwYWNlICB8IOe8lueggeWZqCBcKCBcbWF0aGNhbHtFfSBcKe+8iOWwhiBcKCB4IFwpIOaYoOWwhOWIsCBcKCB6IFwp77yJ44CB6Kej56CB5ZmoIFwoIFxtYXRoY2Fse0R9IFwp77yI5bCGIFwoIHogXCkg5pig5bCE5ZueIFwoIFx0aWxkZXt4fSBcKe+8iSB8CnwgTGF0ZW50IFNwYWNlIHwg5omp5pWj6L+H56iL77yIRGlmZnVzaW9uIFByb2Nlc3PvvInvvJvljrvlmapVIC0gTmV077yIRGVub2lzaW5nIFUgLSBOZXQgXCggXGVwc2lsb25fXHRoZXRhIFwp77yJ77yb5Y675Zmq5q2l6aqk77yIZGVub2lzaW5nIHN0ZXDvvInvvJvkuqTlj4nms6jmhI/lipvvvIhjcm9zc2F0dGVudGlvbu+8jOWQqyBcKCBRL0tWIFwp77yJ77yb5byA5YWz77yIc3dpdGNo77yJ77yb6Lez6LeD6L+e5o6l77yIc2tpcCBjb25uZWN0aW9u77yJ77yb5ou85o6l77yIY29uY2F077yJIHwKfCBDb25kaXRpb25pbmcgfCDor63kuYnlnLDlm77vvIhTZW1hbnRpYyBNYXDvvIzlkKtUZXh077yJ77yb6KGo56S677yIUmVwcmVzZW50YXRpb25z77yM5ZCrSW1hZ2Vz77yJ77yb5p2h5Lu25qih5Z2XIFwoIFx0YXVfXHRoZXRhIFwpIHwKCuadpea6kO+8mlJvYmluIFJvbWJhY2gg562J44CKSGlnaCAtIFJlc29sdXRpb24gSW1hZ2UgU3ludGhlc2lzIHdpdGggTGF0ZW50IERpZmZ1c2lvbiBNb2RlbHPjgIsoMjAyMinvvIzlm73ph5Hor4HliLjnoJTnqbbmiYA=
收藏
行业数据
2025-11-25
原图定位
查看详情
输入图像→CNN提取特征→带注意力的RNN(LSTM)→逐词生成描述
fCDmraXpqqQgfCDmk43kvZwv57uE5Lu2ICAgICAgICAgICAgICAgICAgICAgfCDor7TmmI4gICAgICAgICAgICAgICAgICAgICB8CnwtLS0tLS18LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tfAp8IDEgICAgfCBJbnB1dCBJbWFnZSAgICAgICAgICAgICAgICAgIHwg6L6T5YWl5Zu+5YOPICAgICAgICAgICAgICAgICB8CnwgMiAgICB8IENvbnZvbHV0aW9uYWwgRmVhdHVyZSBFeHRyYWN0aW9uIHwgQ05O5o+Q5Y+W54m55b6B77yM6L6T5Ye6MTTDlzE0IEZlYXR1cmUgTWFwIHwKfCAzICAgIHwgUk5OIHdpdGggYXR0ZW50aW9uIG92ZXIgdGhlIGltYWdlIHwg5bim5rOo5oSP5Yqb55qEUk5O77yITFNUTe+8ie+8jOWvueWbvuWDj+S4jeWQjOWMuuWfn+aWveWKoOazqOaEj+WKm+acuuWItiB8CnwgNCAgICB8IFdvcmQgYnkgd29yZCBnZW5lcmF0aW9uICAgICAgfCDpgJDor43nlJ/miJDlm77lg4/mj4/ov7AgICAgICAgICB8CgrotYTmlpnmnaXmupDvvJpLZWx2aW4gWHUg562J44CKU2hvdywgQXR0ZW5kIGFuZCBUZWxsOiBOZXVyYWwgSW1hZ2UgQ2FwdGlvbiBHZW5lcmF0aW9uIHdpdGggVmlzdWFsIEF0dGVudGlvbuOAiygyMDE1Ke+8jOWbvemHkeivgeWIuOeglA==
收藏
行业数据
2025-11-25
原图定位
查看详情
在MHA、GQA中大量存在于keysvalues中的KV缓存(带阴影表示),MLA
fCBNdWx0aS1IZWFkIEF0dGVudGlvbiAoTUhBKSB8IEdyb3VwZWQtUXVlcnkgQXR0ZW50aW9uIChHUUEpIHwgTXVsdGktUXVlcnkgQXR0ZW50aW9uIChNUUEpIHwgTXVsdGktSGVhZCBMYXRlbnQgQXR0ZW50aW9uIChNTEEpIHwKfC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS18CnwgVmFsdWVzICAgICAgICAgICAgICAgICAgICB8IFZhbHVlcyAgICAgICAgICAgICAgICAgICAgICB8IFZhbHVlcyAgICAgICAgICAgICAgICAgICAgfCBWYWx1ZXMgICAgICAgICAgICAgICAgICAgICAgICAgICAgfAp8IEtleXMgICAgICAgICAgICAgICAgICAgICAgfCBLZXlzICAgICAgICAgICAgICAgICAgICAgICAgfCBLZXlzICAgICAgICAgICAgICAgICAgICAgIHwgS2V5cyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwKfCBRdWVyaWVzICAgICAgICAgICAgICAgICAgIHwgUXVlcmllcyAgICAgICAgICAgICAgICAgICAgIHwgUXVlcmllcyAgICAgICAgICAgICAgICAgICB8IFF1ZXJpZXMgICAgICAgICAgICAgICAgICAgICAgICAgICB8CgrotYTmlpnmnaXmupDvvJrngavlsbHlvJXmk47lvIDlj5HogIXnpL7ljLrvvIzlhazkvJflj7figJzlpKfmqKHlnovoh6rnhLbor63oqIDlpITnkIbigJ3vvIzkuJzljJfor4HliLg=
收藏
行业数据
2025-05-14
原图定位
查看详情
不同注意力机制KV缓存和性能对比
fCAgQXR0ZW50aW9uIE1lY2hhbmlzbSAgfCAgS1YgQ2FjaGUgcGVyIFRva2VuICgjIEVsZW1lbnQpICB8ICBDYXBhYmlsaXR5ICB8CnwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS18CnwgTXVsdGktSGVhZCBBdHRlbnRpb24gKE1IQSkgfCAybmhkaGxsIHwgU3Ryb25nIHwKfCBHcm91cGVkLVF1ZXJ5IEF0dGVudGlvbiAoR1FBKSB8IDJuZ2RobCB8IE1vZGVyYXRlIHwKfCBNdWx0aS1RdWVyeSBBdHRlbnRpb24gKE1RQSkgfCAyZGhsIHwgV2VhayB8CnwgKipNTEEgKE91cnMpKiogfCBcKChkX2MgKyBkX2heUilsIOKJiCBcZnJhY3s5fXsyfWRfaGxcKSB8IFN0cm9uZ2VyIHwKCui1hOaWmeadpea6kO+8mkRlZXBTZWVrLVYy5oqA5pyv5oql5ZGK77yM5rCR55Sf6K+B5Yi456CU56m26Zmi
收藏
行业数据
2025-05-09
原图定位
查看详情
MLA通过低秩联合压缩键值(Key-Value),将它们压缩为一个潜在
5oqx5q2J77yM5oiR5peg5rOV5o+Q5Y+W5Zu+54mH5Lit55qE5pWw5o2u5oiW55Sf5oiQ6KGo5qC844CC5aaC5p6c5L2g5pyJ5YW25LuW6Zeu6aKY5oiW6ZyA6KaB5biu5Yqp77yM6K+35ZGK6K+J5oiR77yB
收藏
行业数据
2025-02-17
原图定位
查看详情
MLA及DeepSeekMOE基础架构
fCDmqKHlnZflkI3np7AgICAgICAgfCDmj4/ov7AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8CnwtLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tfAp8IFRyYW5zZm9ybWVyIEJsb2NrIHwg5YyF5ZCrRmVlZC1Gb3J3YXJkIE5ldHdvcmvjgIFSTVNOb3Jt5ZKMQXR0ZW50aW9u5qih5Z2XICAgICAgICAgICAgIHwKfCBGZWVkLUZvcndhcmQgTmV0d29yayB8IOWJjeWQkeS8oOaSree9kee7nCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8CnwgUk1TTm9ybSAgICAgICAgIHwg5qCH5YeG5YyW5bGCICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8CnwgQXR0ZW50aW9uICAgICAgIHwg5rOo5oSP5Yqb5py65Yi2ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfAp8IERlZXBTZWVrTW9FICAgICB8IOWMheWQq+WkmuS4quS4k+Wutu+8iEV4cGVydO+8ieWSjOi3r+eUseWZqO+8iFJvdXRlcu+8iSAgICAgICAgICAgICAgICAgICAgIHwKfCBNdWx0aS1IZWFkIExhdGVudCBBdHRlbnRpb24gKE1MQSkgfCDlpJrlpLTmvZzlnKjms6jmhI/lipvmnLrliLbvvIzljIXlkKtNdWx0aS1IZWFkIEF0dGVudGlvbuaooeWdlyAgICAgICAgICAgICB8CnwgTXVsdGktSGVhZCBBdHRlbnRpb24gfCDlpJrlpLTms6jmhI/lipvmnLrliLYgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwKCui1hOaWmeadpea6kO+8mkRlZXBTZWVrLVYzIOiuuuaWh+OAgeawkeeUn+ivgeWIuOeglOeptumZog==
收藏
行业数据
2025-02-10
原图定位
查看详情
Minimax-01使用的线性注意力机制将计算复杂度降低至O(N)
fCDnrpfms5XnsbvlnosgICAgICAgfCDorqHnrpflpI3mnYLluqYgfAp8LS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS18CnwgU29mdG1heCBBdHRlbnRpb24gfCBPKE7CsmQpICAgICB8CnwgTGluZWFyIEF0dGVudGlvbiAgIHwgTyhOZMKyKSAgICAgfAoKCui1hOaWmeadpea6kO+8muOAik1pbmltYXgtMDHmioDmnK/miqXlkYrjgIvvvIznlLPkuIflro/mupDnoJTnqbY=
收藏
行业数据
2025-02-08
原图定位
查看详情
图7.缩放点积注意力(ScaledDot-ProductAttention)原理和Softmax公式
fCDlm743Lue8qeaUvueCueenr+azqOaEj+WKm++8iFNjYWxlZCBEb3QtUHJvZHVjdCBBdHRlbnRpb27vvInljp/nkIblkowgU29mdG1heCDlhazlvI8gfCAgfAp8IC0tLSB8IC0tLSB8Cnwg6L6T5YWl5p+l6K+i77yIUe+8ieOAgemUru+8iEvvvInlkozlgLzvvIhW77yJ77yM6L+b6KGM5LiA5qyh55+p6Zi15LmY5rOV77yITWF0TXVs77yJIHwgIHwKfCDov5vooYznvKnmlL7mk43kvZwgfCAgfAp8IOWPr+mAieeahOaOqeeggeaTjeS9nCB8ICB8CnwgU29mdG1heOaTjeS9nCB8ICB8Cnwg5pyA5ZCO5YaN6L+b6KGM5LiA5qyh55+p6Zi15LmY5rOV77yITWF0TXVs77yJ5b6X5Yiw6L6T5Ye6IHwgIHwKfCBBdHRlbnRpb24oUSxLLFYpID0gc29mdG1heCggUUtUL+KImmRrIClWIHwgIHwKfCAx77yJ6aaW5YWI77yM6K6h566X5p+l6K+i5ZCR6YePUeS4jumUruWQkemHj+eahOi9rOe9rktU55qE54K556ev77yM5b6X5Yiw5LiA5Liq5pyq57yp5pS+55qE5b6X5YiG55+p6Zi1IHwgIHwKfCAy77yJ6ZqP5ZCO77yM5bCG6L+Z5Liq5b6X5YiG55+p6Zi155qE5q+P5Liq5YWD57Sg6Zmk5Lulc3FydChkaynov5vooYznvKnmlL7vvIzlvpfliLDnvKnmlL7lkI7nmoTlvpfliIbnn6npmLUgfCAgfAp8IDPvvInmjqXnnYDvvIzlr7nnvKnmlL7lkI7nmoTlvpfliIbnn6npmLXlupTnlKhTb2Z0bWF45Ye95pWw77yM5bCG5q+P5Liq5YWD57Sg6L2s5o2i5Li65qaC546H5YC877yI5Y2z5rOo5oSP5Yqb5p2D6YeN77yJ77yM6L+Z5Lqb5p2D6YeN6KGo56S65LqG5q+P5Liq5L2N572u55qE6YeN6KaB5oCn56iL5bqm77yM5LiU5omA5pyJ5p2D6YeN5LmL5ZKM5Li6MSB8ICB8CnwgNO+8ieacgOWQju+8jOWwhuW+l+WIsOeahOazqOaEj+WKm+adg+mHjeS4juWAvOWQkemHj1bov5vooYzliqDmnYPmsYLlkozvvIzlvpfliLDmnIDnu4jnmoTms6jmhI/lipvovpPlh7ogfCAgfAoK6LWE5paZ5p2l5rqQ77yaR29vZ2xlIEJyYWlu77yM6LSi6YCa6K+B5Yi456CU56m25omA
收藏
行业数据
2025-01-20
原图定位
查看详情
图8.多头注意力(Multi-HeadAttention)原理和MHA公式
fCDlm744LuWkmuWktOazqOaEj+WKm++8iE11bHRpLUhlYWQgQXR0ZW50aW9u77yJ5Y6f55CG5ZKMIE1IQSDlhazlvI8gfCAgfAp8IC0tLSB8IC0tLSB8Cnwg5aSa5aS05rOo5oSP5Yqb77yITXVsdGktSGVhZCBBdHRlbnRpb27vvInmnLrliLbnmoTlhazlvI/ooajovr7vvJrlsIblpJrkuKrms6jmhI/lipvvvIhoZWFkMeWIsGhlYWRo77yJ55qE6L6T5Ye66L+b6KGM5ou85o6l77yIQ29uY2F077yJ77yM54S25ZCO5YaN6YCa6L+H5LiA5Liq5Y+v5a2m5Lmg55qE5Y+C5pWw55+p6Zi1VzDov5vooYznur/mgKflj5jmjaLvvIzlvpfliLDmnIDnu4jnmoTlpJrlpLTms6jmhI/lipvovpPlh7rjgIIgfCAgfAp8IE11bHRpSGVhZChRLCBLLCBWKSA9IENvbmNhdChoZWFkMSwgLi4uLCBoZWFkaClXXjAgfCAgfAp8IHdoZXJlIGhlYWRpID0gQXR0ZW50aW9uKFFXXlFpLCBLV15LaSwgVldeVmkpIHwgIHwKfCDov5nph4znmoRoZWFkaeihqOekuuesrGnkuKrms6jmhI/lipvlpLTnmoTorqHnrpfnu5PmnpzjgIJBdHRlbnRpb27mmK/ln7rmnKznmoTms6jmhI/lipvorqHnrpflh73mlbDvvIjlpoLliY3pnaLmj5DliLDnmoTnvKnmlL7ngrnnp6/ms6jmhI/lipvnrYnvvInvvIxXXlFpLCBXXktpLCBXXlZpIOaYr+WPr+WtpuS5oOeahOWPguaVsOefqemYte+8jOeUqOS6juWvuVEsIEssIFbov5vooYznur/mgKflj5jmjaLvvIzku47ogIzlvpfliLDmr4/kuKrms6jmhI/lipvlpLTnmoTnibnlrprovpPlhaXjgILpgJrov4fov5nnp43mlrnlvI/vvIzmr4/kuKrms6jmhI/lipvlpLTlj6/ku6XlhbPms6jovpPlhaXmlbDmja7nmoTkuI3lkIzmlrnpnaLmiJblrZDnqbrpl7TvvIzlop7liqDmqKHlnovnmoTooajovr7og73lipvlkozlr7nkv6Hmga/nmoTmjZXmjYnog73lipvjgIIgfCAgfA==
收藏
行业数据
2025-01-20
原图定位
查看详情
MLA和MHA在困难测评集上性能和KVcache对比
fCBCZW5jaG1hcmsgKE1ldHJpYykgfCAjIFNob3RzIHwgU21hbGwgTW9FIHcvIE1IQSB8IFNtYWxsIE1vRSB3LyBNTEEgfCBMYXJnZSBNb0Ugdy8gTUhBIHwgTGFyZ2UgTW9FIHcvIE1MQSB8CnwtLS0tLS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tLS18CnwgIyBBY3RpdmF0ZWQgUGFyYW1zICB8IC0gICAgICAgfCAyLjVCICAgICAgICAgICAgICB8IDIuNEIgICAgICAgICAgICAgIHwgMjUuMEIgICAgICAgICAgICAgfCAyMS41QiAgICAgICAgICAgICB8CnwgIyBUb3RhbCBQYXJhbXMgICAgICB8IC0gICAgICAgfCAxNS44QiAgICAgICAgICAgICB8IDE1LjdCICAgICAgICAgICAgIHwgMjUwLjhCICAgICAgICAgICAgfCAyNDcuNEIgICAgICAgICAgICB8CnwgS1YgQ2FjaGUgcGVyIFRva2VuICgjIEVsZW1lbnQpIHwgLSAgICAgICB8IDExMC42SyAgICAgICAgICAgIHwgMTUuNksgICAgICAgICAgICAgfCA4NjAuMksgICAgICAgICAgICB8IDM0LjZLICAgICAgICAgICAgIHwKfCBCQkggKEVNKSAgICAgICAgICAgIHwgMy1zaG90ICB8IDM3LjkgICAgICAgICAgICAgIHwgMzkuMCAgICAgICAgICAgICAgfCA0Ni42ICAgICAgICAgICAgICB8IDUwLjcgICAgICAgICAgICAgIHwKfCBNTUxVIChBY2MuKSAgICAgICAgIHwgNS1zaG90ICB8IDQ4LjcgICAgICAgICAgICAgIHwgNTAuMCAgICAgICAgICAgICAgfCA1Ny41ICAgICAgICAgICAgICB8IDU5LjAgICAgICAgICAgICAgIHwKfCBDLUV2YWwgKEFjYy4pICAgICAgIHwgNS1zaG90ICB8IDUxLjYgICAgICAgICAgICAgIHwgNTAuOSAgICAgICAgICAgICAgfCA1Ny45ICAgICAgICAgICAgICB8IDU5LjIgICAgICAgICAgICAgIHwKfCBDTU1MVSAoQWNjLikgICAgICAgIHwgNS1zaG90ICB8IDUyLjMgICAgICAgICAgICAgIHwgNTMuNCAgICAgICAgICAgICAgfCA2MC43ICAgICAgICAgICAgICB8IDYyLjUgICAgICAgICAgICAgIHwKCui1hOaWmeadpea6kO+8muOAikRlZXBTZWVrLVYyOiBBIFN0cm9uZywgRWNvbm9taWNhbCwgYW5kIEVmZmljaWVudA==
收藏
其它
2024-06-12
原图定位
查看详情
局部因果注意力机制的可视化(左上角为全量注意力计算,其余为各种局部注意力算法)
fCDlm77ooajnvJblj7cgfCDms6jmhI/lipvmnLrliLbnsbvlnosgfCDmj4/ov7AgfAp8IDotLTogfCA6LS06IHwgOi0tOiB8CnwgKGEpIHwgRGVuc2UgQXR0ZW50aW9uIHwg5YWo5bGA5rOo5oSP5Yqb6K6h566XIHwKfCAoYikgfCBCbG9jay13aXNlIEF0dGVudGlvbiB8IOWdl+eKtuazqOaEj+WKm+acuuWItiAoQj0zKSB8CnwgKGMpIHwgU2xpZGluZy1XaW5kb3cgQXR0ZW50aW9uIHwg5ruR5Yqo56qX5Y+j5rOo5oSP5Yqb5py65Yi2ICh3PTMpIHwKfCAoZCkgfCBEaWxhdGVkLVdpbmRvdyBBdHRlbnRpb24gfCDohqjog4Dnqpflj6Pms6jmhI/lipvmnLrliLYgKHc9MywgZD0yKSB8CnwgKGUpIHwgR2xvYmFsLUxvY2FsIEh5YnJpZCBBdHRlbnRpb24gKFNwZWNpYWwgVG9rZW4pIHwg5YWo5bGALeWxgOmDqOa3t+WQiOazqOaEj+WKm+acuuWItiAo54m55q6K5qCH6K6wKSB8CnwgKGYpIHwgR2xvYmFsLUxvY2FsIEh5YnJpZCBBdHRlbnRpb24gKEdsb2JhbCBUb2tlbikgfCDlhajlsYAt5bGA6YOo5re35ZCI5rOo5oSP5Yqb5py65Yi2ICjlhajlsYDmoIforrApIHwKfCAoZykgfCBBdHRlbnRpb24gU2luayB8IOazqOaEj+WKm+S4i+ayiSAoU3RyZWFtTExNKSB8CnwgKGgpIHwgU2hpZnQtU2hvcnQgQXR0ZW50aW9uIHwg56e75L2NLeefreazqOaEj+WKm+acuuWItiAoTG9uZ0xvUkEpIHwKCui1hOaWmeadpea6kO+8muOAikFkdmFuY2luZyBUcmFuc2Zvcm1lciBBcmNoaXRlY3R1cmUgaW4gTG9uZy1Db250ZXh0IExhcmdlIExhbmd1YWdlIE1vZGVsczogQSBDb21wcmVoZW5zaXZlIFN1cnZleeOAi++8jEh1YW5n
收藏
其它
2024-03-29
原图定位
查看详情
最新数据
全球全年龄段奶粉市场规模(亿美元)
收藏
行业数据
2026-07-30
原图定位
查看详情
全球酸奶及发酵乳制品市场规模(亿美元)
收藏
行业数据
2026-07-30
原图定位
查看详情
全球药用级益生菌市场规模(百万美元)
收藏
行业数据
2026-07-30
原图定位
查看详情
全球益生菌膳食补充剂市场规模及增速
收藏
行业数据
2026-07-30
原图定位
查看详情
全球拥有AKK专利菌株的核心企业
收藏
行业数据
2026-07-30
原图定位
查看详情
V3 通过多头潜在注意力(Multi-Head Latent Attention, MLA)机制提升推理效率。MLA 方法借助低秩键值联合压缩,可使 MLA 的性能优于 MHA,但所需的 KV 缓存量显著减少,从而减少模型所需算力。在自回归生成(如文本生成)过程中,模型需逐步生成每个 token,并缓存所有历史 token 的 Key 和 Value 矩阵以供后续计算。若以图书馆管理作类比,图书馆有多个独立研究小组(每个小组对应一个“头”),MHA 方法中每个小组需要查阅完整的书籍库(原始 Key 和Value)。此方法的优点是每个小组能深度研究不同领域(捕捉多样化信息),成果全面。但缺点是 1)存储压力大,每个小组都要复制一套完整书籍(KV 缓存占用高);2)空间浪费,图书馆需要为每个小组开辟独立书架(显存爆炸)。MLA 类似在图书馆引入智能压缩技术,将书籍转换为精华摘要(低秩压缩),每个小组(头)根据摘要展开研究,必要时还原细节,这样可以在降低 KV 缓存的同时维持模型性能。
行业数据
原图定位
当前图表所属:
计算机行业AIGC系列报告之DeepSeek深度解析:DeepSeek推动高性能AI普惠AI生态繁荣发展-250204(17页).pdf
2025年 DeepSeek行业报告:开源模型追平o1,训练成本557.6万美元驱动AI普惠趋势分析
DeepSeek-V3MoE架构示意
MLA与其他注意力机制的优劣对比
DeepSeek-V3训练成本测算
下载图片或Excel,需升级至尊VIP会员或「预见者」VIP
立即去升级
客服
商务合作
小程序
服务号
回到顶部
折叠
微信扫码登录
微信扫码登录
双重福利,扫码领取!
关注公众号『三个皮匠报告』,免费领取报告
1.100+最新优质行业报告
2.每周免费精选报告分享
手机快捷登录
账号登录
记住密码
忘记密码?
验证即登录,未注册将自动创建账号
获取验证码
收不到验证码注意事项
登录代表您已阅读并同意
《三个皮匠服务条款》