6548 - Privacy for AI from NP-Hard Problems -Universal Compute on Encrypted Data-.pdf

1、Rohit Khera Privacy for AI from NP-Hard StatementsPrivacy for AI from NP-Hard StatementsRohit KheraDistinguished Engineer,Cryptography Engineering,Marvell Technology CYBER SECURITY&DATA PROTECTIONFalse Dichotomy?Hardware+Software CountermeasuresMembership in NP(Efficiently verifiable statements)Intu

2、ition from a worked-out example:Encrypted Substring Search Generalize by applying intuition to other applications:Transformers and neural attention in the encrypted domain,activation functions,Boltzmann function Call to action Homomorphic Encryption(HE)Homomorphic Encryption(HE)encryptencryptdecrypt

3、Consistency and Soundness Circuits,Homomorphic Property,Succinctness +AUBoolean:Operands:Arithmetic:Operands:For brevity and succinctness:Well only describe the polynomial circuit for the function f()over the plaintextPattern:Target:Example:Search for an Element in an Array Search:Use Gate+=(1)Negat

4、e search pattern modulo 5:(2)Add negated search pattern element-wise to target:Zero polynomial indicates match =Transformer Circuits in the Encrypted DomainLets model the feed forward layer in a toy transformer block with embedding dimension 3:WeightsEmbeddingEncode the row(4,-2,7)and the hidden sta

5、te vector(3,-4,2)as degree 2 polynomials.Lagrange interpolation w/distinguished x-coordinates x=1,2,3 (1,4)(2,-2)(3,7)Encrypted Transformers:Combine Stone-Weierstrass Theorem and the Homomorphic Property Any continuous function(eg.Transformer inference),can be approximated by polynomials Stone-Weier

6、strass Theorem(SWT):For particular algebras C(X),there exist sub-algebras that are dense in C(X)+Challenges:The Curse of DimensionalityValue transform,single head(GPT-3):Value transform,all heads(GPT-3):Collaborate with silicon vendors to a