领域自适应LLM模型.pdf

1、Domain-adaptive LLMs for Chip DesignTutorial 1:AI Assisted Hardware Design-Will AI Elevate or Replace Hardware Engineers?Hanxian HuangAdvisor:Prof.Jishen ZhaoUniversity of California San Diego1Outline 1.Why domain-adaptive LLMs 2.Domain-adaptive LLM techniques 3.Domain-adaptive LLMs for Chip design

2、tasks21.Why domain-adaptive LLMs (1).Leverage Extensive Knowledge and Capabilities of Pretrained Models Natural language understanding and generation Reasoning and problem solving Instruction following Code generation Multi-modality3Pathways Language Model(PaLM):Scaling to 540 Billion Parameters for

3、 Breakthrough Performance(Google Research)1.Why domain-adaptive LLMs (2).Incorporate Specialized Knowledge4LLM4EDA:Emerging Progress in Large Language Models for Electronic Design Automation(Zhong et al.)2.Domain-adaptive LLM techniques5 1)In Model Training Stage Deep integration of domain-specific

4、knowledge 2)In Model Fine-tuning Stage Enhancing model performance for specific tasks 3)In Model Inference Stage Adapts the model dynamically in real-time,without altering the models underlying parameters Relies on external guidance or data retrieval to produce desired(domain-specific)outputsCost,Re

5、source,#Domain data,Impact on model paramsIntegration depth,Model capability,Flexibility2.Domain-adaptive LLM techniques6 1)Model Training Stage Domain-Adaptive Pre-Training2.Domain-adaptive LLM techniques7 2)Model Fine-tuning Stage Task-specific Fine-tuning Adapter Modules2.Domain-adaptive LLM tech

6、niques8 3)Model Inference Stage Prompt engineering In-context learning,Chain of thought Agent:Multi-turn feedback loop,Self-correction Retrieval-Augmented Generation3.Domain-adaptive LLMs for Chip design tasks9 How to choose the domain-adaptive techniques?Consider:(1)The models you have access to-Th