《美国能源部：2024能源部生成式人工智能参考指南（第二版）（英文版）（61页）.pdf》由会员分享，可在线阅读，更多相关《美国能源部：2024能源部生成式人工智能参考指南（第二版）（英文版）（61页）.pdf（61页珍藏版）》请在三个皮匠报告上搜索。
1、ge!Department of Energy Generative Artificial Intelligence Reference Guide Version 2 June 2024 Record of Changes Version Date Author/Owner Description of Change V 1.0 September 22,2023 Office of the Chief Information Officer DOE Generative AI Reference Guide v1 For Internal Release V 2.0 April 26,20
2、24 Office of the Chief Information Officer DOE Generative AI Reference Guide v2 For Public Release June 2024 Table of contents 1.Document at a Glance.1 2.Executive Summary.2 3.Purpose and Scope.2 4.Federal Guidelines and References.3 5.Background on Generative Artificial Intelligence.4 6.Opportuniti
3、es to Apply Generative AI.9 7.Operationalization.12 8.Key Considerations and Best Practices.20 9.Conclusion.41 10.Appendices.41 June 2024 Page 1 1.Document at a Glance This is a reference guide for the use of generative AI and shall not be interpreted as a policy.As such,it does not prescribe specif
4、ic actions.This document was developed with a general audience in mind and does not currently include targeted considerations for specialized roles,including Research&Development(R&D)and Management&Operating(M&O)staff.Deeper considerations for these roles may be addressed in a next iteration of this
5、 document.Generative AI(GenAI)is an incredibly powerful tool that has enormous potential to enable scientific progress,to enhance productivity of the Department of Energy(DOE)workforce,and to drive the DOE mission of innovation with emerging technology.GenAI is best used to provide a first draft or
6、to help find options or alternatives rather than being relied upon to produce an accurate and unbiased final output.Per Executive Order 14110 on the Safe,Secure,and Trustworthy Development and Use of Artificial Intelligence,federal agencies are discouraged from imposing broad general bans or blocks
7、on agency use of generative AI.The DOE is in the process of considering which GenAI services will be permitted for use based on comprehensive risk assessments.As decisions on services are made,specific guidelines for usage will be established.All existing DOE rules concerning data management and use
8、 should be followed.Contact the OCIO or the DOE Chief Privacy Officer with questions.Legal questions should be directed to DOEs Assistant General Counsel for Technology Transfer and Intellectual Property or a contractors cognizant legal counsel.Continue to use common sense and follow existing rules
9、regarding data and information management when using GenAI.Have a human in the loop to review outputs for accuracy,ethical considerations,quality,and to check for potential bias.Reference specific federal guidance(Section 4).Refer to the key applications and use cases to understand examples how GenA
10、I might be applied to drive value and innovation at DOE(Section 6).Your role in the organization(i.e.,general user,data scientist,leadership)is a factor in the key considerations and best practices that are most relevant to you(Section 7.2).Keep key considerations and best practices in mind to appro
11、priately manage risks associated with AI and GenAI(Section 8).Refer to the Best Practices Checklist to guide your use of GenAI(Section 8.11).Any reference to a specific GenAI model or product in this document should not be construed as an endorsement of the model or any of its potential outputs.As G
12、enAI continues to evolve,DOE will have to stay agile and adjust to the constantly changing landscape of opportunities,risks,and best practices.This guidance will be updated regularly to reflect the most current thinking.June 2024 Page 2 2.Executive Summary The Department of Energy(DOE)Generative Art
13、ificial Intelligence Reference Guide version 21 is being issued as a reference on generative AI(GenAI),a relatively newer AI technology that can produce various types of content,for the entire DOE complex,including federal employees and contractors at laboratories and DOE sites.Key stakeholders and
14、subject matter experts(SMEs)from across the DOE organization established a tiger team to collaborate on the development of this document.The coordinated effort has provided a variety of perspectives from various DOE roles and functions that are woven throughout.Continued collaboration and involvemen
15、t from a variety of stakeholders will benefit future iterations of this document and drive AI innovation at the DOE.This document is not a policy or directive,but rather a reference guide to help stakeholders from across DOE understand how to responsibly use GenAI.This document and the guidance with
16、in will be updated regularly as GenAI technology and the regulatory environment surrounding it continue to evolve.In light of the complexity of GenAI and the pace at which research and commercial advancements are being made,leveraging the expertise of researchers and SMEs will be vital.This guide is
17、 not a replacement for legal advice,therefore any legal questions related to the use of GenAI should be directed to cognizant DOE or contractor legal counsel.GenAI holds promise for furthering the departments mission,but it also poses risks.In employing GenAI,one must be aware of the capabilities an
18、d limitations of the technology and should keep in mind that the user,not the GenAI technology,remains responsible for any actions or outputs resulting from the use of GenAI technologies.Users should therefore not rely on GenAI systems for making decisions;rather,they should use the systems to infor
19、m them.This document comprises helpful information that can be used to spread awareness throughout DOE about the responsible use of GenAI.Topics include background of GenAI,a summary of existing laws and mandates pertaining to GenAI(at the time of publishing),fundamental topics on the responsible us
20、e of GenAI(including organizational roles,data,and service models),potential use cases,and the most prominent risks and best practices surrounding this emerging technology.From data scientists to leadership to general users,everyone across DOE has a role to play in the responsible use of GenAI techn
21、ology.After reading this document,the reader should have a newfound or heightened awareness of their role in the responsible use of GenAI,as well as foundational knowledge of GenAI solutions,the key considerations and risks that should be accounted for,and the current best practices to mitigate risk
22、s and responsibly use GenAI technology.3.Purpose and Scope The purpose of this document is to provide an understanding of the key benefits,considerations,risks,and best practices associated with GenAI in the context of the DOE.This document is intended to serve as a valuable reference on GenAI to al
23、l groups within the DOE environment,offering an overview of the specific risks,considerations,responsibilities,and recommendations that are associated with various organizational roles.The scope of this document is that it serves as the second version of a reference guide highlighting GenAI-specific
24、 risks and best practices.This document is a true reference guide,not indicative of a policy or directive.The best practices recommended in this document do not supersede any laws,regulations,“People sometimes ask the question,Is AI our friend or is AI our enemy?And my answer to that is,I think AI i
25、s our friend,but just like any good relationship,there are boundaries.2-Gardy Rosius,DOE Deputy CIO June 2024 Page 3 or existing DOE policies.As such,this document includes a discussion of background information,key concepts and definitions,and opportunities for applying GenAI,as well as a discussio
26、n of key considerations,risks,best practices,and recommendations.This document does not include any prescriptive,mandatory actions,as these will be captured in existing and future policies.Furthermore,this document is meant to supplement,but not replace,the existing regulations surrounding GenAI.4.F
27、ederal Guidelines and References GenAI is evolving rapidly:its underlying technology continues to advance,the variety of GenAI tools available on the market continues to grow,and GenAI is becoming increasingly accessible to the public.As this evolution accelerates,the need grows for awareness of the
28、 potential impacts of GenAI,as well as identification and mitigation of the associated risks.Several federal publications on AI and GenAI have been issued in recent years.These documents are the first point of reference for this document,providing guardrails for how GenAI can be used in the federal
29、Government.Notably,Executive Order 14110 on the Safe,Secure,and Trustworthy Development and Use of Artificial Intelligence was recently published on October 30,2023.EO 14110 contains a variety of directives which apply to the Department of Energy,including actions that DOE is required to lead or is
30、required to collaborate with other agencies to deliver.The Director of the Office of Management and Budget issued a Memorandum for the Heads of Executive Departments and Agencies on Advancing Governance,Innovation,and Risk Management for Agency Use of Artificial Intelligence in March of 2024 which s
31、hould also serve as a key reference document.This reference guide does not supersede existing law or policy or and is not intended to conflict with any relevant pending legislation.As with all policies,including those not discussed in this document,staff members should review and continue to adhere
32、to DOE policies,procedures,and guides to ensure compliance with laboratory/DOE information requirements.Employees must also continue to follow existing requirements,such as those regarding quality,information security,and research integrity.Staff members must work with appropriate laboratory/DOE SME
33、s and compliance organizations,such as the Office of General Counsel(GC),the Office of Export Control,the Classification Office,the Office of Environment,Health,Safety,and Security(EHSS),and others as appropriate.The reference guide will be updated as new policies and guidelines are issued.Summaries
34、 and selected details of the below references can be found in Appendix E at the end of this document.Existing relevant federal resources and references include:1.Office of Management and Budget Memorandum for the Heads of Executive Departments and Agencies,Advancing Governance,Innovation,and Risk Ma
35、nagement for Agency Use of Artificial Intelligence,March 2024 2.Executive Order 14110 on Safe,Secure,and Trustworthy Development and Use of Artificial Intelligence,October 2023 3.Generative Artificial Intelligence and Data Privacy:A Primer,Congressional Research Service(CRS),May 2023 4.Generative Ar
36、tificial Intelligence and Copyright Law,Congressional Research Service(CRS),May 2023 5.National Artificial Intelligence Advisory Committee(NAIAC)Year 1 Report,May 2023 6.AI Risk Management Framework,National Institute of Standards and Technology(NIST),January 2023 7.Advancing American AI Act,Decembe
37、r 2023 June 2024 Page 4 8.AI Training for the Acquisition Workforce Act,October 2022 9.Blueprint for an AI Bill of Rights,Office of Science and Technology Policy(OSTP),October 2022 10.Secure Software Development Framework(SSDF V.1,1),NIST,February 2022 11.AI Accountability Framework for Federal Agen
38、cies,GAO,June 2021 12.National AI Initiative Act,January 2021 13.Executive Order 13960 on Promoting the Use of Trustworthy AI in the Federal Government,December 2020 14.AI in Government Act,September 2020 15.Executive Order 13859 on Maintaining American Leadership in AI,February 2019 16.John S.McCai
39、n National Defense Authorization Act,Section 1051 for Fiscal Year 2019 17.E-Government Act of 2002 Please refer to Congress.gov to view the status of proposed and pending AI legislation.5.Background on Generative Artificial Intelligence 5.1 AI,Generative AI,and GPT Artificial intelligence has advanc
40、ed tremendously since it was first introduced in the 1950s.Its growth has surmounted two plateaus in advancement which occurred when the vision for the application of AI was broader than the functional ability at the time(i.e.,there was not enough computing power or data and no sufficiently advanced
41、 algorithms to operationalize the vision).In recent years,AI has gained increasing public attention,becoming a hot topic in technology,as well as in American and international society.Figure 1:Illustrative definitions of artificial intelligence,machine learning,deep learning,and generative AI.Defini
42、tion sources:Artificial Intelligence,3 Machine Learning,4 Generative AI,5 and Deep Learning6 June 2024 Page 5 Like AI,GenAI is not new,but has been gaining momentum since the introduction of generative adversarial networks(GANs),a type of machine learning algorithm,in 2014.This development enabled t
43、he creation of image generative models.Two additional recent advancements,transformers and large language models(LLMs)have further accelerated GenAIs evolution and adoption.Transformers are a deep learning model that adopts the self-attention mechanism,differentially weighting the significance of ea
44、ch part of the input data.7 In essence,they are a technique that seeks to help AI models determine what to pay attention to.Large language models(LLMs)use self-supervised learning to learn from large amounts of unstructured and unlabeled text data.These models are trained on large bodies of data,all
45、owing for one model to be used for multiple use cases.The 2017 emergence of the transformer,as well as progress made with convolutions and recurrences for performance and training speed,led to the generative pre-trained transformer(GPT)evolving into todays LLMs.GPT,the type of AI that has been at th
46、e center of the most visible activity in recent years,is based on neural networks,which are a type of machine learning(ML)model built to mimic the biological neural networks that comprise the brains of humans and animals.GPT is a family of LLMs built on deep neural network(DNN)architecture that have
47、 been fine-tuned using natural language processing(NLP)and reinforcement learning from human feedback(RLHF)techniques,as depicted in Figure 2.ChatGPT is the state-of-the-art consumer-facing AI model built on the GPT.It can answer user-prompted questions,generate stories,summarize text like books or
48、articles,and search text based on conceptual queries.Note that ChatGPT is currently available within DOE for use by request based on mission need.Additional guardrails may be developed and implemented in the future as appropriate.Foundation models,as termed by Stanford University researchers,are tra
49、ined on massive amounts of unlabeled data using a transformer algorithm that can be fine-tuned to a wide-ranging array of downstream tasks.To further specialize the models,data scientists can either independently train or fine-tune a foundation model to build task-specific models,which are models de
50、signed to be effective at specific tasks.Figure 3 shows the high-level relationship between foundation models and task-specific models.Additional AI-related definitions can be found in Appendix K:Glossary.As depicted in Figure 3,using foundation models as a starting point and including techniques su
51、ch as supervised fine tuning,instruction tuning,and RLHF,task-specific models that fit the situation at hand are built.The situation at hand may include specifics of the business case,modalities(e.g.,text,image/video,speech,auto coding,etc.),solution architecture,use case-specific data,and intended
52、use.Since OpenAI launched ChatGPT Deep neural networkDNNGPTNatural language processingNLPReinforcement learning and human feedbackRL HFFigure 2:Illustrative depiction of GPT AUTO CODINGSPEECHMULTI-MODALIMAGE/VIDEOTEXTFoundation modelsFine-tuning through RLHFTask-specific modelsFine-tuning through RL
53、HFGenerative AIFigure 3:GenAI:foundation vs.task-specific models June 2024 Page 6 in November 2022,new GenAI models built to be task-specific specializing in different industries,sub-industries,or types of functional applications have been rapidly entering the market and are generally either generic
54、 or built on-premises in a localized environment.User-friendliness and easy access by the general public via the internet have helped make GenAI models increasingly popular.Most models are unimodal,meaning they focus on a single form of information,like text,speech,or computer code.Multimodal models
55、 can learn from multiple forms of input and produce multiple forms of output.Refer to Table 1 below for a list of the various modalities and a sample list of applications and task-specific(or tailored)models currently available in the market.Note that Table 1 does not distinguish between foundation
56、and task-specific models.Category Modality Description Examples of applications Examples of task-specific models Unimodal Text Generation of human-like text from text prompts ChatGPT,Bard,Claude 2,Bing Jasper,copy.ai,NukeLM8 Unimodal Image/video Generation of various images and videos based on text
57、prompts DALL-E 2,Midjourney,Stable Diffusion(Automatic1111),Stability.ai Midjourney,Craiyon,Stable LM 2 1.6B Unimodal Speech Generation of synthesized speech from text prompts,speech recognition Thundercontent,Cleanvoice Voice synthesis,podcast.ai,Speechmatics Unimodal Auto coding Generation of code
58、(e.g.,Python,Java,JavaScript)from text prompts GitHub Copilot,Amazon CodeWhisperer,Codebots,OpenAI codes,ChatGPT,Bard GitHub Copilot,Tabnine,Cogram Multimodal Multimodal Generation of various outputs where the model learns from a variety of sources,including text,images,and audio Gato,Mural by Googl
59、e,GPT-4,GPT-5 Azure Open AI Service,Google Vertex AI,AWS Solutions,IBM Garage Table 1:GenAI modalities June 2024 Page 7 5.2 Trends GenAI and its underlying techniques are rapidly evolving and advancing,and GenAI adoption is exploding at a similar pace.Major breakthroughs have already been made with
60、GenAI technology since its introduction.For example,OpenAI released GPT-4 in March 2023,and by July 2023 a trademark was filed for GPT-5 which suggested a variety of potential new capabilities for the next iteration of the language model.The list includes features that expand ChatGPT further beyond
61、text-to-text GenAI and into the multimodal space,including artificial production of human speech and text,audio-to-text conversion,voice and speech recognition,and development and implementation of artificial neural networks.9 Note that many of these functionalities,such as speech recognition,predat
62、e the emergence of GenAI,but can now be enhanced via GenAI GPT solutions are expected to continue to advance at an aggressive pace.Similarly,between March 2023,when Anthropics GenAI solution Claude was introduced to the market,and May 2023,large strides were made in the solutions processing speed.Fi
63、gure 4:Source:Gartner,“Predicts 2024:The Future of Generative AI Technologies,”Arun Chandrasekaran,Anthony Mullen,Lizzy Foo Kune,Nicole Greene,Jim Hare,Leinar Ramos,Anushree Verma,February 28,202410 GenAI is already rapidly transforming areas like marketing and media,while in other areas,it is still
64、 in an emerging state.The list of potential use cases(explored further in Section 6:Opportunities to Apply GenAI)continues to grow as GenAI continues progressing in its abilities to generate multiple forms of media,including text,image,video,speech,music,and programming code.Although GenAI has alrea
65、dy gained a huge amount of traction from a multitude of organizations and in myriad aspects of society,it is truly still in its infancy.Expect rapid developments with GenAI capabilities and with the proliferation of its potential use cases and applications to continue.As GenAI continues to evolve,th
66、e market and the organizations that adopt it will have to stay agile and adjust to the constantly changing landscape of opportunities,regulations,risks,and best practices.The rapid evolution of GenAI brings many potential benefits,but also many risks and unknown effects.Although a variety of GenAI-r
67、elated risks have already emerged,expect some risks to become more pronounced and new risks to appear as GenAI adoption accelerates.Establishing risk management strategies,documenting and sharing best practices,and encouraging awareness throughout the organization of GenAI-associated risks and recom
68、mendations will be critical steps in successfully adopting GenAI and realizing the many benefits it can offer.5.3 Value Proposition GenAI use cases and potential applications are growing rapidly.Put simply,the value of GenAI is to fill the role of an automated“copilot”for creating materials in vario
69、us forms of media,including text,image,video,and programming code.Within DOE,this means that GenAI may be able to augment work produced by humans with speed.Once adopted,GenAI may change existing human roles within the organization without necessarily replacing them.Section 6 of Executive Order 1411
70、0 includes a variety of mandates on exploring the effects of AI on workers rights and economic stability.Additional information may become available as that reporting is completed.GenAI is forecasted to be a major element in the professional world in the coming years and to realize human-level perfo
71、rmance sooner than previously anticipated.Gartner predicts that By 2026,75%of businesses will use generative AI to create synthetic customer data,up from less than 5%in 2023.By 2027,more than 50%of the GenAI models that enterprises use will be domain-specific specific to either an industry or busine
72、ss function up from approximately 1%in 2023.By 2027,more than half of the selection of development assets from technology marketplaces will be performed by generative AI orchestration.June 2024 Page 8 When hypothetically used as a copilot for DOE employees,GenAI has the potential to help employees w
73、ith day-to-day tasks,including(but not limited to)finding information more quickly with its search functionality,generating summaries of meetings and lengthy documents,and drafting emails and other correspondence.These simple examples are areas where GenAI technology is already proficient at managin
74、g certain tasks very quickly and at scale.GenAI may be able to produce research or content outlines and starting points for content to allow DOE employees more time to focus on refinement and development of the product.The future workplace will likely include a symbiotic relationship between human e
75、mployees and GenAI.As AI technologies become integrated with day-to-day working tools(an example may eventually include Office365)and are therefore less visible to the user,this relationship may change or require additional exploration of risk and usage permissions.GenAI solutions can perform routin
76、e tasks much faster than humans(although this introduces risks regarding accuracy,reliability,and“hallucinations,”which are discussed in Section 8:Key Considerations and Best Practices).GenAI may create more time and space for DOE employees to add value to their work,empowering them to optimize thei
77、r time during their workweek.GenAI is expected to provide capabilities that will allow DOE to innovate more quickly.For example,GenAI can use large sets of relatively unexplored data and content to derive actionable insights that can help drive business value.There are four primary functions of the
78、GenAI text-to-text capability.Any of these functions can be used on its own,or they can be“bundled”together for a solution.Understanding these four functions can help explain how GenAI might be applied as a copilot in the DOE workplace.1.Summarization:The GenAI summarization capability can take a la
79、rge amount of text and summarize it into a shorter and more digestible format.While the model might not always exactly deliver on requests for summaries of specific character or word lengths,it can create a close match.The summarization function can also help extract and summarize specific aspects o
80、f a larger piece of text for example,summarizing only the parts of a larger news article that mention a specific organization or topic.2.Inference:The inference functionality generally involves making predictions or solving problems.Examples of the GenAI inference functionality include asking the mo
81、del to infer the sentiment of a given piece of text(e.g.,positive or negative sentiment)or to make an inference on whether there is a specific type of information within text(e.g.,the brand of an item in a review of the product,or whether an article contains references to a specific government entit
82、y).Note that inference functionalities carry a specific set of risks.Text analysis and inference,specifically if those inferences relate to a specific individual,should be used with considerable caution and only in specific scenarios.All information systems that contain personal information should h
83、ave a completed Privacy Impact Assessment(PIA)on record.One of the more complex questions asked during a PIA is whether the system will add(create,acquire,or infer)information about the person that was not directly collected and isnt officially part of the record.Inference capabilities may also misc
84、ategorize or mischaracterize views or statements made by individuals,and any output should be reviewed by a human co-pilot.3.Transformation:GenAI text-to-text models can transform text in a variety of ways.Translation is one application,as GenAI models are typically familiar with hundreds of languag
85、es in varying degrees of proficiency.Text can be translated into multiple languages simultaneously and adjusted based on formality and the intended audience.The model can also transform a piece of text to reflect a new tone or audience,such as turning a casual greeting into a formal business memo.Te
86、xt can also be edited for grammar and spelling.In addition,text can be transformed into another format,including coding languages,such as changing a block of input from JavaScript Object Notation(JSON)to Hypertext Markup Language(HTML).4.Expansion:The fourth function of text-to-text GenAI is expandi
87、ng upon a given piece of text or topic,adding to or creating content,or providing additional information on an area of interest.June 2024 Page 9 Examples of expansion include using GenAI to write a response to a constituent query based on the subject and the sentiment of the query or to write a long
88、er-form essay or article about a prompt-given topic.Note that expansion functionalities are most susceptible to risks related to copyright and intellectual property concerns(see Section 8.7)and to AI hallucinations(see Section 8.10).6.Opportunities to Apply Generative AI 6.1 Key Applications When co
89、nsidering the opportunities for GenAI use cases and applications,keep in mind that GenAI solutions are multimodal and can generate text,image,audio,video,and programming code.A variety of use cases for each form of media(modality)are already being adopted.Within each of the modalities,there are seve
90、ral viable use cases that can be potentially applied at DOE.Figure 5 provides several of the best fitting GenAI applications for four modalities.Figure 5:GenAI key applications for text,speech,image/video,and code 6.2 Use Cases for DOE(Illustrative)The table below expands on ideas for use cases cate
91、gorized by modality and includes examples of where GenAI could be used at DOE.For an up-to-date and more detailed look into AI use cases being applied at DOE,see the DOE 2023 AI Use Case Inventory(includes use cases for various AI capabilities,mostly in the data analytics and research space,and is n
92、ot limited to GenAI).Additional use cases for other GenAI applications may become available as the inventory matures.For several science examples,refer to Appendix F.Per Executive Order 14110 on the Safe,Secure,and Trustworthy Development and Use of Artificial Intelligence,the Director of the Office
93、 of Management and Budget(OMB)will issue instructions to DOE and other federal agencies for the collection,reporting,and publication of agency AI use cases on an annual basis in alignment with Section 7225(a)of the Advancing American AI Act.11 Regarding the implementation of these and other use case
94、s,Executive Order 14110 Section 10.1(f)(i)states that with appropriate guardrails in place,it is recommended that access be provided to“secure and reliable GenAI capabilities,at least for the use of experimentation and routine tasks which do not have a rights impact.June 2024 Page 10 Generative AI U
95、se Case Examples Text functionalities(e.g.,summarization,inference,expansion,transformation)Summarization Summarize contracts,proposals,reports,stakeholder comments,and regulatory documents Build or enhance internal search tools Inference Conduct sentiment analysis from an interaction such as an ema
96、il(e.g.,positive or negative sentiment)Expansion Create first drafts of contracts,drafts,business presentations,memos,emails,responses to questions,and optimized Request for Proposals(RFPs)Provide additional advice or information on a topic Transformation Translate documents,contracts,and communicat
97、ions into one or more other languages Assist with writing programming code and documentation Evaluate and identify errors in code Translate code from one programming language into another Perform auto-completion of code Image functionalities(e.g.,generation/creation,interpretation)Generation/creatio
98、n (e.g.,text-to-image or image-to-image)Create an image based on a text description Create a visual for a product,campaign,cover page,newsletter,logo,promotional material Interpretation (e.g.,image-to-text)Create a description of a visual used in a presentation,e.g.,recognize the image is a depictio
99、n of a system and use the visual caption as part of the visual description Audio functionalities(e.g.,speech-to-text,text-to-speech,audio creation/generation)Transcription (e.g.,speech-to-text)Transcribe learning resource videos for consumption as text Transcribe meeting minutes Generation/creation
100、(e.g.,text-to-speech)Create an audio voiceover for an educational training Generate custom sounds or audio clips Audio editing (e.g.,speech-to-speech)Edit an audio clip without having to rerecord the clip Translate existing speech in an audio or video clip into a different language using an AI-gener
101、ated voice or the voice of the speaker in the existing audio June 2024 Page 11 Video functionalities(e.g.,interpretation(video-to-text),creation/generation)Interpretation (e.g.,video-to-text,speech-to-text)Review video used in a proposal or in a meeting where video is included and provide a summary
102、of the video Scan videos to identify vulnerabilities and alert security(in the context of security solutions that use cameras)Creation/generation (e.g.,text-to-video and/or image-to-video)Create videos for training materials or presentations,potentially paired with the use of AI avatars Abbreviation
103、/condensation/translation(e.g.,video-to-video)Create a trailer(a short video)to summarize or abbreviate a longer video Use an existing video to generate the same video in other languages Selected potential GenAI use cases Generate interview questions(e.g.,text expansion)Use case Create a first draft
104、 of interview questions for candidate assessment based on a given job description Considerations Evaluate the first draft produced by GenAI to ensure alignment with the intended purpose of the interview Create meeting minutes(e.g.,audio transcription)Use case Generate written meeting minutes for a D
105、OE meeting from an audio recording of the meeting Considerations Disclose that the meeting is being recorded to participants to manage legal and ethical risks Enhance informational videos(e.g.,video personalization)Use case Use GenAI to enhance informational videos by adding voice narration,graphics
106、,captions,or translations Considerations Personal likenesses may only be used with proper legal consent.However,there are significant ethical and legal risks surrounding the creation and release of deepfakes.Any addition of presenters should be synthetic(not a“likeness”of any one person)unless there
107、 has been significant collaboration with the subject and legal experts.Voice narration,translations,and captions should be vetted for correctness and completeness.June 2024 Page 12 7.Operationalization 7.1 Operationalization at a Glance This section provides foundational knowledge of three key conce
108、pts surrounding GenAI before exploring Key Considerations and Best Practices in Section 8.The three concepts introduced in this section are organizational roles,public vs nonpublic data,and service models.Different roles throughout the organization have specific responsibilities and considerations w
109、hen it comes to GenAI.As a best practice to mitigate privacy and security risks,users should not input nonpublic(sensitive)data into a GenAI system unless the appropriate processes have been undertaken to ensure that the rights and potential uses of the data are permitted,or they are using a tool wh
110、ich is appropriately configured and approved for their use case.This best practice is critical for public or commercial systems where the model,inputs,and outputs are not under DOEs direct control.There are several ways to approach service models.The key is to determine whether or not DOE controls t
111、he GenAI model and the outputs and whether inputs are added to the models training data.Specific considerations apply to either case.7.2 Organizational Roles Everyone has an important role to play when considering and implementing a new GenAI solution or using an existing GenAI tool.Whether a genera
112、l user or an AI systems specialist,every employee should be cognizant of their role and any specific considerations that may apply to their role related to the development and use of GenAI technologies.Below is an introductory set of roles across the organization with corresponding descriptions.Note
113、 that this list is not exhaustive,and that in many cases these roles use language which is specific to DOE but may have applications in other organizations.Consider developing a RACI(Responsible,Accountable,Consulted,Informed)Matrix to clearly define the roles and responsibilities for each specific
114、GenAI solution.The descriptions listed below are responsibilities to be considered when drafting more explicit requirements,not requirements in themselves.For additional information on the AI Lifecycle referenced in this table,see Appendix G.Many of these roles are still developing within the DOE.Fo
115、r example,the roles and responsibilities of the Chief Artificial Intelligence Officer(CAIO)and the Responsible Artificial Intelligence Officer(RAIO)vary across organizations.In some organizations,one person may take on both the CAIO and RAIO roles,while in DOE,these are currently two distinct and em
116、erging roles(at the time of publication of this document).At DOE,the RAIO reports to the Chief Intelligence Officer(CIO),while the CAIO reports to the Secretary of Energy.Organizational role name Description General user Regardless of whether a person may hold one of the specific roles listed below,
117、almost anyone in DOE may be or soon may become a general user of GenAI.For general use,it is key to understand the nature of the information input into the model,the intended purpose of the model,and any restrictions related to the input data or to a users role in the organization.Where needed,repor
118、t observed issues of accuracy,fairness,or bias in a models output.The general user may also include the human in the loop to verify outputs to ensure both responsibility(ethics and limited bias)and accuracy,especially when a GenAI system generates outputs for humans to consume,takes action prompted
119、by outputs,or draws conclusions based on outputs.June 2024 Page 13 Organizational role name Description AI developer The AI developer is charged with designing,coding,and iteratively improving new GenAI applications in collaboration with other roles such as data scientists,user experience designers,
120、cybersecurity specialists,project sponsors,and leadership.The AI developer creates the systems and AI solutions,as opposed to the data scientist who develops the underlying models.AI developers should consider the unique implications surrounding GenAI technologies and consult with AI SMEs as needed
121、to implement AI-specific best practices.AI policy and governance staff AI policy and governance staff advise on the creation of new AI policy and governance based on organizational,technical,and legislative need for Department-wide adoption and implementation of responsible,ethical,and trustworthy A
122、I frameworks,principles,procedures,and practices.They ensure that policies reflect the best practices in AI and address any security,risk,or privacy concerns as well as responsible and ethical AI principles.This role is emerging and evolving very quickly,and there will likely be additional responsib
123、ilities associated with it as adoption of GenAI progresses.AI portfolio manager The AI portfolio manager oversees all AI capabilities and projects currently in the pipeline for their organization or departmental element.This role is critical to limiting redundancy of AI solutions which may have simi
124、lar functions.The portfolio manager should understand the current and near-future AI landscape to identify trends and risks in proposed capabilities.This role is accountable for the entire pipeline of all AI initiatives at the highest level and is responsible for all of the policies and processes as
125、sociated with the AI pipeline.This role also manages funding and budget for all AI initiatives.AI subject matter expert(SME)The AI subject matter expert(SME)advises others on best practices and risk considerations for GenAI technologies.This role needs to understand the technologies involved for a g
126、iven use case and provide advisory services to other roles including data scientists,leadership,and data engineers to share knowledge with the appropriate team members.This role may be involved throughout any stage of the AI lifecycle.For example,the SME may be involved in the initial planning stage
127、 to ensure that a given business problem is a good fit for a GenAI solution,during the development or implementation stages to ensure technical or process efficiency and quality,or by providing insights on how to best educate users on responsible GenAI usage.Business analyst The business analyst is
128、tasked with coordinating efforts across project or development teams to design,launch and operate GenAI capabilities.This role is an internal user who is responsible for the translation and coordination of needs and tasks between the business users and the AI/ML development team.This role translates
129、 business needs into technical requirements and helps business users effectively use the output as the system was designed.The business analyst may also identify potential business use cases for AI technologies throughout the course of their daily responsibilities.Chief Artificial Intelligence Offic
130、er(CAIO)The Chief Artificial Intelligence Officer(CAIO)is a role defined in Executive Order 14110 Section 10.1(b)(i)charges the Chief AI Officer with“coordinating their agencys use of AI,promoting AI innovation in their agency,managing risks from their agencys use of AI,and carrying out the responsi
131、bilities described in Section 8(c)of Executive Order 13960(“Promoting the Use of Trustworthy Artificial Intelligence in the Federal Government”)and in Section 4(b)of Executive Order 14091.”They are also charged with overseeing AI Governance Boards as required for their agency,overseeing risk managem
132、ent activities for government uses of AI,and making recommendations to June 2024 Page 14 Organizational role name Description agencies to reduce barriers to the responsible use of AI,including AI-specific barriers of adoption for information technology infrastructure,data,workforce,budgetary restric
133、tions,and cybersecurity processes.Given that this is an emerging roll,the governance process and responsibilities for this role are still in development at the DOE.Contracting officer The contracting officer facilitates the purchase of GenAI tools,platforms,technology,and services for DOE and uses t
134、heir knowledge to ensure that the best fitting GenAI tools are selected.This role ensures that new tools acquired for DOE are rigorously tested,meet policy and security requirements,are not duplicative of other ongoing efforts at the organization,and are aligned with any existing policies,procedures
135、,and strategies for the organization.Contracting professionals are also concerned with the details of new and existing contractual agreements and terms of service(ToS)agreements with third-party providers of GenAI solutions and services to ensure that risk is appropriately shared between DOE and ser
136、vice providers.This role also collaborates with legal specialists/legal counsel and organizational stakeholders.Cybersecurity specialist The cybersecurity specialist is charged with the security,safety,and resiliency of organizational(or related)systems.This role should be involved from the very beg
137、inning of any GenAI initiative to ensure that the solution design has sufficient protection measures in place and will not interfere with existing security measures as a result of the projects requirements.This role ensures that any solution which is planned and designed has a high probability of be
138、ing successfully operationalized.At each phase of the AI lifecycle,this role needs to ensure security features are properly maintained.Cybersecurity professionals may also focus how AI can be used to bolster cybersecurity organization-wide and to prevent or respond to adversarial attacks in non-AI s
139、ystems.Data engineer The data engineer ensures that the appropriate data is available to data scientists and that the data is as trustworthy,fair,and free of bias as possible.This role should have an understanding of data structure,environment,management pipeline,and data quality in terms of sourcin
140、g,depth,and breadth in data that is used for building GenAI systems.Data engineers who are involved in the training of newly developed or purchased models may also assist in data tagging and other implementations to meet organizational data governance policies,and they have the responsibility to def
141、ine and implement the right plan and architecture for the data.Data engineers implement the appropriate checks into the data management pipeline to ensure that the quality standards(typically defined in a policy)are met at each phase of development.Data scientist The data scientist uses data from va
142、rious sources to assist in making organizational decisions and reaching sector-related conclusions.This role is usually responsible for developing models and must understand in detail the intended purpose and outputs of their models to ensure proper functionality,any regulatory or privacy considerat
143、ions related to a given model or project,and the differences between training/validation and production/live data.When developing or training a GenAI system,data scientists are responsible for ensuring the quality,representativeness,and lack of bias in outputs from the training data set.The data sci
144、entist should explore data provided by data engineers and apply the best methodologies and tools to reach the objective of the project with the data given.Data scientists may take on other roles from June 2024 Page 15 Organizational role name Description within this list,including solution architect
145、,developer,AI SME,and more.They may also take on the role of prompt engineer in the case of GenAI.Development operations(DevOps)engineer The DevOps engineer is responsible for the processes that help DOE improve the efficiency of developing,testing,operationalizing,and updating technology.This role
146、helps to facilitate these processes with knowledge of emerging technology,project management skills,and team communication.The DevOps engineer collaborates with other technical roles to ensure functionality during the transfer of the solution from the pilot to the production environment while monito
147、ring for potential risks and vulnerabilities(e.g.,data drifts,software or model drifts,and security issues).The DevOps engineer is also responsible for ongoing model/system management depending on organizational policy or guiding principles.This role may be part of a larger team which includes a var
148、iety of skillsets within the larger umbrella of DevOps,e.g.,build architect,release manager,infrastructure engineer,automation architect,among others.Executive sponsor The executive sponsor is a leadership position and is responsible for communicating to the organization and raising awareness about
149、the importance of prioritized strategic initiatives.This role makes sure that the resources are available for any prioritized GenAI project and elevates the initiative at hand to a higher priority level.This role is responsible for securing initial funding for the project at hand and for ensuring th
150、e appropriate stakeholders are involved with the initiative and aligned on its goals.The executive sponsor may occasionally fill the role of a project director,which is more hands on in nature.Information technology(IT)/systems professional The information technology professional is charged with ove
151、rseeing IT systems across the organization.The focus for IT professionals is on both implementing and maintaining new GenAI technology and providing prospective users with recommendations and best practices for GenAI use.This role has different responsibilities depending on the stage of the AI lifec
152、ycle.For example,during the development phase,the IT professional helps build the sandbox.Leadership There are many different potential responsibilities that may pertain to the leadership role.Leadership sets the strategic direction,priorities,goals,and mission objectives for the organization in col
153、laboration with federal officials.Leadership is concerned with understanding GenAI at the executive level,understanding the broad regulatory landscape as it pertains to GenAI,and building awareness and understanding of GenAI and its use cases across the organization.Leadership at the Department leve
154、l and at the Departmental Element(DE)/site level ensures that training on GenAI usage,limitations,and risks are both available and encouraged for all potential GenAI users across DOE.Leadership also facilitates the creation of a central,collaborative mechanism for sharing knowledge,collaborating on
155、initiatives,reporting observations of bias,unreliability,security issues,and other concerns in GenAI platforms as a means for organizational learning.This group may also include leaders who dont interact directly with AI technologies.Legal specialist in AI and emerging technology Legal professionals
156、 specializing in emerging technology and AI navigate the intricate realm of AI in law,focusing on the implications and requirements unique to emerging technology and AI.They are tasked with comprehending the nuances of AI models,utilizing them effectively for legal tasks,and aligning these applicati
157、ons with established ethical standards and legal frameworks.They ensure that the use of AI within legal operations not only optimizes efficiency and accuracy,but also maintains robust privacy protocols,June 2024 Page 16 Organizational role name Description addresses potential risks,and adheres to le
158、gal regulations and standards.Legal specialists are responsible for staying up to date on existing and pending legislation to proactively protect the agency and prepare for what is to come.They also serve as advisors for the entire GenAI team.Management and operating(M&O)staff The M&O contract staff
159、 oversees National Labs and needs to understand the legal,technical,and procurement risks undertaken as an organizational third party while monitoring the performance of GenAI solutions in operation.This role focuses on the maintenance,monitoring,and overall usage of GenAI system once deployed.In th
160、e design and development stages,this role ensures that the requirements are realistic and accounted for in the solution build.Note that M&O staff includes a wide range of people,including researchers,executives,cybersecurity specialists,and more.Many of the roles on this list may apply to select M&O
161、 staff.Product manager The product manager is responsible for understanding and prioritizing organization-relevant market opportunities for AI use cases,in collaboration with use case owners and leadership.This role is responsible for managing the activities and the team for the specific GenAI initi
162、ative,product,or service at hand.There may be a range of product managers as the AI portfolio expands.Program manager The program manager is a unique role associated with research institutions.This role writes funding opportunity announcements,reviews proposals,makes funding recommendations,and mana
163、ges awards,among other responsibilities.For the Office of Science,these awards are primarily to universities and National Laboratories and focus on fundamental research.While proposals and awards may include the development and use of AI for science and engineering,there is also an opportunity for G
164、enAI to assist the program manager in the performance of their duties.Project director The project director is charged with committing time and resources to oversee and review the development of new GenAI capabilities in a project-based environment,which includes making high-level decisions on the d
165、irection and goals of the project,how the project is managed and structured,and how GenAI technologies may contribute to other ongoing initiatives.Project directors may have overlap with the leadership role.Project directors provide direct requirements depending on the use case and ensure that all r
166、oles in the GenAI team are invited to collaborate and that initiatives are completed in alignment with the overall strategy.Research scientist The research scientist investigates open-source and published research and/or conducts studies and experiments.Research scientists are especially concerned w
167、ith the technical accuracy of the output when utilizing a GenAI tool in their work,as well as understanding the copyright and publication considerations for any research done using a GenAI tool.Responsible and ethical use of GenAI tools is a key concern for research scientists.Note that the roles of
168、 many DOE research scientists differ in their advancement of science in using AI.Refer to Appendix F for references to additional resources on DOE R&D and advancement of science.At this time,researchers reviewing grants as part of the NSF merit review process are unable to use AI as an aid in any ca
169、pacity.12 Responsible AI Officer(RAIO)The RAIO is responsible for managing an AI risk management program,collaborating with the appropriate officials to establish or update processes to evaluate the performance of AI systems,overseeing DOE compliance with requirements to manage AI risks,and conducti
170、ng risk assessments when June 2024 Page 17 7.3 Public vs Protected Data With all forms of AI,and especially in the context of GenAI,there are critical considerations surrounding the use of data and information.Data(meaning recorded information,regardless of form or the media on which it may be recor
171、ded,including both technical data and computer software)can be broadly categorized into two classes:public and protected.Protected data includes information that is protected from public distribution and/or certain uses,and includes sensitive data,private data,proprietary data,confidential data,and
172、copyrighted works.Confidential data is further explored in Section 8.6 and should not be conflated with classified information which has also been marked as confidential.For definitions of various types of sensitive data,refer to Appendix J:Examples of Protected Data.Some types of protected data are
173、 only protected and/or nonpublic for a defined period of time while other types of protected data might remain protected and nonpublic indefinitely.Public data is information that can be freely used and distributed by anyone with no legal restrictions regarding access or usage.The key takeaway is th
174、at any type of protected data or nonpublic information should not be shared or input into any public or commercial(non-DOE-controlled)GenAI system.DOE controls the GenAI system if it is a closed,or proprietary,AI system.A closed AI system is developed and controlled Organizational role name Descript
175、ion needed.The RAIO is also responsible for coordinating implementation of the nine Trustworthy AI Principles set forth in Section 3 of EO 13960.Given that this is an emerging role,the governance process and responsibilities for this role,as well as their implementation,are still in development at t
176、he DOE.Solution architect The solution architect oversees the integration of GenAI technologies into the overall organizational IT infrastructure.This may require implementation of additional data governance or security measures,as well as an awareness of the types of data flows and access which are
177、 or are not permitted with GenAI technologies.This role helps design how the system will look,and function based on the given requirements and intended purpose.This role thinks about how the GenAI model will be integrated and operationalized with upstream and downstream systems and ensures that ever
178、ything goes smoothly when the successful POC moves to production.Use case business owner The use case business owner is responsible for establishing a business case for a new or expanded AI solution,communicating the needs of the business to the development and/or procurement team,collaborating with
179、 other functions to select a solution for the use case,and assisting in the implementation and maintenance of the GenAI solution as needed.This role is involved on a daily basis with the given use case and may be responsible for providing the data for the development of the solution.User experience(
180、UX)designer The user experience(UX)designer is responsible for creating the human-centered user interface of a technological solution or product,including the design of the components and features that control how a user interacts with the product.UX designers focus on how to tailor the user interfa
181、ce to meet the needs of the target end-users of the solution,to improve the quality of the customer experience,and to make the user interface as simple and effective to use as possible for the target end user of the product.It is essential for this role to embed a human-centered design approach in t
182、he user interface in the final product.June 2024 Page 18 by a single organization that has full control and ownership over the system.Exceptions may apply if the GenAI system is protected by a confidentiality agreement between DOE and the vendor and DOE has sufficient rights to use the protected or
183、nonpublic data.13 This recommendation is a critical best practice for DOE use of GenAI.Even for DOE controlled GenAI systems,users should ensure they have the appropriate rights to use any nonpublic or protected data being utilized.The key takeaway for public data is that the possibility of the publ
184、icly available information input into the GenAI system becoming incorporated into the AI model may not pose a threat since the data is already public.However,it is important to remember the risk of plagiarism and copyright infringement and to verify publicly available data even though it can be used
185、 freely.There are many existing laws,mandates,and internal DOE policies and trainings that provide valuable guardrails on the general use of data,especially policy on the protection and sharing of information.These should be given special attention when using GenAI.Additional resources regarding dat
186、a include:DOE Resources:CUI Slicksheet,Controlled Unclassified Information “Protected Critical Infrastructure Information(PCII)Information Program,”Cybersecurity and Infrastructure Security Agency(CISA)Federal Acquisition Regulation(FAR),Acquisition.gov Improper and unauthorized use or disclosure of
187、 protected data or nonpublic information can lead to legal liability for both DOE and the individual responsible for the unauthorized use of disclosure.In some cases,the liability for individuals can include potential civil and criminal penalties.7.4 Service Models Using GenAI solutions can come wit
188、h a variety of risks depending on who has control of the solution in use.The major distinction comes down to the following question:Does DOE control or not control the solution?(Refer to Section 7.3 directly above for a brief discussion on the distinction between a DOE-controlled system vs.non-DOE-c
189、ontrolled system.)Systems that are off-the-shelf and have not been purchased by and customized for DOE or publicly accessible(e.g.,ChatGPT,Bard)will come with a higher set of risks versus an internally built application that is protected within the boundary of DOE.It is important to understand the t
190、ype of application in use and to comply with existing DOE policies and guidelines when it comes to the use of technology.To clarify the instance of an application that is to be used,or for any other systems-related questions,contact the Responsible AI Official(RAIO),the Chief AI Officer(CAIO),the OC
191、IO Supply Chain Risk Management(SCRM)Team,or the local Information Technology(IT)team.Robust understanding and documentation of the information detailed below is critical for implementing the best practices listed in Section 8.The following are questions to consider before using a GenAI application:
192、What are the specific benefits the use of this solution would provide?Who built the application and/or model?Is this solution developed and/or controlled by DOE?If not,is there a tool which is?Where is the application hosted?How was the model trained?How was the data selected and collected?What data
193、 was used to train the model and as of what date?June 2024 Page 19 Is the use permissible under the data rights available given the source of the data or for type of information?Which model,platform,and methodology were used?Is this a public application,is it in a private secure cloud,or is it opera
194、ting inside a DOE secure network?Is the application an off-the-shelf product?When information is supplied to the application,what is the risk of the information becoming public?How was the model validated?Are the terms of service for the GenAI system“federally compatible”within the guidelines establ
195、ished by the General Services Administration(GSA)?Beyond that,do the terms of service for data-handling meet appropriate privacy or confidentiality requirements?What are the limits of shared or transferred risk and accountability between DOE and all involved parties?For systems built internally for
196、DOE that routinely process DOE business information,on-premises or hybrid solutions,the following additional considerations are needed before operationalizing GenAI:Has the application been approved by DOE cybersecurity standards and procedures(e.g.,Authorization to Operate(ATO)has been issued,Priva
197、cy Impact Assessment(PIA)in place,etc.)?Is the application FedRAMP-approved?Is the model fully self-contained with no third-party retrieval?Is there a service contract or agreement in place?June 2024 Page 20 8.Key Considerations and Best Practices 8.1 Key Considerations and Best Practices at a Glanc
198、e This section provides an overview of seven considerations where certain risks are known to arise with GenAI technologies.Each of the seven subsections provides a brief description of the consideration,public and illustrative examples of where and how risks may arise,specific risks,and best practic
199、es to mitigate those risks.The seven topics covered in this section are security and resilience,privacy,confidentiality,intellectual property,safety,fairness and bias,and AI hallucinations and interpretations.At the end of this discussion in Section 8.11:Best Practices Checklist,a summary of best pr
200、actices is provided.Note that challenges may arise when implementing best practices,and keep in mind that best practices for GenAI will continue to emerge and evolve.Reference the DOE AI Risk Management Playbook(AIRMP)for more ideas regarding AI-related risks and risk mitigation strategies.8.2 Intro
201、duction There are always specific considerations,unique risks,and best practices that should be given attention when embarking on a journey to innovate with technology.AI has more unique considerations than non-AI technologies due to the complex nature of the models and their reliance on datasets.Ge
202、nAI is even more complex and therefore comes with even more nuanced considerations,risks,and risk mitigation strategies.The entire organization needs to understand the complex nature of GenAI risks and best practices to maximize the benefits of GenAI while minimizing its risks.Each role listed in Se
203、ction 7.2:Organizational Roles,including general users,has GenAI-specific considerations and best practices associated with it.It is essential to build awareness throughout the organization of these roles,responsibilities,and best practices.This section provides details on seven key considerations a
204、nd best practices pertaining to GenAI.Section 8.3:AI Risk Management introduces the seven characteristics of trustworthy AI systems outlined in the National Institute of Standards and Technology(NIST)Artificial Intelligence Risk Management Framework(NIST AI RMF 1.0).These seven characteristics are u
205、sed as a framework to discuss seven key considerations surrounding GenAI,each of which has its own subsection(Sections 8.4 8.10).8.3 AI Risk Management When developing and deploying new GenAI technologies or when incorporating GenAI functionalities into existing systems,it is critical to understand
206、both existing risk management considerations and unique risks associated with GenAI.GenAI introduces an additional layer of risk considerations,including hallucinations,misinterpretations,training poisoning,prompt injection,deepfakes,and intellectual property infringement.Keep in mind that risks not
207、 specific to GenAI may become more pronounced when GenAI is integrated into the technology ecosystem.It is best to design GenAI systems to be secure,responsible,and trustworthy at the onset of any GenAI initiative,and effective AI risk management is a critical component of achieving these goals.When
208、 employed appropriately,AI risk management also allows users and developers to understand the limitations and ambiguities of AI and to enable the selection of appropriate,responsible,and viable AI use cases.AI risk management differs in several ways from non-AI technology risk management practices.R
209、isk management program governance typically includes a set of metrics to measure performance and progress based on public and historical data.However,AI use cases do not generally have reliable metrics to use in comparison,and metrics may not fully capture relevant factors or impacts.There is also a
210、 lack of consensus on how to define clear metrics for reliability or trustworthiness in AI systems.14 Another difference is that AI systems designed to augment human actions(which have existing risk management criteria)act differently from the human thought process,which can make specific AI risk Ju
211、ne 2024 Page 21 management requirements difficult to operationalize.Finally,prioritization of AI risk resources may be decided differently than with non-AI risk management strategies.Prioritization metrics for AI systems may include those that interact with humans,that have downstream effects on saf
212、ety,or that have training sets that include personally identifiable information(PII).15 The NIST Artificial Intelligence Risk Management Framework(NIST AI RMF 1.0)is an excellent resource to become familiar with AI-related risk management and responsible AI practices.The NIST AI RMF is cited heavily
213、 throughout the latest Executive Order 14110,which requires the Secretary of Energy to collaborate with the Secretary of Commerce,Secretary of Homeland Security,and others to develop guidelines and best practices for developing and deploying safe,secure,and trustworthy AI systems,including by develo
214、ping a companion resource to the NIST AI RMF for GenAI.The risk management practices outlined in the NIST AI RMF are considered the current standard by the federal government.The NIST AI RMF outlines three core concepts to emphasize in responsible AI development:human centricity,social responsibilit
215、y,and sustainability.16 With these core concepts of responsible AI in mind,AI risk management can enable responsible usage,practices,and processes by encouraging employees across the DOE ecosystem to practice critical thinking about potential risks and unexpected impacts of AI.A critical overarching
216、 theme in designing,developing,and deploying AI in a way that maximizes its benefits while adequately managing its risks is trustworthiness.Trustworthy AI is a concept reflected in numerous relevant federal publications,including Executive Order 13960 on Promoting the Use of Trustworthy Artificial I
217、ntelligence in the Federal Government and Executive Order 14110 on the Safe,Secure,and Trustworthy Development and Use of Artificial Intelligence.The NIST AI RMF lists seven characteristics of trustworthy AI systems to guide AI risk management and responsible AI development.These seven trustworthy A
218、I characteristics defined by NIST are:Safe,Secure and Resilient,Explainable and Interpretable,Privacy-Enhanced,Fair with Harmful Bias Managed,Valid and Reliable,and Accountable and Transparent(refer to Figure 6).For additional information on the NIST AI RMF,refer to Appendix H or the full online pub
219、lication.The seven characteristics of trustworthy AI as outlined by the NIST AI RMF are used as a framework for the discussion of key considerations that follows in Sections 8.4 8.10.In the following subsections,key considerations and best practices for GenAI are presented in seven high-risk areas:S
220、ecurity and Resiliency,Safety,Privacy,Confidentiality,Intellectual Property,Fairness and Bias,and Hallucinations and Misinterpretations and are mapped against the NIST AI RMFs seven trustworthy AI characteristics listed above.All seven DOE considerations have trustworthy implications and aspects.Whe
221、n the best practices corresponding to the seven DOE key considerations are thoughtfully applied,GenAI systems will be trustworthy by design.The seven DOE key considerations are mapped against the seven NIST AI RMF trustworthy AI characteristics as follows:Figure 6:Seven characteristics of trustworth
222、y AI systems outlined in the NIST AI RMF 1.0 June 2024 Page 22 DOE key consideration NIST AI RMF 1.0 trustworthy AI characteristic(s)Security and Resiliency Secure and Resilient Safety Safe Privacy Privacy-Enhanced Confidentiality Secure and Resilient;Safe Intellectual Property Secure and Resilient;
223、Accountable and Transparent Fairness and Bias Fair with Harmful Bias Managed Hallucinations and Misinterpretations Accountable and Transparent;Valid and Reliable;Explainable and Interpretable The subsequent sections go into depth on the aforementioned seven DOE key considerations for GenAI.Section 8
224、.11 provides a checklist to summarize the highest priority best practices for all seven topics,as well as more general best practices not listed in the seven subsections.June 2024 Page 23 8.4 Security and Resiliency While GenAI solutions have emerged as innovative tools to drive science,operation,an
225、d business transformation,they also introduce security risks that should be carefully addressed and mitigated.AI systems that can maintain confidentiality,integrity,and availability through protection mechanisms that prevent unauthorized access and use,including covert modification of training data
226、or the foundational models,may be said to be secure.AI systems are said to be resilient if they can withstand unexpected adverse events or unexpected changes in their environment or use or if they can maintain their functions and structure in the face of internal and external change and degrade safe
227、ly and gracefully when necessary.17 Examples Examples Illustrative example Applications backended by GenAI services that are then installed on devices can automatically join meetings or access other data and services.These apps make phishing/smishing attempts more realistic and convincing with deepf
228、ake imagery and voice.These should all be covered in existing cybersecurity practices and regulation but are emergent and have an increased threat landscape.Public example In spring 2023,a vulnerability in ChatGPTs source code exposed users sensitive information and allowed adversarial players to vi
229、ew users chat history.Some of the data that was exposed included names,email addresses,credit card types,payment addresses,and chat histories.The potential aftermath of this incident includes the exposure of private data(pertaining to both individuals and businesses),damaged reputation,and legal rep
230、ercussions.Read more here:Generative AIs first data breach:OpenAI takes action,bug patched|Markets and Markets Key considerations Personally identifiable information(PII),as well as sensitive,confidential,proprietary,or otherwise protected information stored by a GenAI system which was entered as pa
231、rt of a prompt or gathered as part of the model training process could be accessed by an attacker or other adversarial players.Adversarial players can use“prompt injection,”a method used by hackers that tricks the system into bypassing specific security or ethical guardrails that have been patched o
232、nto foundational models,to manipulate GenAI systems to output unauthorized information.GenAI tools such as ChatGPT can be tricked into generating malware or ransomware programming code.Adversarial players can poison data to create vulnerabilities in the system.Deepfakes,or digitally forced images or
233、 videos,can be created using GenAI.Best practices Monitor and test the GenAI system for vulnerabilities,threats,failures,etc.and work to develop methods of test and securing AI systems more efficiently.Develop and implement detection features that can identify threats,failures,and attacks on the sys
234、tem and notify personnel.18 Definition June 2024 Page 24 Train users to understand the security risks associated with AI,including the potential for malicious use or adversarial attacks,as well as risks to input and output validation and data integrity.Develop and regularly update robust,secure syst
235、ems for sites to defend against threats and plan for system resilience to ensure that AI systems can recover from potential attacks or failures.For GenAI systems that DOE has developed or for which DOE has compiled a specialized training set,establish a program to conduct adversarial testing via“red
236、 teaming,”which involves actively seeking out examples of where the GenAI system fails,retraining the model on these examples,and continuing this iterative process until the team closes the loop on identifying failures.19 Regularly update provisions to the systems risk management plan to reflect the
237、 newest risks.Isolate GenAI systems as much as is practical and avoid allowing GenAI systems to directly control other systems(especially real-world physical systems).Additional resources NIST AI Risk Management Framework DOE Artificial Intelligence Risk Management Playbook June 2024 Page 25 8.5 Pri
238、vacy Protecting privacy is fundamental to preserve the publics trust in the government.The federal government strives to hold the highest standards in collecting,maintaining,using,and disseminating peoples personal information.The privacy consideration involves protecting the security of personal in
239、formation to ensure its accuracy,relevance,timeliness,and completeness,avoiding unauthorized disclosure,and ensuring that no system of records concerning individuals,no matter how insignificant or specialized,is maintained without public notice.20 Examples Examples Illustrative example During a glob
240、al pandemic with human contact and exposure concerns,an organization seeks to understand where resources are physically present for purposes of cleaning and disinfection.An AI model is used to consume and train on input data.As the data is fed into the model,the output can provide information in a w
241、ay that unintendedly exposes individuals geolocation.This situation now requires additional legal notifications to individuals regarding privacy.Public example In 2021,Canadian privacy officials found that American-based company Clearview AI was collecting photos of Canadian citizens,including child
242、ren,without their knowledge or consent for use in a facial recognition software which was used by law enforcement agencies to identify persons of interest or victims.The AI utilized billions of photos found on the internet and social media accounts to attempt to identify the person.Even after use of
243、 the technology was halted,Clearview continued to use pictures of Canadian citizens.Experts warned that not only was the storage of data against privacy laws,but that the tool itself could easily be misused.This issue also highlights the concept of consent in collecting training data.Read more here:
244、U.S.technology company Clearview AI violated Canadian privacy law:report|CBC News Key considerations AI platforms and service providers may share user information with third parties,including vendors,service providers,affiliates,or other users,without informing the user.Information entered into a Ge
245、nAI system may become part of its training data set.Thus,any proprietary,sensitive,personally identifiable,confidential,or otherwise protected data entered as part of a prompt could be used in outputs for other users of the system.Best practices Integrate privacy programmatic considerations into a w
246、ide range of functions,including but not limited to information security,records management,strategic planning,budget and acquisition,contractors and third parties,workforce,training,incident response,and risk management.Continue to adhere to existing privacy policies and procedures.Iteratively revi
247、ew new privacy policies and recommended procedures to ensure they are accounted for in GenAI use cases.Exercise data minimization practices by taking steps to anonymize data and limit the collection,storage,and reuse of personal information.Definition June 2024 Page 26 Do not include protected data
248、or nonpublic information(as part of the input to any commercial or open GenAI system.Describe clearly and accurately,and share in an accessible way,how the department uses,manages,and collects information.Clearly document who creates,contributes to,and has access to that information,and communicate
249、this to all people who entrust government with their data and information.21 Implement full lifecycle stewardship of data,which is the practice of securing and protecting data,metadata,and information throughout its lifecycle.That includes collection,storage,use,control,processing,publication,transf
250、er,retention,and disposition.22 .Develop tailored GenAI privacy trainings for employees with access to protected data(e.g.,sensitive data,private data,confidential data,limited rights data,proprietary data etc.)or any other nonpublic information.23 Conduct a privacy impact assessment(an analysis of
251、how information is handled to ensure handling conforms to applicable legal,regulatory,and policy requirements regarding privacy,to determine the risks and effects of creating,collecting,using,processing,storing,maintaining,disseminating,disclosing,and disposing of information in identifiable form in
252、 an electronic information system,and to examine and evaluate protections and alternate processes for handling information to mitigate potential privacy concerns)as necessary when developing and implementing a new technology.24 Differential-privacy guarantees,or protections that allow information ab
253、out a group to be shared while provably limiting the improper access,use,or disclosure of personal information about particular entities,should be understood when in place,including how these guarantees affect data shared with,used to train,or created by GenAI technologies.25 GenAI can also facilita
254、te or interact with privacy-enhancing technologies(PETs),or“any software or hardware solution,technical process,technique,or other technological means of mitigating privacy risks arising from data processing,including by enhancing predictability,manageability,disassociability,storage,security,and co
255、nfidentiality.These technological means may include secure multiparty computation,homomorphic encryption,zero-knowledge proofs,federated learning,secure enclaves,differential privacy,and synthetic-data-generation tools.This is also sometimes referred to as“privacy-preserving technology.”26 Understan
256、d how GenAI may be used to increase privacy in newly developed or existing capabilities,or how the introduction of GenAI into a system may affect existing PETs.Per the NIST AI RMF,“privacy-enhancing technologiesas well as data minimizing methods such as de-identification and aggregation for certain
257、model outputs,can support design for privacy-enhanced AI systems.Under certain conditions such as data sparsity,privacy-enhancing techniques can result in a loss in accuracy,affecting decisions about fairness and other values in certain domains.”27 Additional resources DOE O 206.1 Department of Ener
258、gy Privacy Program,January 16,2009 Office of Science and Technology(OSTP),Blueprint for an AI Bill of Rights,October 2022 June 2024 Page 27 8.6 Confidentiality As noted in Section 7.3,the word“confidential”when used in this Guide falls outside of the NSI definition.Confidential information input int
259、o a GenAI tool may be stored or processed by the tool or its providers,revealing confidential information to unauthorized personnel.Within US government documents,the word“Confidential”has a specific National Security Information(NSI)definition that relates to the level of the severity of harm if a
260、document marked“Confidential”is inappropriately shared.In this Guide,use of the word“confidential”is outside of the NSI context.Instead,it is associated with the non-governmental business environment.Confidentiality,as discussed in this Guide,is defined as“preserving authorized restrictions on acces
261、s and disclosure,including means for protecting personal privacy and proprietary information.”28 The DOE Operations Security Handbook states that there are two primary characteristics of a piece of information that determine whether that information is safe for public disclosure or whether it should
262、 be considered sensitive.These two primary characteristics for“determining suitability for release of information”are sensitivity and risk.Sensitivity:“If the information is released to the public,it should not reveal or identify sensitive information,activities,or programs.”Sensitive information ca
263、n also be defined as information that could be used by adversaries to the detriment of the organization,its employees,the public,or the nation.Sensitivity gauges the level of harm that could ensue from release.Risk:“Information that may be used by adversaries to the detriment of employees,the public
264、,the department,or the nation should not be approved for release.This determination should be based on sound risk management principles focused on preventing potential adverse consequences.”In terms of the definition of sensitivity presented above,risk the likelihood of such harm.Together,these two
265、characteristics suggest that the term“sensitive information”is a categorical term which includes other specific types of sensitive information.29 Refer to Appendix J for a list of protected data types and definitions.Examples Examples Public example 1 Both Apple and Samsung have instituted restricti
266、ons on the use of OpenAIs ChatGPT and Microsofts GitHub Copilot by some of their employees because of concerns over the potential for employees to mishandle and leak confidential company data.This move aligns with a growing trend of companies and governments worldwide imposing restrictions on the us
267、e of GenAI platforms.In April 2023,OpenAI released a series of updates to ChatGPT that enabled better privacy controls after some nations voiced their concerns.Read more here:Apple Restricts Employee Use of ChatGPT,Joining Other Companies Wary of Leaks|WSJ Public example 2 Samsung Electronics banned
268、 the use of any AI-powered chatbots and ChatGPT by its employees because of concerns about sensitive internal information being leaked.This decision comes after an accidental leak of sensitive source code through ChatGPT,prompting the company to issue a memo banning the use of GenAI tools.Even thoug
269、h the exact severity of the leak is unknown,data shared with chatbots may be stored on servers owned by outside companies operating the service,such as like OpenAI,without the ability for Samsung to access or delete the data.Read more here:Samsung Bans ChatGPT Among Employees After Sensitive Code Le
270、ak|Forbes Definition June 2024 Page 28 Key considerations Information entered into a GenAI system may become part of its training data set.Thus,any confidential data entered as part of a prompt could be used in outputs for other system users,which could result in unintended exposure or misuse of thi
271、s information.Confidential information should not be used in any way which could lead to the information being shared outside of its intended or authorized used.Data provenance,digital rights management,and understanding of data rights are critical to responsible GenAI management.Digital rights mana
272、gement programs should be robust in order to avoid unauthorized data usage.GenAI systems can store data and information input as prompts indefinitely.Adversarial players can hack the system to gain access to any stored confidential data.Best practices Do not input or disclose protected data or nonpu
273、blic information,as part of a prompt when using a public GenAI tool,unless you can validate rights to use it in this way from the originator.Refer to the most recent versions of DOE information security policies for specific guidance.Seek advice from your organizations legal department.Do not rely o
274、n GenAI to generate confidential or mission-critical information or data,as the information used to train AI models may not be accurate,complete,or without bias.Review and continue to adhere to existing policies,procedures,and guides to ensure compliance with National Laboratories and DOE informatio
275、n requirements.Additionally,continue to follow existing requirements,such as those regarding quality,information security,and integrity.Work with appropriate DOE and National Laboratory SMEs and compliance organizations,such as the Office of General Counsel,the Office of Export Control,the Classific
276、ation Office,the Office of Environment,Health,Safety,and Security(EHSS),and others as appropriate.Continue to follow existing cybersecurity and privacy procedures and iteratively review existing policies and procedures to understand how confidentiality should be applied generally within an organizat
277、ion,to ensure that new policies and regulations are implemented in issued procedures,and to stay current with the most up-to-date requirements.Encourage prompt engineering training for GenAI users to learn the best ways to structure prompts that generate more accurate outputs(refer to Appendix I for
278、 more details on prompt engineering).Review output produced from GenAI tools to ensure that any issues surrounding confidentiality are identified and addressed.Practice secure storage and processing of protected data and nonpublic information and implement access controls.Clearly define who has acce
279、ss to the data and the purpose of its use every time.Note that contract solicitation responses are proprietary(confidential)information.Refer to existing DOE training on confidentiality that employees are required to take.It is imperative that users understand the legal rights the Government(has or
280、doesnt have)in the data inputted into AI tools and/or which are used to train data LLMs.Additional resources DOE mandatory training on confidentiality:CUI-100DE Controlled Unclassified Information Overview June 2024 Page 29 8.7 Intellectual Property GenAI tools introduce risks surrounding intellectu
281、al property(IP),including copyright and data protections,as the tools can access copyrighted works and generate outputs that closely resemble content from these works.It should also be noted that the intersection of GenAI and existing laws,regulations,and policies regarding IP and copyright is dynam
282、ic and evolving.This guide is not a replacement for legal advice;therefore,any legal questions related to the intersection of GenAI and IP should be directed to cognizant DOE or contractor legal counsel.Intellectual property(IP)is intangible property that is the product of an original thought,includ
283、ing inventions,designs,writings,images,and names,much of which is protectable by statutory and contractual rights,including patents,copyrights,trade secrets,and trademarks(intellectual property rights or IPR).Data may also be protectable as IP,typically as copyrightable compilation,if selected and a
284、rranged in a unique and original way,such as with a dataset.Data in such form may be copyrightable and licensable.Copyright is not a single right,but a bundle of rights that include not only reproduction,but also provide the copyright owner the right to prevent others from adapting,distributing to t
285、he public,performing,and displaying the copyright work(including digitally).Intellectual property issues have significant legal,financial,and ethical implications.Examples Examples Public example 1 Comedian Sarah Silverman and two authors filed a class-action lawsuit on July 7,2023,against OpenAI an
286、d Meta,accusing them of copyright infringement for the use of their protected work in the companies training datasets.According to the lawsuit,“copyrighted materials were copied and ingested as part of training.”While the outcome is still pending,the context surrounding this lawsuit is of great impo
287、rtance,as the training dataset can include copyrighted materials“without permission by scraping illegal online“shadow libraries”that contain the text of thousands of books,”as mentioned by The New York Times.Read more here:Sarah Silverman Sues OpenAI and Meta Over Copyright Infringement|The New York
288、 Times Public example 2 In Andersen v.Stability AI et al.,a case filed in 2022,three artists filed a lawsuit against several GenAI vendors on the grounds that the GenAI systems used their original works as part of a training set.Users of these systems were able to generate works very similar to the
289、original artists works.If a court rules that the GenAI-generated works are derivative and unauthorized,considerable penalties may apply.Read more here:Generative AI has an intellectual property problem|Harvard Business Review Key considerations Under current law,an invention created solely by an AI
290、tool is not able to be patented or copyrighted because the output is not created by a human.30 In August of 2023,the United States Copyright Office,a subsidiary of the Library of Congress,released a request for public comment on the interaction of AI and copyright law which provides a picture of the
291、 types of discussions which are ongoing in this area of legal study.31 Inventors should discuss specific cases with a patent attorney,as the Department of Energy policy is unable to determine legality.The U.S.Copyright Office has also issued several statements informing creators that it will not reg
292、ister copyrights for works produced by a machine or computer program.Definition June 2024 Page 30 On February 13,2024,the United States Patent and Trademark Office(USPTO)issued new guidance that explains that while AI-assisted inventions are not categorically unpatentable,the inventorship analysis s
293、hould focus on human contributions,as patents function to incentivize and reward human ingenuity.Patent protection may be sought for inventions for which a natural person provided a significant contribution to the invention,and the guidance provides procedures for determining the same.32 In August o
294、f 2023,the United States Copyright Office,a subsidiary of the Library of Congress,released a request for public comment on the interaction of AI and copyright law which provides a picture of the types of discussions which are ongoing in this area of legal study.33 Inventors should discuss specific c
295、ases with a patent attorney,as the Department of Energy policy is unable to determine legality.The U.S.Copyright Office has also issued several statements informing creators that it will not register copyrights for works produced by a machine or computer program.GenAI tools are trained on huge sets
296、of scraped data,and that training forms the basis for the models responses to prompts.Therefore,GenAI tools may generate outputs that contain plagiarized or copyrighted information.Considerable care should be taken not to infringe intellectual property rights,or violate other protections,when inputt
297、ing data into GenAI prompts or otherwise using data to train LLMs.Best practices Adhere to existing policies and procedures regarding copyright issues,and continue to monitor for changes in copyright laws,policies,and recommended procedures that apply to GenAI tools.Have a human in the loop,preferab
298、ly someone who has knowledge of GenAI,to validate generated output sources and prevent plagiarism and/or copyright issues.Use caution when using the outputs of a model in other work,and keep in mind that large language models will not reliably tell you if their sources are in the public domain or no
299、t.When GenAI solutions play a role in creating an idea,approach,or invention at DOE or a National Laboratory,employees must clearly identify the specific contribution(e.g.,attribution in a report,laboratory record,invention disclosure)and cite the GenAI as part of their research methodology.It may b
300、e essential to determining patentability or copyrightability to know specific attributions to humans versus AI technologies.Several style guides and publishing houses are developing guidance on how to appropriately credit AI tools in written work(See the additional resources below).Employees should
301、follow these guidelines where they exist.Educate users on the challenge of AI generating content that may infringe on existing copyrights and promote an understanding of intellectual property rights with regard to GenAI.Use secondary tools to identify and validate sources,context,and citations,parti
302、cularly in cases where the user has some prior knowledge.Avoid using GenAI output to create website content unless the origin of the training data is verified as appropriate for the given use,as it may contain or have been trained on confidential or sensitive data.See Section 8.6:Confidentiality.Emp
303、loy best practices for prompt engineering(refer to Appendix I for additional information on prompt engineering).Additional resources The Use of Copyrighted Materials by Government Employees,Department of Energy Congressional Research Service:Generative Artificial Intelligence and Copyright Law Gener
304、ative AI Has an Intellectual Property Problem “How to cite ChatGPT,”Timothy McAdoo,APA Style,April 7,2023 June 2024 Page 31 8.8 Safety GenAI systems must be designed to be safe for system users and society in general.Outputs resulting from GenAI systems should not compromise the safety of individual
305、s or their health or property.AI systems are safe if they do“not under defined conditions,lead to a state in which human life,health,property,or the environment is endangered.”34 Safe operation of AI systems is achieved through:Responsible design,development,and deployment practices Clear informatio
306、n to deployers on responsible use of the system Responsible decision-making by deployers and end users(e.g.,via reinforcement learning,defined in Section 5.1)Explanation and documentation of risks based on empirical evidence of incidents35 Examples Examples Public example 1 In 2022,it was reported t
307、hat Tesla vehicles utilizing the AI-assisted Autopilot functionality had been involved in 273 crashes during the previous year.These included crashes with other cars and motorcycles,and pedestrian and driver deaths.The autopilot functionality includes the ability to maintain speed and safe distance
308、behind other cars,to stay within their lane lines,and to make lane changes on highways.According to Tesla,however,human drivers are supposed to keep their eyes on the road and their hands on the wheel,with the technology serving as an assistant.This human oversight is critical to safe operation of t
309、he vehicle.Read more here:Teslas running Autopilot involved in 273 crashes reported since last year|The Washington Post Public example 2 President Biden announced that leading AI companies,such as OpenAI,Alphabet,and Meta,“have made voluntary commitments to the White House to implement measures such
310、 as watermarking AI-generated content to help make the technology safer.”This will enable the identification of when content was generated by AI,and most importantly,the identification of deepfakes that can spread misinformation or be used to defraud individuals.The companies also made pledges to te
311、st systems thoroughly before release and to focus on protecting users privacy.These commitments are steps toward ensuring safeguards in GenAI.Read more here:OpenAI,Google,others pledge to watermark AI content for safety,White House says|Reuters Key considerations Various types of risks involving saf
312、ety might necessitate custom AI risk mitigation strategies depending on the context and the severity of the potential risks.Safety relates mostly to the use and application of the system.Safety risks can arise from both negligence and deliberately malicious intent.Establish mechanisms to support rep
313、roducibility and ability to scrutinize outputs for accuracy and consistency through versioning and provenance of training inputs,model parameters,data corpus,and other key system elements.Definition June 2024 Page 32 Best practices Adopt a safety-by-design approach and mentality by considering safet
314、y risks throughout the AI lifecycle,starting as early as possible during the planning and design phases.Do not use GenAI for malicious or deceptive activities,such as the creation of malware,identity theft,or identity impersonation.Develop safety measures for AI system deployment,including checks ag
315、ainst harmful and unintended uses.Leverage the guidelines for safety in the transportation and healthcare fields and align with the existing sector-or application-specific guidelines or standards in AI safety risk mitigation strategy(e.g.,the NIST AI Risk Management Framework(NIST AI RMF).Have a hum
316、an in the loop throughout the AI lifecycle.Human oversight,validation,and verification,are a combined,iterative process beginning in the planning and design phase and continuing throughout the AI lifecycle,including after the model is deployed.Have the appropriate responsible AI solutions controls i
317、n place.Consider using the prompt(any modality,such as a sensor)as an interface with the system about the current state that can be used as a control for increasing safety.Account for secondary usage of outputs by ensuring that any caveats,considerations,and/or assumptions are tacked on to the outpu
318、t so that the output will not be inadvertently misused.Intentional malicious misuse is outside of our control.Per the NIST AI RMF,“AI safety risk management approaches should take cues from efforts and guidelines for safety in fields such as transportation and healthcare and align with existing sect
319、or-or application-specific guidelines or standards.”36 June 2024 Page 33 8.9 Fairness and Bias GenAI introduces challenges in defining,measuring,and addressing concerns about fairness and bias in a number of ways.Fairness in AI involves addressing issues such as harmful bias and discrimination to fo
320、ster equality and equity.Standards of fairness can be complex and difficult to define because perceptions of fairness differ among cultures and may shift depending on application.37 GenAI systems should be designed to be fair so that individuals or groups are not systematically disadvantaged through
321、 AI-driven decisions.Achieving fairness in AI can be challenging,as it requires careful consideration of different types of bias and using the technology in a way that avoids favoritism or discrimination,particularly to humans.Bias refers to the systematic and consistent deviation of an algorithms o
322、utput from the true value or from what would be expected in the absence of bias.38 Bias is a component of fairness and comes in many forms,going beyond lack of demographic balance or data representativeness.NIST has identified three major categories of AI bias to be considered and managed:systemic,c
323、omputational and statistical,and human-cognitive.Each of these can occur in the absence of prejudice,partiality,or discriminatory intent.Systemic bias can be present in AI datasets,the organizational norms,practices,and processes across the AI lifecycle,and the broader society that uses AI systems.C
324、omputational and statistical biases can be present in AI datasets and algorithmic processes,and often stem from systematic errors due to nonrepresentative samples.Human-cognitive biases relate to how an individual or group uses AI system information to decide or fill in missing information,or how hu
325、mans think about an AI systems purposes and functions.Human-cognitive biases are omnipresent in decision-making processes across the AI lifecycle and system use,including the design,implementation,operation,and maintenance of AI.39 While fairness and bias are closely related concepts,they differ in
326、important ways.The key difference is that while bias can be unintentional,fairness is inherently a deliberate and intentional goal.In other words,bias can be viewed as a technical issue,while fairness is a social and ethical issue.40 Examples Examples Public example 1 Stable Diffusions text-to-image
327、 GenAI solution has been identified by Bloomberg as a model that contributes to biased racial and gender stereotypes.Bloomberg used Stable Diffusions tool to create thousands of images pertaining to crime and employment.In this analysis,the model was prompted with text to create images of workers fo
328、r 14 jobs 300 images for seven jobs generally considered as“high-paying”in the U.S.and 300 images for seven jobs generally considered“low-paying”as well as three topics related to crime in the U.S.The analysis discovered that images generated for the high-paying jobs were of people with lighter skin
329、 tones.In contrast,images generated with darker-skinned Public example 2 A class action lawsuit was filed against HR and financial management software provider Workday,alleging that the software produced a screening system that resulted in racial bias.The lawsuit alleges that Workday“unlawfully offe
330、rs an algorithm-based applicant screening system that determines whether an employer should accept or reject an application for employment based on the individuals race,age,and/or disability.”The plaintiff of the lawsuit states that Workdays AI tools rely on algorithms that may be riddled with human
331、 bias.Read more here:Workday wants racially biased recruitment algorithm claim thrown out|The Register Definition June 2024 Page 34 Examples Examples subjects were created by the solution in response to prompts like“fast-food worker.”The conclusion of this analysis implies that racial and gender rep
332、resentation in various career images was significantly different than the representation in the actual careers.For instance,about 3%of the images generated for the prompt“judge”were women,whereas in reality,about 34%of American judges are women.Read more here:Humans are biased.Generative AI is even
333、worse|Bloomberg Key considerations Fairness needs to be defined for every use case at the beginning of the design and planning phase,as it can mean different things in different contexts.Outputs from GenAI systems can and will produce bias if bias is included in the training,validation,or test datasets,which is usually the case.Bias can be introduced at any point in the AI lifecycle.Non-stationary
0731-84720580
sgpjbg002
工作日 9:30 - 18:00
报告分类
新质生产力
ChatGPT
新能源汽车
大模型
Sora
机器人
微短剧
芯片
薪酬
白皮书
低空经济
数据要素
创新药
人工智能
AI产业
信息科技
互联网
消费经济
汽车交通
电商零售
传媒娱乐
医药健康
投资金融
能源环境
地产建筑
传统产业
英文报告
其它
扫码登录
手机快捷登录
账号登录
