Technology Assessment Design Handbook - Gao

1m ago
0 Views
0 Downloads
1.83 MB
65 Pages
Last View : 1m ago
Last Download : n/a
Upload by : Francisco Tran
Transcription

HANDBOOKTECHNOLOGYASSESSMENTDESIGNHANDBOOKHandbook for Key Steps and Considerationsin the Design of Technology AssessmentsGAO-21-347GFebruary 2021

ContentsPrefaceChapter 1Chapter 2Chapter 31The Importance of Technology Assessment Design81.1 The Purpose of Technology Assessment1.2 Importance of Spending Time on Design89Technology Assessment Scope and Design112.1 Sound Technology Assessment Design2.2 Stages and Considerations for Technology AssessmentDesign2.2.1 GAO Technology Assessment Design Examples11Approaches to Select Technology Assessment Design andImplementation Challenges3.1 Ensuring the Design and Implementation of TechnologyAssessments Result in Useful Products for Congress and OtherPolicymakers3.2 Determining the Policy Objective and Measuring PotentialEffects3.3 Researching and Communicating Complicated Issues3.4 Engaging Relevant Stakeholders12182223242526Appendix IObjectives, Scope, and Methodology27Appendix IIGAO’s Expertise with Technology Assessments42Appendix IIISummary of Steps for GAO’s General Engagement Process45Appendix IVExample Methods for Technology Assessment48Page iGAO-21-347G Technology Assessment Design Handbook

Appendix VAppendix VIOverview of Scope and Design of Policy Options for TechnologyAssessments55GAO Contact and Staff Acknowledgments59TablesTable 1: Summary of GAO’s Technology Assessment ProcessTable 2: Examples for Technology Assessment Objectives thatDescribe Status and Challenges to Development of aTechnologyTable 3: Examples for Technology Assessment Objectives thatAssess Opportunities and Challenges that May Resultfrom the Use of a TechnologyTable 4: Examples for Technology Assessment Objectives thatAssess Policy Implications or Options Related to aTechnologyTable 5: Challenges to Ensuring the Design and Implementationof Technology Assessments Result in Useful Products forCongress and Other PolicymakersTable 6: Challenges to Determining the Policy Objective andMeasuring Potential EffectsTable 7: Challenges to Researching and CommunicatingComplicated IssuesTable 8: Challenges to Engaging Relevant StakeholdersTable 9: Select Examples of Methodologies for TestimonialEvidenceTable 10: Select Examples of Methodologies for DocumentaryEvidenceTable 11: Select Examples of Methodologies for PhysicalEvidence419202123242526495152FiguresFigure 1: Summary of Key Stages and Considerations ofTechnology Assessment DesignFigure 2: Summary of Key Stages for Design of Policy Options forTechnology AssessmentsPage ii1356GAO-21-347G Technology Assessment Design Handbook

AbbreviationsAICRSEMSOTAS&TSTAATATRLartificial intelligenceCongressional Research ServiceEngagement Management SystemOffice of Technology Assessmentscience and technologyScience, Technology Assessment, and Analyticstechnology assessmenttechnology readiness levelThis is a work of the U.S. government and is not subject to copyright protection in theUnited States. The published product may be reproduced and distributed in its entiretywithout further permission from GAO. However, because this work may containcopyrighted images or other material, permission from the copyright holder may benecessary if you wish to reproduce this material separately.Page iiiGAO-21-347G Technology Assessment Design Handbook

Letter441 G St. N.W.Washington, DC 20548PrefaceThe Government Accountability Office (GAO) provides Congress, federalagencies, and the public with non-partisan, objective, reliable informationto help the government save money and work more efficiently andeffectively. Science and technology (S&T) issues figure prominently inproblems that Congress confronts, and one component of the assistanceGAO provides to Congress is the production of technology assessments(TA). GAO TAs analyze recent S&T developments, highlight potentialeffects of technological change, and strive to make S&T concepts readilyaccessible to policymakers.The Technology Assessment Act of 1972 (Public Law 92-484)established the Office of Technology Assessment (OTA) as an analyticalsupport agency of the Congress. OTA was defunded in 1995. 1 In 2002,GAO began conducting TAs, 2 and in 2008, established a permanent TAfunction. 3 In 2019, GAO created the Science, Technology Assessment,and Analytics (STAA) team by pulling together and building upon existingelements and expertise within GAO. 4 For more details on GAO’sexpertise with technology assessments, see Appendix II.The TA Design Handbook provides GAO staff and others with tools toconsider for supporting robust and rigorous TAs, while following internalGAO guidance. 5 This handbook is particularly important given the needfor GAO to provide insight and foresight on the effects of technologiesand corresponding policy implications related to a wide range of S&Tissues. Other organizations may also find portions of this handbook usefulas they consider or conduct TAs, although their needs, approaches, andrelationships with stakeholders and government bodies may differ.1SeeLegislative Branch Appropriations Act, 1996, Pub. L. No. 104-53, 109 Stat. 514, 526(1995).2SeeH.R. Rep. No. 107-259, at 47 (2002) (Conf. Rep.) (directing the Comptroller Generalto obligate funds for a pilot program in technology assessment).3See Consolidated Appropriations Act, 2008, Pub. L. No. 110-161, 112 Stat. 1844, 2249(2007) (providing up to 2.5 million of amounts appropriated to GAO for technologyassessment studies).4SeeH.R. Rep. No. 115-929, at 213 (2018) (Conf. Rep.) (encouraging GAO to reorganizeits technology and science function by creating a new, more prominent office within GAO).5SomePage 1of these tools may be useful to teams that design other types of engagements.GAO-21-347G Technology Assessment Design Handbook

Technology AssessmentsHistory of Science and Technology Workat GAOThe Government Accountability Office (GAO)has conducted science and technology (S&T)work for close to 50 years, includingtechnology assessments for almost twodecades. In 2018, Congress encouragedGAO to form an S&T-focused team,recognizing that the scope of technologicalcomplexities continue to grow significantly andthere is need to bolster capacity of, andenhance access to, quality, independentscience and technological expertise forCongress. On January 29, 2019, GAOformally created the Science, TechnologyAssessment, and Analytics (STAA) team bypulling together and building upon existingelements within GAO. Since then, STAA hasprovided over 50 products to Congress,including technology assessments covering awide range of science, technology, andinformation technology issues. In addition,STAA has also worked collaboratively withother teams at GAO on about 275 productssince its creation.New technologies can have a range of effects, potentially both positiveand disruptive, that TAs can explore. 6 GAO has broadly defined TA as thethorough and balanced analysis of significant primary, secondary,indirect, and delayed interactions of a technological innovation withsociety, the environment, and the economy and the present and foreseenconsequences and effects of those interactions. 7 GAO TAs share somecommon design principles with GAO’s general audit engagementprocess, which is centered on intentional and purpose-driven design. 8Source: GAO-20-246G and review of GAO product line data. GAO-21-347G6Inthis context, GAO refers to technology broadly as the practical application ofknowledge in a particular area, and the resulting capability given by that application.Technology may also refer to a manner of accomplishing a task using technicalprocesses, methods, or knowledge as well as the specialized aspects of a particular fieldof endeavor.7Thereis no single agreed-upon typology/taxonomy of or approach to TAs. Examples ofdifferent TA approaches found in the literature include, but are not limited to: strategic,early-warning, future-oriented, classical or expert, real-time, constructive, andparticipatory. For example, expert TAs may emphasize expert knowledge, andparticipatory TAs may emphasize stakeholder and public involvement. Please refer toAppendix IV for further discussion of example methodologies that can be used with theseapproaches.8Forexample, both TAs and GAO’s general audit engagement process includes a robustinitiation and design process that considers factors such as: requester’s interests andstakeholder input, the current state of knowledge, and relevant and appropriatemethodological considerations in defining and investigating appropriate researchquestions. Also part of GAO’s general audit engagement process is internal messagedevelopment and agreement, along with external review. Design decisions areimplemented and revisited throughout the audit engagement process. Refer to AppendixIII for a summary of the typical GAO engagement process, of which design is a part.Page 2GAO-21-347G Technology Assessment Design Handbook

While general design principles are shared across GAO’s product lines,TAs are distinct from other GAO products due to their specialized content,scope, and purpose, which warrant different considerations. 9 Table 1highlights some similarities and differences between TAs and other GAOproduct lines, including where TAs follow aspects of GAO’s general auditengagement process, and where TAs may further emphasize certainsteps or require additional steps during the engagement process. 109Examplesof other GAO products include performance audits, financial audits, and otherroutine non-audit products.10Other product lines may emphasize these elements as well, depending on engagementneeds.Page 3GAO-21-347G Technology Assessment Design Handbook

Table 1: Summary of GAO’s Technology Assessment ProcessSteps in plain text are process steps for both general audit and TA products. Steps in bold italics are either additional process steps or a particularemphasis for technology assessments (TA).aPhaseStepsInitiation Design Messagedevelopment External review Discussion with congressional requesters, if applicable, regarding scope and focus of the engagementbConsideration of technology state, relevant stakeholder expertise, and potential policy implicationsConsideration of whether policy options may be appropriate for inclusionPerformance of initial researchConsideration of relevant sections of GAO’s quality standards and GAO methodological and technicalstandards and guidesConsultation with GAO subject matter experts and internal stakeholders, as neededDiscussion with agency officials and expertsIdentification of and consultation with external experts, such as science, policy, and industryexperts, who may also serve as external reviewerscIdentification of possible policy options, if appropriateCollection and analysis of evidenceAssessment of evidence and research resultsDevelopment of draft findingsOngoing engagement with external expertsConduct and discuss policy options assessment, if appropriatedRequest views from relevant third parties, if applicable, and request comments from relevant federalagencies, as appropriateRequest comments from external experts, and others as appropriateSource: GAO-20-246G and additional review of GAO product lines. GAO-21-347GaNot all steps have been included in this table.GAO performs work for Congress that is initiated through requests, legislation (i.e., statutorymandates), and Comptroller General Authority (i.e., GAO-initiated work). In addition, GAO conductswork in response to requests for technical assistance (e.g., briefings on prior or ongoing work,responses to technical questions, short-term analysis of agency programs or activities, detailedfollow-up, and hearing support).bcFor example, GAO has contracted with the National Academies of Sciences, Engineering, andMedicine to help GAO identify experts on various scientific topics and leverage National Academiesassistance to convene GAO expert meetings.Unlike GAO’s general audit products, which often contain recommendations, TAs may include policyoptions designed to enhance benefits or mitigate challenges of a technology.dSTAA has taken a number of steps to account for the unique nature ofTAs and related S&T work. The effects of technological interactions canhave implications, and recognizing this, GAO includes policy options insome of its products. Policy options are defined as a set of alternatives ormenu of options that may enhance benefits or mitigate challenges of atechnology, and which policymakers such as legislative bodies,government agencies, standards-setting organizations, industry, andPage 4GAO-21-347G Technology Assessment Design Handbook

other groups, could consider taking. 11 A first step for developing policyoptions is to define a policy objective, which guides the development ofthe options by stating the overall aim, and by helping to identify thelandscape and scope of the options. 12 GAO is continuing to exploreapproaches to making policy options a standard feature in its S&T workand has included them in four TAs to date. 13 GAO’s experiences withthose TAs inform this update to the handbook, and we have includedconsiderations related to the development of policy options that teamsmay wish to consider at each phase of TA design.Objectives, Scope, andMethodologyThis handbook elaborates on GAO’s approach to TA design. It outlinesthe importance of TA design (Chapter 1), describes the process ofdeveloping TA design (Chapter 2), and provides approaches to select TAdesign and implementation challenges (Chapter 3). The handbookgenerally follows the format of the 2012 GAO methodology transferpaper, Designing Evaluations. 14 This is an update to the handbookpublished in December 2019. 15 We are updating this handbook to includethe experiences of GAO teams and relevant literature since the handbookwas initially published as well as comments made by external experts andthe public submitted between December 2019 and December 2020.11Policy options are for policymakers to consider and take action on at their discretion. Inaddition, GAO TAs strive to list likely policy options supported by analysis, but the list maynot be exhaustive, and policymakers may choose to consider other policy options notlisted by GAO.12TA teams may identify more than one policy objective if the technology’s landscape andscope is particularly complex. Alternatively, if a team chooses not to include a policyobjective as part of a TA’s scope and design, the final product will not include policyoptions.13GAO,Artificial Intelligence in Health Care: Benefits and Challenges of Technologies toAugment Patient Care, GAO-21-7SP (Washington, D.C.: Nov. 30, 2020); GAO, 5GWireless: Capabilities and Challenges for an Evolving Network, GAO-21-26SP(Washington, D.C.: Nov. 24, 2020); GAO, Artificial Intelligence in Health Care: Benefitsand Challenges of Machine Learning in Drug Development, GAO-20-215SP (Washington,D.C.: Dec. 20, 2019); and GAO, Irrigated Agriculture: Technologies, Practices, andImplications for Water Scarcity, GAO-20-128SP (Washington, D.C.: Nov. 12, 2019).14Designing Evaluations describes designs of program evaluations. See GAO, DesigningEvaluations: 2012 Revision, GAO-12-208G (Washington, D.C.: Jan. 2012).15GAO,Technology Assessment Design Handbook, GAO-20-246G (Washington, D.C.:Dec. 4, 2019).Page 5GAO-21-347G Technology Assessment Design Handbook

The following summarizes the approach we used to identify anddocument TA design steps and considerations for this handbook, and toupdate the handbook. For more information, please refer to Appendix I:Objectives, Scope, and Methodology. Reviewed select GAO documents, including Designing Evaluations(GAO-12-208G), published GAO TAs, select GAO products utilizingpolicy analysis approaches to present policy options, and other GAOreports Reviewed select Office of Technology Assessment reports Reviewed select Congressional Research Service reports Reviewed select English-language literature regarding TAs andrelated to development and analysis of policy options Consulted with external experts and performed outreach, includingholding an expert meeting to gather input regarding TA design,soliciting comments from external experts who contributed to GAOTAs published since 2015, and soliciting comments from the public 16 Reviewed experiences of GAO teams that have successfullyassessed and incorporated policy options into GAO products and TAdesign, including related to challenges to TA design andimplementation and possible solutionsTo update this handbook, we conducted our work from March 2020 toFebruary 2021 in accordance with all sections of GAO’s QualityAssurance Framework that are relevant to our objectives. The frameworkrequires that we plan and perform the engagement to obtain sufficientand appropriate evidence to meet our stated objectives and to discussany limitations in our work. We believe that the information and dataobtained, and the analysis conducted, provide a reasonable basis for anyfindings and conclusions in this product.16Wesolicited comments on this handbook from December 2019 to December 2020.Page 6GAO-21-347G Technology Assessment Design Handbook

If you have any questions concerning this handbook, please contactTimothy M. Persons or Karen L. Howard at (202) 512-6888 [email protected] or [email protected] Contact points for our Officesof Congressional Relations and Public Affairs can be found on the lastpage of this handbook. Key contributors to this handbook are listed inAppendix VI.Timothy M. Persons, PhDManaging DirectorScience, Technology Assessment, and AnalyticsChief Scientist, GAOKaren L. Howard, PhDDirectorScience, Technology Assessment, and AnalyticsFebruary 18, 2021Page 7GAO-21-347G Technology Assessment Design Handbook

Chapter 1: The Importance of TechnologyAssessment DesignChapter 1: The Importance of TechnologyAssessment DesignThis chapter underscores the importance of TA design, outlining reasonsfor performing TAs and for spending time on their design. This chapter isbased on reviewing the results of a literature search, an expert meeting,select GAO reports, the experiences of GAO teams, and public andexternal expert comments. For details, see Appendix I: Objectives,Scope, and Methodology.1.1 The Purpose ofTechnologyAssessmentTAs are significant given the growing effects of S&T on society, economy,and other areas. Technological change can be positive, but alsodisruptive, making it critical for Congress and other policymakers tounderstand and evaluate the effects of technology—for example, toensure national security and global competitiveness are maintained.GAO TAs are often requested by Members of Congress andcongressional committees, who may use them to, among other things,make resource allocation decisions to address research gaps or to informlegislation or the development of a national strategy. 17TAs help Congress and other policymakers understand and evaluate theeffects of technology by: Highlighting potential short, medium, and long-term effects of atechnology Elaborating on and communicating the challenges and benefitsassociated with a technology, including early insights into the potentialeffects of a technology Highlighting the status, viability, relative maturity, and public andprivate uses of a technology Supporting planning and evaluation of federal investments in S&T Describing the regulatory environment of a technology Exploring ethical, legal, and social questions that may arise from theapplication of a technologyTechnologies present opportunities and challenges that may vary,depending in part on the policy context in which they are assessed.Therefore, part of a TA is considering the policy context surrounding a17GAO’scongressional protocols expand on how GAO performs work pursuant to law oras directed by congressional committees. For more information, see GAO’s CongressionalProtocols. GAO-17-767G. Washington, D.C.: July 17, 2017. For examples of researchquestions and objectives from published GAO TAs, see tables 2-4.Page 8GAO-21-347G Technology Assessment Design Handbook

Chapter 1: The Importance of TechnologyAssessment Designgiven technology. Recognizing this, GAO may, where appropriate, identifyand analyze policy options as part of its TAs, along with other informationand analyses, to clarify and summarize policy-related issues andchallenges to support policymakers’ decision-making. In this situation,policy options can be defined as a set of alternatives or menu of optionsthat policymakers, such as legislative bodies, government agencies,standards-setting organizations, industry, and other groups, couldconsider taking. 18 Policy options can be used to articulate a range ofpossible actions a policymaker could consider that may enhance benefitsor mitigate the challenges of a technology, in the context of a given policyobjective. Policy options in TAs are addressed to policymakers broadly,not to a specific agency or entity, and do not endorse a particular courseof action. They are not recommendations or matters for congressionalconsideration, which GAO traditionally makes in other product lines. 191.2 Importance ofSpending Time onDesignAs part of the TA process, developing a written TA design, including awritten work plan, helps teams agree on and communicate a clear plan ofaction to the team’s advisers, requesters, and other stakeholders. 20Written TA designs also help guide and coordinate the project team’sactivities and facilitate documentation of decisions and procedures in thefinal product. In addition, focusing the TA on answering specificresearchable questions can assist teams to define and select theappropriate scope and approach, ensuring usefulness of the product tothe intended users. More specific reasons for spending time onsystematically designing a TA include: Enhance its quality, credibility, and usefulness Ensure independence of the analysis Ensure effective use of resources, including time18As stated previously, policy options are for policymakers to consider and take action onat their discretion. In addition, GAO TAs strive to list likely policy options supported byanalysis, but the list may not be exhaustive, and policymakers may choose to considerother policy options not listed by GAO.19Althoughpolicy options in GAO TAs do not endorse a particular course of action, GAO’sanalysis of multiple feasible alternatives is intended to demonstrate that various policyoptions have trade-offs, with each potentially fulfilling certain goals more than others. Thisinformation could help policymakers choose options based on these trade-offs and whichgoals they hope to achieve.20For example, see Appendix III for additional information on the design and project plantemplates that GAO uses as part of its quality standards. GAO TAs use these templates.Page 9GAO-21-347G Technology Assessment Design Handbook

Chapter 1: The Importance of TechnologyAssessment DesignData collection and quality assurance of data can be costly and timeconsuming. A thorough consideration of design options can ensure thatcollection and analysis of the data are relevant, sufficient, and appropriateto answer the researchable question(s), and helps to mitigate the risk ofcollecting unnecessary evidence and incurring additional costs.Page 10GAO-21-347G Technology Assessment Design Handbook

Chapter 2: Technology Assessment Scopeand DesignChapter 2: Technology Assessment Scope andDesignThis chapter highlights design stages, cross-cutting considerations, andGAO examples for sound TA design. To ensure that the information andanalyses in TAs meet policymakers’ needs, it is particularly useful tooutline the stages and considerations involved in sound TA design, whileremaining aware of the iterative and nonlinear process of designing a TA.The information presented in this chapter is based on reviewing theresults of a literature search, an expert meeting, select GAO reports,experiences of GAO teams, and public and external expert comments.For details, see Appendix I: Objectives, Scope, and Methodology.2.1 SoundTechnologyAssessment DesignBelow are questions to consider for a sound TA design. Reflecting onthese questions may help teams make important decisions (like selectingan appropriate design), weigh various factors, and ensure quality TAs. Does the design address congressional and other policymakers’needs? Will the design yield a quality, independent, balanced, thorough, andobjective product? 21 Will the design likely yield information that will be useful tostakeholders? Will the design likely yield valid conclusions on the basis of sufficientand credible evidence? Will the design yield results in the desired time frame? Will the design likely yield results within the constraints of theresources available? How will policy options be identified and assessed, if applicable?21TAteams can use their organization’s existing protocols and policies to ensureindependence of analysis.Page 11GAO-21-347G Technology Assessment Design Handbook

Chapter 2: Technology Assessment Scope andDesign2.2 Stages andConsiderations forTechnologyAssessment DesignFigure 1 outlines three stages of TA design, and the subsequent sidebarelaborates on seven considerations. While Figure 1 presents TA designas a series of stages, actual execution is highly iterative and nonlinear. Aswith all GAO engagements, teams need to be prepared to re-visit designdecisions as circumstances change. 22 In addition, TAs must meet GAO’srigorous quality standards, which are designed to ensure that all GAOproducts provide accurate, credible, and balanced information. GAO’squality standards and standards of evidence apply to all GAO products,including TAs.22Refer to Appendix III for a summary of the typical GAO engagement process, of whichdesign is a part.Page 12GAO-21-347G Technology Assessment Design Handbook

Chapter 2: Technology Assessment Scope andDesignFigure 1: Summary of Key Stages and Considerations of Technology Assessment DesignPage 13GAO-21-347G Technology Assessment Design Handbook

Chapter 2: Technology Assessment Scope andDesignStage 1: Determine the ScopeCross-Cutting ConsiderationsBelow are some considerations for the teamto think about while designing a technologyassessment (TA) and throughout the processof performing the TA. This list is notexhaustive, and some of the considerationsmay not be unique to TAs.The iterative nature of TA design: Ascircumstances change and new informationcomes to light, it may be necessary to revisitscope and design.Congressional and policymakers’ needs:Assess needs and interests of congressionalrequester(s) and other potential policymakers,as applicable.Resources: These include staff availability,staff expertise, and time available. Trade-offsmay need to be considered, such as betweenresources and potential scope.During this stage, TA teams will make scoping decisions. Scopingdecisions are informed by an initial situation analysis that may be usedto: 23 Develop an initial understanding of the technology (such as the stateof the technology) and context of the technology (such as social,political, legal, and economic factors) Identify internal and external stakeholders 24 Identify other preliminary activities (such as initial interviews andidentifying potential data sources) Inform the purpose of the work and possible objectives (also knownas research questions), including the policy objective, if applicable Identify potential issues to be researched and assessedIndependence: This includes potential orperceived threats to independence, includingconflicts of interest, bias, and implicit bias.Engaging internal and external stakeholders:Consider and consult with relevant internaland external stakeholders as early as possibleand during all design stages.Potential challenges: Consider potentialchallenges to design and implementation ofthe TA, such as: (1) possible changes inoperating environment; (2) characterizing orquantifying anticipatory factors, uncertainty,and future condition(s); and (3) lack of orlimitations with data. See Chapter 3 for morespecific examples.Communication strategy: Consider potentialusers of the product(s) and how informationregarding the TA will be communicated. Howresults are communicated can affect how theyare used, so it is important for TA teams todiscuss communication options.Source: GAO-20-246G. GAO-21-347G23An initial situation analysis may entail a preliminary literature search, early interviewswith experts, and review of relevant GAO bodies of work, among other methods. SeeAppendix IV for additional information regarding methods.24Stakeholders are described later in this section, and include a wide range of internal andexternal stakeholders who advise, review, contribute to, and may be affected by the work,including the possible policy objective and subsequent policy options.Page 14GAO-21-347G Technology Assessment Design Handbook

Chapter 2: Technology Assessment Scope andDesignDefining ScopeTo make scoping decisions teams may find ituseful to define and delineate scopeaccording to, for example:TA teams may identify a possible policy objective, as applicable, basedon the congressional request, evidence, and other factors. Teams willneed to ensure that the policy objective is balanced by not generating abias for a single potential course of action. 25 The policy objective servesto guide the development of policy options by stating their overall aims,and helping to identify the landscape and scope of policy options. Type, part, or level of maturity of thetechnology Timeframe, economic sector(s), orgeography Types of effects and outcomes Institutional considerations, such asprevious work by GAO and otherorganizationsDuring this stage, and as early as possible, teams identify and may startengaging with relevant internal and external stakeholders, including thoserelated to the potential policy objective. Such stakeholders could include: Availability of information, includingpossible proprietary nature of information Degree of knowledge, lack of info

1.2 Importance of Spending Time on Design 9 Chapter 2 Technology Assessment Scope and Design 11 2.1 Sound Technology Assessment Design 11 2.2 Stages and Considerations for Technology Assessment Design 12 2.2.1 GAO Technology Assessment Design Examples 18 Chapter 3 Approaches to Select Technology Assessment Design and Implementation Challenges .

Related Documents:

This report is one of two providing Congress with background on the GAO bid-protest process. It analyzes (1) trends in bid protests filed with GAO, (2) why companies protest, (3) the impact bid protests have on acquisitions, (4) the most common grounds for GAO to sustain a protest, and (5) trends in bid protests filed against DOD.

GAO -17 665. For more information, View . contact Marcia Crosse at (202) 512-7114 or [email protected] Why GAO Did This Study . According to data from CDC, an agency within the Department of Health and Human Services (HHS), among chil

GAO studies and policy documents and program evaluation literature. To ensure the guide's competence and usefulness, drafts were reviewed by selected GAO, federal and state agency evaluators, and evaluation authors and practitioners from professional consulting firms. This paper updates a 1991 version issued by GAO's prior Program .

Anna Irvine, CCEA . Slide 2 About GAO GAO Best Practice Guides (Overview) Agile Guide –Agile Adoption . (PARM), update or develop new guidance on Agile methodologies to describe how Agile teams can estimate the relative complexity of user stories Recommendation 10 Ensure that

Congress consider revisiting the property screening requirements under the Act. View GAO-15-688T. For more information, . adopt space design standards to optimize federal domestic office space usage. The National Strategy . government’s office and warehouse space between fiscal years 2012 and

Security and Governmental Affairs and the House of Representatives Committee on Oversight and Government Reform, has brought much- . changes bring GAO’s 2013 High Risk List to a total of 30 areas. Throughout the past two decades, attention to highrisk areas has - . for GAO to narrow the scope of the high-risk issue.

Defense Authorization Act for Fiscal Year 2019 includes a provision for GAO to review the cost savings achieved from the competition and award of the CNS contract. GAO’s report examines the extent to which (1) CNS achieved proposed cost savings from fiscal year 2014 through fiscal year

'Operation Desert Slorm: Full Army Medical Capability Not Achieved (GAO/NSIAD-92-175, Aug. 18, 1992). Page 1 GAO/NSIAI) '3 189 Operation Desert Storm. B-253207 the predicted number of casualties, or had the ground war started earlier . Page 3 GAO/

Highlights of GAO-18-296, a report to congressional requesters April. 2018. U.S. DEPARTMENT OF AGRICULTURE . Additional Data Analysis Could Enhance Monitoring of U.S. Cattle Market . What GAO Found . Supply and demand factors, such as a drought that affected the price of cattle feed, aff

Curriculum Vitae Yongjun Gao 5 _ 28. 2000, Gao Y J, Mu Z G, Liu Y L, Huang B L,2000, K-Ar Dating of Mariana Trough Basalts and Its Significa

agency’s entire portfolio of facilities. View GAO-13-222. To view the e-supplement online, click on GAO-13-223SP. For more information, contact Mark Goldstein at (202) . broad overview of the makeup of the gover

Phased Retirement Programs, Although Uncommon, Provide Flexibility for Workers and Employers . What GAO FoundWhy. Participation of older workers in the labor market has increased in the last decade, according to GAO analysis . Further, most individuals ages 61 to 66 who were still working maintained a full -time work schedule.

selected security controls for three of the six systems GAO assessed. DOD’s risk . GAO as Critical to Inventory Management Operations 6 Table 2: Extent to Which the Defense Logistics Agency Addressed . We did not use the updated version of this guidance in our review, because we focu

Director, Federal Mediation and Conciliation Service. Comptroller General of the United States Page 2 GAO/GGDS&27 Report on OPM . Page 3 GAO/GGD-3&27 Report. on OPM . Appendix I . List of Fiscal Year 1985 GAO Reports On . federal jobs

Guidance Would Help DOD Programs Better Communicate Requirements to Contractors . What GAO Found . Since GAO's 2018 report, the Department of Defense (DOD) has taken action to . and more effective than trying to add, or bolt on, cybersecurity protections late in the development cycle or after a system is fielded. Moreover, because

Improving Enterprise Architecture Management (Version 2.0) August 2010 . . and (3) 59 elements, or building blocks, of EA management that are at the core of an EA program. (See the figure below for a . Page ii GAO-10-846G Executive Guide. Page 1 GAO-10-846G . United States Government Accountability Office. Washington, DC 20548 .

TRANSIT ASSET MANAGEMENT FTA Should Clarify Performance Data and Develop a Plan to Guide Future Program Improvements . Committee on Banking, Housing, and Urban Affairs, U.S. Senate September 2020 GAO-20-686 . Why GAO Did This Study . In 2019, the Department of Transportation (DOT) reported a 98

and Medicaid services and financing and improving care coordination. Thirteen states are participating. GAO was asked to examine care coordination under the demonstration. GAO examined (1) how integrated care organizations—which are health plans or other entities—are implementing care coordination and (2) what, if any,

and Medicaid services and financing and improving care coordination. Thirteen states are participating. GAO was asked to examine care coordination under the demonstration. GAO examined (1) how integrated care organizations—which are health plans or other entities—are implementing care coordination and (2) what, if any,

Zecharia Sitchin these aliens had been coming here for a long time and even brought civilization to Planet Earth. Civilization? No, barbarism, cursed Roland. Today, with millions of claimed UFO sightings encounters with aliens alleged kidnappings investigators everywhere were coming right out and calling it an epidemic.