Operational Testing Of Agile Programs In A Time Of Change At DHS - ITEA

DHS SCIENCE AND TECHNOLOGY Operational Testing of Agile Programs in a Time of Change at DHS Mr. Wayne Dumais Test Area Manager Office of Test and Evaluation Science and Technology Directorate

Background Five Agile development programs identified February 2016: ICE – Student Exchange Visitor Information System FEMA – National Flood Insurance Program FEMA – Grants Management Modernization TSA – Technology Infrastructure Modernization USCIS – Verification Modernization DHS Science and Technology Directorate MOBILIZING INNOVATION FOR A SECURE WORLD 2

Agile Pilots TSA TIM ICE SEVIS Technology Infrastructure Modernization (TIM) Program Improving programmatic outcomes Student Exchange Visitor Information System (SEVIS) Modernization Program Coordinating document reviews, adjudication and HQ collaboration Process Improvement Sub-IPT USCIS E-VERIFY FEMA NFIP National Flood Insurance Program (NFIP), IT Phoenix Program Metrics Sub-IPT Verification Modernization (VER MOD) Program DHS DHSHQ HQAgile AgileIPT IPT Accelerating document approval cycles Establishing best practices for technology and program delivery Testing and Evaluation Sub-IPT FEMA GMM Evaluating opportunities for process improvement Cybersecurity Sub-IPT DHS Science and Technology Directorate MOBILIZING INNOVATION FOR A SECURE WORLD Grants Management Modernization (GMM) Program Monitoring and measuring performance Software Delivery Maturity Model Sub-IPT 3

Where Are They? 1 Need 2 3 Obtain Analyze/Select 2B Produce/Deploy/Support 2C LRIP SEVIS FOC – 4QFY20 NFIP FOC – 4QFY20 GMM FOC – 4QFY21 TIM Ver Mod DHS Science and Technology Directorate MOBILIZING INNOVATION FOR A SECURE WORLD FOC – 4QFY21 FOC – 4QFY19 4

Agile – More Testing, More Often Programs want to avoid the big IOT&E\FOT&E event Programs will have periodic program reviews Operational testers Involved with user story generation (operational view) Operational requirements identified/tagged OTA approval of UAT plan, definition of “done” OTA has a vote at the Release Readiness Review Multiple OA’s through the development to provide status to PM Each pilot structures OT a little differently 1 Need 2 3 Obtain Analyze/Select 2B Produce/Deploy/Support 2C LRIP DHS Science and Technology Directorate MOBILIZING INNOVATION FOR A SECURE WORLD 5

Agile OTAs DHS as a department does not have a centralized OTA Some Components do – TSA, CBP, DNDO Others contract for OTA services Contractors with Operational and Agile Test experience are rare Industry often lacks governance and oversight FFRDCs seem expensive to some PMs DOT&E requires OTDs to have Agile training DOT&E provides Scrum Master Certification training opportunities for PMs\OTAs DHS Science and Technology Directorate MOBILIZING INNOVATION FOR A SECURE WORLD 6

Growing Pains Acquisition Staffing Plans do not include Agile expertise “On Loan” expertise is not available to programs from CIO, OSE Agile requires commitment across program and acquisition support functions (CPO, Finance, COR, etc.) Components disagree on how to implement Agile process Agile view of operational requirements Agile view of major acquisition documents Agile view of Operational T&E, developing a new paradigm for testing with DHS (integrated testing) Lack of common terminology for different Agile processes DHS Science and Technology Directorate MOBILIZING INNOVATION FOR A SECURE WORLD 7

Risk Undeveloped full guidance for the Agile process No common methodology to implement Agile Increased cost Potential breach issues IPTs brought together to develop guidance as programs move forward through the ALF\Agile process Experienced OTAs Workflow and volume of work for small OTA teams covering multiple Agile development teams Increased process time due to OTA inclusion DHS Science and Technology Directorate MOBILIZING INNOVATION FOR A SECURE WORLD 8

Questions DHS Science and Technology Directorate MOBILIZING INNOVATION FOR A SECURE WORLD 9

Agile requires commitment across program and acquisition support functions (CPO, Finance, COR, etc.) Components disagree on how to implement Agile process Agile view of operational requirements Agile view of major acquisition documents Agile view of Operational T&E, developing a new paradigm for testing with DHS (integrated .