NYSTEC Testing Oversight Of ES&S EVS 6.0.4

2y ago
31 Views
2 Downloads
1.05 MB
32 Pages
Last View : 2m ago
Last Download : 2m ago
Upload by : Ophelia Arruda
Transcription

NYSBOE: NYSTEC OVERSIGHT REVIEW OF ES&S EVS 6.0.4.1ReportNYSTEC Testing Oversight ofES&S EVS 6.0.4.1Prepared for:Thomas Connolly, Director of Election OperationsBrendan Lovullo, Deputy Director of Election OperationsNew York State Board of Elections40 North Pearl StAlbany, NY 12207December 10, 2020

NYSBOE: NYSTEC OVERSIGHT REVIEW OF ES&S EVS 6.0.4.1ACRONYMS AND TERMSCOTSCommercial Off the ShelfCVECommon Vulnerability and ExposuresDREDirect Recording ElectronicES&SElection Systems and Software, LLC.EVSES&S Voting SystemFCAFunctional Configuration AuditHAVAHelp America Vote ActNYSBOENew York State Board of ElectionsPCAPhysical Configuration AuditSLISLI Compliance, a Division of Gaming Laboratories International, LLC.TDPTechnical Data PackageVVSGVoluntary Voting System Guidelines i

NYSBOE: NYSTEC OVERSIGHT REVIEW OF ES&S EVS 6.0.4.1Table of Contents1INTRODUCTION. 12EXECUTIVE SUMMARY . 13ES&S EVA 6.0.4.1 RELEASE DESCRIPTION . 23.1 Components in the Current NYS ES&S EVS Configuration . 23.2 Component Enhancements/Additions. 24SLI TESTING . 34.14.24.34.45Documentation Review. 3Source Code Review. 3Security Review Test . 3Functional Testing . 4DISCREPANCIES FOUND BY SLI. 45.1 SLI Findings . 46OPEN DISCREPANCIES . 86.1 SLI Discrepancy ESS6041-21. 96.1.16.1.26.1.36.1.46.1.5Overview . 9Discrepancy . 9Previous Guidance . 10Analysis of Discrepancy, Based on Previous Guidance. 11List of Compensating Controls . 126.2 SLI Discrepancy ESS6041-18. 136.2.16.2.2Overview . 13Discrepancy . 136.3 SLI Discrepancy ESS6041-12. 146.3.1Overview . 14 ii

NYSBOE: NYSTEC OVERSIGHT REVIEW OF ES&S EVS 6.0.4.17NYSTEC ACTIVITIES. 158ISSUES FOUND IN TEST PLANS AND TEST CASES . 178.1 Test Plans . 178.2 Test Cases. 189ISSUES FOUND IN REPORTS . 209.1 Overview of Findings from AtSec . 219.2 Overview of Findings from Cyber Castellum. 2210RISKS SPECIFIC TO EXPRESSVOTE XL . 2310.110.210.310.410.510.611Barcodes. 23Shared Printer and Scanner Path . 24Voters Do Not Review Paper Audit Trails . 25Alternative Languages Do Not Print on Activation Card . 25Integrated Zebra Technologies, QR Code Scanner . 25“AutoCast” (Cast Ballot Without Viewing Card) . 26DOCUMENTS REFERENCED . 27List of TablesTable 1, Count of Discrepancies . 5Table 2, List of Discrepancies . 5Table 3, NYSTEC Response to Code Review Plan . 17Table 4, Issues Found by NYSTEC in Test Cases . 18Table 5, Issues Found by AtSec . 21Table 6, Issues Found by Cyber Castellum . 22Table 7, List of Referenced Files . 27 iii

NYSBOE: NYSTEC OVERSIGHT REVIEW OF ES&S EVS 6.0.4.11 IntroductionThe New York State Board of Elections (NYSBOE) has asked NYSTEC, as a security expert, to perform anindependent review of work conducted by SLI Compliance (SLI) for the 6.0.4.1 version of the ElectionSystems and Software, LLC. (ES&S) EVS Voting System. Specifically, NYSTEC was tasked with reviewingSLI’s functional and security tests, based on the SLI-provided source code and documentation.The ES&S EVS 6.0.4.1 Voting System contains new devices and software, in addition to significantmodifications to hardware and software from the previous NYSBOE-certified ES&S Voting System. Thisvoting system suite consists of software applications, central count devices, and accompanyingfirmware, as well as commercial off the shelf (COTS) hardware and software.This report includes: A summary of the current ESS EVS System in use in New York State, as well as the changesbrought in by the new 6.0.4.1 version that is currently undergoing testing.The list of SLI deliverables reviewed by NYSTEC.The two discrepancies found by SLI during its testing, which remain open.The specific review work performed by NYSTEC.The issues found by NYSTEC (and its subcontractor) in its review work, as well as theresolutions to those issues.Risks Specific to the ExpressVote XL2 Executive SummarySLI tested the functionality and security of the ES&S EVS Voting System, based on VVSG version 1.0(2005) and 2019 NYS voting laws and regulations. NYSTEC reviewed SLI’s requirement mapping, testplans, test cases, discrepancies (findings) and reports. Based on that review, NYSTEC believes that SLIadequately tested the functionality and security of the ES&S EVS Voting System. Nearly all ofdiscrepancies found by SLI during testing were adjudicated appropriately by ES&S, SLI, and the NYSBOEOperations Unit. The only remaining open discrepancies are:ID # ESS6041-18, “Alternative Languages.”ID # ESS6041-12, “Electronic and Paper Record Display.” 1

NYSBOE: NYSTEC OVERSIGHT REVIEW OF ES&S EVS 6.0.4.1One discrepancy—ID # ESS6041-21, “Cryptography: Crypto mapping, FIPS Mapping, CryptographicSoftware”—has been addressed via the compensating controls in place on the system. For details, seesection 6.1, “Discrepancy ESS6041-21,” of this report.3 ES&S EVA 6.0.4.1 ReleaseDescription3.1 Components in the Current NYS ES&S EVS Configuration Electionware – an end-to-end election management software application that provideselection definition creation, ballot formation, equipment configuration, result consolidation,adjudication, and report creation. Composed of five software groups: Define, Design,Deliver, Results, and Manage.DS200 – a paper-based polling place digital scanner and tabulator that simultaneously scansthe front and back of a paper ballot and/or vote summary card in any of four orientationsfor conversion of voter selection marks to electronic Cast Vote Records (CVRs).DS850 – a paper-based polling place central scanner and tabulator that simultaneouslyscans the front and back of a paper ballot and/or vote summary card in any of fourorientations for conversion of voter selection marks to electronic CVRs.3.2 Component Enhancements/Additions ExpressVote XL – a hybrid paper-based polling place voting device that providestouchscreen vote capture, incorporates the printing of the voter’s selections as a CVR, andtabulates scanning into a single unit. Capable of operating in either marker or tabulatormode, depending on the configurable mode selected in Electionware.DS450 – a paper-based polling place central scanner and tabulator that simultaneouslyscans the front and back of a paper ballot and/or vote summary card in any of fourorientations for conversion of voter selection marks to electronic CVRs.Electionware Reporting Module – used for results consolidation, Election Night reporting,and ballot/write-in adjudication. Includes a new Electionware Touch Screen Ballot moduleto lay out ballots for the ExpressVote XL Marker and Tabulator. 2

NYSBOE: NYSTEC OVERSIGHT REVIEW OF ES&S EVS 6.0.4.14 SLI TestingThis section reviews the various testing performed on ES&S EVS 6.0.4.1 by SLI.4.1 Documentation ReviewFrom SLI: “NYSBOE ESS EVS Voting System Documentation Review Test Report v1.2.pdf”:SLI reviewed the documentation supplied in the EVS 6.0.4.1 TDP to verify compliance against VVSG1.0 and NY 2019 Election Law requirements. SLI traced in a set of internally developed test caseswhere each NY 2019 Election Law requirement is met by the vendor documentation. In addition, SLIused a set of internally developed PCA document review forms to trace and demonstrate where eachVVSG 1.0 requirement is met by the vendor documentation based on changes in the TDP.4.2 Source Code ReviewFrom SLI: “NYSBOE ESS EVS 6041 Voting System Source Code Review Test Report v1.1.pdf”:SLI conducted a source code review against the EVS 6.0.4.1 voting system. The review consisted of acomparison of the EVS 6.0.4.0 source code that previously underwent a full source code review by SLICompliance for Federal certification against ES&S delivered EVS 6.0.4.1 source code for this New YorkState Board Of Elections (NYSBOE) project. All changed code was reviewed against the VVSG 1.0requirements. All source code delivered for the EVS 6.0.4.1 project was reviewed against the NYSelection code.4.3 Security Review TestFrom SLI: “NYSBOE ESS EVS Voting System Security Review Test Report v1.1.pdf”: The security test suites are tests for verifying whether a voting system complies withpertinent requirements in the VVSG 1.0 and NY 2019 Election Law requirements. These suitesincorporate system security provisions, unauthorized access, deletion or modification ofdata, audit trail data, and modification or elimination of security mechanisms.The vendor documentation was reviewed to ensure sufficient detail is present to operatethe voting system in a secured manner. Where the vendor statements assert the votingsystem is secured via mechanisms and seals, procedures tested the presence andeffectiveness of such controls.The security test report identifies the specific threats that were assessed and the associatedrisk if a flaw or exception was identified in a voting system. The tests were designed to 3

NYSBOE: NYSTEC OVERSIGHT REVIEW OF ES&S EVS 6.0.4.1 ensure that the voting system meets or exceeds the security requirements in the VVSG 1.0and NY 2019 Election Law requirements.Security testing included testing each individual component of the system and the system asa whole. As such, each type of precinct device, central count device, EMS, tally, reportingapplication, etc., was subjected to review, as was the system as a whole and its interactionsbetween components.4.4 Functional TestingFrom SLI: “NYSBOE ESS EVS 6041 Voting System Functional Test Report v1.2.pdf”:1. Evaluation of Prior VSTL EAC Certification TestingThe ES&S EVS 6.0.4.1 voting system is based on a branch of ES&S voting systems that originatedwith the fully tested and EAC certified EVS 6.0.0.0 voting system. Subsequent EAC certifiedversions of the EVS 6.0.0.0 voting system, EVS 6.0.2.0 and EVS 6.0.4.0, were certification testedby SLI for changes to the original fully tested EVS 6.0.0.0 voting system during each respectiveEAC test campaign.2. VVSG 1.0 “Should to Shall” Functional TestingThe ES&S EVS 6.0.4.1 voting system was functionally tested to a specific subset of VVSG 1.0requirements. As required by NYSBOE, all VVSG 1.0 requirements where the word “should”appears was replaced with “shall”. Custom test cases were created and executed by SLI to testthis functionality.3. NY 2019 Election Law Functional TestingAs the ES&S EVS 6.0.4.1 voting system contains new devices and software in addition tosignificant modifications to hardware and software from the previous NYSBOE certified ES&Svoting system, the full EVS 6.0.4.1 system was tested against all functional NY 2019 Election Lawrequirements.5 Discrepancies Found by SLI5.1 SLI FindingsSLI reports its testing findings as “Discrepancies.” In code review, a discrepancy occurs when the sourcecode does not meet defined requirements or specifications. In all other testing, a discrepancy occurswhen an element of the voting system does not meet defined requirements or specifications.Table 1 shows the count of each type of discrepancy reported by SLI. 4

NYSBOE: NYSTEC OVERSIGHT REVIEW OF ES&S EVS 6.0.4.1TABLE 1, COUNT OF DISCREPANCIESFUNCTIONALCONFIGURATIONAUDIT (FCA)DOCUMENTATIONNumber IT (PCA)DOCUMENTATION(TDP)1110TOTAL26No security or code audit discrepancies were found. Note that this is not unexpected, as the ESS systemhas gone through many rounds of previous testing and updates. Table 2 shows the synopsis of eachfinding. Note that finding ESS6041-23 was entered by SLI in error and removed from the finaldiscrepancy list.TABLE 2, LIST OF DISCREPANCIESISSUE KEYSUMMARYRESOLUTIONESS6041-27EMS COTS in PIP do not matchTDP.This issue was resolved in the final EVS 6.0.4.1 TDPsubmission.ESS6041-26Cerberus FTP server version notsupported with Windows 2008.This issue was resolved in the final EVS 6.0.4.1 TDPsubmission.ESS6041-25WSUS offline update instructionmissing.This issue was resolved in the final EVS 6.0.4.1 TDPsubmission.ESS6041-24Windows 7 support messageduring workstation setup.This issue was resolved in the final EVS 6.0.4.1 TDPsubmission.ESS6041-22EVS system lock/keycombinations are not unique.Per NYSBOE, this should be addressed by fieldprocedures guide or similar; closing, as this is not afunctional issue.Resolution was provided by NYSBOE during a callwith SLI on 8/7/2020.ESS6041-21Cryptography: crypto mapping,FIPS mapping, cryptographicsoftware.Per NYSBOE, this is addressed with compensatingcontrols. See section 6.1 of this report for moreinformation.ESS6041-20Ballot approval and storage.Per NYSBOE, a ballot, activation card, or votesummary card is considered stored as long as it isphysically contained within the device.ExpressVote XL will tabulate the summary cardwhen the cast button is pressed, just prior to thecard entering the attached ballot bin or container.Since storage occurs when the card is inserted, thisis not considered an issue. 5

NYSBOE: NYSTEC OVERSIGHT REVIEW OF ES&S EVS 6.0.4.1ISSUE KEYSUMMARYRESOLUTIONResolution was provided by NYSBOE during a callwith SLI on 8/7/2020.ESS6041-19Unique identifier.Per NYSBOE, this is not applicable to ExpressVoteXL.Resolution was provided by NYSBOE during a callwith SLI on 8/7/2020.ESS6041-18Alternative languages.This discrepancy is unresolved.Per ES&S, ExpressVote XL has been certifiednumerous times by the Election AssistanceCommission (EAC) and several individual states asmeeting the EAC Voluntary Voting SystemGuidelines, the Voting Rights Act of 1965 (VRA),and the Help America Vote Act of 2002 (HAVA), aswell as individual state law requirements.Per NYSBOE, this is a discrepancy only whenworking with languages other than English. Thiswill be brought up for further review by thecommissioners.ESS6041-17Protective counters are notdisplayed at all times.Per NYSBOE, although the requirement indicatesthat the counters must be located such that theyare visible to the inspectors and watchers at alltimes while the polls are open, it is not necessaryto display protective counters during the votingsession, because the on-screen display during thevoting session must be private.Resolution was provided by NYSBOE during a callwith SLI on 8/7/2020.ESS6041-16Write-in stamps and stickers.This has been closed. The intent of thisrequirement is to prevent a person from handingout stickers to voters to place within the write-inspot. It is not the intent of the requirement thatthe voting system itself prohibit the use of stickersor stamps.Resolution was provided by NYSBOE during a callwith SLI and NYSTEC on 7/15/2020.ESS6041-15Write-in nominated candidate.This has been closed. This requirement is forelection officials rather than the voting system.Resolution was provided by NYSBOE during a callwith SLI and NYSTEC on 7/15/2020.ESS6041-14Maximum number of ballotsallowed.This has been closed. This requirement is for thevoter and voting procedures, not the voting 6

NYSBOE: NYSTEC OVERSIGHT REVIEW OF ES&S EVS 6.0.4.1ISSUE KEYSUMMARYRESOLUTIONsystem. Any voter who spoils three (3)ballots/activation cards is not eligible to receiveanother ballot.Resolution was provided by NYSBOE during a callwith SLI and NYSTEC on 7/15/2020.ESS6041-13Rejected paper records.Per NYSBOE in an email to SLI dated 8/31/2020,this issue can be closed. The necessaryinformation is written to the log.ESS6041-12Electronic and paper recorddisplay.This discrepancy is unresolved.ES&S takes exception to

1. Evaluation of Prior VSTL EAC Certification Testing The ES&S EVS 6.0.4.1 voting system is based on a branch of ES&S voting systems that originated with the fully tested and EAC certified EVS 6.0.0.0 voting system. Subsequent EAC certified versions of the EVS 6.0.0.0 voting system, EVS 6.0.2.0 and EVS 6.0.4.0, were certification tested

Related Documents:

Feb 26, 2010 · Support Element fJRISE . Fort Gillem. Georgia. GANG Intel! program had improved dramatically due to tw o events: a 1999 vie pointed up the need for oversight and was turned into a teaching tc appointment of an Intelligence Oversight officer in May 2000. SBAB Intelligence Oversight programs were adequate., 2) ce Oversight

Overview of civilian oversight of armed forces Civilian oversight of the armed forces and whistleblower protection has been a topic of renewed global interest within the last decade, sparked by the Manning and Snowden revelations of US wartime and surveillance practices. The core issue that sets civilian oversight of the security

Domestic Policy and National Security Committee, expert oversight is carried out by the Office of the National Security Council, and civilian oversight that is carried out by the Council for the Civilian Oversight of the Security-Intelligence Activities. In addition to external oversight, SOA has a system of internal

1 2019 Department of Children, Youth, and Families Oversight Board Legislative Report RCW 43.216.015 (20) Prepared by: Crista Johnson, Executive Director, Department of Children, Youth, and Families Oversight Board This report has not been approved by the Governor's Policy Office or the Office of Financial Management, and is being submitted directly from the DCYF Oversight Board

EN 571-1, Non-destructive testing - Penetrant testing - Part 1: General principles. EN 10204, Metallic products - Types of inspection documents. prEN ISO 3059, Non-destructive testing - Penetrant testing and magnetic particle testing - Viewing conditions. EN ISO 3452-3, Non-destructive testing - Penetrant testing - Part 3: Reference test blocks.

Assessment, Penetration Testing, Vulnerability Assessment, and Which Option is Ideal to Practice? Types of Penetration Testing: Types of Pen Testing, Black Box Penetration Testing. White Box Penetration Testing, Grey Box Penetration Testing, Areas of Penetration Testing. Penetration Testing Tools, Limitations of Penetration Testing, Conclusion.

HOW A POERFUL E-COMMERCE TESTING STRATEGY 7 HITEPAPER 4.3 Obtaining Strong Non-Functional Testing Parameters Retailers also need to focus on end-user testing and compatibility testing along with other non-functional testing methods. Performance testing, security testing, and multi-load testing are some vital parameters that need to be checked.

Trustee Joy Harris Jane Gardener Simon Hebditch Trustee Sarah Howell- Davies Jill Batty Cartriona Sutherland treasurer Verity Mosenthal Jenny Thoma Steve Mattingly Trustee Anne Sharpley Lynn Whyte Katy Shaw Trustee Sandra Tait Tina Thorpe Judith Lempriere The position of chair is contested so there will be an election for this post Supporting Statements David Beamish Standing for Chair I .