Demonstration Of Supervisory Control And Data Acquisition .
ARL‐CR‐0773 MAY 2015US Army Research LaboratoryDemonstration of Supervisory Control and DataAcquisition (SCADA) Virtualization Capability inthe US Army Research Laboratory (ARL)/SustainingBase Network Assurance Branch (SBNAB)US Army Cyber Analytics Laboratory (ACAL)SCADA Hardware Testbedprepared by Daniel T SullivanRaytheon Company22260 Pacific BlvdDulles, VAandEdward J Colbert, PhDICF International7125 Thomas Edison Drive #100Columbia, MDunder contract W911QX‐14‐F‐0020Approved for public release; distribution unlimited.
NOTICESDisclaimersThe findings in this report are not to be construed as an official Department of theArmy position unless so designated by other authorized documents.Citation of manufacturer’s or trade names does not constitute an officialendorsement or approval of the use thereof.Destroy this report when it is no longer needed. Do not return it to the originator.
ARL‐CR‐0773 MAY 2015US Army Research LaboratoryDemonstration of Supervisory Control and DataAcquisition (SCADA) Virtualization Capability inthe US Army Research Laboratory (ARL)/SustainingBase Network Assurance Branch (SBNAB)US Army Cyber Analytics Laboratory (ACAL) SCADAHardware Testbedprepared by Daniel T SullivanRaytheon Company22260 Pacific BlvdDulles, VAandEdward J Colbert, PhDICF International7125 Thomas Edison Drive #100Columbia, MDunder contract W911QX‐14‐F‐0020Approved for public release; distribution unlimited.
Form ApprovedOMB No. 0704‐0188REPORT DOCUMENTATION PAGEPublic reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining thedata needed, and completing and reviewing the collection information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing theburden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302.Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currentlyvalid OMB control number.PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.1. REPORT DATE (DD‐MM‐YYYY)2. REPORT TYPE3. DATES COVERED (From ‐ To)May 2015Final07/2014–12/20144. TITLE AND SUBTITLE5a. CONTRACT NUMBERDemonstration of Supervisory Control and Data Acquisition (SCADA)Virtualization Capability in the US Army Research Laboratory(ARL)/Sustaining Base Network Assurance Branch (SBNAB)US Army Cyber Analytics Laboratory (ACAL) SCADA Hardware TestbedW911QX-14-F-00206. AUTHOR(S)5d. PROJECT NUMBER5b. GRANT NUMBER5c. PROGRAM ELEMENT NUMBERDaniel T Sullivan and Edward J Colbert, Ph.D5e. TASK NUMBER5f. WORK UNIT NUMBER7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)8. PERFORMING ORGANIZATION REPORT NUMBERRaytheon Company22260 Pacific BlvdDulles, VA 20166ARL-CR-0773ICF International7125 Thomas Edison Drive #100Columbia, MD 210469. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)10. SPONSOR/MONITOR'S ACRONYM(S)US Army Research LaboratoryATTN: RDRL-CIN-S2800 Powder Mill RoadAdelphi, MD 20783-113811. SPONSOR/MONITOR'S REPORT NUMBER(S)12. DISTRIBUTION/AVAILABILITY STATEMENTApproved for public release; distribution unlimited.13. SUPPLEMENTARY NOTESARL POC: Robert J Reschly14. ABSTRACTIn support of the US Army Research Laboratory (ARL) mission to conduct cybersecurity research to protect Industrial ControlSystems (ICS), the ARL Sustaining Base Network Assurance Branch (SBNAB) constructed a Supervisory Control and DataAcquisition (SCADA) hardware testbed to simulate the network traffic between human machine interface (HMI) andprogrammable logic controller (PLC) components. The HMI and PLC components were instantiated with software andinstalled in multiple virtual machines (VMs) to emulate 6 conceptual manufacturing plant processes. Two experiments wereconducted: Validate the virtualized network performance by creating and capturing HMI–PLC network traffic over a 24-h periodin the virtualized network and inspect the packets for errors. Test the interoperability of physical network elements with the virtualized network. In this test, a simulated threatactor used a laptop computer to connect to the virtualized production network and send malicious Modbus networkcommands to create a manipulation of view attack.The results of both experiments are PASS. The experiments validated the capability to establish a SCADA hardware testbedusing virtualization and this infrastructure is now part of the ARL SBNAB US Army Cyber Analytics Laboratory (ACAL).15. SUBJECT TERMSSCADA, Modbus, virtualization16. SECURITY CLASSIFICATION OF:a. REPORTUnclassifiedb. ABSTRACTUnclassifiedc. THIS PAGEUnclassified17. LIMITATIONOFABSTRACT18. NUMBEROFPAGESUU3219a. NAME OF RESPONSIBLE PERSONDaniel T Sullivan19b. TELEPHONE NUMBER (Include area code)301-394-0248Standard Form 298 (Rev. 8/98)Prescribed by ANSI Std. Z39.18ii
ContentsList of FiguresivList of TablesivAcknowledgmentsv1. Background and Motivation12. Description of Test12.1 Test Processes12.2 Virtual Representation of the MRE SCADA system22.3 PLC Configuration42.4 HMI Configuration53. Execution of MRE Test93.1 Network Virtualization Subtest93.2 Simulated Cyber Attack104. MRE Test Results114.1 Network Virtualization Subtest114.2 Simulated Cyber Attack115. Conclusions136. References14Appendix A. Experiment Hardware and Software15Appendix B. ModbusPal Tables17List of Symbols, Abbreviations, and Acronyms23Distribution List24iii
List of FiguresFig. 1Process map for MRE SCADA system .2Fig. 2Testbed architecture .3Fig. 3Field network VMs .4Fig. 4Overall plant HMI dashboard .6Fig. 5Chicken cooker dashboard .7Fig. 6Vegetable cooker dashboard .7Fig. 7Meal preparation dashboard .8Fig. 8High-pressure processing dashboard .8Fig. 9Main conveyor belt dashboard .9Fig. 10 Product packaging dashboard .9Fig. 11 Meal preparation dashboard before cyber attack .12Fig. 12 Meal preparation dashboard after cyber attack .12Fig. 13 Meal preparation alarm panel after cyber attack.13List of TablesTableNetwork virtualization test results .11Table A-1Hardware list .16Table A-2Software list .16Table B-1Configuration and measurements for chicken cooker PLC .18Table B-2Configuration and measurements for vegetable cooker PLC .19Table B-3Configuration and measurements for meal preparation andpackaging PLC .20Table B-4Configuration and measurements for high-pressure processingPLC 21Table B-5Configuration and measurements for main conveyor belt PLC.22Table B-6Configuration and measurements for packaging PLC .22iv
AcknowledgmentsWe greatly appreciate Dr Alexander Kott, Curtis Arnold, and Chuck Smith forsupporting Industrial Control Systems/Supervisory Control and Data Acquisition(ICS/SCADA) research at the US Army Research Laboratory (ARL). We aregrateful to Kin Wong and Carlos Mateo for help with the SCADA lab design andordering lab equipment. Max Turk, Akhil Oniha, and James Herron were veryhelpful in setting up hardware and software in the testbed.v
INTENTIONALLY LEFT BLANK.vi
1.Background and MotivationThis report describes a test experiment executed on the US Army ResearchLaboratory (ARL) Sustaining Base Network Assurance Branch (SBNAB)Supervisory Control and Data Acquisition (SCADA) hardware testbed. Thisinitial test experiment has been executed to demonstrate SCADA virtualizationcapability on the testbed. The SCADA hardware testbed is part of the US ArmyCyber Analytics Laboratory (ACAL), which provides hardware and networkinfrastructure and other support needed for collaboration between ARL and othergovernment and commercial institutions.In this test, we use a software-emulated programmable logic controller (PLC) andpublic domain human machine interface (HMI) controller software instead ofactual PLC hardware and vendor-based HMI software. Both PLC and HMIcontroller software run inside virtual machines (VMs), allowing the entireSCADA system to be virtualized. In the future, real PLC hardware andcommercial HMI software will also be used in ACAL SCADA testbed researchexperiments.This initial test of the ACAL SCADA testbed emulates network traffic found inSCADA systems (or Industrial Control Systems [ICS]), as we demonstrate below.2.Description of Test2.1 Test ProcessesThe SCADA system emulated in this test is that of a conceptual Meals-Ready-toEat (MRE) manufacturing process. The process map for the system is illustratedin Fig. 1 and shows 6 PLCs controlling various pieces of machinery used toproduce the MREs.1
Fig. 1Process map for MRE SCADA system2.2 Virtual Representation of the MRE SCADA systemThe software used in the test emulates the traffic sent and received by PLCs andHMIs found in MRE SCADA processes. The PLCs control machinery and receivesensor inputs from physical plant components. HMIs are computers runningcontrol software that frequently polls a PLC for status information about thecontrolled process. A human plant operator monitors the HMI computer andsoftware. HMIs may also provide a capability for the human operator to manuallycontrol a process, if needed.In this experiment, the HMI and PLC components function within VMs. Thetestbed topology of VMs used for the MRE SCADA test is depicted in Fig. 2. Sixpairs of PLCs and HMIs have been constructed inside a virtual network, and all12 VMs are connected to virtual switches. An attacker, who also has access to thevirtual network via a virtual switch, can initiate attacks on the MRE SCADAsystem.2
Fig. 2Testbed architectureA more detailed diagram of the simulated SCADA network is shown in Fig. 3 andadditional information is presented in Appendix A. Experiment Hardware andSoftware. In this experiment, each HMI polls a simulated PLC using the industrialModbus transmission control protocol (TCP). The HMI software used in this testis the open source Mango Automation application,1 while the simulated PLCsoftware is the open source ModbusPal Java application. When queried using theModbus TCP protocol, ModbusPal reports coil and holding register values in amanner similar to a real PLC. For each HMI–PLC pair, Modbus network trafficwill be captured by the tcpdump utility. This captured traffic is used to check ifpacket loss or network errors occur in the virtualized hosts or network during theexperiment.3
Fig. 3Field network VMsThe HMI and PLC VMs are hosted by a VMware ESXi hypervisor on a DellR610 server. For each HMI–PLC pair, a virtual switch connects the 2 VMs. Eachvirtual switch is part of the same virtual network. The virtual network is alsomapped to one of the host machine’s network interface cards (NICs) and this NICallows external access to the virtual network, for example, to the attacker.2.3 PLC ConfigurationEach ModbusPal virtual PLC instance must be configured with a set number ofholding registers and coils to simulate the corresponding process presented inFigs. 1 and 2. ModbusPal was configured with an Extensible Markup Language(XML)-based text file where holding registers and coils are defined and valuesspecified. The values of holding registers and coils can be controlledprogrammatically within ModbusPal.In Appendix B. ModbusPal Tables, we list the detailed configuration informationfor each of the 6 PLCs controlling the 6 processes (see Fig. 2):1. Chicken cooker4
2. Vegetable cooker3. Meal preparation and packaging4. High-pressure processing5. Main conveyor belt6. Packaging2.4 HMI ConfigurationEach virtualized Mango HMI polls its respective ModbusPal PLC for its values ofcoils and holding registers every 10 seconds (sec). The Mango software will sendModbus TCP requests to ModbusPal to request values of all holding registers andcoils configured for this experiment. A graphical dashboard will also beconfigured to provide situational awareness, see Fig. 4 for the overall dashboard,which represents the view typically seen in industrial plants.5
6Fig. 4Overall plant HMI dashboard
Snapshots are shown of the 6 Mango HMIs in Figs. 5–10, illustrating the HMIdashboards of each of the 6 processes.Fig. 5Fig. 6Chicken cooker dashboardVegetable cooker dashboard7
Fig. 7Fig. 8Meal preparation dashboardHigh-pressure processing dashboard8
Fig. 9Main conveyor belt dashboardFig. 10 Product packaging dashboard3.Execution of MRE TestThe MRE test consists of 2 parts—a network virtualization test and a simulatedcyber-attack. A description of these 2 subtests follows.3.1 Network Virtualization SubtestIn this subtest, we will validate that each HMI–PLC pair of VMs has networkconnectivity and that the network paths are configured correctly.9
Step 1: For each automation process, once the Mango HMI begins polling itsrespective ModbusPal instance, capture the traffic over a 24-hour (h) period usingtcpdump.Step 2: During the 24-h polling process, perform spot checks to verify the MangoHMI is receiving measurements in compliance with the values listed inTables 2–7.Step 3: Use Wireshark to inspect the 24-h tcpdump captures and check for anyInternet Control Message Protocol (ICMP) error messages in the tcpdump files.The condition for PASS requires that no ICMP error messages exist in thetcpdump files. The condition for FAIL requires that one or more ICMP errormessages are found. If ICMP error messages in the tcpdump files are discovered,investigate the reasons and correct the configuration.3.2 Simulated Cyber AttackThis subtest simulates a cyber-attacker sending malicious Modbus messages to aPLC to change the values of coils. The Modbus protocol does not have securitycapabilities to authenticate messages or prevent replay attacks.2 As a result,anyone (insider or external threat actor) who has knowledge of the process mapcan send malicious Modbus messages to a PLC and impact an automationprocess. External threat actors can gain knowledge of the process map and PLCladder logic by conducting reconnaissance of the plant network prior to an attack.In this subtest, we will conduct a manipulation of view attack on the MealPreparation ModbusPal instance. In a real plant environment, this attack wouldcause the production process to stop while plant operators investigate the cause.Step 1: On an external laptop connected to the experiment network, use the Perl“mbtget” script to change the Meal Preparation PLC Robot Arm and SealingSystem coil values to “0” (“Off” state).Step 2: Monitor the Meal Preparation process Mango HMI dashboard. Thedashboard should show the Robot Arm and Sealing System processes are in an“Off” state and an alarm should be visible. The test is a PASS if the MealPreparation dashboard shows both processes are “Off” and alarm symbols aredisplayed.10
4.MRE Test Results4.1 Network Virtualization SubtestThe tcpdump data captured over a 24-h period of each HMI-simulated PLC pairwere examined using Wireshark. The table presents the number of packetscaptured and examined for each HMI and simulated PLC pair. The number ofnetwork errors are also listed. Because no network errors were found, all testswere a PASS.TableAutomation ProcessChicken cookerVegetable cookerMeal preparationHigh-pressureprocessingMain conveyor beltNetwork virtualization test resultsNumber of Mango HMI–ModbusPal Packets Capturedover 24 h172,904172,806172,818172,807Number ofNetworkErrors0000Test Results(PASS/FAIL)172,8030PASS86,4020PASSProduct packagingPASSPASSPASSPASSFor each automation process, the Mango HMI polled its respective ModbusPalapplication every 10 sec. In each polling period, Mango HMI issued a Modbuscoil read request and waited for the response. After receiving the coilmeasurements, the Mango HMI sent a holding register read request to itsrespective ModbusPal application. Therefore in each 10-sec poll interval,2 Modbus read requests are sent and 2 responses are received by the HMI.The number of Modbus packets for the Product Packaging process was much lessthan the other automation processes because Product Packaging only used holdingregisters. Therefore, in each 10-sec polling interval, Mango sent only one Modbusmessage compared to 2 in the other automation processes.4.2 Simulated Cyber AttackWe show the Meal Preparation HMI dashboard during normal operations andafter the attacker has sent malicious traffic, in Figs. 11 and 12, respectively.11
Fig. 11 Meal preparation dashboard before cyber attackFig. 12 Meal preparation dashboard after cyber attackThe attack was simulated using the Perl mbtget script, which sent Modbus coilwrite messages to the Meal Preparation ModbusPal to set the coil values to “0”(turn the process offline). The small yellow triangles with an “!” symbol in the12
upper part of Fig. 12 are alarms that have consequently sounded in the Mangodashboard for this critical process.Fig. 13 is a screen capture of the Meal Preparation HMI alarm panel. The loss ofthe Robot Arm and Sealing System processes are listed as critical alarms.Fig. 13 Meal preparation alarm panel after cyber attackThis attack would have resulted in a shutdown of the Meal Preparation process ifthis were an actual plant. The test result is PASS.5.ConclusionsThis experiment demonstrates that virtualization of SCADA components is aneffective means to simulate a production plant’s network traffic and create cyberattack scenarios. The VMs and guest operating systems with their applicationsemulated the automation components found in a plant and zero packets were lostby the virtual network. The virtual environment enabled us to simulate a cyberattack on a commonly used Modbus industrial protocol. We will leverage theresults of this experiment in future tests to protect critical infrastructure.13
6.References1. Mango Automation. Version 2.4.2, Intelligent Automation Systems, re/. [accessed Oct 8 2014].2. Modbus. Digital Bond Incorporated. s/modbus-2/. [accessed Jan 142015].14
Appendix A. Experiment Hardware and Software15
Table A-1 presents each hardware component with a description of its use andoperating system.Table A-1 Hardware listPlatformMac laptop and desktopDell R610Dell R710Dell PowerConnect 6224FunctionRemote access to VirtualMachines (VMs), configureapplications for experimentHosts ESXiSoftware development and testingof applicationsNetwork switchOperating SystemOS X Mavericks(Version 10.9)ESXi 5.5 hypervisorCentOS 6.5Dell FirmwareThe software for this experiment is presented in Table A-2 for each hardwareplatform. This experiment will use US Army Research Laboratory (ARL)licensed, as well as open source software and operating systems.Table A-2 Softw
Supervisory Control and Data Acquisition (SCADA) hardware testbed. This initial test experiment has been executed to demonstrate SCADA virtualization capability on the testbed. The SCADA hardware testbed is part of the US Army Cyber Analytics Laboratory (ACAL), which provides hardware and network
1F 2 In March 2017, the CFPB published its first special edition of Supervisory Highlights dedicated to consumer reporting issues. 2F 3 This special edition of Supervisory Highlights reports on mor e recent supervisory findings in this area. Recent supervisory reviews of compliance with the FCRA and Regulation V have identified new
Chapter 5 SUPERVISORY COMMITTEE Examination 0 Determine the necessary supervision and examination scope based 0 bjectives on the review of the supervisory committee audit, internal audit reports and risk management reports Determine whether the supervisory committee audit and verification meets the requirements
grading position – flow chart. 14. determine pay plan. general schedule. determine supervisory role. fws. determine supervisory role. wage employee. wage supervisor. wage leader. supervisory. lead. non-supervisory. determine applicable occuaptional series. determine grade and title. apply classification standard
A. Supervisory Control and Data Acquisition Systems Industrial Control Systems (ICS) are often found in in-dustries, such as electric, water, oil, natural gas, chemical, transportation, etc. Supervisory Control and Data Acquisition (SCADA) systems are examples of ICS systems, which are generally used in controlling dispersed assets using central-
The work in general can be divided into two parts, optimization and supervisory control. Optimization of cogeneration systems is discussed in chapter 2 through 4, while supervisory control is discussed in chapters 4 through 7. Optimization of cogeneration systems is a well-known topic.
OSU South Centers is uniquely positioned to provide applied research, onsite demonstration, and comprehensive grower training through its demonstration vineyard, world class training facilities and a state-of-art digital diagnostic lab. Our demonstration vineyard has been the
1 Lab meeting and introduction to qualitative analysis 2 Anion analysis (demonstration) 3 Anion analysis 4 5. group cation anion analysis 5 4. group cation (demonstration) 6 4. group cation anion analysis 7 3. group cation (demonstration) 8 3. group cation anion analysis 9 Mid-term exam 10 2. group cation (demonstration)
Keyboards Together 2 Music Medals Bronze Ensemble Pieces (ABRSM) B (T) In the Meadow Stood a Little Birch Tree Trad. Russian, arr. Mike Cornick: p. 3 B (T) Jazz Carousel Jane Sebba: p. 4 B (T) Heading for Home John Caudwell: p. 5 B (T) Don’t Mess with Me! Peter Gritton: p. 6
