SANS Institute Information Security Reading Room

Many organizations keep these capabilitiesInternal Onlyin-house (see Figure 4). This choice islikely indicative of organizations that haveSecurity administrationconcerns about entrusting data to externalentities or have experienced failures withoutsourcing the capability and broughtSecurity road map and planningof how internal business processes270Security architecture and engineering(of systems in your environment)it back in-house. The most effective pentesting requires a strong understanding282265Incident response252Remediation251SOC architecture and engineering(specific to the systems running your SOC)operate and what the “crown jewels” ofthe business are. Cookie-cutter pen testengagements often miss the mark, becausethey lack this knowledge. These types ofCompliance supportregulatory or industry requirement to penThreat researchtest at least annually.Purple-teamingpeople leverage outsourcing. Severaltelephone interview respondents233Security monitoring and detectionDigital forensicsinterviews to shed further light on how238Data protection and monitoringpen tests are typically done to meet aHere we turn to some of our 1Otherwere MSSPs. Other respondents were36050100organizations that used MSSPs for monitoring and Tier 1 response. This gives a nice point andcounterpoint on the perspective of MSSPs for security monitoring and detection.150200250300Figure 4. Internal SOCCapabilitiesThe common thread from the MSSPs was that a new customer would invariably consume ahigher level of SOC resources for the first six to nine months—until standard use cases weretuned to match the business operations:“The early days of a new SOC customer can be a little bit hairy. The use casedevelopment won’t be great. It’ll be producing alerts that aren’t working real well.It’ll start to taper off as detection development improves and the efficiency of thework improves. Twenty use cases in month 1 will produce maybe twice as muchconsumption as 20 use cases at month 9.”One customer of an MSSP for managed detection cited the need to communicateeffectively with the service provider to achieve value:“Make sure that your metrics for tracking the success of your SOC/securityorganization take into account contributing factors, such as incident communicationand tasks assigned to other teams inside and outside of the organization, and thatthose parts are centrally documented. Having disjointed mixtures of communicationinternally and between you and your MSSP bouncing between email, IM, word ofmouth and your CMS/ticket system can diminish a manager’s visibility into day-today and week-to-week interactions between the SOC and the other technical teamsin the company. This makes it more difficult to understand where to focus effortfor improving the interaction between their people and processes to improve theAction ItemsDefine an outsourcing strategyif you don’t have one, andcompare the capabilities youintend to outsource with whatyour peers are doing. Paycareful attention to articulatingneeds to providers if youintend to outsource, and keepreinforcing those expectationsand assessing performance.If you haven’t figured out thedetails of what you need fromthe service provider, anticipate6–12 months of on-ramp timeto achieve a normal steadystate of operations.organization as a whole.”Common and Best Practices for Security Operations Centers: Results of the 2019 SOC Survey6

Incident HandlingIncident Response Capability (N 282)Once the SOC identifiesa potential issue, initialverification is typically doneby the SOC, which hands theincident off to a responseteam to conduct preliminarycontainment actions andfurther investigation. This iswhen the incident response(IR) process begins. Most ofIncident response is a fully integrated part of our internalSOC capability.We use internal incident responders who perform responseas an ad hoc duty when needed.We use internal incident responders with whom we aretrying to integrate our internal SOC but haven’t yet done so.We use dedicated internal incident responders, but theyare separate from the SOC, with no plans to integrate.We use internal incident responders who work with ourexternal SOC services provider.We pull our incident responders from our external SOCservices provider.6Other611564372525Incident response is pulled from a services provider that isnot part of our SOC services engagement.the respondents keep IR in-4025house (266 of 282 responses,5075100125Figure 5. IR Capabilitiesor 94%). Of the internal responders, most (204 of 266 responses, or 77%)IR teams are part of the SOC. See Figure 5.Action ItemsKnowledge ManagementDo a tabletop walk-though of a common incidentscenario and one that is more unusual. Usethat walk-through to demonstrate that the IRstrategy you have in place is the optimal one foryour organization. If it is not optimal, build animprovement plan to get better.During telephone interviews conducted with a sample of the surveyrespondents, we asked what knowledge management tools they usedto document process-related knowledge across the team and supportboth repeatability of operations and the ability to quickly bring onnew analysts. Smaller SOCs (fewer than five analysts) relied on moreinformal methods such as “one gigantic OneNote document” or theuse of SharePoint. Larger SOCs were commonly using Jira for troubletickets and were using Confluence for collaboration. Larger SOCs thatwere integrated with IT or the NOC tended to use ServiceNow or BMCRemedy for trouble tickets and had no access to Confluence. SharePointdominated these large, integrated SOCs.None of the interviewees was using a formal playbook, although onewas budgeting to move from SharePoint to a formal playbook solution.MSSPOf our 517 responses, 302 (58%) of the SOCs represented in the surveyaren’t service providers. The SOC is primarily an internal phenomenonAction ItemsDevelop your system for capturing tribal loreinto documented internal guidance for newand seasoned staff. Capture the pain pointsfrom onboarding new SOC staff so the nextiteration has a smoother transition into effectiveperformance within the SOC. Document thenecessary and optional training for staff.Document details of high-profile incidents thathave occurred in the past so new SOC membersunderstand the organization’s past negativeexperiences and can try to avoid them.in our survey’s population, with 412 of the 517 respondents (80%) statingtheir “customers” are internal to the organization. Roughly three outof four (74%) of those internal entities do not self-identify as a serviceprovider to the organization. See Table 1.Table 1. MSSP Self-IdentificationYes, customers outside of my organization105Yes, internal service provider110No302Answered517Common and Best Practices for Security Operations Centers: Results of the 2019 SOC Survey7

For those who consider themselves internal service providers, 75 ofAction Items111 (68%) are the mandatory provider, meaning that members of theorganization are required to purchase services from the SOC and may notDetermine whether becoming a service providerfor your organization is the right way to offeryour SOC service. Such a model is tenable onlywhen the SOC is somewhat mature and theorganization has a good security culture. The“internal MSSP” approach will drive maturity,efficiency, performance and customer orientation.If you launch this strategy too soon, you risklosing the funding needed to achieve maturity asconstituents move to external providers.hire an outside service.Technology CoverageWhich assets are monitored by the SOC (and which are not) is typicallybased on resource constraints. Because organizations cannot defendeverything, it is interesting to see when organizations choose to leaveassets exposed or less protected.Budget and staffing constraints often mean that SOCs focus on IT systems only, andnot operational technology (OT) or other specialized systems. Only a small numberof SOCs (10%) say they have all of the smart systems present in their environmentcovered by the SOC. See Figure 6.Leaving smart systems unprotected iscommon practice per the above chart. OnlyDoes your SOC support nontraditional computing devices such as smart sensors,building devices, building monitoring, manufacturing, industrial control systemsand other items considered as part of the Internet of Things? (N 353)62 of the 353 respondents said they knowthey’re monitoring “smart systems.” Abouta third of the respondents (121) said theyknow they don’t monitor these systemsand intend not to monitor them. “Unsure”NowNo. We have no plans to support smartsystems.We haven’t assessed and inventoriedsmart systems yet, but we plan to.Partly. Our SOC supports some of ourconnected, at-risk smart systems.and “we haven’t inventoried them yet ”are implied risk decisions that result fromfailing to integrate security into the ITIn the next 12 months1219291Unsure.79Yes. Our SOC supports all of our at-risksmart systems.procurement and deployment process.62Other702550SOCs struggle to monitor and track current75100125Figure 6. Support forNontraditional Devicesassets. Having an accurate inventory of all endpoints and users in a network can bea challenge. The root of the problem comes from the fact that IT operations has thesame problem—even IT organizations that have maturedenough to establish configuration management databases(CMDBs) rarely find that the CMDB is even 80% accurateat any given time. SOC asset inventory approaches thatrely on host-based agents can at best match this level.SOCs that add network scanning or credentialed accessDo you have a full inventory of endpoints on your networkso that, if you have an issue with a specific IP address, you’reable to correlate that asset to a known system owner and/orresponsible user? (N 314)100%approaches are often in the position of telling IT operations76–99%that the CMDB is incomplete or out of date. The increased51–75%use of infrastructure-as-a-service (IaaS) by IT has created26–50%blind spots for traditional network scanning approaches,however. SOCs need to develop the capability of integratinginformation from inventory and asset management toolsavailable in all IaaS offerings. This seems to be a perennialfailure of SOCs, as seen in Figure 7.16106774225% or lessWe don’t correlate.2816Unknown029255075100125Figure 7. Endpoint InventoryMapped to Asset OwnersCommon and Best Practices for Security Operations Centers: Results of the 2019 SOC Survey8

A significant percentage of endpoints cannot be correlated to a specific user, hamperingresponse and remediation operations. Not much has changed on this front since lastyear; as seen in Table 2, the values are nearly identical.Table 2.Year-Over-Year Endpoint Mapping Capabilities20192018Unknown2928We don’t correlate.162525% or 1619314296The best way to address the monitoring of and response to new technologies is toensure that SOC teams are aligned with the IT operations of the organization. Althoughwe saw some improvement this year, most SOCs still aren’t fully leveraging the potentialof interactions with the NOC.If you aren’t consistentlyleveraging this “sibling” inyour organization, you’reWhat is your SOC’s relationship to your network operations center (NOC)? (N 363)Our SOC and NOC teams work together only when there is an emergency.We don’t have a NOC.missing efficiency and78Our NOC team is an integral part of our detection and response,although our SOC and NOC activities are not technically integrated.knowledge opportunities.An encouraging portion(34%) of SOCs are capableof doing this, with 122 of 36374Our SOC and NOC teams have very little direct communication.49Our NOC team and SOC team are kept well-informed through integrativedashboards with shared information, APIs and workflow, where needed.48respondents saying they areThere is no relationship.very tightly coupled to the governance structure of theorganization. No single pattern emerged from the survey orthe interview responses. A few points did come across fromour interviews.No SOC manager reported having to work with a “zero-basedbudget” and justify SOC staffing and technology budgets from255075100Figure 8. SOC/NOC RelationshipSee Figure 8.How organizations acquire security funding for SOCs is80effectively working together.Funding for SOCs20Othereither fully integrated orscratch each year.86Action ItemsLeverage native capability or add external monitoring softwareto all new cloud, IoT and mobile projects for coverage. Vendorshave solutions ready to help you. Play catch-up, if necessary,to monitor devices that are already deployed. Continueto expand coverage of all standard IT systems, and moreclosely align with IT operations to keep pace with changingorganizational demands. If your organization says it can’tdo this, look to other institutions that have accomplishedcloser integration for examples of how to accomplish thiseffort. There is usually a managed operational capabilityand consensus on inclusion of security in place beforetechnological solutions can be deployed effectively.Common and Best Practices for Security Operations Centers: Results of the 2019 SOC Survey9

Some SOCs are funded as a “tax” on business units,whether or not the business unit decided to use the SOCservices. This provided an incentive to business units to usethe centralized SOC services and provided a stable base offunding. This model was commonly used when centralizednetwork services or IT in general were an automatic cost.SOCs using MSSP services were generally able to simplypass along increases in prices from the MSSP. MSSPs oftenprovide metric and benchmark data across their customersAction ItemsIdentify potential funding vehicles that are currently unutilizedor underutilized. Make use of metrics to demonstrate valueprovided by the SOC. Look for ways to share your newlyacquired assets with NOC and governance, risk managementand compliance (GRC) teams to drive closer coordination andunify efforts.that allow MSSP customers to justify new or increasedfunding in internal security controls and operations.SOC SizeAnalysts (N 355) 1,000Security managers often ask how many staff members101–1,000are required to run a SOC effectively. It is our intention tounderfunded and not performing well, so the number ofemployees based on this consensus might not reflect theNumber of FTEscourse. All SOCs are not equal. The other SOCs may be626–100provide some numbers that will enable you to compareyour SOC with others. There’s a danger in doing so, of33411–25576–10702–51231status or maturity of your organization. More sophisticated20 1 (part-time)and persistent attackers might be targeting yourorganization rather than focusing on this survey’s other15Unknownrespondents—meaning you need more people to thwart2702550this adversary. Caveat lectorem.75100125Figure 9. Full-Time Analysts WhoUse SOC Systems and ServicesOverall ResponsesWe asked respondents to describe the size of their SOCsAnalysts Needed to Maintain (N 355)in two general staff roles: analysts and those involved inmaintaining the SOC systems.The number of analysts employed in SOCs falls primarilyin the two-to-five range (123 responses, or 35%). This isresponses to the survey, as seen in Figure 9.Similarly, the number of those assigned to maintainsystems also falls mostly in the two-to-five range (119, or34%), as seen in Figure 10.5101–1,000526–100Number of FTEsnot calibrated based on organization size, just overall 1,0002211–25386–10612–51191 1 (part-time)3924Unknown035255075100125Figure 10. FTEs Needed to MaintainSOC Systems and ServicesCommon and Best Practices for Security Operations Centers: Results of the 2019 SOC Survey10

Adjusted Responses byOrganization SizeTaking into account the organization size is probablya worthwhile dimension to add to provide a morerelevant comparison. Table 3 provides a glimpse atNumber of Analysts vs. Organization Size 1,000101–1,00026–10011the number of SOC team members.Table 3.SOC Team Analysts by Organization SizeOrganization Size(by Workforce Size)Common Numberof Analysts 10,0002–5 (99 of 204)10,000–15,0006–10 (9 of 20)15,001–100,00011–25 (23 of 67) 100,00026–100 (13 of 37)These numbers are within typical norms for IT and ITMore than 100,00053550,001–100,0002headcount for IT staff and 3% of IT staff headcount40000Overall business governance and how IT services00are governed and delivered are usually the biggestin interesting outliers. The winner this year: the811,001–2,00010077of this paper want to visit your SOC to see how it5functions. See Figure 11.82is greater than 100,000, but there’s only one part101–1,00020732respondent who indicated that the organization sizetime analyst in the SOC. If that’s you, the authors166441factors affecting staffing ratios. This question’scorrelation to organization size always results8522,001–5,000147755,001–10,000lower staff levels with higher budgets for trainingand tools often provide higher levels of service.852falls in that range is not strictly budget-driven—8000Since we’re talking about the number of people inthe SOC, we want to address effective hiring andFewer than 10000010810retention of the right SOC analysts and maintainers.5261053Hiring and Retention InterviewQuestions Insights 1 (part time)2for security. The spread for the majority falls between2–5% for each of the ratios. Where an 00006–107300typically shown that a 10,000 employee organizationstaff.3 This represents an average of 3% of employee6400security staffing. Surveys by Gartner and others havewill have on the order of 300 IT staff and 9 security11–251015Figure 11. Number of Analysts byOrganization Size152025Figure 11. Number of Analysts byOrganization SizeRespondents said that stability of personnel in the3 www.gartner.com/document/code/316640?ref grbody&refval 3832268 [Subscription required

Aug 05, 2019 · used interchangeably when people describe a security operations center. We asked what the SOC does internally, via outsourcing, or both. The ability to identify and respond to issues is the key aspect of the SOC and is frequently an internal capability. Architecture