The No-Nonsense Guide To Security Operations Metrics

2y ago
3.47 MB
5 Pages
Last View : 29d ago
Last Download : 2y ago
Upload by : Eli Jorgenson

The No-NonsenseGuide to SecurityOperations Metrics

The No-Nonsense Guideto Security Operations MetricsKPIs That Actually Work for Assessing Your SOC Progress and Driving ExecutiveSupport (Plus Actionable Advice for Improving Those Metrics)As the saying goes, there are two types of companies: Thosethat know they have been hacked – and those that just haven’tdiscovered it yet. The adage admittedly isn’t perfect, nor fair,as there are countless companies practicing defense in depthand performing due diligence by continuously assessing thethreat landscape and effectively putting people, process andtechnology to work to address them.But because every organization in the world is a target forcyber threats and data breaches, information security is arare discipline in which what should be the traditional hallmarkof success – “Nothing bad happened!” – isn’t an option.As a result, success becomes more difficult to define andmeasure, and companies must think long and hard aboutthe appropriate metrics for their business and technicalgoals. Still, metrics are an important part of cybersecurityand security operations programs. Being able to measureyour progress shows how well your security programis functioning and helps justify to executive leadershipand other stakeholders the security operations center(SOC) resources you require.Security teams are constantly asking for more budget forresources to improve their day-to-day operations.Yet, the SANS Institute found in its 2018 Security OperationsCenter Survey that just 54% of SOCs collected metrics,and most weren’t “business-relevant effectiveness metrics.”SANS said that without clear measurements, SOCsmay run into funding resistance due to their inabilityto communicate to management the value of the SOC andoverall security program.Knowing that securing more funding for additional hiresand new technologies rests on providing data-based proofraises an obvious question: Why aren’t more SOCs diligentlytracking their performance? For starters, many securityoperations teams say the reporting they provide requires asignificant amount of work to pull together. Between non-stopalerts, burned-out analysts, an overdependence on manualprocesses, skills shortages and security tools that don’tinteroperate, defining success – never mind approachingmetrics from a business perspective – can seemvirtually impossible.But for security operations to grow within your company,attesting to its value both internally and across the greaterorganization will be tantamount to your individual success.BEFORE YOU MEASUREThe first step to building your enterprise cybersecuritymetrics and security operations KPIs is setting clear directionas to what you are collecting and why. You will need a vision,long-term objectives and strategy before you can achieveexecutive stakeholder buy-in for your metrics program.Going the reverse route will result in a barrage of questionsand little in the way of support. Reduce the friction andexpedite approvals by clearly articulating a solid planand the concrete role leadership support plays. (In caseyou are wondering, yes, not only are you ultimately seekingexecutive-level support for your SOC, but also endorsementfor collection of data related to it.)Outside the executive suite, some stakeholders within the ITdepartment may feel a metrics program adds pressureto their teams because of the added visibility into their dayto-day operations. No one enjoys feeling like another groupwithin the organization is keeping tabs on them.Building and presenting your program to alleviate this concernis paramount to minimizing pushback. Framing the processand rationale as a way you are assisting with tighteningprocesses and technology for the organization as a whole isoften a good starting point. Also, go in prepared with a clearoutline of stakeholder roles and responsibilities. You’ll needto answer questions like:1. If an issue is determined via the metrics program,what is each stakeholder’s responsibility with regardto remediation efforts?2. How will information be reported to stakeholders?3. Will there be service-level agreements (SLAs) for solvingand correcting concerns within the metrics?

BUSINESS-ORIENTEDMETRICSEarlier, we established that no organization is immune from asuccessful cyberattack. But that doesn’t mean catastropheis inevitable. Those businesses that can reduce the harm thata successful compromise can impose will be able to stave offcostly consequences and revert to normal operating conditionsin a timely manner. Here are a few metrics that will help supportyour risk mitigation strategies, as well as convey risk to seniorleaders and the board.Time to Detection, Containment and EradicationKPIs in this category include mean time to detect, or MTTD,which reflects the amount of time it takes your team todiscover a potential security incident. Mean time to respond,or MTTR, is the time it takes to control, remediate and/oreradicate a threat once it has been discovered. And dwell timecaptures the entire length of a security incident, reflectingthe duration from when an attacker first enters your networkto the time they are removed and you have returned to aknown-good state.The impact of dwell time cannot be understated: It is duringthis window that adversaries have unrestricted access on yourenvironment, allowing them not just lateral movement butalso the ability to commit any number of damaging actions,including data theft, network and user reconnaissance, andadditional malware infections.Staff RetentionThe security operations center is a pressure-packed place,home to sophisticated threat assaults, disparate detectiontools firing off countless alerts and a widening skills chasm.But for as active a hub it is, your SOC can be home to greatdisaffection and mental exhaustion due to the humdrum tasksinvolved with working individual alerts. No analyst wants to bestuck doing the tedious stuff for long, and you shouldn’t wantthat either. The goal is to get your analysts partaking in morespecialized disciplines, like threat hunting and eradication. Andbecause of the much-maligned skills shortage, you’ll want toensure you offer a thriving environment in which to work.Remember: if the burnout doesn’t get them first, a competingemployer could.Other Business-Oriented Metrics to ConsiderMetricQuestions it can help answerFalse PositiveRatesHow many false positives is yourSOC experiencing in general or,more specifically, per product?This will help determine overalleffectiveness in deploymentand configuration of tools.Distribution ofRoot CausesWhat are the causes ofincidents? Root cause analysiscan help you determine whatneeds to change in termsof people, processesor technology.ThreatAttributionThis is likely the mostsophisticated of assessmentoptions for the SOC, reserved forthose security hubs experiencingelite proficiency. How well youunderstand the details of a threatcan be a strong indicator of theproficiency of your securityoperations, and it’s no surprisethat certain organizations willwant to add this metric to theirmeasurement repertoire. But bearin mind that threat attributionis complex and involvesorganizations using advancedintelligence and forensicevidence to determine the origin,motivations and sponsorsof attacks.Alert ReductionWith the average company receiving tens of thousandsof threats per week, not only are SOCs overwhelmed,they are also experiencing alert fatigue. Two typesof investigations generally occur in the SOC: alert-basedand threat-centric. Alert-based represents a one-to-onerelationship between case and alert. An alert comes in,enrichment and automation happen for the alert, and theanalyst completes the investigation. Analyst efficiency mayimprove, but the biggest problem in the SOC, the volumeof alerts, is left unresolved. To address alert volume aswell as meet the SOCs objectives, you must do more thansimply enrich alerts and automate some tasks. This involvesleveraging a threat-centric approach to investigations thatlooks for contextual relationships in the alerts and,if identified, groups these alerts into a single case.This will help you reduce the huge numbers of alerts youmust regularly confront.Service-Level AgreementsHave you been meeting the SLAs as jointly defined with thebusiness? For the SOC to win executive support, it must showit is aligned with business goals, including risk posture andcompliance requirements. SLAs help provide supervisedaccountability and ensure that the SOC is adequatelycommunicating with key departments, including compliance,legal, IT and human resources.

OPERATIONAL METRICSIMPROVEMENT METRICSNumber of Alerts and Incidents HandledAnalyst ImprovementDocumenting the number of alerts and incidents your teamconfronts is an obvious bellwether metric. Is the numberincreasing or decreasing? What types of tickets and casesare coming through? And what are their severity levels?Whether they are Tier 1, 2 or 3 – or grouped by functionalexpertise, a personnel structure that is becoming morecommon according to a recent Cyentia Institute-Siemplifyreport - analysts are the bedrock of any SOC. Not only arethey the frontline responders and coordinators to alerts andevents, they also must bear other responsibilities for thewider organization. This includes assisting with compliancerequirements, helping to establish companywide securitypolicies and staying up to date on the latest threats. Be sureto create individual metrics that can track your analysts’progress to determine their composite impact.In addition to helping to determine the overall success of yourprogram, tracking alert and incident counts and closureswill help determine false positive rates, decision-makingspeed, and whether you are operating with an appropriateheadcount (i.e. are there too many events being handled perSOC analyst?), as well as other bottlenecks.In addition, it will allow you to pinpoint specific areas of thebusiness from which a lopsided number of issues may beemanating. Isolating threat distribution will enable yourteam to help the company determine which departmentsmay be ripe for remedies, such as additional securityawareness training.You should also document who beyond the SOC was needed torespond to incidents, as well as the escalation level requiredand how the response played out. SIEM and log data will helpdiscern incidents, but security automation, orchestration andresponse (SOAR) can help centralize the work by providing fullvisibility into your detection tools. In addition, SOAR will helpmake your team more effective by lending more efficiency toyour caseload.If you’re feeling inspired, you can stretch this metric to includehow many alerts were closed per shift, offering a gauge ofyour individual team’s adeptness. More later on some othermetrics that specifically can be used to evaluate SOCteam performance.For example, which Tier 1 analysts are most commonlyescalating to Tier 2 analysts? This can help to identify analystperformance, capacity issues and analysts who requireadditional training (on specific topics or in general).Other Improvement Metrics to ConsiderMetricQuestions it can help answerFalse PositiveRates PerProductAre you utilizing your covetedhigher-tiered resources onlywhen necessary because Tier1 analysts (combined withautomation technology like SOAR)is handling anything that they canand should?Handling TimePer AlertWhich type of alert/product istaking the longest to properlyaddress? This will help youdetermine where you shouldspend time optimizing andbuilding new processes/playbooks moving forward.Handling TimePer StageAre your analysts moving alerts ina timely manner across commonstages: queue, false positives,escalation, review, opened case,closed investigation, resolution?Handling TimePer AnalystWhat are the response times andcounts for individual analysts?PlaybookUsage RatesWhich best practice workflowsare you most commonlyleveraging to help handle andinvestigate alerts and respond toincidents?Most CommonEntitiesWhat are the most prevalenthosts/IP addresses/users thatappear in malicious cases? Thiscould indicate they deservespecial attention.Other Operational Measurements to ConsiderMetricQuestions it help answersDistribution ofProductsFrom which products are themost alerts originating?Distribution ofCases by TierHow many cases are handledper tier (1, 2 or 3)?Distribution ofOutcomesHow many false positives arebeing marked as real issues andvice versa?Distribution ofAlert TypesWhat types of alerts occurredover a given time period?

AFTER YOU MEASUREOnce your cybersecurity metrics program is in swing, you’llhave to aggregate the data you collect to output metricsreports. The reports should be sent to stakeholders andinclude a clear representation of what is being measured, itspriority, what its baseline was and how it has changed overtime. Producing these reports requires analysis to get a fullunderstanding of the numbers to have the ability to explainprogress, shortfalls and fluctuations.Be prepared for your reports to take into account exceptions,adjusting variables and areas where combining data may muddythe waters. Often these arise from manual and inconsistentprocesses. The ability to automate response and remediationprocesses can limit skewed metrics, streamline reporting,improve predictability and allow for better data hygiene whenspeaking with stakeholders. SOAR technology can lend a bigassist here.Your deltas between a current metric and the establishedbaseline – either positive or negative – will show changewithin your organization and should be reviewed by your keyconstituencies. Positive improvements should get just as muchattention as negative metrics, with the goal of applaudingthe hard work of those who are improving the security of theorganization. Not only can this go a long way toward buildingconfidence with stakeholders and boost morale amonganalysts, metrics improvements in one area can shed lighton how to make improvements in others.Remember: Metrics are an important part of your cybersecurityand security operations programs and being able to measureyour progress shows how well your team is functioning. Havingkey stakeholders brought to review your vision and strategywill assist with getting other teams to cooperate in your datacollection. The more you can automate metric collection, aswell as in broader security operations processes, the quickeryou can respond and produce reports.FINAL CONSIDERATIONSIf you outsource security operations to an MSSP.If you’re relying on a managed security services provider forsome or all of your security operations, make sure you are inregular communication with the vendor and that your metricsprogram takes into account the fact that an outside partner ishandling many of the things that need to be measured. Work tocentrally document this interaction process, keeping backand-forth limited to a single channel if possible.Security automation, orchestration and response (SOAR)Earlier we mentioned SOAR, but it’s worth referencing again.Most security managers and CISOs likely have been introducedto SOAR solutions by now, but if you haven’t here is the basicrundown of what a SOAR is designed to do.SOARs take alerts from a detection/alerting tool, generally aSIEM, and using APIs gathers data from a variety of sourcesto “enrich” alerts. It then follows predefined playbooks (akarunbooks) to take automated or semi-automated actionsto either fully investigate and respond to an alert or get thealert ready for analyst investigation. SOAR solutions are notintended to replace detection/alerting technologies – or evenSIEMs for that matter. Instead, they act as a virtual analystwith the intent to improve analyst, and thus SOC, efficiency.You should rely on SOAR platforms that deliver robustreporting and business intelligence. Security operationsteams no longer need to rely on lengthy, manual efforts forreliable metrics. KPI dashboards provide a clear view of casesbeing worked, as well as SOC mean time to detect, meantime to respond and dwell time so teams can more easilyidentify ways to improve productivity and effectiveness. Andperhaps most importantly, security orchestration gives SOCmanagement the tools they need to demonstrate the valuesecurity operations brings to the organization overall.For more information, visit

of investigations generally occur in the SOC: alert-based and threat-centric. Alert-based represents a one-to-one relationship between case and alert. An alert comes in, enrichment and automation happen for the alert, and the analyst completes the investigation. Analyst efficiency may improve, but th

Related Documents:

May 02, 2018 · D. Program Evaluation ͟The organization has provided a description of the framework for how each program will be evaluated. The framework should include all the elements below: ͟The evaluation methods are cost-effective for the organization ͟Quantitative and qualitative data is being collected (at Basics tier, data collection must have begun)

Silat is a combative art of self-defense and survival rooted from Matay archipelago. It was traced at thé early of Langkasuka Kingdom (2nd century CE) till thé reign of Melaka (Malaysia) Sultanate era (13th century). Silat has now evolved to become part of social culture and tradition with thé appearance of a fine physical and spiritual .

On an exceptional basis, Member States may request UNESCO to provide thé candidates with access to thé platform so they can complète thé form by themselves. Thèse requests must be addressed to esd rize unesco. or by 15 A ril 2021 UNESCO will provide thé nomineewith accessto thé platform via their émail address.

̶The leading indicator of employee engagement is based on the quality of the relationship between employee and supervisor Empower your managers! ̶Help them understand the impact on the organization ̶Share important changes, plan options, tasks, and deadlines ̶Provide key messages and talking points ̶Prepare them to answer employee questions

Dr. Sunita Bharatwal** Dr. Pawan Garga*** Abstract Customer satisfaction is derived from thè functionalities and values, a product or Service can provide. The current study aims to segregate thè dimensions of ordine Service quality and gather insights on its impact on web shopping. The trends of purchases have

More Nonsense Pictures, Rhymes, Botany, etc. By Edward Lear. 1894 Originally published 1872 Original Cover Click for larger version. Introduction Nonsense Botany One Hundred Nonsense Pictures And Rhymes Twenty-Six Nonsense Rhymes And Pictures INTRODUCTION. In offering this little book--the third of its kind--to the public, I am glad to take the

Chính Văn.- Còn đức Thế tôn thì tuệ giác cực kỳ trong sạch 8: hiện hành bất nhị 9, đạt đến vô tướng 10, đứng vào chỗ đứng của các đức Thế tôn 11, thể hiện tính bình đẳng của các Ngài, đến chỗ không còn chướng ngại 12, giáo pháp không thể khuynh đảo, tâm thức không bị cản trở, cái được

Stuff And Nonsense The Jumblies Songs and Dances Inspired by the Nonsense of Edward Lear Music and Book by John Mills 1) What is ‘Stuff And Nonsense The Jumblies’ All About? Meet Edward Lear – an enthusiastic, extroverted writer, poet, illustrator, musician, traveller . but sometimes ?