Outcome-Driven Metrics

Introduction

As the adoption of cloud services continues to rise globally, Gartner predicts that the combined markets for IaaS, PaaS, and SaaS will grow by over 17% annually through 2027. This shift necessitates a corresponding change in security strategies and technologies; traditional methods are becoming less effective as organizations move towards cloud-native and SaaS solutions. The focus now needs to be on platform configuration and identity risk, requiring a new security approach and spending model. These efforts must be supported by business-relevant, not just technology-focused, metrics.

The cloud represents more than just remote computing; it is a complex network of interconnected services. Therefore, investments in cloud security need to be measured differently. Security and risk management leaders must align their cloud security investments with business outcomes. By customizing and applying the outcome-driven metrics (ODMs), leaders can assess and adjust their current security posture to combat risks and achieve business-appropriate results. In this blog we will explore ODMs that will help guide future investments in cloud security control.

The Necessity of Outcome-Driven Metrics

The Necessity of Outcome-Driven Metrics

An outcome-driven metric (ODM) is designed to align security controls and their associated investments with the desired security outcomes of a business, providing clear visibility into the effectiveness of security measures.

Why Use ODMs?

  • Measure Outcomes: ODMs quantify cybersecurity outcomes based on specific investments, aligning measurements with desired protection goals to reflect both protection levels and investment value.
  • State Validation: They help compare the current security state with desired levels, indicating improved protection when ODMs increase and reduced protection when they decrease.
  • Investment Guidance: ODMs serve as tools for directing investments, allowing organizations to either increase spending for better protection or save money with lower protection levels.

Key Areas of Focus

  • Third-Party Risk Assessment: Crucial for managing and securing cloud usage, especially relevant for SaaS platforms, but also applicable to IaaS, PaaS, and individual services.

Business Continuity: Metrics should ensure operational continuity despite potential outages. While long-term outages of major providers are unlikely, planning for short-term disruptions of smaller providers or specific services is essential.

Major Categories of Outcome Driven Metrics:

  1. Governance Metrics: Metrics that oversee and manage cloud usage, typically on a per-tenant basis. Each cloud provider, regardless of delivery model, starts with at least one tenant used by an organization. Most environments are multitenant, with organizations often managing multiple tenants within a single provider.
  2. Operational Metrics: Metrics that assess the actual usage and configuration of each cloud tenant.
  3. Identity Metrics: Metrics that focus on identity management, crucial for cloud security at both the account and individual level. Effective management of access and permissions enhances overall security posture.

Practical Examples of Outcome-Driven Metrics for Cloud Security

A) Cloud Governance ODMs

You should have a reliable estimate of how much of your cloud infrastructure is actively monitored. Without comprehensive tracking of these assets, all other metrics become irrelevant, as substantial risks may exist beyond visibility and control. This challenge is exacerbated by the fact that much of cloud adoption in organizations is driven not by IT but by business units, which may lack direct accountability.

Many of the metrics listed below assume visibility into “known cloud accounts,” assuming these represent the majority of an organization’s cloud presence. To identify additional accounts, utilize compensating controls such as active approval processes, expense audits, and technical measures like security service edges and network firewalls. For optimal metric effectiveness, these controls should aim to provide a comprehensive view of all existing accounts.

Metric  Category  Calculation  Description 
Cloud Account Accountability  Coverage  Number of cloud accounts without documented owner / number of known cloud accounts * 100  Ownership of cloud accounts ensures accountability for managing how accounts are used and configured. 
Cloud Account Usage and Risk  Coverage  Number of cloud accounts without risk assessment / number of known cloud accounts * 100  The usage and capabilities of cloud accounts frequently change. Understanding these changes is crucial for maintaining continuous security. 

 

B) Cloud Operation ODMs

In managing cloud operations, standard security metrics are essential, but they can vary based on how your cloud services are set up. These metrics directly affect how well your security measures work. One common issue is that measuring these metrics properly is hard without the right tools. These metrics cover your entire cloud setup, but it’s often necessary to look at them account by account or group them based on how important they are to get a clear picture.

Metric  Category  Calculation  Description 
Real-time Cloud Workload Protection  Protect  Percentage of workloads protected by real-time runtime visibility controls out of the total number of known workloads, calculated as (Number of protected workloads / Total known workloads) * 100  Critical workloads, specifically managed by the consumer (not the cloud provider like PaaS databases), require visibility into dynamic components such as memory and running processes. Real-time protection typically involves agent-based methods. 
Runtime (non-real-time) Cloud Workload Protection  Protect Percentage of workloads protected by runtime non-real-time visibility controls (agentless scanning) out of the total number of known workloads, calculated as (Number of protected workloads / Total known workloads) * 100  Not all workloads necessitate continuous real-time visibility. Alternative approaches, such as agentless scanning, still offer sufficient protection for many workloads. 
 

C) Cloud Identity ODMs

Cloud Identity ODMs Identity management in the cloud encompasses more than just user identities, particularly in Infrastructure as a Service (IaaS). Workloads possess their own machine identities and privileges, necessitating lifecycle management and governance. In Software as a Service (SaaS), identity often stands as the primary control directly managed by the application consumer. Cloud providers observe that overprivileged identities are a prevalent issue. Like operational metrics, accurately assessing many of them can be challenging without appropriate tools. Therefore, we have identified categories of tools that may be necessary.

Metric  Category  Calculation  Description 
Workload Access to Sensitive Data 
Protect  (Percentage of workload identities accessing sensitive data / Total workload identities)  Cloud workloads have their own machine identities, which often outnumber user identities. Compromised workloads with high privileges pose significant risks.  
Active Multi-Factor Authentication (MFA) Users  Protect  Percentage of user identities accessing cloud tenants using Multi-Factor Authentication (MFA)  MFA is a crucial security measure for user accounts accessing cloud services across the organization. 

Steps to Put Outcome-Driven Metrics into Practice

Define Success:

  • Clearly articulate the desired outcomes for the initiative or project.
  • Ensure these outcomes are specific and measurable.

Set Goals:

  • Establish achievable goals and benchmarks derived directly from the defined outcomes.
  • Create a roadmap for success and facilitate progress tracking.

Choose Metrics:

  • Select measurable indicators closely linked to the defined outcomes.
  • Metrics should effectively measure progress, effectiveness, and areas needing improvement.

Monitor:

  • Consistently track selected metrics to assess strategy effectiveness over time.
  • Use monitoring to drive continual improvement and adaptation as necessary.

Final Thoughts

Visibility and understanding of the cloud services used within an organization are crucial for effective cloud security efforts and metric development. While some metrics used on-premises can be adapted for cloud environments, the dynamic and business-driven adoption of cloud services requires a different approach. Cloud-specific outcome-driven metrics (ODMs) offer a framework for investing in cloud controls based on targeted security outcomes, rather than simply allocating a percentage of overall cloud spending.

Automation plays a critical role in managing these controls in the rapidly evolving cloud landscape. It’s essential to automate tracking, reporting, and configuration management metrics whenever possible. Organizations are often hesitant to automate remediation in production environments to avoid disrupting business operations. Achieving many of the metrics discussed requires substantial automation capabilities.

Receive the latest news in your email
Table of content
Related articles