Security Metrics

Group 12

Security Metrics
An interpretation of data by management that leads to changes in security policies, systems, practices, etc. Pros: Allows management to make decisions based on trends Allows management view of scope of security threats Provides a realism to the threat to security Cons: Interpretation left to management Data can be misinterpreted

What are security metrics?

There are a wide variety of factors Examples: Number of port scans on one or more servers Number of stolen laptops Number of password lockouts on an application due to repeated failed attempts Number of people who have failed to take required security training Number of servers running with known security vulnerabilities

Security Metrics vs. Measurements

Measurements Measurements provide single-point-in-time views of specific, discrete factors. Measurements are generated by counting. Metrics

While metrics are derived by comparing, to a predetermined baseline, two or more measurements taken over time. Metrics are generated from analysis.

In other words, measurements are objective raw data metrics are either objective or subjective human interpretations of those data.

What to do with the data?

Compare results with other organizations Assume metrics are comparable Identify differences that may suggest deviation Let results speak for themselves - No one wants to be at the back of the pack Compare results over time Identify trends Use as an indicators to make security-related decisions Similar to Doctors, diagnostic tests influence decision-making

How to use metrics to move a company forward

Try to get agreement on common metrics with other organizations including willingness to share metrics Private companies can do privately Keep alert to data which suggests that a metric may be quantifiably related to a specific risk Look for trends over time and take action if the trends appear to be problematic

What NOT to do with the data?

Use them as performance imperatives, so work focuses on trying to move the meter In essence, that which is measurable takes higher priority that than which is not Panmunjom problem argument over shape of the metrics rather than over substance Create specific requirements Require % reduction (or increase) per year usually inadvisable Measure overall security status of organization at a point in time Relating metric to risk is fraught with problems

Data Loss

Up to May

Reported data loss due to security breaches is not slowing down in the least bit, as the graph points out. Whats more, these statistics only include publicly reported breaches. One can only imagine how many security breaches are unreported by organizations wanting to avoid public scrutiny.

The Question
Security metrics are supposedly a way for upper management and IT departments to converse intelligently about in-house security programs. Why aren't the metrics working?

Factors to why Security Metrics are not effective

Metrics or measurement systems can be costly to develop, implement and maintain An essential factor of security metrics is that it must be objective and tangible. There is a subtle but important distinction between measuring subjective factors and measuring subjectively. Information security is a complex area which makes it difficult but not impossible to identify useful metrics.

Useful Security Awareness Metrics

Malware statistics Computer audit statistics Control Self-Assessment IT Help Desk/Incident statistics Firewall statistics System and network vulnerability statistics Response to security awareness activities

Questions?