Vous êtes sur la page 1sur 10

Overcoming Common Causes for SIEM Deployment Failures

  • in

    • Archived Published: 21 August 2014 ID: G00260858

Analyst(s): Oliver Rochford


Implementing SIEM continues to be fraught with difficulties, with failed and stalled deployments common. Security managers can avoid the six most common failures by following these best practices.


Key Challenges

Security information and event management (SIEM) is a use-case-driven technology

and requires careful and meticulous planning to avoid stalled or failed deployments. Selecting a SIEM solution is predicated on a clear understanding of the scope,

objectives and associated use cases. Buying before planning can doom a deployment before it has even begun. Many organizations underestimate the amount of prework that is required and hit a

hard stop once this becomes clear. The necessary resources and expertise for effective operations are routinely misjudged, and SIEM is often selected even though the feasibility of running it in- house was not assessed.


IT security planners and managers

Evaluate requirements including for ongoing operations and follow a

formalized planning process before selecting a SIEM solution. Define the scope of the SIEM deployment and associated objectives.

Plan for an initial six- to 12-month road map encompassing the deployment of the

SIEM solution and the phased implementation of five to seven use cases. Follow an output-driven SIEM model, and include only the data required to address

distinct use cases. Simulate use cases to verify their soundness and effectiveness, as well as to test the incident response readiness of security personnel.

Consider involving an external service provider if required by resource or expertise constraints or strategy.

Table of Contents


Due to their complexity and rigid requirements, SIEM projects often do not live up to users' expectations, and failed or abandoned deployments are not uncommon. Moreover, Gartner frequently speaks to clients who are purchasing their second or third SIEM solution after finding that their incumbent solution does not meet their requirements. Lack of planning and underappreciation of the operational and infrastructure requirements, as well as underestimating the required resources and skills, are the most common root causes for failed SIEM projects.

Even a successful SIEM deployment whether the company acquired it or is running and maintaining it is an expensive and resource-intensive proposition. Approaching a SIEM project without sufficient planning and without following a formalized process will make it even costlier, with the risk of fewer benefits.

This research points out how to overcome the six most common pitfalls that Gartner encounters as root causes for failed and stalled SIEM deployments. Avoiding these and following the best practices that accompany them will help ensure an effective and successful SIEM deployment.


Undertake Careful Planning to Avoid Stalled or Failed SIEM Deployments

Pitfall 1: Failure to Plan Before Buying

Despite the common perception that SIEM solutions are complex and expensive, many organizations buy their solution without following best practices and first evaluating and scoping the project to determine that a given solution will meet all of their requirements. The chance of successfully implementing a SIEM project without prior planning is slight; the necessary investment in time, resources and potential additional costs will far outweigh the perceived benefit of moving forward without planning. Gartner commonly encounters

organizations where a SIEM solution was acquired and has been quietly gathering dust ever since.

The majority of SIEM solutions provide solid security information management (SIM) and security event management SEM capabilities. However in terms of available out-of-the- box, third-party integrations integrated workflows, auxiliary capabilities such as network flow (NetFlow) and network monitoring, including, for example, data derived from deep packet inspection (DPI) or NetFlow analytics (see "Use NetFlow Analysis to Gain More Value From SIEM" ) will vary widely. More importantly, without sufficient planning, correct scoping becomes guesswork.

Planning is also important to prove feasibility. In the course of planning, you may come to the conclusion that the inherent requirements of SIEM mean that this is not a realistic option for your organization. If that is the case, there are better options available from using a managed service to investing in something else first for lower effort and reduced cost-risk.

SIEM is not suitable for everyone. It requires expertise and dedicated resources, it relies on a sane, well-formed operational environment, and it will not compensate for shortcomings in investment, operational execution or skills.

Best Practice: Use a Formalized Planning Approach

Gartner recommends the following approach to planning and implementing a SIEM (see "Planning for an SIEM Technology Deployment" ):

Form a core project team whose primary responsibilities include defining the goals,

scope, and deployment phases of the project, and identifying initial stakeholders. Define monitoring objectives and the initial scope of deployment.

Determine the use cases that will be covered.

Define the data collection, retention, reporting and monitoring requirements.

Assess how many and which data sources will be included to determine the required scale of the solution whether in terms of events per second, back-end storage or processing power.

An organization should then use this foundation to:

Conduct an environmental assessment.

Assess the architectural requirements to determine what effort will be required to

integrate data sources, how many collector instances will be needed to accommodate segmented networks or geographically dispersed locations, or whether other organizational groups such as networking or application support should be included in the planning. Define monitoring and incident response processes.

Create the relevant processes and policies in order to assess how many resources will

be required to actually run the solution. Select the technology (see "Magic Quadrant for Security Information and Event

Pitfall 2: Failure to Define Scope

Attempting to deploy SIEM without a predefined scope can be sardonically expressed as: no scope, no hope. The scope provides the basis for all that will follow planning, deploying, implementing and maturing the SIEM. It will determine the choice of solution, the architectural requirements, the necessary staffing, and the processes and policies. Deployment without a defined scope is like building a house without a foundation. Not only is the process of building fraught with danger; the house will eventually crumble.

Attempting to define the scope afterward will be a costly venture. The technological debt incurred especially in terms of possibly having selected an unsuitable SIEM solution for a scope retrospectively defined may not be able to be paid. Gartner frequently speaks to clients who have to write down their past investments in SIEM due to this particular problem.

Best Practice: Define Scope and Objectives Based on Drivers

SIEM scope and associated objectives will be determined by the drivers for deploying SIEM and will facilitate the creation and design of use cases. Driverless SIEM will never provide a return on investment.

Alongside a variety of less common niche drivers, there are two primary drivers for SIEM:

compliance and threat management (see "How to Implement SIEM Technology" ).

Compliance When compliance is a driver, the scope will be determined by the compliance requirements and the regulatory mandates. Typical mandates include log management or user activity monitoring by regular inspection of reports and monitoring for policy violations. The main objective is to comply with regulations.

Basic use cases include the collection and retention of log data, as specified by a regulatory mandate, the occasional generation and review of user activity reports, with advanced use cases ranging from the automated auditing of policy violations to daily log reviews.

Threat Management Threat management scope can span many different objectives and use cases, for example, detecting spear phishing and malware attacks, or monitoring third-party or privileged users. Depending on the specific objective, the scope will typically include network security devices such as firewalls, intrusion detection/prevention systems (IDS/IPS), Web application firewalls or proxies security applications like vulnerability assessment and data loss prevention (DLP), as well as various security, audit and applications logs from systems or database servers.

Some examples for threat management use cases are real-time monitoring for external network threats, authentication failures, and user activity and behavior profiling and monitoring. These will be accompanied by incident response processes and automated cross-data correlation and alerting among others (see "Planning for an SIEM Technology Deployment" Note: Some of the preceding references link to documents that have been archived; some content may not reflect current conditions).

Niche There are also a number of niche use cases with their own scope for example, fraud or suspicious transaction monitoring, or industrial control system

monitoring. Fraud management is often focused on the application layer and associated event sources and can expand the relevant stakeholders and involved participants considerably.

Pitfall 3. Overly Optimistic Scoping

To obtain the maximum value out of a SIEM, it may seem tempting to do everything with it at once. SIEM solutions can ingest and process vast amounts of event data and provide capabilities to manage these accordingly. Critically though, the only effective way to scale the SIEM to be the effective platform for organization-wide monitoring and incident management is to do this in stages. Every use case has distinct required data sources and subsequent correlation rules, alerts, reports and dashboards.

Attempting to throw everything at the SIEM at once and hoping to be able to clean up later is one of the most common causes for stalled SIEM deployments. In many cases, an environmental assessment of that scale is a daunting and costly task that inevitably involves many different organizational units and stakeholders. Architectural and operational changes that may result from the use-case requirements will be complex and difficult to implement across the organization.

Best Practice: Construct an Initial Road Map of Five to Seven Use Cases

Gartner recommends that IT security managers planning to implement SIEM identify between five and seven use cases that will be used to construct an initial road map. A realistic time frame for this sort of implementation is six to 12 months, depending on use-case complexity and depending too on whether additional technologies or stakeholders are required. Then further use cases can be added going forward. Once the process has become sufficiently formalized and practiced, many organizations find their ability to implement new use cases becomes more efficient and effective.

It is however, not uncommon to have a combination of compliance, threat management or niche drivers. In this situation, the use cases should be prioritized according to business needs. Advanced users can also attempt to logically group use cases according to shared requirements such as needing the same data sources.

Common use cases include authentication tracking and compromised account detection, tracking compromised and infected systems, malware detection by using network connectivity logs to identify outbound connections, and tracking system changes and other administrative actions.

Pitfall 4: Monitoring Noise

SIEM is not log collection, where the goal is to capture and store all logs from all devices and applications without discrimination. Yet a common mistake is to approach it this way, thinking it will be easy to make sense of all of this data once it is in the SIEM. The predictable result is that what should be an exercise in reducing noise actually amplifies it and generates more of it. Finding a needle in the haystack does not benefit from increasing the amount of hay.

In addition, beyond out-of-the-box correlation rules, reports and dashboards that cover common basic use cases, SIEM has to be configured to look for and recognize the activity or events you may be seeking. It is not a magic bullet throwing data at it and hoping it will automatically illuminate every security problem in your environment will yield nothing but disappointment and disillusionment.

Certain anomaly detection approaches such as statistical analysis, deviation from baseline and outlier identification can potentially benefit from larger datasets, but even then the specific type of data is relevant. Despite some basic overlap in functionality and approach, SIEM is not big data.

Best Practice: Employ Output-Driven SIEM

To facilitate the targeted and focused collection and analysis of only relevant events and data, SIEM should be output-driven, which means that event data is captured only if it is required for a predetermined output or result. Log and event sources must be admitted based only on specific use cases, correlation rules, reports and dashboards. For example, for the typical use case of monitoring for suspicious outbound connectivity and data transfers, firewall and Web proxy logs and network flow data are required. The use case would not, however, benefit from the inclusion of Web application access or Dynamic Host Configuration Protocol (DHCP) logs.

Except for special use cases, the log collection and management infrastructure must be implemented first. This will allow the controlled selective filtering and forwarding of relevant and in-scope event data to the SIEM, permitting an output-driven approach. If there is a requirement to gather all logs, for example, for later forensic usage or to fulfill nonsecurity requirements, this can be done on the log collection and management tier without polluting the SIEM.

Output-driven SIEM requires careful forward planning, know-how around real-world threats (or regulatory requirements and controls), and expertise around the processes and technologies to be monitored. Often this requires the involvement of stakeholders outside of the security organizations (see "Using SIEM for Targeted Attack Detection" ). However, a case can be made that these are general requirements for an effective SIEM deployment.

This investment in planning will yield a far more effective SIEM deployment compared to doing this haphazardly. Output-driven SIEM has several benefits and advantages beyond preventing the SIEM being inundated with useless data.

More focused use cases with selective data ingestion reduce the amount of personnel required to watch and manage the SIEM and also minimize the required data collection. This allows for more cost-effective scaling of the SIEM solution.

If you do not know what you are looking for, SIEM will not provide a lot of additional value especially for the potential costs involved.

Pitfall 5: Lack of Sufficient Context

Monitoring an intrusion detection system (IDS) via SIEM will add some powerful capabilities: It will identify and group attacks originating from a repeated source, for

example, or it will apply threat intelligence to an otherwise anonymous IP address to indicate whether that IP has been seen in other attacks. But without further context and data sources for correlation, such as access to the application server logs to verify if the attack seen on the IDS has been successful, this is as good as it will get. SIEM does not see anything that you do not provide.

Best Practice: Follow a Formal Use-Case Implementation Process

Generally, these pitfalls can be avoided by following a formal process for use-case implementation that includes the following:

Use-case selection Determine and select the initial use case(s), which identify

what you are trying to monitor or achieve. Data collection needed Identify the scope of required data.

Log source configuration needed Configure necessary log and data sources to

send the data to, or fetch it from, the SIEM system. SIEM content creation, preparation and selection Construct the SIEM content

(correlation rules, reports, dashboards, etc). Definition of operational processes required Define the operational processes to

manage this specific use case if required. Test the use case Generate the anticipated behavior or activity, and verify that

everything is configured correctly (see Note 1). Refine the content and processes loop Remediate and refine any issues and SIEM content.

Pitfall 6: Insufficient Resources

A successful SIEM deployment requires skilled people. Once you begin looking, you will inevitably find things; and these findings will require a response and management. Additionally, a SIEM solution does not run itself. At a bare minimum, it requires ongoing tuning and maintenance to reflect changes in the environment, "threatscapes," compliance mandates or the gathered data itself. There are three main duties associated with the operation of SIEM (see "Security Information and Event Management Architecture and Operational Processes" ):

Run This entails managing and maintaining the underlying infrastructure for the

SIEM, ensuring that patches have been applied, that there is sufficient storage or that users are added or deleted. Typically, this is an engineering task, especially when following segregation of duties per ITIL, for example. Watch This encompasses real-time monitoring of alerts and events and

responding, investigating and escalating incidents. A typical role title would be security analyst. Tune This aspect focuses on the ongoing optimization and tuning of correlation rules, reports and signatures, and even processes involved with the watch duties described above. Often this is done by a senior security analyst or third line.

Each one of the duties requires time and a different skill set, and each has to be performed on a continual basis.

While some airplane mechanics may be able to fly a plane, you would probably prefer a fighter pilot to fly it into combat if the need ever arose. The same is the case with SIEM. The maintenance and administration of the architecture remains under the engineering umbrella and will feel familiar to anyone managing other types of systems or network infrastructure. However, using SIEM making sense of the data that is coming in, analyzing and responding to incidents, and constructing use cases and correlations rules requires a security analyst. Most organizations will require several security analysts and will find that they are in short supply 1 and come with a high price tag.

Not only must you have sufficient knowledgeable staff to manage and maintain the SIEM, but you must also be prepared for the additional work that results from the SIEM. Incidents must be investigated and issues remediated, and these tasks are seldom the responsibility of the security organization; they require other departments and teams.

For a typical midsize bank, a minimum staff of eight is required to run a dedicated 24/7 security monitoring operation, with two analysts per shift (not necessarily dedicated full-time equivalents) working three days and having four days off, reversing the week after. In total, there are four 12-hour shifts per week, without taking into consideration vacation, sickness or staff turnover. In addition, this does not include any managers, and it also does not account for how much work is actually there. Rather, it is the minimum to allow real-time, 24/7 monitoring.

Best Practices: Limit the Scope and Engage an External Service Provider

SIEM is not a deploy-and-forget technology, and it does not run itself. Instead, SIEM is a force multiplier. It will allow certain tasks and use cases to be done more efficiently and effectively, but it will not run by itself. There are, however, a few strategies that can be employed to operate SIEM with a low staff contingent:

Limit the scope Adapting the scale of what is being monitored to align with the available resources is a viable option. This applies to the number of monitored data sources, as well as the scope of the use cases. Use cases should be concise and focused so they can be automated via correlation rules as much as possible, and there should be a realistic estimate of how many use cases in total can be managed by the available staff.

Successfully implementing at least limited use cases can provide risk reduction where it is most needed and can also be used to build a business case for expanding the monitoring scope.

Engage an external security service provider Two very basic questions must be answered in order to decide whether involving a service provider is a good solution:

  • 1. Can I provide a better service or capability for the same or less cost than the service provider? Can I provide it all?

  • 2. Does the risk reduction I gain from delegating this to an external provider outweigh the risks associated with handing over control?

If the answer to these two questions is affirmative, engaging a managed security services provider (MSSP) will be a viable solution for many use cases, for example, log collection and

management for compliance, or external threat monitoring for threat management (see "Choose Advanced Capabilities From Managed Security Services Providers With Care" ).

Fully resident staffing is also offered by many providers, but it can be prohibitively expensive, with prices ranging from $100,000 to $350,000 per annum per analyst. 2

Additional research provided by Mark Nicolett, Kelly Kavanagh and Anton Chuvakin

Gartner Recommended Reading

Some documents may not be available as part of your current Gartner subscription.


1 In "Critical Times Demand Critical Skills" (Frost and Sullivan in partnership with Booz/Allen/Hamilton, 2013), the Security Analyst skill set was identified as representing the greatest shortage, with 47% of the respondents naming that position as their top need.

2 Based on pricing reviewed by Gartner during client interviews.

Note 1


While testing may seem daunting to some, without it, it is impossible to know whether what you intend on monitoring will actually be feasible. Similarly, simulated attacks must be used to verify that sufficient forensic data is being gathered to allow effective incident or breach investigation, especially sufficient to be used in legal proceedings.

External penetration tests prove an ideal testing bed to ensure that attacks are even detectable with your current deployment. It is something that has to be verified before a breach occurs afterward it is too late.

© 2014 Gartner, Inc. and/or its Affiliates. All Rights Reserved. Reproduction and distribution of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Although Gartners research may discuss legal issues related to the information technology business, Gartner does not provide legal advice or services and its research should not be construed

or used as such. Gartner shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The opinions expressed herein are subject to change without notice.

Why Gartner

Gartner delivers the technology-related insight you need to make the right decisions, every day.

or used as such. Gartner shall have no liability for errors, omissions or inadequacies in theFind out more  Taha Khan |Log Off |  Select a Gartner siteWhy GartnerAnalystsResearchEventsConsultingAboutAboutCareersContactGartner Blog NetworkIT GlossaryNewsroomPoliciesPrivacySite IndexWebinars © 2017 Gartner, Inc. and/or its Affiliates. All Rights Reserved. " id="pdf-obj-9-10" src="pdf-obj-9-10.jpg">

 <a href=Taha Khan |Log Off |  Select a Gartner siteWhy GartnerAnalystsResearchEventsConsultingAboutAboutCareersContactGartner Blog NetworkIT GlossaryNewsroomPoliciesPrivacySite IndexWebinars " id="pdf-obj-9-33" src="pdf-obj-9-33.jpg">

or used as such. Gartner shall have no liability for errors, omissions or inadequacies in theFind out more  Taha Khan |Log Off |  Select a Gartner siteWhy GartnerAnalystsResearchEventsConsultingAboutAboutCareersContactGartner Blog NetworkIT GlossaryNewsroomPoliciesPrivacySite IndexWebinars © 2017 Gartner, Inc. and/or its Affiliates. All Rights Reserved. " id="pdf-obj-9-131" src="pdf-obj-9-131.jpg">

© 2017 Gartner, Inc. and/or its Affiliates. All Rights Reserved.