LRQA Nettitude Blog

How do you know if your SIEM/SOC is effective?

Posted by Rowland Johnson on Nov 15, 2017 11:00:00 AM

Many organizations have gone out and bought SIEM appliances which are either in-house or outsourced to an external security operations center. We have highlighted the top five areas for organizations to review, when they deploy SIEM technology, or utilize a security operations center function. This guide is designed to help improve SIEM coverage, and provide confidence to the organization that they are getting the most from their security operations center providers.

How do you know if your SIEM-SOC is effective - BLOG-1.jpg
How are logs/alerts being generated and handled?

One of the first things that organizations need to do is determine which logs need to be fed in to a SIEM appliance or SOC function. There are some obvious sources of information such as firewall logs or IPS logs when they are present. Equally, many organizations will naturally choose to select Windows server logs and feed them into their tool sets. However, beyond this it is often surprising that the log sources can really vary from client to client and deployment to deployment. There is a general perception that more logs are better than less logs, and on the whole we wouldn’t challenge this approach. However, organizations should really be thinking about leveraging quality log sources, as opposed to purely focusing on volume.

We would encourage organizations to think about the attack surface, and collect quality logs from the touch points that are most readily accessible through threat actors. For web servers, this should be more than system logs, and instead look at application logs, that provide rich information about application server response codes. For mail servers, it should include logs that detail file attachments, web links, and wider mail conversation metadata.

Organizations should endeavor to gather log data from end user machines. These devices are often ignored, despite being the first type of resource that is targeted in a spear phishing or social engineering type of attack.

Organizations should also gather log data from core assets, including application and database logs that are associated with these key functions. In addition, organizations should review their external cloud based estate, and determine how they collect log data from external cloud based service providers. We often find external cloud services are overlooked, and there is an assumption that the security logging and monitoring is delivered by the cloud provider themselves.  We would encourage you to review the terms of service with your external service providers, and determine what they are responsible for and what you are responsible for, then build a logging and monitoring plan accordingly.

CREST has issued a guide on the types of logs that should be collected by organizations and it describes why each log source provides rich information to a security operations center. Download the CREST guide today.


Logs are not enough!

Many organizations implement SIEM technology and assume that this will be sufficient for them to detect and respond to an attack. In Nettitude’s opinion, there is a lot more than just technology required to maintain a robust detection and response capability.

Once logs are captured, rules need to be built to determine what type of action should be carried out. As a consequence, there is a significant reliance on people and process to get the most out of a SIEM implementation. The process around what to do when an alert is generated is often the most overlooked aspect of a SIEM deployment. Who responds to the alert? A dedicated analyst? The network administrator? The systems owner? Who validates that the alert is a true positive or a false positive? How is the SIEM updated and tuned to learn from previous incidents? Who is responsible for defining follow up activity if certain types of events are detected? For instance, if 1GB of data is seen to be sent to AWS, what are the next steps? Is there a play book, and how should it be executed?

We really believe that the process around SIEM technology is often the most under-defined aspect of a SIEM deployment, despite being one of the most important features that will determine its overall success.  The lesson here is – think about the process, define the process, and then test the process!


Did the business identify critical assets, and align them?

This may seem like a really obvious statement, however it never ceases to surprise Nettitude that many organizations cannot agree on what the critical assets are. If you speak to Finance, they may describe the critical assets as an ERP platform or a payroll application. If you speak to a network administrator, they may describe it as the file server, a directory services device or a key database. And if you speak to a user, they may describe critical assets as e-mail systems or CRM services. Organizations need to define a consolidated view on what their critical assets are, and then align their detection and response tooling and processes towards these assets. This doesn’t mean that they should forget about other resources in the network, however they should have a clear understanding of the critical assets and the attack paths that could be leveraged to gain access to these assets.


Have you conducted a SOC maturity assessment?

Not all SOC’s are the same, and unfortunately there is frequently significant asymmetry of understanding between SOC providers and buyers of these services. It is not uncommon for buyers of SOC services to measure SOC capability by their hours of operation. The assumption being that a 24/7, 365 days a year SOC is more effective than one that operates from 9-5, Monday to Friday. In parallel, many organizations assume that SOC maturity might be measured by having SOC redundancy, disparate data centers, and high numbers of first line analysts. Although these are sensible questions for buyers to ask, in Nettitude’s opinion they do not measure the maturity or effectiveness of the security operations center itself. 

Organizations should undertake a SOC maturity exercise to gain a better understanding of the effectiveness of the SOC service they have built or are subscribing to. This maturity assessment should look at the threat landscape and the threat modelling that has been conducted. It should look at the devices that are monitored and it should look at how network traffic is reviewed and assessed. It should look at the people and the process that operate the SOC, and it should evaluate both reactive and proactive activities in unison. Nettitude provides a range of SOC maturity services to help organizations measure the effectiveness of security operations centers.

Have you conducted targeted attacks to verify detection and response capability?

This is an overarching theme that is often missing when organizations deploy SIEM technology or subscribe to an external managed security operations center function. There is an expectation that it will work, and that it will provide visibility when an organization is being targeted by external threat actors. In our experience there is often a disconnect in what the SOC can see, and the type of traffic that is generated by threat actors techniques, tactics and procedures.  We actively encourage all organizations to conduct simulated attack assessments against their environment so as to determine what levels of visibility their SOC and SIEM appliances have across their estate. This activity should identify blind spots, and allow the people, process and technology to better align to the threat landscape.

What Nettitude can do for your business

Nettitude offers a full suite of detection and response services for clients across the globe. As well as providing consulting about SIEM technology, we deliver Security Operations centers that deliver services 24 hours a day, 365 days a year, from multiple locations across the globe. To find out more about how Nettitude can help, fill in the contact form below and a consultant will be in touch. 

Subscribe Here!

About LRQA Nettitude

LRQA Nettitude is the trusted cyber security provider to thousands of businesses around the world. We stop at nothing to keep your data and business secure in an age of ever-evolving cyber threats.

Recent Posts

Posts by Tag

See all