Security testing (including scanning, penetration testing, red teaming, and more), is often seen as a compliance bug bear. However, if your security team wants to provide a level of assurance to the business that if an attack was to take place, you are well placed to both defend and detect it, it is essential that you perform the right kind of testing for your business. In this post, we’ll take a look at the types of testing you should be deploying within your organization by explaining what each type of test does and what it can (and can’t) deliver for you.

Is there one type of testing?

There are various security testing types that you can do:  Automated scanning, organization-wide red team activity, social engineering on your employees, third party penetration testing, physical testing on your offices, drive by wireless testing, or focused application security testing at the end of development cycles, to name a few.

They all have their place, so using a more strategic approach can help ensure the efforts you deploy give you the results you need.

Why is there not one type of security testing?

There is often a tension between objectives in a security test. On one hand, you want to get as much coverage and as many potential issues identified as possible. But that requires a very noisy and wide coverage of attacks to be done in a short space of time. On the other hand, you also want to know what a real attacker would be able to do. And this would be the opposite of the first approach, often involving a single attack to a targeted resource, making every effort to be unobserved.

Both approaches are important and the tension between them is shown in the diagram below.

Figure 1 – Tension between testing and real world attacks

To meet these objectives, a range of types of security testing have emerged (See Figure 2).

Together they provide a comprehensive security assurance package. A mature assurance process will include elements at all the levels shown.

Figure 2 – Security Assurance Testing Pyramid

Each of these types of testing are now discussed in more detail.

1: (Continuous) Vulnerability Assessment

What is Vulnerability Scanning?

Vulnerability Scanning is an essential part of your basic needs within a cyber security assurance program. It will identify changing exposures within your systems and networks that are easily identifiable by cyberthreat actors, and which can often be fixed through known policy updates, configuration changes and patching.

It is an automated scan of all of your IP addresses that seeks to identify known weaknesses and configuration issues that could be used by malicious users to attack your organization.

2: Penetration Testing

What is Penetration Testing?

Security Penetration Testing takes a much more in depth look at your environment. It builds significantly on an automated scan by using a certified penetration tester to drive the behaviors that real malicious users would take. It has a much deeper coverage than a scanning tool and approaches the test from a much more real world perspective. The impact of any vulnerabilities is shown and demonstrated. 

It is an essential part of a security assurance program that is used to verify in depth that the building blocks of your environment have been solidly constructed and implemented. Tests often focus on a particular part of your environment such as a web application, IP address range, or network segment of a system (such as Wi-Fi, specific products, etc.).

1. Seeks to exploit the vulnerabilities identified. What is the real impact of these issues? What would a malicious user actually be able to do with them?
2. Human led vantage point. A penetration tester will log into web applications, probe the business logic of your systems, and test the assumptions and logic within your applications. It will lift up the covers and test the conventions around your authentication, encryption, security policy and more. A scan will only look at what’s available from its single starting position. A penetration test on the other hand, will seek to test all the available attack points regardless of pre/post authentication, initial permissions, etc. It will look at the surrounding configuration and environment and seek to approach the attack from the position of what a real attacker might do. Vulnerabilities will be combined to a greater effect. Information gathered in one area may well be used to exploit another in ways an automated scan could never do.

Security pen tests do not replace vulnerability scans. But they should be done when changes or updates have been applied. They are often used to verify the development practices or security features deployed are actually in place, can’t be bypassed and are effective.

3: Red Team Security

What is Red Team Testing?

Red Team Testing is a concept that is used to validate the wider security posture and assurance levels of an organization. It builds on Penetration Testing by having a much wider scope and remit both in terms of attack surfaces looked at, and in the level of controls that are tested.

The starting point is not individual components within your environment (as with a penetration testing) but rather looks holistically at your whole environment and seeks to find out if your assets of value can be compromised.

1. Scope is defined by objectives, rather than a narrow set of IP addresses or URLs. For example, can you access data on our customers? Can you affect the availability of system ABC? Can you change the transactions flowing from A to B?
2. Wider business polices and controls are tested. For example, physical access policy into the building, the patching policy, the induction process, the detection and response process or the effectiveness of other security controls will be checked. A penetration test does not look at or seek to stress these aspects by design.
3. Approach in a way a real threat actor would. The Red Team looks at the environment through the attacker’s eyes. How would they behave? How would they react and adapt? What would they do to meet their objectives?

The primary purpose of the Red Team is to validate the organization’s effectiveness from the customer’s perspective. The existing strategy deployed by the customer is stressed and tested to see how the overall security posture reacts and behaves. Red teaming may be more or less structured, and a wide range of approaches can be taken, as would be done in real life by a threat actor.

4: Threat Intelligence Led Testing

What is Cyber Threat Intelligence (STAR) Testing?

Cyber threat intelligence led testing, or STAR (Simulated Targeted Attack & Response) testing as it’s also known, takes things one step up again from red teaming. Although looking at the whole of your organization and what a real world threat actor might do is helpful from a strategic impact level, it is still not how a real cyber threat actor will operate.

Threat intelligence led testing moves the starting point from what you already know, into the world view of your attackers. This type of testing uses advanced malware intelligence to build credible, realistic scenarios of who and how you might be attacked. These are then used as a basis to build realistic simulations. Once testing is completed, a threat intelligence report looking at what took place, and how you detected and responded is delivered. The objective here is very much on the assurance level you have that a real attack is (or is not) managed effectively.

1. Based on Threat Intelligence. What is your real attack surface? What about beyond your borders in the open internet, social media, third parties’ activities and historically available information?

Are you likely to endure an advanced persistent threat attack (APT) any time soon? Who is likely to come after you and how? Who would they target and how will they initially compromise you? How will they then move around your environment? How will they communicate?

2. Simulates realistic scenarios. Once the threat is understood, the testing team will simulate these scenarios. Custom code, infrastructure and methods may need to be created to mimic the threat actors in question. It will be staged over a longer time period and actions will be done to mimic what would happen in a real attack.
3. Assesses your ability to detect and respond. Once the testing has been completed, an analysis of what took place vs what you were able to detect and respond to will be provided. Any lessons learned and recommendations for the future will be documented and discussed.

STAR testing addresses the most realistic type of simulations and can mimic the most sophisticated of cyber threat actors. In some areas, (such as UK Financial Services) it has been adopted as a standard (Bank of England’s CBEST program).

Summary

A mature assurance process will use all elements of the security testing pyramid at different times and for different objectives. 

Figure 3 – Table of Testing Attributes