What Is a Threat-Led Penetration Test (TLPT)?

Download the Guide as a PDF.

What Is a Threat-Led Penetration Test (TLPT)?

Threat-Led Penetration Tests (TLPT) are enhanced security tests reserved for financial entities whose failure would have systemic effects and which are most likely to be targeted by malicious actors. For anyone wishing to maximise their profits or destabilise a part of the financial system, an international bank is more appealing than a relatively modest management company.

Treat-Led Penetration Tests are performed under the supervision of the authorities. They are, therefore, a mechanism for verifying the actual resilience of the financial entities that are essential to the functioning of our financial system.

Resilience tests raise the level of security of regulated entities. At the same time, TLPTs aim to verify that the book has done its job and that the entities' security is effective.

What does this mean?

As it stands, the formula "threat-led penetration testing" is impressive but somewhat cryptic. Let's start with DORA's official definition:

'Threat-led penetration testing (TLPT)' means a framework that mimics the tactics, techniques and procedures of real-life threat actors perceived as posing a genuine cyber threat that delivers a controlled, bespoke, intelligence-led (red team) test of the financial entity's critical live production systems. – DORA, Article 3(17)

A TLPT is, therefore, a large-scale Red Team exercise — and not a mere pentest — which must simulate all the elements of an actual attack and test the entire attack surface of regulated entities. The organisation's attack surface is composed of three main areas:

• The physical attack surface, such as premises, which are vulnerable to on-site intrusions;
• The human attack surface relates to people whom social engineering, phishing, and collusion attacks can target.
• The digital attack surface, i.e. everything that a cyber attack can target.

A TLPT must test each of these elements to be valid and effective. The Red Team must, therefore, carry out assaults against information systems in production, social engineering attacks (phishing, baiting, watering holes, and more), physical intrusion attempts into premises, and much more. In a proper Red Team exercise, the objective is to reach the target by any means necessary.

The Mandatory Criteria of a TLPT According to DORA

The official definition clarifies the concept, but the technical details remain unclear. Fortunately, Article 26 specifies some mandatory criteria for a Threat-Led Penetration Test:

• The test must cover "several or all critical or important functions" of the financial entity;
• The scope is defined by the entity and approved by the competent authorities; some text
  • Where third-party ICT services are in the scope, the entity must take "the necessary measures and safeguards to ensure the participation" of the service provider involved. Full responsibility remains with the entity.
• The Penetration Testing is on live production systems;
• The penetration test is carried out every three years. The competent authorities may reduce or increase this frequency for a given entity according to its risk profile or operational circumstances.
• Once complete, the financial entity must submit a summary of the relevant findings, corrective action plans, and documentation to the competent authorities. The documentation will demonstrate that the penetration test was performed per the requirements. some text
  • In return, the authorities issue an attestation "to allow for mutual recognition of threat-led penetration tests between competent authorities".

TIBER-EU Framework and ESAs' Standards to Guide Operational Requirements

While DORA lays down the guidelines for Threat-Led Penetration Testing, it is another European framework that guides operational requirements: TIBER-EU.

TIBER-EU specifies several criteria, such as the scope of the test or the methodologies and approach to be followed for each phase, from construction to remediation. It also details the requirements applicable to Threat Intelligence and Red Teaming service providers involved in test execution. Certification for TIBER-EU test providers has yet to be recognised and will be formalised at the European level over the coming months.

Regarding TLPTs, DORA gives the European Supervisory Authorities (ESAs) responsibility for developing regulatory technical standards based on the TIBER-EU framework. The first version of these standards, still in the draft phase, was published in November 2023 and will be submitted to the European Commission on 17 July 2024.

The same draft also specifies that countries that have already adopted the TIBER-EU framework may continue to use it. It will be amended to reflect specific measures introduced by the ESAs.

In this article, we will discuss the compliance aspects that we consider the most essential. But if you'd like to know all the operational requirements of a TLPT, we can only recommend that you read:

• The draft Regulatory Technical Standards specifying elements related to Threat-Led Penetration Testing, published by the ESAs;
• the TIBER-EU information page published by the ECB;
• the TIBER-EU framework guidelines for providing Threat Intelligence and Red Teaming services.

We also advise you to consult the national transposition of the TIBER framework in your country, if available. To date (January 2024), 16 countries have published a national implementation guide — Austria, Belgium, Denmark, Finland, France, Germany, Iceland, Ireland, Italy, Luxembourg, the Netherlands, Norway, Portugal, Romania, Spain and Sweden.

Identification Based on a Two-Level Approach

The ESA's RTS introduce a two-layer approach to the identification of financial entities required to carry out a TLPT:

• Specific criteria and thresholds are defined for financial entities operating in the main sub-sectors of finance and playing a systemic role;
• Other criteria are specified to allow the competent authority to include other financial entities that do not operate in a vital sub-sector but still meet the said criteria.

While the competent authority can include entities in the TLPT obligation at will, it can also exclude them. For example, it may decide to exclude regulated entities that are not in a position to do so, that does not have a certain degree of systemic importance, and that need to be technologically mature enough to avoid jeopardising the continuity of core financial services.

The Complete List of Entities Required to Carry Out TLPTs

As mentioned earlier, entities operating in a significant sub-sector of the financial world that play a systemic role must carry out TLPTs. Now, the question is: what are these sub-sectors?

We must refer to Article 2 of the draft ESA regulatory technical standards here. Rather than paraphrasing, we provide you with the list of entities concerned.

TLPT authorities shall require all of the following financial entities to perform TLPT:

• Credit institutions identified as global systemically important institutions (G-SIIs) by Article 131 of Directive 2013/36/EU of the European Parliament and the Council or as other systemically important institutions (O-SIIs) or that are part of a G-SIIs or O-SIIs;
• Payment institutions, exceeding in each of the previous two financial years EUR 120 billion total value of payment transactions as defined in point (5) of Article 4 of Directive (EU) 2015/2366 of the European Parliament and of the Council;
• Electronic money institutions, exceeding in each of the previous two financial years EUR 120 billion of the total value of payment transactions as defined in point (5) of Article 4 of Directive (EU) 2015/2366 or EUR 40 billion of the total value of the amount of outstanding electronic money;some text
  • Central securities depositories;
  • Central counterparties;
  • Trading venues with an electronic trading system that meets at least one of the following criteria:
• At a national level, the trading venue which has the highest market share in terms of turnover in equity, in equity-like financial instruments in bonds and other forms of securitised debts, in derivative contracts or other non-equity financial instruments in each of the preceding two financial years;
• At the Union level, the trading venue whose Union market share in terms of turnover in equity, equity-like financial instruments in bonds and other forms of securitised debts in derivative contracts or other non-equity financial instruments exceeds 5% in each of the preceding two financial years; where the trading venue is part of a group, the turnover of financial instruments on all trading venues about the same group and established in the Union shall be considered.
• Insurance and reinsurance undertakings that subsequently meet the following criteria, identifying: some text
  • First, the undertakings exceeding in each of the previous two financial years EUR 500 million of Gross Written Premium (GWP);
  • Secondly, undertakings that fulfil the previous point included in the 90th percentile of the Gross Written Premiums (GWP) distribution, including all undertakings having reported Gross Written Premiums above the average of the Gross Written Premiums of all insurance and reinsurance undertakings established in the Member State calculated separately for the following activities: some text
    • Life other than life Similar-To-Health (SLT) and reinsurance life;
    • Non-life other than non-life Similar-To-Health (NSLT) and reinsurance non-life;
    • Health is calculated as the sum of life Similar-To-Health (SLT) and non-life Similar-ToHealth (NSLT) and;
    • Reinsurance is calculated as the sum of reinsurance life and reinsurance non-life.
  • Third, insurance and reinsurance undertakings that fulfil point (ii) and whose total assets are equal or higher to the 10 % of the sum of the total assets valued according to Article 75 of Directive 2009/138/EC of the insurance and reinsurance undertakings established in the Member State belonging to the activity type identified as referred to in the previous point;

Once again, it is essential to remember that this list is only the first level of identification of entities subject to the TLPT obligation. The competent authorities are entirely free to designate financial entities other than those mentioned above based on factors that would be too laborious to list here. If you wish to read more about these conditions, please refer to Article 2(3) of the ESA's draft technical standards for TLPTs.

"Pooled Testing" for ICT Service Providers

An ICT service provider must be excluded, as this would compromise the security of its services and the test quality. Similarly, a provider may be the supplier of different financial entities. In such cases, DORA allows for an exception.

The provider may conduct a group test involving several financial entities to which it provides its services. The provider must then:

• agree this test in writing with the various entities;
• sign contractual agreements with an external tester;
• ensure that the test covers all ICT services supporting the critical functions of the financial entities;
• conduct the test under the direction of a designated financial entity.

‍

In such cases, Article 26(4) states that the pooled test "shall be considered TLPT carried out by the financial entities participating in the pooled testing.

How Often Should a Threat-Led Penetration Test Be Performed?

As mentioned above, designated financial entities must carry out a TLPT every three years. Competent authorities may, at their discretion, reduce or increase this frequency for a designated entity, depending on its risk profile or operational circumstances.

What Should Be the Duration of a TLPT?

This question is difficult to answer precisely, as the duration of a TLPT will depend on its scope and the complexity of the entity targeted. Nevertheless, the ESAs' draft technical standards provide for various deadlines, making it reasonable to expect a TLPT to last 6 to 12 months.

For example, the scope specification document must be returned to the TLPT authority within six months of receiving the obligation to carry out a test. The active testing phase (Red Teaming) must last at least 12 weeks. We'll return to the details of each phase in a moment.

Who Are the Stakeholders in a TLPT?

The ESA's draft RTS foresees five participants for a TLPT under DORA:

• The TLPT Cyber Team;
• The Control Team;
• The Blue Team;
• The Threat Intelligence (TI) provider;
• The Red Team.

The TLPT Cyber Team

The TLPT Cyber Team is an internal regulator team, aka the TLPT authority. At least two test managers, one principal and one alternate, must be assigned to each TLPT.

The Control Team

The Control Team is an internal team within the financial entity being tested and appointed by it. It is responsible for ensuring liaison with the regulator's TLPT Cyber Team and for everything related to the proper completion of the exercise, such as managing external service providers, risk assessment, day-to-day operational management of testing activities, risk management, etc.

The Control Team must have the necessary mandate within the financial entity to guide all aspects of the TLPT without disclosing its content to other teams. To reduce the risk of leakage, it should be as small as possible.

The Blue Team

For example, the Blue Team is the financial entity's defensive security team—the SOC. The Blue Team must not be informed of the TLPT's existence until the end of the active testing phase.

The Threat Intelligence Provider

The Threat Intelligence (TI) provider is an external service provider responsible for conducting threat intelligence missions.

The Red Team

The Red Team comprises the testers who will carry out the attacks as part of the TLPT. These testers may be external or internal to the entity.

It is worth pointing out that the ESAs' draft RTSt prefers to use the term "testers" in its broadest sense rather than "Red Team" in the mind of the TIBER-EU framework, which is stricter regarding the choice of Red Team members. For example, the TIBER-EU framework doesn't authorise using in-house testers.

What Are the Different Phases of a TLPT?

Once again, we must examine the ESAs' draft RTS to learn more about the different stages of a Threat-Led Penetration Test (TLPT).

A TLPT consists of three main phases:

• The preparation phase;
• The testing phase;
• The closure phase.

The Preparation Phase

During this phase, the entity under testing must form its internal Control Team and choose its Threat Intelligence and Red Teaming providers—unless the testers are internal. The scope of the TLPT, along with its code name and the communications channels, is defined. The entire content of the preparation phase is in Article 6 of the draft ESAs' RTS.

The Testing Phase

The testing phase is in two parts:

• The first part is dedicated to the Threat Intelligence scenarios;
• The second one is dedicated to the Red Team executing these scenarios.

The Threat Intelligence Phase

The testing phase begins with producing the Threat Intelligence scenarios used during the TLPT. The threat intelligence provider must propose several scenarios that differ according to the identified threat actors and associated tactics, techniques, and procedures. The scenarios must target each of the critical functions.

Three scenarios are to be selected by the Control Team based on the following:

• The recommendation by the threat intelligence provider and the threat-led nature of each scenario;
• The input provided by the test managers;
• The feasibility of the proposed scenarios for execution based on the expert judgment of the testers;
• The financial entity's size, complexity, overall risk profile, nature, scale, and complexity of its services, activities, and operations.

These scenarios must be compiled into a Threat Intelligence report and sent to the TLPT authority for validation.

The Red Teaming Phase

Once the choice of scenarios is made, it's time to move on to the Red Teaming phase.

Testers draw up a "Red Team Test Plan," which is a detailed plan of the attacks. This plan comes from the scope specification document and the TI scenarios selected. The exact content of this plan is detailed in Appendix IV of the ESAs' draft standards, which we recommend you consult.

The entity's internal Control Team and the TLPT authority must validate the Red Team test plan.

Next comes the active testing phase, the execution of the attacks, which must be proportionate to the scope and complexity and last at least 12 weeks.

Throughout the active testing phase, testers must report on the operation's progress to the Control Team and the TLPT Cyber Team at least once a week. The test can only be blended with the agreement of all parties, including the internal control team, the testers, the TI provider, and the TLPT authority.

The entire content of the testing phase is detailed in Article 7 of the ESAs' draft RTS.

The Closure Phase

The closure phase begins at the end of the active testing phase. It consists mainly of drafting various reports and joint exercises between the Red and Blue Teams, the financial entity's defensive security teams. During the closure phase, the Blue team is made aware of the existence of the TLPT.

A Red Team Report

Within four weeks of the end of the testing phase, the attacking team must send a Red Team report to the Control Team, which must forward it without delay to the Blue Team. The exact content of the Red Team report is detailed in Appendix V of the draft AES standards.

A Blue Team Report

Within four weeks of receiving this report, the defensive team must, in turn, draw up a Blue Team report. This report is provided to the Control Team and then forwarded to the attacking team and the TLPT Cyber Team managers. The exact content of this Blue Team report is detailed in Appendix VI of the draft AES standards.

Purple Teaming and Final Report

Within four weeks of the previous step, the Red and Blue Teams meet to examine the offensive and defensive actions. The Control Team, which will have previously identified the topics to be addressed, must then direct a Purple Team exercise, a joint exercise between the defence and attack teams.

This Purple Teaming can include methods such as tabletop exercises, catch-and-release exercises, war games or the joint development of proofs of concept (PoC) for selected TTPs.

Finally, the Control Team must prepare a summary report of the TLPT and its conclusions, which must be sent to the TLPT authority within a maximum of 12 weeks after the end of the testing phase. The exact content expected of this final report is detailed in Appendix VII of the ESAs' draft standards.

Remediation Plans

The Control Team must also send remediation plans to the TLPT authority within 16 weeks of the end of the active testing phase.

These remediation plans must include the following:

• a description of the identified shortcomings;
• a description of the proposed remediation measures and their prioritisation and expected completion, including where relevant measures to improve the identification, protection, detection and response capabilities;
• a root cause analysis;
• the financial entity's staff or functions responsible for the implementation of the proposed remediation measures or improvements;
• The risks associated with not implementing the above measures and, where relevant, risks associated with implementing such measures.

Finally, the TLPT authority must issue a TLPT certificate to the financial entity, specifying which critical systems were included in the scope.

The entire content of the closure phase is detailed in Article 9 of the ESAs' draft RTS.

In-House or Outsourced Testing?

There's one question we haven't yet addressed concerning TLPT: Should testing be outsourced, or can it be carried out in-house? The short answer is that it depends.

Article 26.8 of DORA allows financial entities to rely on in-house testers, but it also introduces an obligation to call in an external tester for every third test. The ESAs' RTS specify that if the Red Team comprises a mix of internal and external testers, the TLPT is considered to have been conducted with internal testers.

One important exception is that credit institutions classified as significant must always turn to an external tester.

An Obligation to Use an External Tester for Significant Credit Institutions

This same Article, 26.8 of DORA, stipulates that "credit institutions that are classified as significant following Article 6(4) of Regulation (EU) No 1024/2013" have no choice but to call in external testers. The real question is: What is a significant credit institution within the meaning of Regulation (EU) n°1024/2013?

The text in question assesses significance according to three criteria:

• the size of the institution;
• its importance for the economy of the Union or of a participating Member State;
• the significance of its cross-border activities.

In practical terms, a significant credit institution is one of the following three conditions is met:

• The total value of the institution's assets exceeds 30 billion euros;
• The ratio between its total assets and the GDP of the Member State is greater than 20% unless the total value of its assets is less than 5 billion euros;
• The ECB designates it as such on its initiative, in which case the institution receives notification from its national competent authority.

If a credit institution meets at least one of these conditions, it is not authorised to carry out TLPTs with an internal tester.

It may seem paradoxical that such organisations should be forced to outsource, even though they possess the most substantial resources—both financial and human—that would allow for in-house testing. The main reason for this is the desire for greater, impartial supervision of the banking institutions essential to the health of the European financial system.

This Regulation (EU) n°1024/2013 quoted by DORA, the one that sets out what a significant credit institution is, was born in the wake of the public debt crisis within the eurozone (2010-2012), itself a knock-on effect of the subprime crisis. Its primary objective was to entrust supervisory powers to an omnipotent third party, in this case, the ECB, without any considerations other than prudential ones. The official name of this legislation is: "Regulation conferring specific tasks on the European Central Bank concerning policies relating to the prudential supervision of credit institutions". Crystal clear.

DORA thus echoes Regulation no. 1024/2013, requiring significant credit institutions to use external testers of the highest quality to avoid conflicts of interest or motivations other than security when performing a TLPT.

The Conditions to Carry Out a TLPT With In-House Testers

Article 27.2 of DORA sets out the conditions that entities must meet when resorting to in-house testers:

• The use of in-house testers has been approved by the relevant competent authority (or by the single public authority designated);
• The financial entity has sufficient resources to conduct a TLPT and has taken care to avoid conflicts of interest throughout the design and execution phases of the test, both of which must be verified by the competent authority;
• The Threat Intelligence provider is external to the financial entity. That means that all Threat-Intelligence assignments must be outsourced.

In addition, Chapter IV of the ESAs' draft regulatory technical standards specifies and reinforces the requirements governing the use of in-house testers.

How do you choose a service provider to perform a TLPT?

As mentioned earlier, financial entities must involve an external tester at least once every three tests.

Requirements Applicable to External Testers

Article 27 of DORA defines the requirements applicable to testers who may perform a Threat-Led Penetration Test.

To be authorised to carry out TLPTs, external testers must:

• Be of the highest suitability and reputability;
• Possess technical and organisational capabilities and demonstrate specific expertise in threat intelligence, penetration testing and red team testing;
• Be certified by an accreditation body such as CREST in a Member State or adhere to formal codes of conduct or ethical frameworks;
• Provide an independent assurance, or an audit report, about the sound management of risks associated with the carrying out of TLPT, including the due protection of the financial entity's confidential information and redress for the business risks of the economic entity;
• Be duly and fully covered by relevant professional indemnity insurance, including against risks of misconduct and negligence.

To these conditions must be added those of the ESAs' draft technical standards, Article 5 of which lays down additional requirements for the selection of external testers:

• The external testers must provide at least five references from previous assignments related to intelligence-led red team tests;
• Red Team staff assigned to the TLPT must: some text
  • Be composed of a manager with at least five years of experience in threat intelligence-led red team testing as well as at least two additional testers, each with red teaming experience of at least two years;
  • Display a broad range and appropriate level of professional knowledge and skills, including knowledge about the business of the financial entity, reconnaissance, risk management, exploit development, physical penetration, social engineering, and vulnerability analysis, as well as adequate communication skills to present and report on the result of the engagement;
  • Have a combined participation in at least five previous assignments related to threat intelligence-led red team tests.