Article 11 Testing phase: red team test

To mirror the TIBER-EU framework methodology, test managers should have the skills and capabilities necessary to provide advice and to challenge tester proposals. Experience under the TIBER-EU framework has proven that it is valuable to have a team of at least two test managers assigned to each test. To reflect that the TLPT is used to encourage the learning experience, to safeguard the confidentiality of tests, and unless they have resources or expertise issues, TLPT authorities are strongly encouraged to consider that, for the duration of a TLPT, test managers should not conduct supervisory activities on the same financial entity undergoing a TLPT.

It is important, for consistency with the TIBER-EU framework, that the TLPT authority closely follows the testing in each of its stages. Considering the nature of the testing and the risks associated to it, it is fundamental that the TLPT authority is involved in each specific phase of the testing. In particular, the TLPT authority should be consulted and should validate those assessments or decisions of the financial entities that may, on the one hand, influence the effectiveness of the test and, on the other hand, have an impact on the risks associated with the test. The fundamental steps on which a specific involvement of the TLPT authority is necessary include the validation of certain fundamental documentation of the testing, and the selection of threat intelligence providers and testers and risk management measures. The involvement of the TLPT authorities, and in particular for validations, should not result in an excessive burden for those authorities and should therefore be limited to those documentation and decisions that directly affect the conduct of the TLPT. Through the active participation in each phase of the testing, the TLPT authorities may effectively assess compliance of the financial entities with the relevant requirements, which should allow those authorities to issue attestations pursuant to Article 26(7) of Regulation (EU) 2022/2554.

The secrecy of TLPT is of utmost importance to ensure that the conditions of the testing are realistic. For that reason, testing should be covert, and precautions should be taken to keep the TLPT confidential, including the choice of codenames that should be designed to prevent the identification of the TLPT by third parties. Should staff members responsible for the security of the financial team be aware of a planned or ongoing TLPT, it is likely that they would be more observant and alert than during normal working conditions, thereby resulting in an altered outcome of the testing. Staff members of the financial entity outside of the control team should therefore only be made aware of any planned or ongoing TLPT where there are cogent reasons and subject to the prior agreement of the test managers, inter alia to ensure the secrecy of the test in case a blue team member has detected the testing.

As evidenced by the experience of the implementation of the TIBER-EU framework, holding in-person or virtual meetings including all stakeholders concerned (financial entities, authorities, testers and threat intelligence providers) is the most efficient way to ensure the appropriate conduct of the testing. In-person and virtual meetings should therefore be held at various steps of the process, and in particular during the preparation phase at the launch of the TLPT and to finalise on its scope, during the testing phase, to finalise the threat intelligence report and the red team test plan and for the weekly updates, and during the closure phase for replaying testers and blue team actions, purple teaming and to exchange feedback on the TLPT.

To ensure the smooth performance of the TLPT, the TLPT authority should clearly present to the financial entity its expectations with respect to the testing. In that respect, the test managers should ensure that an appropriate flow of information is established with the control team within the financial entity, and with the TLPT providers.

To enable testers to conduct a realistic and comprehensive testing in which all attack phases are executed and flags are reached, sufficient time should be allocated to the active red team testing phase. On the basis of the experience gathered with the TIBER-EU framework, the time allocated should be at least 12 weeks and should be determined taking into account the number of parties involved, the TLPT scope, the resources of the involved financial entity or entities, any external requirements, and the availability of supporting information supplied by the financial entity.

During the active red team testing phase, the testers should deploy a range of tactics, techniques, and procedures (TTPs) to adequately test the live production systems of the financial entity. The TTPs should contain, as appropriate, reconnaissance (i.e. collecting as much information as possible on a target), weaponization (i.e. analysing information on the infrastructure, facilities, and employees and preparing for the operations specific to the target), delivery (i.e. the active launch of the full operation on the target), exploitation (i.e. where the testers’ goal is to compromise the servers, networks of the financial entity and exploit its staff through social engineering), control and movement (i.e. attempts to move from the compromised systems to further vulnerable or high value ones), and actions on target (i.e. gaining further access to compromised systems and acquiring access to the previously agreed target information and data, as previously agreed in the red team test plan).

While carrying out a TLPT, testers should act considering the time available to perform the attack, resources, and ethical and legal boundaries. Should the testers be unable to progress to the programmed next stage of the attack, occasional assistance should be provided by the control team, upon agreement of the TLPT authority, in the form of ‘leg-ups’. Leg-ups can broadly be categorised in information and access leg-ups and may consist of the provision of access to ICT systems or internal networks to continue with the test and focus on the following attack steps.

During the active red teaming in the testing phase, if necessary to allow for the continuation of the TLPT as a last resort in exceptional circumstances and once all alternative options have been exhausted, a collaborative testing activity that involves both the testers and the blue team, should be used. In the context of such limited purple teaming exercise, the following methods can be used: ‘catch-and-release’, where testers attempt to continue the scenarios, get detected and then resume the testing, ‘war gaming’, which allows for more complex scenarios to test strategic decision-making, or ‘collaborative proof-of-concept’ which enables testers and blue team members to jointly validate specific security measures, tools, or techniques in a controlled and cooperative environment.