Penetration tests as defined by the PCI DSS Glossary are an attempt to identify ways to exploit vulnerabilities to circumvent or defeat the security features of system components. Penetration testing includes network/infrastructure and application testing as well as controls and processes around the networks/infrastructure and applications. It could be both done from outside the environment (external testing) and from inside the environment (internal testing).

PCI DSS requirements 11.3.1.b, 11.3.2.b and 11.3.4.c look simple at first glance. However, when applied these are difficult to assess as the level of interpretation can be quite high. You have to ensure that external, internal and segmentation penetration tests are performed correctly. This should be done by a qualified internal resource or a qualified external party. If applicable, the tester must prove organizational independence, and not required to be a QSA or ASV.

Organizational independence of the tester (if applicable)

Let’s analyze this aspect in more detail. Organizational independence perhaps is the easiest to deal with. The note ‘if applicable’ (see table below) is a bit confusing, as it may seem this applies only when a qualified internal resource is performing the penetration test. If you read the reporting instructions within the Report on Compliance you will notice that requirements 11.3.1.b (external penetration test) and 11.3.2.b (internal penetration test) start with a question whether the penetration test is performed by an internal resource. If the answer is ‘No’, then the assessor is allowed to skip the checks of qualification and organizational independence. However, requirement 11.3.4.c (segmentation penetration test) requires verification of qualification and independence of the tester regardless if it’s an internal resource or an external third party.

How to choose penetration testing team Advantio

In fact, the independence requirement may be very much applicable to third parties. The PCI Security Standards Council (SSC) Penetration Testing Guidance (dated September 2017) clarifies that in situations where a third party company is performing the PCI DSS assessment for the entity, the party cannot perform the penetration test if they were involved in the installation, maintenance, or support of target systems. Careful consideration should be made in each case to decide whether a third party can or should perform penetration tests for you if this third party was in any way involved in implementation, maintenance and/or support of your cardholder data environment. Would you want a company that develops an application for you to be performing a penetration test for that application? The conflict of interests is clear. In case some critical vulnerabilities are found the third party might not be willing to show their mistakes, but rather to fix them without letting you know. It may seem like the issue is tackled, but you wouldn’t get a full picture.

When it comes to a qualified internal resource, of course, you should make sure that the tester does not report to the business line, which owns the area being tested. In other words, someone from the network team should not be performing penetration tests of network devices, or developers should not be testing in-house developed applications. Ideally, this role is best suited within the security team, which is usually situated in the organizational structure with a different reporting line from IT implementation, maintenance and/or support areas.

Qualifications of the tester

When we talk about a qualified internal resource or qualified external third party, it becomes even more subjective. We’ll give you some guidance on what you need to evaluate when choosing a penetration testing third party to make sure you end up meeting the requirements:

  • How many years of experience do the penetration tester and the company he/she works for have? Whenever possible, you should pay most attention to relevant, i.e. industry specific, experience. E-commerce or brick-and-mortar merchants, call centres, hospitality, ATM environments, etc. all have their own specific issues and weaknesses, that should be taken into account when conducting penetration tests.
  • Is the penetration tester familiar with technologies that are in scope (including any exotic flavours of operating systems, databases, programming languages, cloud implementations and so on)? For example, does the penetration tester understand the concept of a NoSQL database you might have in your environment, and will he/she know how to test for injection vulnerabilities for NoSQL databases?
  • References from other clients, especially from the same industry and similar size companies, are very valuable, as well as the possibility to review any sanitized sample penetration reports.
  • A penetration test methodology should always be shared by a penetration testing third party. Ideally, even before the contract is signed, so you would know what to expect during penetration testing.
  • A list of industry certifications held by the penetration tester may also strengthen the trust in qualification required. However, it should never be looked at in isolation to other points mentioned above. Some common penetration testing certifications are – OSCP, CEH, GPEN, GWAPT, GXPN, CREST, CESG, CHECK.

So how can a QSA (Qualified Security Assessor) assess whether an internal resource is qualified enough to perform the penetration tests? Basically, the tester has to demonstrate his/her qualification to the QSA. This can be done by interviewing penetration testers to identify relevant previous experience, educational background, and reviewing professional penetration test reports. An activity output should also include a well-documented penetration test methodology, which should be properly followed during activity.

Other things to consider

A note in the above table by the PCI SCC that the penetration tester doesn’t have to be a QSA or an ASV (Approved Scanning Vendor) emphasizes the fact that if someone tries to sell you penetration testing purely based on the fact that they are a QSA or an ASV company, you should stay away from them.

One important aspect that is not usually offered or at least advertised by penetration testing third parties is the possibility for your staff to watch and learn during the penetration test. Most of the penetration tests nowadays are performed remotely, however, sometimes it’s worth considering spending slightly more (i.e. tester’s travel expenses) for the possibility of having the tester onsite and letting your staff learn by watching. Of course, this has to be agreed in advance. This kind of learning experience may be extremely valuable for your staff and, if planned well, even form a part of their training program. The tester onsite also facilitates an early view of found issues and the ability to react faster if critical ones are identified.

In most cases, companies undergoing PCI DSS assessments choose third parties to perform required penetration tests. We think it’s always the right approach. While it is possible to perform a good network/infrastructure penetration test using internal resources, it’s always challenging to expect a good application penetration test from someone, who is not a full-time penetration tester. In reality, only companies with a very liberal security budget (a rarity nowadays) can afford to employ and retain their own qualified penetration testers.

If you are looking for a trusted team of security professionals and ethical hackers to perform a penetration test for your company, fill out our scoping questionnaire and our experts will come back to you with your personalized quote. 

Get Your Personalized Penetration Test Quote Now

Irmantas Brazaitis

Written by Irmantas Brazaitis

Information security professional with 12+ years of experience in information security consulting, 2+ years of experience in the role of information security manager, and 2+ years of experience in large scale governmental IT project management. Developed and delivered PCI DSS Implementation and ATM Security training sessions in 15+ different countries. Certified as Qualified Security Assessor (QSA) by PCI Security Standards Council, holding CISM, CISA and CISSP certifications in good standing.