Penetration tests help identify and address weaknesses in a company's security before attackers can exploit them. But for such a test to be truly useful, it's important to understand why you're conducting it, the methodology you're using, and what you'll do with the results.
In this article we will look into:
A company with poor cybersecurity is easy prey for competitors, hacktivists, ransomware, and other attackers. No FSTEC or ISO certificates will protect against cybersecurity incidents. Only practice—penetration testing—reveals the true picture.
Of course, if a company employsone and a half excavators, three accountants with laptops, and the only service is a business card website, it's too early to talk about a pentest. But if the infrastructure includes at least a few servers and the business depends on data, offensive security becomes a necessity.
A black box is a blindfolded game. The penetration tester simulates the actions of an external attacker who knows nothing about the company's infrastructure or its services. All they have is the organization's name—for example, "Roga i Kopyta" LLC.
Sometimes in such pentests, the customer provides basic input data: a limited scope (for example, a list of domains) or an ASN (Autonomous System Number) by which the company's external services can be identified.
Interaction with the client under this method boils down to regular health checks: to ensure everything is going according to plan and that the checks aren't interfering with the normal operation of services. If the infrastructure load is increasing, pentesters reduce the intensity of attacks. The client's information security team can also request signatures to distinguish pentest activity from real threats.
A corporate network or application is truly terra incognita for it , so the contractor's perspective remains fresh, unclouded by redundant information. This ensures thorough inspections that cover a variety of attack types and techniques.
However, black box training has its drawbacks. Even the most realistic exercises are no substitute for real combat: a black box doesn't cover every attack scenario. A real attacker can easily send phishing emails impersonating government agencies or other companies or hack employees' personal devices, but for a penetration tester, such "dirty" methods are usually prohibited .
Moreover, if a penetration tester fails to penetrate the external perimeter, vulnerabilities in internal services remain unnoticed and, therefore, not included in the report. This is another significant drawback of the methodology.
This approach also works well in companies with a transparent role model. Imagine a situation where a system administrator has virtually unlimited rights. In this case, digging into access rights and potential internal attackers becomes pointless. The only option is to focus on external cyberthreats and hope the all-powerful administrator doesn't decide to go dark.
Black box testing is also a popular choice for companies with limited cybersecurity budgets. The logic is simple: the deeper penetration testers delve into the internal infrastructure, the longer the audit takes, the more resources it requires, and the more expensive it is for the business.
In this case, testing is conducted from the perspective of an insider—an employee, client, administrator, or contractor of the company who already has limited access to the system. The penetration tester is provided with some information about the infrastructure or application in advance and is assigned an account with certain privileges within the system.
In a "gray box" test, the emphasis is on checking the robustness of the role model: whether certain roles are granted too broad powers, whether an ordinary user can escalate their privileges and gain moderator or administrator rights.
Sometimes, the client's security team shares additional details: known vulnerabilities, unusual system behavior, and past incidents. This "case history" helps immediately focus on problem areas and avoid wasting time on reconnaissance where everything is already clear.
To obtain the necessary input, the pentest team sends questionnaires to the client's information security team in advance, asking questions about infrastructure, technologies, and roles. The more carefully the client fills out the questionnaire, the easier it will be for the pentesters to select tools, assemble a team, and estimate the project timeline.
An account with access rights allows you to simulate the actions of an attacker who is already inside the network—for example, a former employee or a hacker who has been entrenched there for a long time.
However, there's a catch. This type of pentest requires significant material, human, and time resources. The client's information security team must assign dedicated staff to collaborate with the pentest team. Without such support, it's impossible to delve deeply into the infrastructure or application.
The "gray box" method is useful in B2B companies where clients or contractors are issued keys and access tokens. A typical example is a taxi aggregator that opens part of its IT systems to partner taxi companies.
The presence of a large number of contractors and a complex role model is another compelling argument in favor of this approach. Aggregators, for example, can have a whole zoo of roles: drivers, taxi company administrators, customer support, end users. Penetration testing helps bring order to this diversity and understand the true capabilities of each role.
The presence of internal admin panels also speaks in favor of gray-box penetration testing. Any back office is also a fully-fledged service with its own logic, role models, and potential vulnerabilities. Gray-box testing allows access to the admin panel and analyzes it from a cybersecurity perspective.
API services are a special case. Many companies build their architecture on GraphQL or other flexible interfaces. Providing them with API documentation allows pentesters to delve deeply into the application's logic and test it from every angle.
White-box testing is like playing a royal flush. The penetration tester has comprehensive information about the infrastructure or application. They have access to:
When testing infrastructure using the white box method, the check often smoothly flows into a kind of internal audit.
The specialist's job is to check security policies, role models, servers, and configurations for compliance with both generally accepted and internal security standards. To do this, they use both manual methods and automated tools.
Classic OSINT and external intelligence are generally absent here. Instead, so-called "internal OSINT" appears—albeit an oxymoron. Internal knowledge bases, Confluence, wikis, and reference materials serve as sources. These often contain a wealth of insights into architecture and stack specifics, role models, and access rights.
This type of pentest helps identify privileged users with excessive rights, detect the lack of necessary logging, and other internal flaws in the role model and security architecture.
Moreover, this methodology significantly simplifies hardening and corrective actions. Unlike a "black box," the report contains specific, detailed instructions for resolving the issues found. For example, it identifies the specific line of code containing the vulnerability and how to mitigate it. Sometimes, it even includes step-by-step guides for building a secure architecture.
Unfortunately, like gray-box testing, this type of pentesting is expensive and not a panacea. Testers are always limited in time, while real attackers sometimes monitor their victims for years. Finding such dormant compromises is an art in itself .
This approach has taken hold in high-risk industries, such as fintech, as it allows businesses to comply with strict regulatory requirements. For example, the Central Bank requires financial institutions to conduct regular, in-depth cybersecurity audits, including internal threat analysis.
Comparison of pentest methodologies
A common scenario: pentesters act as external attackers for the first one or two weeks. If they manage to break through, the full-blown "gray" phase begins.
Sometimes, an external attack can successfully hijack a privileged account. This is an opportunity that can't be missed: the test switches to "insider" mode. And if the source code is successfully obtained, the methodology automatically evolves to "white-box" mode.
Another common scenario: testing using two methodologies is launched simultaneously and conducted in parallel. For example, a company decides to simultaneously test the security of its infrastructure or application against both external threats and hypothetical internal attackers.
For example, if a client suddenly requests that all actions be logged and transferred to the SOC, testers have to change their approach on the fly. Time is wasted on routine tasks, some logs are lost, and the results suffer. It's better to agree on such details in advance—then the team will be prepared and will be maintaining the logbook from day one.
It's important that specialists don't respond formally, but share details. This allows you to choose the appropriate format: black box, white box, gray box, or a combination of these.
Over time, penetration testing becomes blurred, and pentesters can miss important vulnerabilities. That's why we at Bastion try to rotate our specialists so we always have a fresh perspective on our infrastructure.
Methodologies like "black box," "gray box," and "white box" aren't just names. They're ways to look at a system from different angles. Each offers its own depth, each revealing something unique. Ideally, they should be combined to provide a complete picture. But even the most thorough penetration test won't save you if the conclusions remain at the level of, "Well, it doesn't seem to be working." Without fixing vulnerabilities, revising the architecture, and fixing bugs, it's just an expensive document in the "Reports" folder.
So, if you're already conducting pentests, great. If you're just planning to, start by asking yourself, "Why exactly do we need this?" But if you have no plans for pentests at all, it might be time to rethink your approach to security.
In this article we will look into:
- What are the differences between black, gray, and white box methodologies?
- which one to choose for specific risks;
- How to prepare for testing so as not to waste money;
- and why even a "green" report in itself is not a reason to breathe a sigh of relief.
A company with poor cybersecurity is easy prey for competitors, hacktivists, ransomware, and other attackers. No FSTEC or ISO certificates will protect against cybersecurity incidents. Only practice—penetration testing—reveals the true picture.
Of course, if a company employs
Black box
A black box is a blindfolded game. The penetration tester simulates the actions of an external attacker who knows nothing about the company's infrastructure or its services. All they have is the organization's name—for example, "Roga i Kopyta" LLC.
OSINT—the collection of publicly available information about a target—becomes the primary reconnaissance method in such circumstances. Penetration testers attempt to extract as much data as possible from public sources to identify vulnerabilities in the company's defenses. They then continue their investigations with more invasive methods: scanning the organization's network perimeter, manually testing entry points, and ultimately finding a way to penetrate.
Sometimes in such pentests, the customer provides basic input data: a limited scope (for example, a list of domains) or an ASN (Autonomous System Number) by which the company's external services can be identified.
Interaction with the client under this method boils down to regular health checks: to ensure everything is going according to plan and that the checks aren't interfering with the normal operation of services. If the infrastructure load is increasing, pentesters reduce the intensity of attacks. The client's information security team can also request signatures to distinguish pentest activity from real threats.
Black box pentesting stages
- OSINT is the collection of open information about an infrastructure or application.
- Active reconnaissance is the probing of an organization's network perimeter using specialized tools. This involves clarifying technical details such as software versions, open ports, and configurations.
- Vulnerability exploitation is an attempt to penetrate an internal network or service.
- Persistence – gaining a foothold in the system: setting up backdoors, creating stable entry points.
- Preparation of a report – a document containing the results of the audit and recommendations.
Advantages and disadvantages of a black box
The main advantage of the black-box method is its extreme closeness to the actions of a real attacker. Penetration testers, like most hackers, must use their entire arsenal to find and exploit vulnerabilities.A corporate network or application is truly terra incognita for it , so the contractor's perspective remains fresh, unclouded by redundant information. This ensures thorough inspections that cover a variety of attack types and techniques.
However, black box training has its drawbacks. Even the most realistic exercises are no substitute for real combat: a black box doesn't cover every attack scenario. A real attacker can easily send phishing emails impersonating government agencies or other companies or hack employees' personal devices, but for a penetration tester, such "dirty" methods are usually prohibited .
Moreover, if a penetration tester fails to penetrate the external perimeter, vulnerabilities in internal services remain unnoticed and, therefore, not included in the report. This is another significant drawback of the methodology.
Who is the black box method suitable for?
First and foremost, the choice of pentest methodology depends on the current information security risks. A black-box approach is appropriate when external attackers pose the greatest threat, and the goal is to test the resilience of infrastructure and services to external attacks.This approach also works well in companies with a transparent role model. Imagine a situation where a system administrator has virtually unlimited rights. In this case, digging into access rights and potential internal attackers becomes pointless. The only option is to focus on external cyberthreats and hope the all-powerful administrator doesn't decide to go dark.
Black box testing is also a popular choice for companies with limited cybersecurity budgets. The logic is simple: the deeper penetration testers delve into the internal infrastructure, the longer the audit takes, the more resources it requires, and the more expensive it is for the business.
Gray box
In this case, testing is conducted from the perspective of an insider—an employee, client, administrator, or contractor of the company who already has limited access to the system. The penetration tester is provided with some information about the infrastructure or application in advance and is assigned an account with certain privileges within the system.
In a "gray box" test, the emphasis is on checking the robustness of the role model: whether certain roles are granted too broad powers, whether an ordinary user can escalate their privileges and gain moderator or administrator rights.
In addition to access rights and accounts, penetration testers are often provided with additional documentation: network diagrams, architecture descriptions, API data, endpoint lists, and internal manuals. All of this helps them quickly understand the context and build a realistic attack scenario.
Sometimes, the client's security team shares additional details: known vulnerabilities, unusual system behavior, and past incidents. This "case history" helps immediately focus on problem areas and avoid wasting time on reconnaissance where everything is already clear.
To obtain the necessary input, the pentest team sends questionnaires to the client's information security team in advance, asking questions about infrastructure, technologies, and roles. The more carefully the client fills out the questionnaire, the easier it will be for the pentesters to select tools, assemble a team, and estimate the project timeline.
Gray-box pentesting stages
The sequence of steps here is almost the same as in the "black box," but with a few additions. So, find four differences:- OSINT is the collection of open information about infrastructure and services.
- Authorization - pentesters log in to the system using the issued accounts.
- Role model analysis - studying access levels, available functions and restrictions.
- Active reconnaissance is a technical analysis of available services, configurations, and interactions between components.
- Vulnerability exploitation is an attempt to gain additional privileges or access to critical data.
- Business logic attacks – checking for incorrect process implementation: authorization errors, improper role handling, etc.
- Persistence – maintaining a foothold in the system: creating hidden entry points or privileged accounts.
- Report - a description of the problems found and recommendations for their resolution.
Advantages and disadvantages
The advantage of this methodology is the ability to immediately focus on potentially vulnerable components and systems. This is made possible by reviewing design documentation at the outset of the audit.An account with access rights allows you to simulate the actions of an attacker who is already inside the network—for example, a former employee or a hacker who has been entrenched there for a long time.
However, there's a catch. This type of pentest requires significant material, human, and time resources. The client's information security team must assign dedicated staff to collaborate with the pentest team. Without such support, it's impossible to delve deeply into the infrastructure or application.
Who is the "gray box" suitable for?
This methodology should be chosen alongside a Compromise Assessment if there is a suspicion of illegitimate activity within the company, or as a complement to an incident investigation. For example, if a data leak occurs and all indications point to a mole within the support team, a penetration tester would be granted access with similar privileges and would test what could be done within the organization as a regular engineer.The "gray box" method is useful in B2B companies where clients or contractors are issued keys and access tokens. A typical example is a taxi aggregator that opens part of its IT systems to partner taxi companies.
The presence of a large number of contractors and a complex role model is another compelling argument in favor of this approach. Aggregators, for example, can have a whole zoo of roles: drivers, taxi company administrators, customer support, end users. Penetration testing helps bring order to this diversity and understand the true capabilities of each role.
The presence of internal admin panels also speaks in favor of gray-box penetration testing. Any back office is also a fully-fledged service with its own logic, role models, and potential vulnerabilities. Gray-box testing allows access to the admin panel and analyzes it from a cybersecurity perspective.
API services are a special case. Many companies build their architecture on GraphQL or other flexible interfaces. Providing them with API documentation allows pentesters to delve deeply into the application's logic and test it from every angle.
White box
White-box testing is like playing a royal flush. The penetration tester has comprehensive information about the infrastructure or application. They have access to:
- source code of applications;
- configs;
- architecture documentation;
- data on backend, frontend, microservices;
- information on DevOps (CD pipelines, etc.);
- role models and access;
- protective equipment and much more.
Another similarity between white-box testing and the previous methodology is its limited scope and focus on analyzing internal infrastructure. Consequently, the penetration tester must be proficient in working with source code.
When testing infrastructure using the white box method, the check often smoothly flows into a kind of internal audit.
The specialist's job is to check security policies, role models, servers, and configurations for compliance with both generally accepted and internal security standards. To do this, they use both manual methods and automated tools.
White-box pentesting stages
The main steps here are more concise:- testing of architecture, source code and development pipelines;
- vulnerability analysis and exploitation;
- report.
Classic OSINT and external intelligence are generally absent here. Instead, so-called "internal OSINT" appears—albeit an oxymoron. Internal knowledge bases, Confluence, wikis, and reference materials serve as sources. These often contain a wealth of insights into architecture and stack specifics, role models, and access rights.
Advantages and disadvantages
The main advantage of the white-box method is its focus on internal security issues. This type of pentesting can uncover logical vulnerabilities that are undetectable from the outside. This reduces the chances of insider attackers harming the business—for example, by hacking the company's back office, stealing its customer base, or leaking confidential information to competitors.This type of pentest helps identify privileged users with excessive rights, detect the lack of necessary logging, and other internal flaws in the role model and security architecture.
Moreover, this methodology significantly simplifies hardening and corrective actions. Unlike a "black box," the report contains specific, detailed instructions for resolving the issues found. For example, it identifies the specific line of code containing the vulnerability and how to mitigate it. Sometimes, it even includes step-by-step guides for building a secure architecture.
Unfortunately, like gray-box testing, this type of pentesting is expensive and not a panacea. Testers are always limited in time, while real attackers sometimes monitor their victims for years. Finding such dormant compromises is an art in itself .
Who is a white box suitable for?
The first and foremost argument in favor of white-box testing is the high risk of insider attacks. The second most common is suspected vulnerabilities in the applications' business logic. Furthermore, this methodology is well suited for testing large, complex applications, where it is difficult to cover all possible attack surfaces at once.This approach has taken hold in high-risk industries, such as fintech, as it allows businesses to comply with strict regulatory requirements. For example, the Central Bank requires financial institutions to conduct regular, in-depth cybersecurity audits, including internal threat analysis.
Comparison of pentest methodologies
How to combine pentest boxes
In practice, the above-mentioned pentest methodologies rarely exist in their pure form. They are combined depending on the objectives and the testing process.A common scenario: pentesters act as external attackers for the first one or two weeks. If they manage to break through, the full-blown "gray" phase begins.
Sometimes, an external attack can successfully hijack a privileged account. This is an opportunity that can't be missed: the test switches to "insider" mode. And if the source code is successfully obtained, the methodology automatically evolves to "white-box" mode.
Another common scenario: testing using two methodologies is launched simultaneously and conducted in parallel. For example, a company decides to simultaneously test the security of its infrastructure or application against both external threats and hypothetical internal attackers.
Some useful life hacks
Finally, here are some practical tips on how businesses and information security teams can work with their pentest team to maximize the benefits of the audit.Conduct pentests regularly
The frequency of pentests depends on the industry, infrastructure, and regulatory requirements. In most cases, a reasonable frequency is once per quarter.Fix any vulnerabilities found
Based on the test results, it is worth setting deadlines for fixing critical issues, and then verifying that they have actually been fixed.Discuss all the details with penetration testers in advance
A professional team always strives to accommodate the client's wishes. However, changing conditions mid-progress complicates the process, increases the timeframe, and reduces the quality of the result.For example, if a client suddenly requests that all actions be logged and transferred to the SOC, testers have to change their approach on the fly. Time is wasted on routine tasks, some logs are lost, and the results suffer. It's better to agree on such details in advance—then the team will be prepared and will be maintaining the logbook from day one.
Don't interfere with penetration testers' work
Security specialists should not interfere with testers as if they were real hackers. Countermeasures are appropriate for Red or Purple Team projects, but not for traditional pentesting. Such actions create obstacles, increase risks, and delay the project.Choose your methodology consciously
Mature companies know exactly what testing format they need, but if their information security processes are still being developed, assistance is needed. We conduct introductory meetings and ask clients to complete a questionnaire. This helps gather data on the infrastructure, threats, goals, and business expectations.It's important that specialists don't respond formally, but share details. This allows you to choose the appropriate format: black box, white box, gray box, or a combination of these.
Change the command when testing in black box format
If the black box analysis reveals few vulnerabilities and the final report is "green," it's too early to relax. Especially if this isn't the first time this team has tested your system.Over time, penetration testing becomes blurred, and pentesters can miss important vulnerabilities. That's why we at Bastion try to rotate our specialists so we always have a fresh perspective on our infrastructure.
Simplify account creation for Gray Box
If penetration testers have to create a new individual entrepreneur account to register as a contractor in the system, this slows down the work process. Such barriers only distract from the core of the work. Businesses should ensure access upfront so the team can focus on testing, not on formalities.Check the infrastructure before white and gray box
If you haven't taken an inventory in a while, it's easy to forget about important hosts, source codes, or domains. Vulnerable services are sometimes managed by contractors and thus out of sight. Before testing, it's helpful to conduct a quick audit. This will allow you to more accurately define your scope, avoid surprises, and provide penetration testers with up-to-date information.Methodologies like "black box," "gray box," and "white box" aren't just names. They're ways to look at a system from different angles. Each offers its own depth, each revealing something unique. Ideally, they should be combined to provide a complete picture. But even the most thorough penetration test won't save you if the conclusions remain at the level of, "Well, it doesn't seem to be working." Without fixing vulnerabilities, revising the architecture, and fixing bugs, it's just an expensive document in the "Reports" folder.
So, if you're already conducting pentests, great. If you're just planning to, start by asking yourself, "Why exactly do we need this?" But if you have no plans for pentests at all, it might be time to rethink your approach to security.
