There's increasing talk about static analysis for vulnerability detection as a necessary development step. However, many also discuss the challenges of static analysis. This was discussed extensively at the last Positive Hack Days , and following those discussions, we already wrote about how a static analyzer works. If you've tried any serious tool, you might have been put off by long reports with confusing recommendations, complex setup, and false positives. So, is static analysis really necessary?
Our experience suggests it is. And many of the problems that arise when first looking at a tool can be easily solved. I'll try to explain what a user can do and what an analyzer should be like to ensure its use is useful, rather than "yet another unnecessary tool that security professionals demand."
So, we've already discussed the theoretical limitations of static analysis. For example, deep static analysis attempts to solve problems of exponential complexity. Therefore, each tool seeks a tradeoff between execution time, resource consumption, the number of vulnerabilities found, and the number of false positives.
Why do we need deep analysis at all? Any IDE quickly finds errors, sometimes even security-related ones, so what are exponential problems anyway? A classic example is an SQL injection (and any other injection, such as XSS, RCE, and the like) that crosses multiple functions (meaning reading user data and executing the query occur in different functions). Finding such a vulnerability requires interprocedural data flow analysis, which is a problem of exponential complexity. You'll agree that without detecting such vulnerabilities, analysis cannot be considered deep. For the same reason, the code must be analyzed as a whole, not in parts—otherwise, interprocedural vulnerabilities can be missed.
In recent years, I've gained considerable experience communicating with (potential) customers of various static analyzers. We also discuss complaints about tools based on initial use (pilot). Most complaints stem, in one way or another, from the theoretical limitations of the technology. Furthermore, tools may simply lack the functionality the user requires. However, in my opinion, analyzers can (and are) moving toward the user in addressing the issues outlined below. But you also need to know how to use analyzers, mitigating the consequences of these same issues—and it turns out, this isn't all that difficult. Let's take this step by step.
Imagine a model situation: you decide to try out a technology or choose a static analyzer—you run a pilot. Of course, you don't trust the vendor's test cases and want to try analyzing your code (at the same time, you can find real vulnerabilities and fix them). You are provided with an installer or a ready-made virtual machine with the system for a short period.
First, you need to run the analysis. You open the interface, and it seems like everything should be straightforward: upload the source code archive and click "analyze." But no: you get several forms with different fields that need to be filled out. You need to specify programming languages, some analyzer settings, select vulnerability packages (how do you know what's in them?), and so on. You pass this test, and the analysis begins. Oh, no—the scan fails. "Format does not meet requirements," "This language requires code compilation," "No files found for scanning"... If you didn't write this code yourself, you'll need to contact the developers for help.
A developer submits source code for testing
. Code build requirements deserve special attention. Most analyzers for a number of languages require that the code be built during analysis (JVM languages—Java, Scala, Kotlin, and the like, C/C++, Objective-C, C#). You can imagine the pain: reproducing the environment of a large project for a build on a new machine. On the other hand, these requirements are justified; they follow from the analysis technology and the specifics of these languages.
How do analyzers solve these problems? First, they make the analysis process as automated as possible. Ideally, you only need to upload a file of any format, and the analyzer should automatically figure out what languages it contains, how to attempt a build, and how to set the remaining default settings to ensure the results are as complete as possible. Clearly, it's impossible to anticipate every possible scenario, but you can try to handle most cases.
Build requirements should be as lenient as possible. For example, for JVM languages, there's no need to require compilation during analysis—it's enough to request the loading of artifacts, i.e., compiled code along with the source code (which is significantly simpler). For XCode, in the case of Objective-C, compilation can be automated in most cases. If compilation fails, the analyzer can attempt a partial analysis. Its results won't be as comprehensive, but it's better than no results at all. It's also convenient if the analysis module can be installed on the developer's machine, where code compilation is already configured, while the architecture should allow the remaining modules and the interface to be moved to a separate machine.
Finally, the analyzer should have the most lenient formatting requirements and handle input files automatically. Source code archives, nested archives, archives from a repository, links to a repository, archives from a production environment, and executable files from a production environment—it's good if the analyzer supports all of these.
However, it's important to remember that the analyzer isn't equipped with artificial intelligence and can't anticipate everything. Therefore, if errors occur, it's worth consulting the manual—it can provide a lot of useful information on preparing code for analysis. Furthermore, the entire process of running a scan when implementing the analyzer is only performed once for each codebase. Most often, the analyzer is integrated into the CI cycle, meaning there won't be any build issues.
Okay, the scan has started. An hour passes – no results. The progress bar hovers somewhere in the middle, with no clear percentage or estimated completion date. The second hour passes – the progress has moved 99 percent and has been there for half an hour. The third hour passes – and the analyzer crashes, reporting insufficient RAM. Or it hangs for another hour and terminates. You might expect the analysis to run as fast as your checkstyle, but here your expectations diverge greatly from reality.
Yes, a good static analyzer can consume a fair amount of resources. I mentioned one reason above: finding complex vulnerabilities is an exponentially difficult task. So, the more resources and the more time it takes, the better the results will be (given a good engine, of course). Both analysis time and resource requirements are truly difficult to predict – the running time of static analysis algorithms depends heavily on language constructs, code complexity, and call depth – characteristics that are difficult to predict in advance.
Resource issues are a necessary evil. You need to be careful about allocating the necessary resources, patiently wait for the scan to complete, and understand that no one can accurately predict the analyzer's resource requirements, even for a given codebase, and you need to be prepared for these parameters to change. Moreover, the required parameters can change even without updating the codebase – due to analyzer updates.
However, an analyzer can still help with this issue. It can separate the resource-intensive part (engines) and the interface across different machines. This will prevent machines from being overloaded with unnecessary programs that would slow them down, while still allowing the system interface to be used regardless of scan load (for example, to view and edit results). It will also allow for easy scaling without reinstalling the entire system (run the analyzer on a new virtual machine, specify the IP address of the main machine, and voila).
Furthermore, the analyzer can allow you to select the analysis depth, disable heavy-duty checks, and use incremental analysis (which checks only changed code, not all code). These features should be used with caution, as they can significantly impact scan results. If you use this functionality, it is recommended to run a full analysis periodically.
Let's move on to the scan results (it's been a while). You anxiously await the number of vulnerabilities in the analyzer window, and are very surprised when you see it. 156 critical, 1260 medium, and 3210 low. You go to the results page and are overwhelmed by the number of issues found. You download the PDF report and see several thousand pages of text. Guess what a code developer would say upon seeing such a canvas?
A security professional delivers a vulnerability report to a developer.
But let's try looking at the results anyway, give it a chance. After examining several dozen occurrences more closely, you begin to understand why there are so many vulnerabilities. A few of them actually look serious, and you realize they need to be fixed. However, you immediately find a dozen false positives. And then there's a huge number of vulnerabilities in the library code. You're not going to fix the libraries! And then you realize how much time you'll spend parsing the results. And this procedure needs to be repeated every day, week, or at least every release. (Not really.)
Let's start with the fact that false positives can be understood in very different ways. Some won't consider only critical vulnerabilities that can be exploited right now as false positives. Others will consider only outright analyzer errors as false positives. Much depends on what you expect from the tool. We recommend reviewing virtually all occurrences, as even a low-level vulnerability that can't be exploited today could become a serious problem tomorrow—for example, if the code or external conditions change. While
reviewing all occurrences is certainly necessary, it still requires a significant amount of work. This is where analyzers can be extremely helpful. A key feature of an analyzer is its ability to track vulnerabilities between scans of a single project, while also being resilient to minor changes typical of code development. This eliminates the need to repeat a lengthy vulnerability review: the first time, you'll spend more time removing false positives and adjusting the severity of occurrences, but subsequently, you'll only need to review new vulnerabilities, which will be significantly fewer.
Okay, but is it really necessary to review all vulnerabilities the first time? We recommend it, but generally speaking, it's not necessary. First, analyzers allow you to filter results by directories and files: for example, when running a scan, you can immediately exclude certain components, libraries, or test code from analysis. This will also impact the speed of the analysis. Secondly, analyzers allow you to filter results by vulnerability, meaning you can limit the set of vulnerabilities when starting a scan. Finally, in addition to severity, the analyzer can also provide a metric such as the probability of a vulnerability being false positive (i.e., its confidence in a given vulnerability). Using this metric, you can filter results.
Software Composition Analysis (SCA) is also worth mentioning (it's now being supported by a growing number of tools at various levels). This technology can detect library usage in your code, identify names and versions, display known vulnerabilities, and even licenses. This technology can separate library code from your own, which also allows you to filter results.
It turns out that the problem of overwhelming analysis results can be addressed, and it's not particularly difficult. While the initial scan of results may indeed take time, subsequent rescans will consume less and less time. However, I'd like to emphasize again that any filtering of results should be approached with caution—you could miss a vulnerability. Even if a library is known, that doesn't mean it's free of vulnerabilities. If a given vulnerability is currently poorly detected (meaning the tool displays many false positives for it), and you disable it, you could miss a real vulnerability when updating the analyzer.
We've dealt with the large report and the false positives. But you want to go further—make sure the analyzer is finding the vulnerabilities you know for sure exist (you may have intentionally introduced them, or another tool may have found them).
First, it's important to understand that the analyzer could fail to find a vulnerability for various reasons. The simplest is that the scan was configured incorrectly (pay attention to error messages). But from an analysis technology perspective, the causes can also vary. A static analyzer consists of two important components: the engine (which houses all the algorithmic complexity and mathematics) and the vulnerability search rule base. One situation is when the engine can detect a vulnerability of a given class, but the vulnerability isn't in the rule base. In this case, adding a rule is usually straightforward. A completely different situation is when the engine doesn't support such a vulnerability at all—in this case, the modifications can be quite significant. I gave the example at the beginning of the article: you'll never find an SQL injection without data flow analysis algorithms.
A static analyzer must implement a set of algorithms in its engine that cover the available vulnerability classes for a given programming language (control flow analysis, data flow analysis, interval analysis, etc.). An important feature is the ability to add custom vulnerability search rules to the tool—this will eliminate the primary cause of a missed vulnerability.
Therefore, if you don't find an existing vulnerability in the scan results, you first need to understand the reason for the missed vulnerability—the vendor can usually help with this. If the cause lies in the rule base or the scan configuration, the situation can be resolved fairly easily. Most importantly, evaluate the depth of analysis, that is, what the engine can actually search for.
Having read this article up to this point, one might assume that working with this tool requires in-depth developer expertise, as it's essential to understand which alarms are true and which are false positives. In my opinion, it all depends on how user-friendly the tool is. If it provides user-friendly and intuitive functionality, clear vulnerability descriptions with examples, links, and recommendations in multiple languages, and if the tool displays traces for data flow analysis vulnerabilities, then working with it won't require in-depth developer expertise or a thorough understanding of all the intricacies of the programming language and frameworks. However, a minimum background in development is still required to read the code.
At the end of this article, we'll briefly touch on one of the most important aspects of tool use, and we'll examine it in detail in subsequent articles. Let's say you've decided to use a static analyzer. However, you have an established development process, both technological and organizational, and you don't want to change it (and no one will).
The tool should have a fully functional non-graphical interface (for example, a CLI or REST API) that allows you to integrate the analyzer into any of your processes. It's helpful if the analyzer has ready-made integrations with various components: plugins for IDEs or build systems, integrations with version control systems, plugins for CI/CD servers (Jenkins, TeamCity), integrations with project management systems (JIRA), or user management systems (Active Directory).
Integrating static analysis into the development process (the so-called SDLC) is the most effective approach if the process is well-established and all participants agree on and understand the purpose of it. Continuously analyzing the code after changes or analyzer updates will help identify vulnerabilities as early as possible. Separating the roles of developers and information security specialists, clearly defining information security requirements, and seamlessly integrating them into the existing process (for example, initially, the system will be advisory in nature) will ensure the tool is used smoothly and effectively. However, manual use of the tool is still necessary if your development model doesn't allow for such a process.
This article provides basic recommendations for getting started with a static analyzer. A good analyzer performs far better than any lightweight checker; it detects problems of a fundamentally different complexity. Therefore, it's important to consider the specific characteristics of static analysis as a technology, but at the same time, choose a specific tool whose functionality mitigates these characteristics as much as possible.
Our experience suggests it is. And many of the problems that arise when first looking at a tool can be easily solved. I'll try to explain what a user can do and what an analyzer should be like to ensure its use is useful, rather than "yet another unnecessary tool that security professionals demand."
About static analysis
So, we've already discussed the theoretical limitations of static analysis. For example, deep static analysis attempts to solve problems of exponential complexity. Therefore, each tool seeks a tradeoff between execution time, resource consumption, the number of vulnerabilities found, and the number of false positives.
Why do we need deep analysis at all? Any IDE quickly finds errors, sometimes even security-related ones, so what are exponential problems anyway? A classic example is an SQL injection (and any other injection, such as XSS, RCE, and the like) that crosses multiple functions (meaning reading user data and executing the query occur in different functions). Finding such a vulnerability requires interprocedural data flow analysis, which is a problem of exponential complexity. You'll agree that without detecting such vulnerabilities, analysis cannot be considered deep. For the same reason, the code must be analyzed as a whole, not in parts—otherwise, interprocedural vulnerabilities can be missed.
In recent years, I've gained considerable experience communicating with (potential) customers of various static analyzers. We also discuss complaints about tools based on initial use (pilot). Most complaints stem, in one way or another, from the theoretical limitations of the technology. Furthermore, tools may simply lack the functionality the user requires. However, in my opinion, analyzers can (and are) moving toward the user in addressing the issues outlined below. But you also need to know how to use analyzers, mitigating the consequences of these same issues—and it turns out, this isn't all that difficult. Let's take this step by step.
Imagine a model situation: you decide to try out a technology or choose a static analyzer—you run a pilot. Of course, you don't trust the vendor's test cases and want to try analyzing your code (at the same time, you can find real vulnerabilities and fix them). You are provided with an installer or a ready-made virtual machine with the system for a short period.
Launching the analysis
First, you need to run the analysis. You open the interface, and it seems like everything should be straightforward: upload the source code archive and click "analyze." But no: you get several forms with different fields that need to be filled out. You need to specify programming languages, some analyzer settings, select vulnerability packages (how do you know what's in them?), and so on. You pass this test, and the analysis begins. Oh, no—the scan fails. "Format does not meet requirements," "This language requires code compilation," "No files found for scanning"... If you didn't write this code yourself, you'll need to contact the developers for help.
A developer submits source code for testing
. Code build requirements deserve special attention. Most analyzers for a number of languages require that the code be built during analysis (JVM languages—Java, Scala, Kotlin, and the like, C/C++, Objective-C, C#). You can imagine the pain: reproducing the environment of a large project for a build on a new machine. On the other hand, these requirements are justified; they follow from the analysis technology and the specifics of these languages.
How do analyzers solve these problems? First, they make the analysis process as automated as possible. Ideally, you only need to upload a file of any format, and the analyzer should automatically figure out what languages it contains, how to attempt a build, and how to set the remaining default settings to ensure the results are as complete as possible. Clearly, it's impossible to anticipate every possible scenario, but you can try to handle most cases.
Build requirements should be as lenient as possible. For example, for JVM languages, there's no need to require compilation during analysis—it's enough to request the loading of artifacts, i.e., compiled code along with the source code (which is significantly simpler). For XCode, in the case of Objective-C, compilation can be automated in most cases. If compilation fails, the analyzer can attempt a partial analysis. Its results won't be as comprehensive, but it's better than no results at all. It's also convenient if the analysis module can be installed on the developer's machine, where code compilation is already configured, while the architecture should allow the remaining modules and the interface to be moved to a separate machine.
Finally, the analyzer should have the most lenient formatting requirements and handle input files automatically. Source code archives, nested archives, archives from a repository, links to a repository, archives from a production environment, and executable files from a production environment—it's good if the analyzer supports all of these.
However, it's important to remember that the analyzer isn't equipped with artificial intelligence and can't anticipate everything. Therefore, if errors occur, it's worth consulting the manual—it can provide a lot of useful information on preparing code for analysis. Furthermore, the entire process of running a scan when implementing the analyzer is only performed once for each codebase. Most often, the analyzer is integrated into the CI cycle, meaning there won't be any build issues.
The analysis process
Okay, the scan has started. An hour passes – no results. The progress bar hovers somewhere in the middle, with no clear percentage or estimated completion date. The second hour passes – the progress has moved 99 percent and has been there for half an hour. The third hour passes – and the analyzer crashes, reporting insufficient RAM. Or it hangs for another hour and terminates. You might expect the analysis to run as fast as your checkstyle, but here your expectations diverge greatly from reality.
Yes, a good static analyzer can consume a fair amount of resources. I mentioned one reason above: finding complex vulnerabilities is an exponentially difficult task. So, the more resources and the more time it takes, the better the results will be (given a good engine, of course). Both analysis time and resource requirements are truly difficult to predict – the running time of static analysis algorithms depends heavily on language constructs, code complexity, and call depth – characteristics that are difficult to predict in advance.
Resource issues are a necessary evil. You need to be careful about allocating the necessary resources, patiently wait for the scan to complete, and understand that no one can accurately predict the analyzer's resource requirements, even for a given codebase, and you need to be prepared for these parameters to change. Moreover, the required parameters can change even without updating the codebase – due to analyzer updates.
However, an analyzer can still help with this issue. It can separate the resource-intensive part (engines) and the interface across different machines. This will prevent machines from being overloaded with unnecessary programs that would slow them down, while still allowing the system interface to be used regardless of scan load (for example, to view and edit results). It will also allow for easy scaling without reinstalling the entire system (run the analyzer on a new virtual machine, specify the IP address of the main machine, and voila).
Furthermore, the analyzer can allow you to select the analysis depth, disable heavy-duty checks, and use incremental analysis (which checks only changed code, not all code). These features should be used with caution, as they can significantly impact scan results. If you use this functionality, it is recommended to run a full analysis periodically.
Results of the analysis
Let's move on to the scan results (it's been a while). You anxiously await the number of vulnerabilities in the analyzer window, and are very surprised when you see it. 156 critical, 1260 medium, and 3210 low. You go to the results page and are overwhelmed by the number of issues found. You download the PDF report and see several thousand pages of text. Guess what a code developer would say upon seeing such a canvas?
A security professional delivers a vulnerability report to a developer.
But let's try looking at the results anyway, give it a chance. After examining several dozen occurrences more closely, you begin to understand why there are so many vulnerabilities. A few of them actually look serious, and you realize they need to be fixed. However, you immediately find a dozen false positives. And then there's a huge number of vulnerabilities in the library code. You're not going to fix the libraries! And then you realize how much time you'll spend parsing the results. And this procedure needs to be repeated every day, week, or at least every release. (Not really.)
Let's start with the fact that false positives can be understood in very different ways. Some won't consider only critical vulnerabilities that can be exploited right now as false positives. Others will consider only outright analyzer errors as false positives. Much depends on what you expect from the tool. We recommend reviewing virtually all occurrences, as even a low-level vulnerability that can't be exploited today could become a serious problem tomorrow—for example, if the code or external conditions change. While
reviewing all occurrences is certainly necessary, it still requires a significant amount of work. This is where analyzers can be extremely helpful. A key feature of an analyzer is its ability to track vulnerabilities between scans of a single project, while also being resilient to minor changes typical of code development. This eliminates the need to repeat a lengthy vulnerability review: the first time, you'll spend more time removing false positives and adjusting the severity of occurrences, but subsequently, you'll only need to review new vulnerabilities, which will be significantly fewer.
Okay, but is it really necessary to review all vulnerabilities the first time? We recommend it, but generally speaking, it's not necessary. First, analyzers allow you to filter results by directories and files: for example, when running a scan, you can immediately exclude certain components, libraries, or test code from analysis. This will also impact the speed of the analysis. Secondly, analyzers allow you to filter results by vulnerability, meaning you can limit the set of vulnerabilities when starting a scan. Finally, in addition to severity, the analyzer can also provide a metric such as the probability of a vulnerability being false positive (i.e., its confidence in a given vulnerability). Using this metric, you can filter results.
Software Composition Analysis (SCA) is also worth mentioning (it's now being supported by a growing number of tools at various levels). This technology can detect library usage in your code, identify names and versions, display known vulnerabilities, and even licenses. This technology can separate library code from your own, which also allows you to filter results.
It turns out that the problem of overwhelming analysis results can be addressed, and it's not particularly difficult. While the initial scan of results may indeed take time, subsequent rescans will consume less and less time. However, I'd like to emphasize again that any filtering of results should be approached with caution—you could miss a vulnerability. Even if a library is known, that doesn't mean it's free of vulnerabilities. If a given vulnerability is currently poorly detected (meaning the tool displays many false positives for it), and you disable it, you could miss a real vulnerability when updating the analyzer.
Let's check the analyzer
We've dealt with the large report and the false positives. But you want to go further—make sure the analyzer is finding the vulnerabilities you know for sure exist (you may have intentionally introduced them, or another tool may have found them).
First, it's important to understand that the analyzer could fail to find a vulnerability for various reasons. The simplest is that the scan was configured incorrectly (pay attention to error messages). But from an analysis technology perspective, the causes can also vary. A static analyzer consists of two important components: the engine (which houses all the algorithmic complexity and mathematics) and the vulnerability search rule base. One situation is when the engine can detect a vulnerability of a given class, but the vulnerability isn't in the rule base. In this case, adding a rule is usually straightforward. A completely different situation is when the engine doesn't support such a vulnerability at all—in this case, the modifications can be quite significant. I gave the example at the beginning of the article: you'll never find an SQL injection without data flow analysis algorithms.
A static analyzer must implement a set of algorithms in its engine that cover the available vulnerability classes for a given programming language (control flow analysis, data flow analysis, interval analysis, etc.). An important feature is the ability to add custom vulnerability search rules to the tool—this will eliminate the primary cause of a missed vulnerability.
Therefore, if you don't find an existing vulnerability in the scan results, you first need to understand the reason for the missed vulnerability—the vendor can usually help with this. If the cause lies in the rule base or the scan configuration, the situation can be resolved fairly easily. Most importantly, evaluate the depth of analysis, that is, what the engine can actually search for.
Competencies
Having read this article up to this point, one might assume that working with this tool requires in-depth developer expertise, as it's essential to understand which alarms are true and which are false positives. In my opinion, it all depends on how user-friendly the tool is. If it provides user-friendly and intuitive functionality, clear vulnerability descriptions with examples, links, and recommendations in multiple languages, and if the tool displays traces for data flow analysis vulnerabilities, then working with it won't require in-depth developer expertise or a thorough understanding of all the intricacies of the programming language and frameworks. However, a minimum background in development is still required to read the code.
Integration into the development process
At the end of this article, we'll briefly touch on one of the most important aspects of tool use, and we'll examine it in detail in subsequent articles. Let's say you've decided to use a static analyzer. However, you have an established development process, both technological and organizational, and you don't want to change it (and no one will).
The tool should have a fully functional non-graphical interface (for example, a CLI or REST API) that allows you to integrate the analyzer into any of your processes. It's helpful if the analyzer has ready-made integrations with various components: plugins for IDEs or build systems, integrations with version control systems, plugins for CI/CD servers (Jenkins, TeamCity), integrations with project management systems (JIRA), or user management systems (Active Directory).
Integrating static analysis into the development process (the so-called SDLC) is the most effective approach if the process is well-established and all participants agree on and understand the purpose of it. Continuously analyzing the code after changes or analyzer updates will help identify vulnerabilities as early as possible. Separating the roles of developers and information security specialists, clearly defining information security requirements, and seamlessly integrating them into the existing process (for example, initially, the system will be advisory in nature) will ensure the tool is used smoothly and effectively. However, manual use of the tool is still necessary if your development model doesn't allow for such a process.
Resume
This article provides basic recommendations for getting started with a static analyzer. A good analyzer performs far better than any lightweight checker; it detects problems of a fundamentally different complexity. Therefore, it's important to consider the specific characteristics of static analysis as a technology, but at the same time, choose a specific tool whose functionality mitigates these characteristics as much as possible.
