Whether you’re researching SAST, DAST, SCA, RASP, one thing is for certain: eventually it all starts to blur into a game of anagram. If you’re struggling to understand how to get your application security testing program off the ground, you wouldn’t be the first. Find comfort in knowing that this is a common struggle.
In this blog, we’ll walk through the spectrum of risk and the types of solutions that are strongest at addressing each risks.
In a 2002 US Department of Defense news briefing, Secretary of Defense Donald Rumsfeld brought much fame and attention to the strategic planning concepts known as the Johari window, “known knowns”, “known unknowns”, and “unknown unknowns”, to explain the lack of evidence linking the government of Iraq with the supply of weapons of mass destruction to terrorist groups.
Rumsfeld leveraged the Johari window to assess US relations with themselves and others. The cyber security market has found that this principle is applicable in software security risk management as well.
The Johari window contains four quadrants as outlined below:
Known to Self |
Unknown to Self |
|
Known to Other |
“Arena” |
“Blind Spot” |
Unknown to Other |
“Facade” |
“Unknown” |
When aligned with the types of vulnerabilities, the chart looks as follows:
Known to Self |
Unknown to Self |
|
Known to Other |
KNOWN VULNERABILITIES (CVEs)
“Arena” |
DEFECTS (CWEs)
“Blind Spot” |
Unknown to Other |
ZERO-DAY VULNERABILITY “Facade” |
UNKNOWN VULNERABILITY “Unknown” |
Figure 1: When combining terms, we reference each state as: “Other” + “Self”. For example the Arena (top left) quadrant is referred to as Known Known, whereas the Facade (bottom left) quadrant is known as Unknown Known. Green quadrants represent easily identified and address risk, yellow quadrants represent moderately challenging risk to identify and address, and red quadrants represent high difficulty in identifying and addressing risk.
Let’s break down these types of vulnerabilities, what they mean, and how to address them.
In order to address the full spectrum of risks you will need three tools: software composition analysis, static analysis, and guide fuzz testing. However, an organization implementing an application security program for the first time may not be looking for the whole enchilada, just the starter package.
To address the full spectrum of risks in the least amount of tooling, we recommend software composition analysis and guide fuzz testing. We also posit that the first implementation should be guide fuzz testing, then software composition analysis. Here’s why.
Let’s start with the more obvious use case. SCA tools can quickly and easily scan and identify Known Known risks. Vendor options include BlackDuck, Snyk, WhiteSource, and SourceClear.
Consider SCA much like the “ctrl + f” function on a Word document. For example, let’s say that you realized you specified “John” in your document rather than “Emily”, and -- drats -- you only realized this after you finished writing your love letter! It’s time to ctrl +f. Easy. Done. Now that you’ve found all the “John”s, you can go in and manually replace (Whoops! Looks like this Word doc version doesn’t come with an automated “Replace” function. ¯\_(ツ)_/¯ ) “John” with “Emily”. But, drats again! You didn’t pay attention and did a blind replace, so now you’re not sure if, in some instances, it made sense to keep “John”.
SCA has a similar experience. This tool is helpful in that it will quickly identify every vulnerable library and tell you which patched version to upgrade to. However, it won’t be able to tell you whether the library is invoked or reachable -- meaning while you may have a vulnerable library, it might not pose a risk at all. And, even if your developers do upgrade to the new version, you won’t know if it addressed the vulnerability without introducing new issues until regression and integration testing occurs.
Regardless of its limitations, SCA still proves to be the strongest at addressing the Known Known risk of any technique available today.
Get a detailed breakdown of the various application security testing techniques, the strengths and weaknesses of each technique, and how each technique complements one another.
Now, for the less obvious use case. Guided fuzz testing is unique from all other testing techniques in that it is able to identify all the different types of unknown risks -- whether it be unknown to others and/or unknown to self -- allowing organizations to get ahead of the threat landscape.
Yes, you read that correctly. Guided fuzz testing is able to address Unknown Unknown, Unknown Known, and Known Unknown risks. I want to clarify that my former statement that static analysis is the most popular tool for addressing Known Unknowns still stands. However, we’re seeing the market open to address this risk, especially as guided fuzz testing grows mainstream and continues to demonstrate its success in this quadrant.
Consider SAST much like a grammar checker function in a Word document. For example, let’s say that the love letter you’re about to write is the first document you’ve written in a decade. In primary school, you learned that understanding the rules of English grammar sets the foundation for apt writing. So, with the grammar checker on your side, you’re feeling confident. However, as you work your way into the second sentence, you find yourself muttering to yourself grammar rhymes like “I before E except after C”. Now, you’re confused. The English language is all about exceptions and the proper use of grammar depends on context. Maybe understanding grammar is only a single pillar in strong writing, but not the foundation. Coding works similarly; The applicability of coding rules largely depends on context. This is where SAST falls apart. While the logic behind SAST is in the right place, it unfortunately goes awry in execution. Static testing directly analyzes the code for coding weaknesses patterns. Because SAST is conducted on applications while they’re in a non-running state, it can only blindly apply coding best practices. SASTs have zero context into how the application will behave in production environments, and, as a result, frequently produce false-positives. These false-positives have desensitized developers to fix request notifications and introduced strain between security and development teams, propelling the notion that security is a productivity blocker for developers. As a result, SAST has grown to have a poor reputation, motivating organizations to explore other solutions.
You may predict that the reason we assert guided fuzz testing should be your first application security testing investment is because it is able to address three of the four types of risk. Wrong. It’s actually able to uncover Known Knowns too -- meaning guided fuzz testing is able to address the full spectrum of risks. However, it should be noted that its strength remains in the Unknown space.
In application security, we believe that vendors should select the solution that is best qualified to address each risk. While organizations can rely on guided fuzz testing to address Known Known vulnerabilities, we recommend organizations eventually consider moving forward with an SCA to drive greater efficiency as a part of their devops workflows. The benefits of guided fuzzing doesn't stop there. SCA and guided fuzz testing offer synergistic value. Guided fuzz testing is able to address some of SCA limitations, such as validating no new issues were introduced as a part of its regression testing. Thus, when guided fuzzing is implemented first, then SCA, many of the aforementioned limitations are mitigated.
Interested in getting started with fuzz testing? Unfortunately, it can be tough to navigate the different types of fuzzers -- from random to template to protocol to guided -- in the market. But, you’re in luck. We’re here to help.
To figure out which fuzzer is right for you, check out this Fuzz Testing Checklist to help build a buyer’s criteria for your organization and environment.
Not quite there yet? No problem. Check out the What is Advanced Fuzz Testing whitepaper to learn more about the different types of fuzz testing.
Not sure about the value of fuzz testing yet? Organizations that take the leap to implement a fuzz testing program don't rip it out. Find out the economical reasons that keep them loving their fuzz testing programs in the Fuzz Testing ROI Framework.
By submitting this form, you agree to our Terms of Use and acknowledge our Privacy Statement.