There are several benefits for using Static Analysis Security Testing (SAST) for your software security. Having previously worked at Coverity (now Synopsys), I’m intimately familiar with the arguments in favor of using SAST. While there have been a lot of successes (such as adoption in the OSS community through Coverity SCAN), I’ve also seen challenges with organizations attempting to adopt SAST as part of their development process. Increasingly, a number of customers I’ve had discussions with are looking for SAST alternatives. Why? In theory, the ability to analyze source code and infer potential defects using SAST in the build process seems like a real step forward in improving the quality of software. However, I can think of at least six challenges to this form of analysis. This raises questions on the efficacy of SAST for organizations focused on immediate benefits.
SAST does not use the actual executable/binary for analysis; it typically uses a representation of your program. This technique, no matter how good the analysis, will always result in false positives (FPs). And it will find defects in paths that the program would never actually implement in a live system.
Why is this important? For programs that are trivial in size, FPs may be manageable, but what happens when you have code bases that are 10MLoC or more? Some of the industry’s best SAST checkers are designed to have FP rates below 5%, but if we use a common metric of 15-50 errors per 1KLoC as posited in Steve McConnell’s Code Complete, the number of potential defects identified by SAST on that 10MLoC is approximately 150k-500k defects!
Of these defects, we can typically expect approximately 7.5k - 25k to be FPs (and that’s if your SAST tool is good).
This leads to a few problems:
Given these six problems, it begs the question - does SAST effectively improve security given the rapid pace of software evolution? There are many organizations that adopt SAST simply to claim that some assurance/quality tools were used as part of their development process, especially if it means externally imposed compliance criteria. Compliance however is not security.
While there are defects that SAST excels at uncovering (think linting/configuration checks that can be performed to prevent insecure use/behavior of some functionality), SAST's problems limit its effectiveness in today's rapid mode of software development, where we’re seeing an exponential increase in source code.
The ability to analyze ever larger codebases exceeds human scale. Another approach is required.
Get a detailed breakdown of the various application security testing techniques, the strengths and weaknesses of each technique, and how each technique complements one another.
Unlike SAST which does not use the actual executable/binary for analysis, fuzzing always tests running code. Modern fuzzers autonomously generate inputs and send them to target applications for behavior verification. When target applications behave unexpectedly, this is a sign of an underlying defect. This method is effective for defect discovery because it ensures defects are not theoretical weaknesses but actual, reproducible behaviors with real consequences.
Developers benefit from fuzzing in a number of ways:
Putting all these together-- the test case, reproducer, context, and efficacy -- modern fuzzers provide developers with everything needed to remediate an identified defect. Issues identified with fuzzing are guaranteed to be actionable.
Another clear benefit of fuzzing is that testing can now happen at machine scale. Back when unit testing was introduced to the SDLC, it fundamentally changed how software was developed. Fuzzing is the next evolution. This time, however, with machines generating test cases instead of developers reasoning about all the possible edge cases within code. Rather than wasting developer time to handcraft unit tests, developers can instead create test harnesses that automatically provide bias-free inputs. These inputs discover unexpected conditions that lead to crashes. The use of fuzzing can reduce significant costs associated with maintaining tests. Advanced fuzzers dynamically generate test suites with minimal investment of developer time.
In addition, the accumulation of dynamically generated tests provide a suite of inputs that can be incorporated as part of regression testing applications prior to deployment. This makes fuzzing something that is easy to incorporate within the modern SecDevOps workflow.
Given all these benefits, a fuzzing-first strategy for most software development offers immediate benefits to the software’s security posture.
Fuzz your software before someone else fuzzes it for you. Want to learn more? Contact us for a demo here.
By submitting this form, you agree to our Terms of Use and acknowledge our Privacy Statement.