Although the term application can be used to refer to a collection of programs, ForAllSecure uses the term to refer to a single binary program.
Collection of external interface points that may be used as penetration points by an adversary. Entry points that can be exploited for unauthorized access are of most interest.
Behavior testing is the process of sending malformed inputs to a running application and observing it target for anomalous behaviors. Anomalous behavior signifies an underlying defect.
Control flow graph (CFG)
A control flow graph (CFG) is a graphical representation of all the paths that might be traversed during a program’s execution. Each node represents a statement, and an edge exists between two nodes.
A dependency is any library file, such as an output from ldd /usr/bin/program, or configuration file, such as the nginx.conf file for NGINX, needed to run an application.
Dynamic analysis is an application testing method whereby a security solution monitors the target program while executing on a real inputs. For example, valgrind is a type of dynamic analysis solution that looks for memory errors while a target runs. Contrast to static analysis.
Dynamic binary translation
Dynamic binary translation is the practice of translating, modifying, and rewriting executables from one architecture to another during runtime.
Fuzzing is the practice of mutating a chosen input A to create a new input B and then running the application with input B. The term “fuzzing” is widely credited to have been coined by Bart Miller.
Mayhem is a portfolio fuzzer. At a high level, the family of fuzzing techniques can be broken down into:
- Black-box fuzzers mutate inputs without knowledge of the program itself, typically at random. The Linux zzuff program is an example of a black-box fuzzer.
- White-box fuzzers use information about the program to derive input B from A. Mayhem includes a white-box fuzzer based on symbolic execution. Symbolic execution is a program analysis technique that uses formal computer science methods to derive the new input B by modeling how the program executes on A.
- Gray-box fuzzers use instrumentation to derive each new input. The afl fuzzer is a gray-boxed fuzzer. Mayhem also includes a gray-box fuzzer.
- Portfolio fuzzers intelligently combine the fuzzing techniques above to maximize coverage.
Harnessing is a practice whereby a user adds code to call specific routines they wish to test within their target executable, or executable-under-test. For example, OpenSSL defines several harnesses in their fuzz directory.
A problem-solving process or technique that employs practical and logical methods for resolution. Although the practice may not be perfect, the outcome is sufficient enough to reach immediate goals.
Instrumentation is a programming practice whereby developers implement code instructions to monitor specific applications within a larger software. Instrumentation enables the ability to monitor and measure an application’s performance and diagnose errors.
Intermediate representation is code that ise used internally by a compiler to represent source code.
A Mayhem configuration file.
The practice of sending unexpected inputs to a target executable to test for incorrect, anomalous, or undesirable behavior. Contrast to positive testing.
A package is a complete chroot environment for running a target application against Mayhem. A package consists of:
- A target
- Application-specific libraries and configuration files needed to run the application
- A Mayhemconfig specifying how to run Mayhem
The practice of running various types of analysis, where the type of each analysis is the same. For example, symbolic execution and fuzzing both take a program and a seed input, and output a new test case. Symbolic execution and fuzzing can be run together as a portfolio analysis.
The practice of sending expected inputs to a target executable to verify for correct or desirable behavior. project A project is a collection of targets. The user can decide which targets to include in a project. One common practice is to put all targets for a single software into one project.
The practice of disassembling and closely examining an application or software to understand how it was manufactured.
A Mayhem run (also called a job) is when Mayhem runs an application within the distributed cluster.
Seed corpus is a set of valid inputs that serve as a starting point for fuzzing a target.
Static analysis is an application testing method whereby a security solution inspects the source code of software for security flaws. Contrast to dynamic analysis.
Symbolic execution is a program analysis technique that uses formal computer science methods to determine the input that triggers a node to execute. Once determined, the valid input is used to derive invalid inputs for negative testing.
A target is an application compiled with the command line to run it. For example, compiling OpenSSL produces the openssl executable. There are three different targets for the one executable. The symbol @@ represents the file to fuzz.
openssl cms -cmsout -inform DER -in @@
openssl sha @@
openssl seed -in @@ -out /tmp/file2 -k foobar
Test coverage, also referred to as code coverage, measures how much a program has been exercised by a test suite. Mayhem uses the edge coverage metric, which measures the number of control flow graph edges that have been exercised. See control flow graph (CFG).