Fuzzy about fuzz testing? This fuzzing tutorial will help

Automation and application security

Application security has historically been hard for both security pros, who must achieve testing and implementation at an ever-increasing cadence, and developers, who fear security processes will cause bottlenecks and roadblocks. A major contributing factor is that the processes and methodologies employed by security teams are often hard to automate, thereby forcing organizations to rely more on personnel skill even as they juggle time constraints.

Consider application threat modeling, which has traditionally served as one of the most versatile and valuable tools in the application security arsenal. Yet, it relies heavily on human expertise to create the requisite data flows, perform the analysis on the intercomponent interaction points and evaluate the impact of the exploitation potential of threat areas discovered.

Application penetration testing also depends on human ingenuity. Consider the test conditions in the "OWASP Web Security Testing Guide" -- not many are viable without a human engineer to manually drive them.

I've deliberately cherry-picked some of the more difficult processes to automate here, but the truth is that even lower-touch approaches, such as static application security testing and dynamic application security testing, rely heavily on human intervention. Consider the effort and expertise required to separate false positives from true ones, ensure appropriate coverage, and create and implement appropriate fixes.

Given this, security teams want approaches that automate elements of the application security process. As the development teams and product areas security teams support become more automated, the pressure is naturally on to respond in kind. What separates the highest-performing application security teams from the merely average are two things:

1. The relationships forged with development partners -- arguably always the most important element.
2. Security teams' ability to automate.

What is fuzz testing?

Fuzz testing, also known as fuzzing, is a fully or partly automated software testing technique that involves exercising the input functionality of an app by sending it many iterations and permutations of test data.

Fuzzing showers a test program with these random inputs to determine the program's stability. Those inputs can come in any form, including edit controls, command-line parameters or API endpoint JSON inputs. It's anything a user can control.

The goal of fuzz testing is to find inputs that generate unexpected glitches, such as crashes, unanticipated behaviors or other anomalous results. The aim is to see how the software reacts to invalid and malicious code and thereby to detect any cybersecurity vulnerabilities or coding errors. Security teams later analyze the subset of input that caused unexpected behaviors to debug and remediate any issues.

Fuzzing helps detect issues that are difficult to discover using other methods. The OWASP testing guide, for example, explains the benefits fuzz testing brings to web application and API security testing -- in particular, detecting attacks such as cross-site scripting (XSS), buffer overflows, format string errors, and SQL and other injection attacks. Inputting these attacks -- and their various iterations and permutations -- manually to test an application is an immense, time-consuming task. Fuzzing handles the testing portion of the job for security teams, leaving them responsible only for analyzing a test's results -- i.e., the fuzzed data.

And, because it operates iteratively over an application, fuzz testing is relatively easy to automate within the constraints of a given input format, helping streamline the release process.

Types of fuzzing

Fuzz testing comes in a variety of forms and flavors, and in the quarter century since the technique originated, organizations have found ways to fine-tune it even further. For this article, blue team fuzz testing is the focus. Let's look at some common fuzzing strategies.

How a fuzzer generates output to test an application is one way to categorize the test. Fuzzing peppers the application inputs with data, so input is critical. Construction of input data can be generation-based or mutation-based. It is useful to understand the distinctions because tools generate data in different ways:

• Generation-based fuzzing is when input data is based on a detailed and comprehensive understanding of expected values as derived from the specific protocol or defined format. Also known as smart fuzzers, generation-based fuzzers can be used, for example, to send a variety of potential inputs to a web form to locate reflected XSS on that form.
  
  Random inputs are not useful. Rather, you want output that has a chance of causing that condition to occur.
• Mutation-based fuzzing lacks understanding of the application's structure. Mutation-based fuzzers, also known as dumb fuzzers, use seeds -- files of predefined inputs in the tool -- derived from known-good input that are expanded upon to create many related data inputs to supply to the application under test.

This model produces more possible inputs, but they are less likely to conform to a specific protocol or set of constraints.

Another differentiator is how much of the targeted software the fuzzer is aware of. As a general rule, the greater the application functionality you can test -- i.e., the more paths through the software -- the more likely you are to encounter an error condition. The different levels of awareness are as follows:

• Black box fuzzing has no knowledge of the underlying software implementation.
• White box fuzzing is typically used in combination with other tools to maintain an awareness of what application paths are being exercised. As a result, it can adapt input to get as close to complete code coverage as possible.
• Gray box fuzzing sits in the middle. It might use techniques such as software instrumentation, performance data or dependency tracing to intuit what paths are being exercised.

Fuzz testing is also categorized by what is being fuzzed:

• Application fuzzing uses a fuzzer to test the various inputs and outputs of a given application.
• Protocol fuzzing uses a fuzzer to craft packets.
• File format fuzzing uses a fuzzer to construct files that are then used as inputs.

How to get started with fuzzing

When it comes to selecting your fuzzing targets and methodology, circumstances largely dictate which approach works best.

Step 1. Is fuzzing necessary?

The first step is to determine if you want to fuzz. Many security teams complement pen tests and other human-led investigations with a fuzz test for thoroughness. Fuzzing helps strategically root out unwanted behavior by exercising the resiliency of input filtering methodologies or the functionality of API endpoints. The same is true when evaluating desktop applications. You might want to ensure, for example, that attackers cannot elevate privilege by attacking a desktop application.

Step 2. What are the targets?

The next step is to select your targets. Choice of targets comes down to why you've decided to fuzz in the first place and, from there, how the application is built. In the case of API endpoints or desktop apps mentioned above, the targets are obvious.

Most of the time, the underlying test rationale for security practitioners is to ensure application resiliency. As a result, the type of application and its construction dictate your targets. If it is a web application, targets include interaction points between the user and/or components where data is exchanged. If you threat-modeled an application, the interaction points noted on your data flow diagram are a useful shortcut to understand where to fuzz. If it's a desktop application, the same is true: Targets include inputs to the application itself and any interconnections to outside interfaces -- for example, API endpoints, shared libraries, etc.

Step 3. Which tools to use?

What testing tools to use is heavily influenced by the previous two steps. Consider the following:

• If you are testing a web application's input fields, for example, an open source tool, such as OWASP Zed Attack Proxy (ZAP), or a commercial counterpart, such as Burp Suite's Intruder, might be useful. Chances are you are working in that tool already and the fuzzing capabilities are tuned for that usage.
• If you are testing a Java application in an automated security testing approach, you might find a tool such as Jazzer to be beneficial because of its ability to analyze code coverage.
• If you're testing a C/C++ command-line utility, something like American Fuzzy Lop could be useful.

Again, what you are doing and what you hope to achieve heavily influence the fuzzer you choose.

Fuzzing is a fantastic way to help automate security testing and ensure you get solid coverage of the testing you perform. It's also particularly useful in web contexts to ensure appropriate input validation is done. Fuzzing is a powerful tool in your arsenal; learning how to use it can be incredibly beneficial.

Fuzz testing steps

After deciding fuzzing is necessary, determining your targets and choosing your tools, it's time to conduct the fuzz test. This involves the following steps:

1. Generate fuzz input. Determine which inputs to test with.
2. Test with fuzz input. Conduct the test by inputting the code into the application.
3. Monitor and analyze system response. Assess how the program responds to the inputs to identify issues in code.
4. Log vulnerabilities. Document the inputs and responses. Share with developers to get the code updated.

Fuzzing tutorial video

As mentioned, security practitioners have multiple fuzzing tools from which to choose. The following video demonstrates how to fuzz-test Damn Vulnerable Web App (DVWA), part of OWASP's Broken Web Application Project, using the fuzzing feature of OWASP's ZAP web app scanner.

Watch for a step-by-step example of fuzzing for SQL injections in DVWA with ZAP in a use case appropriate during an application pen test.

Ed Moyle is a technical writer with more than 25 years of experience in information security. He is currently the systems and software security director at Drake Software.