Speed up Python Fuzzing

20 Dec 2014

When fuzzing an application, Python is a very handy language since it allows for fast customisation of fuzzing parameters. Furthermore there exist many frameworks for interfacing with for example the network (think Scapy) or other external programs. The disadvantage of using Python for fuzzing is that it is generally slower than using a native language like C. The reason for that is that C is compiled in native code while Python (even Python byte-code) needs an interpreter to execute. In this blog post I demonstrate a method for speeding up fuzzing when using Python, without losing much of the benefits of the language.

Consider the following Python program:

import os


devnull = open(“/dev/null”, “w”)
def run():
        pid = os.fork()
        if pid == 0:
                args = [“/usr/bin/tar”, “-h”]
                os.dup2(devnull.fileno(), 1)
                os.execvp(“/usr/bin/tar”, args)
        else:
                os.waitpid(pid, 0)

if name == “main“:
        for x in range(0,1000):
                run()

This is a simple program to display the help of the tar program natively present on many linux systems. I execute this program on my 2011 MacBook Pro with SSD upgrade, using python 2.7.6. I use the native tar program included with the distribution.

Timing results in: macmax:fuzz max$ time python test.py real 0m2.213s user 0m0.960s sys 0m1.280s

To execute the tar help 1000 times, takes 2.213 seconds. That is approximately 452 executions per second.

Now consider the following program in C which performs the exact same task:

#include
#include
#include <sys/types.h>
#include <sys/stat.h>
#include


void run(int devnull) {
        int pid = fork();
        int status = 0;
        if (pid == 0) {
                dup2(devnull, 1);
                execl(“/usr/bin/tar”, “/usr/bin/tar”, “-h”, (char*)0);
        } else {
                waitpid(pid, &status, 0);
        }
}

int main() {
        int x=0;
        int devnull = open(“/dev/null”, O_WRONLY);
        for(x=0;x<1000;x++) run(devnull);
        return 0;
}

Now, this code obviously has to be compiled first, losing all of the benefits of being able to edit it quickly as in Python. Running this program gives us the following timing:

macmax:fuzz max$ time ./test


real 0m1.564s
user 0m0.751s
sys 0m0.768s

To execute the tar help via C, 1000 times needs 1.564 seconds which is about 1,5 times as fast as Python. This amounts to approximately 639 executions per second compared to only 452 using Python.

What if we could maintain the flexibility of a Python script, while having the speed increase C has to offer? We can!

One of the reasons you might think of that makes Python fuzzing slow is that the Python interpreter has to be started. However, if we increase the amount of iterations the fuzzer runs on, we should see an increase in iterations per second. However, this is not the case. Increasing the iterations to 20k results in 439 executions per second (compared to 452 when using 1k iterations). There is no increase and the amount of iterations is roughly the same. Therefore we can refute this hypothesis.

So… What if we had a faster way of spawning a process via Python (in our case the tar process). Luckily there is, it is called vfork. The following code was obtained via mwrinfosecurity.com blog:

import ctypes
import os


class PosixSpawn():
        def init(self):
                self.libc = ctypes.cdll.LoadLibrary(“libc.dylib”)
                self._posix_spawn = self.libc.posix_spawn
                self._posix_spawn.restype = ctypes.c_int
                self._posix_spawn.argtypes = (
                        ctypes.POINTER(ctypes.c_int),
                        ctypes.c_char_p, ctypes.c_void_p, ctypes.c_void_p,
                        ctypes.POINTER(ctypes.c_char_p),
                        ctypes.POINTER(ctypes.c_char_p)
                )
                # dirty hack: hardcoded struct sizes
                self.attrs = self.libc.malloc(336)
                self.actions = self.libc.malloc(80)
                self.devnull = open(“/dev/null”,“wb”)
                self.env = [x+“=”+os.environ[x] for x in os.environ] + [ 0 ]

    def execute(self, exe, args):
            pid = ctypes.c_int()
            args = [exe] + args + [ 0 ]
            argv = (ctypes.c_char_p * 5) (*args)
            env = (ctypes.c_char_p * ( len(self.env) ))(*self.env)
            self.libc.posix_spawnattr_init(self.attrs)
            self.libc.posix_spawnattr_setflags(self.attrs, 0x40)
            self.libc.posix_spawn_file_actions_init(self.actions)
            self.libc.posix_spawn_file_actions_adddup2(self.actions, self.devnull.fileno(), 1)
            self._posix_spawn(ctypes.byref(pid), ctypes.c_char_p(exe),
                    self.actions, self.attrs,
                    ctypes.cast(argv, ctypes.POINTER(ctypes.c_char_p)),
                    ctypes.cast(env, ctypes.POINTER(ctypes.c_char_p)))
            status = ctypes.c_int()
            self.libc.waitpid(pid.value, ctypes.byref(status), 0)


if name == “main“:
        ps = PosixSpawn()
        exe = “/usr/bin/tar”
        for x in range(0,1000):
                ps.execute(exe, [“-h”])

It uses a Python ctypes which allows calling c libraries (actually any “foreign” library or DLL) and wraps this in pure Python. The vfork() function is a faster way to fork a process. What it does is skip the copying of memory pages of the calling process into the child process, which is exactly what we need here. This almost doubles the executions per second to 870 (compared to 452 using regular Python fork()).

So there you have it, a fast way to fuzz binaries with the flexibility of Python and the speed (almost) of C.