Introduction to Golang fuzzing

28 Oct 2019

First steps in fuzzing

A few weeks ago, I was starting to get interested into fuzzing. I read many articles describing what fuzzing is, and when it is used. I found interest in libFuzzer, ASAN, and Go-Fuzz. The problem is, when you see people finding many security issues with these tools, you immediately want to try them by yourself. So I tried some of them. I started with Go-Fuzz.

Find the first target

To make my first steps, I needed to find a case study to fuzz, in a language that I know. Since several months, I write and read Go every day, and that’s a language that I like, so why don’t I choose a Go target. To improve my chance of finding bugs, I needed to find a target that deals with raw data, and that does not expect bad input. The classic target is a parser. But there is a constraint, I have to find a target that has not been fuzzed by Google (with OSS-Fuzz for example), because if I do, I will not find anything, obviously. So let’s try something like a JSON parser, like jsonparser. I used this parser when I was intern a few months ago, it worked very well, but does it deal fuzzing?

Golang fuzzing

go-fuzz from Dmitry Vyukov is awesome. Its use is very simple: you find a target function, you call it (with input generated by the fuzzing engine) into a Fuzz function, and that’s it. You just need to prepare a corpus in which you create expected or unexpected inputs for your target function. I strongly advise you to read documentation and related articles on the go-fuzz Github page. Moreover, it can work with libFuzzer, what is very great.

Fuzzing

Identify the target

After reviewing the jsonparser’s code, I identified several functions that could be vulnerable, one of them is the GetInt function. The function parses a JSON input as a bytes slice (perfect for go-fuzz) and identifies an integer in the JSON, with a given key. Very easy to fuzz.

Corpus

The corpus was a little bit tricky to understand for when I started fuzzing, but it is very simple. You create a directory in which you create files containing well-formed or malformed inputs. In my case, I created two files that will be used by the engine to build its fuzzing inputs and new corpus files. I created the file 1, containing:

{"a":[{"b":1},{"d":2}],"c":1}

And a file 2 containing

{"a":"toto", "b":2,"c":1}

Running the fuzzer

Launching the fuzzer is very simple, just call your target function wrapped inside Fuzz function:

package fuzz

import (
    "github.com/buger/jsonparser"
)

func Fuzz(data []byte) int {
    jsonparser.GetInt(data, "a", "b")
    return 0
}

Once your corpus is fed:

$ go-fuzz-build
$ go-fuzz corpus/

Results

After a few seconds of fuzzing, crashers start appearing:

2019/10/28 21:37:50 workers: 8, corpus: 31 (1s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 3s
2019/10/28 21:37:53 workers: 8, corpus: 31 (4s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 76, uptime: 6s
2019/10/28 21:37:56 workers: 8, corpus: 31 (7s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 76, uptime: 9s
2019/10/28 21:37:59 workers: 8, corpus: 31 (10s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 76, uptime: 12s
2019/10/28 21:38:02 workers: 8, corpus: 31 (13s ago), crashers: 0, restarts: 1/906, execs: 3627 (242/sec), cover: 76, uptime: 15s
2019/10/28 21:38:05 workers: 8, corpus: 32 (2s ago), crashers: 1, restarts: 1/906, execs: 3627 (201/sec), cover: 76, uptime: 18s
2019/10/28 21:38:08 workers: 8, corpus: 32 (5s ago), crashers: 1, restarts: 1/1303, execs: 19549 (931/sec), cover: 77, uptime: 21s
2019/10/28 21:38:11 workers: 8, corpus: 32 (8s ago), crashers: 1, restarts: 1/1221, execs: 19550 (814/sec), cover: 77, uptime: 24s
2019/10/28 21:38:14 workers: 8, corpus: 32 (11s ago), crashers: 1, restarts: 1/1221, execs: 19550 (724/sec), cover: 77, uptime: 27s
2019/10/28 21:38:17 workers: 8, corpus: 32 (14s ago), crashers: 1, restarts: 1/1086, execs: 19552 (652/sec), cover: 77, uptime: 30s
2019/10/28 21:38:20 workers: 8, corpus: 32 (17s ago), crashers: 3, restarts: 1/1086, execs: 19552 (592/sec), cover: 77, uptime: 33s
2019/10/28 21:38:23 workers: 8, corpus: 32 (20s ago), crashers: 3, restarts: 1/1598, execs: 47959 (1332/sec), cover: 77, uptime: 36s
2019/10/28 21:38:26 workers: 8, corpus: 32 (23s ago), crashers: 3, restarts: 1/1547, execs: 47960 (1230/sec), cover: 77, uptime: 39s
2019/10/28 21:38:29 workers: 8, corpus: 32 (26s ago), crashers: 3, restarts: 1/1547, execs: 47960 (1142/sec), cover: 77, uptime: 42s
2019/10/28 21:38:32 workers: 8, corpus: 32 (29s ago), crashers: 3, restarts: 1/1453, execs: 47964 (1066/sec), cover: 77, uptime: 45s
2019/10/28 21:38:35 workers: 8, corpus: 32 (32s ago), crashers: 3, restarts: 1/1453, execs: 47964 (999/sec), cover: 77, uptime: 48s
2019/10/28 21:38:38 workers: 8, corpus: 32 (35s ago), crashers: 3, restarts: 1/988, execs: 63259 (1240/sec), cover: 77, uptime: 51s
<...>

Let’s have a look to crashers. Crash1:

panic: runtime error: index out of range [3] with length 2

goroutine 1 [running]:
github.com/buger/jsonparser.searchKeys(0x2010000, 0x7, 0x7, 0xc00008ae50, 0x2, 0x2, 0x1202008)
    /Users/thomasleroy/Software/go/src/github.com/buger/jsonparser/parser.go:264 +0xef5
github.com/buger/jsonparser.internalGet(0x2010000, 0x7, 0x7, 0xc00008ae50, 0x2, 0x2, 0x300000002, 0xc000000180, 0xc00008ad70, 0x1032180, ...)
    /Users/thomasleroy/Software/go/src/github.com/buger/jsonparser/parser.go:876 +0x3b5
github.com/buger/jsonparser.Get(0x2010000, 0x7, 0x7, 0xc00008ae50, 0x2, 0x2, 0x1052ce2, 0x10444ec, 0x1054d20, 0xc00008ae08, ...)
    /Users/thomasleroy/Software/go/src/github.com/buger/jsonparser/parser.go:870 +0x8b
github.com/buger/jsonparser.GetInt(0x2010000, 0x7, 0x7, 0xc00008ae50, 0x2, 0x2, 0x3a23685800000000, 0x5db751b9, 0xc00008ae70)
    /Users/thomasleroy/Software/go/src/github.com/buger/jsonparser/parser.go:1140 +0x8b
github.com/buger/jsonparser/fuzz.Fuzz(0x2010000, 0x7, 0x7, 0x4)
    /Users/thomasleroy/Software/go/src/github.com/buger/jsonparser/fuzz/fuzz.go:8 +0x95
go-fuzz-dep.Main(0xc00008af48, 0x1, 0x1)
    go-fuzz-dep/main.go:36 +0x1ad
main.main()
    github.com/buger/jsonparser/fuzz/go.fuzz.main/main.go:15 +0x52
exit status 2

Crash 2:

panic: runtime error: index out of range [2] with length 2

goroutine 1 [running]:
github.com/buger/jsonparser.searchKeys(0x4010000, 0x6, 0x6, 0xc00008ae50, 0x2, 0x2, 0xffffffffffffffff)
    /Users/thomasleroy/Software/go/src/github.com/buger/jsonparser/parser.go:264 +0xef5
github.com/buger/jsonparser.internalGet(0x4010000, 0x6, 0x6, 0xc00008ae50, 0x2, 0x2, 0x300000002, 0xc000000180, 0xc00008ad70, 0x1032180, ...)
    /Users/thomasleroy/Software/go/src/github.com/buger/jsonparser/parser.go:876 +0x3b5
github.com/buger/jsonparser.Get(0x4010000, 0x6, 0x6, 0xc00008ae50, 0x2, 0x2, 0x1052ce2, 0x10444ec, 0x1054d20, 0xc00008ae08, ...)
    /Users/thomasleroy/Software/go/src/github.com/buger/jsonparser/parser.go:870 +0x8b
github.com/buger/jsonparser.GetInt(0x4010000, 0x6, 0x6, 0xc00008ae50, 0x2, 0x2, 0x19f505000000000, 0x5db751ba, 0xc00008ae70)
    /Users/thomasleroy/Software/go/src/github.com/buger/jsonparser/parser.go:1140 +0x8b
github.com/buger/jsonparser/fuzz.Fuzz(0x4010000, 0x6, 0x6, 0x3)
    /Users/thomasleroy/Software/go/src/github.com/buger/jsonparser/fuzz/fuzz.go:8 +0x95
go-fuzz-dep.Main(0xc00008af48, 0x1, 0x1)
    go-fuzz-dep/main.go:36 +0x1ad
main.main()
    github.com/buger/jsonparser/fuzz/go.fuzz.main/main.go:15 +0x52
exit status 2

And crash 3:

program hanged (timeout 10 seconds)

SIGABRT: abort
PC=0x7fff65b6b8f6 m=0 sigcode=0

goroutine 0 [idle]:
runtime.pthread_cond_wait(0x11af788, 0x11af748, 0x0)
    runtime/sys_darwin.go:378 +0x39
runtime.semasleep(0xffffffffffffffff, 0x0)
    runtime/os_darwin.go:63 +0x85
runtime.notesleep(0x11af548)
    runtime/lock_sema.go:173 +0xe0
runtime.stopm()
    runtime/proc.go:1928 +0xc0
runtime.findrunnable(0xc000022000, 0x0)
    runtime/proc.go:2391 +0x53f
runtime.schedule()
    runtime/proc.go:2524 +0x2be
runtime.exitsyscall0(0xc000000180)
    runtime/proc.go:3141 +0x116
runtime.mcall(0x10513c6)
    runtime/asm_amd64.s:318 +0x5b

goroutine 1 [runnable]:
github.com/buger/jsonparser.blockEnd(0x4010006, 0x1, 0x1, 0x7d7b, 0xffffffffffffffff)
    /Users/thomasleroy/Software/go/src/github.com/buger/jsonparser/parser.go:182 +0x294
github.com/buger/jsonparser.searchKeys(0x4010000, 0x7, 0x7, 0xc00007ee50, 0x2, 0x2, 0xffffffffffffffff)
    /Users/thomasleroy/Software/go/src/github.com/buger/jsonparser/parser.go:286 +0xd91
github.com/buger/jsonparser.internalGet(0x4010000, 0x7, 0x7, 0xc00007ee50, 0x2, 0x2, 0xc000022000, 0xc000000180, 0xc00007ed70, 0x1032180, ...)
<...>

This is clearly an infinite loop.

Interpretation

Regarding these crashes, we can see that the two first ones are very similar. They are actually the same bug, but triggered twice. Crashers create an ìndex out of range in the searchKeys function, what is a kind of bug we were looking for. Since this is Golang code, these bug could just lead to deny of service (by making the app using jsonparser crash), but in C that would have been far more dangerous. The first bug could lead to an out-of-bound read or write, depending if the code tried to read or write in the array. Let’s report this bug with an issue. The third crash is different. The crasher triggers an infinite loop in the BlockEnd function. Again, that could lead to a deny of service. Let’s report this bug too.

Conclusion

Finding these bugs was awesome, especially with fuzzing. This was the very first time I used fuzzing, and that’s even cooler than I expected. Because, with fuzzing, you find crashes with a kind of magic. You don’t even need to deeply understand the code you fuzz, which is time-saving but bad. I will try to spend time to understand the code that I fuzzed to spot were the bugs are, to pursue what I started.