Published my first PyPI package today, called lafleur.
#lafleur is a specialized CPython JIT fuzzer that uses a coverage-guided, evolutionary approach. It executes test cases, observes their effect on the JIT's state by analyzing verbose trace logs, and uses that feedback to guide its mutations, becoming smarter at finding interesting code paths over time.
Let me know if you use it or have any questions.
https://pypi.org/project/lafleur/
https://github.com/devdanzin/lafleur
AFL++ 4.35c release! Complete hidden coverage gathering, GUIFuzz++ support, IJON for qemu, various fixes! https://github.com/AFLplusplus/AFLplusplus/releases/tag/v4.35c #fuzzing #fuzzer
There's a researcher, Jiang Yuancheng, who's doing a great work finding CPython crashes and memory leaks: https://github.com/python/cpython/issues?q=is%3Aissue%20author%3AYuanchengJiang
They've come up with a very clever idea for a new way of fuzzing, made a fine tool out of it, and are reaping great results.
Fuzzing can be a diminishing returns endeavor: you only have so many bugs to find. Their approach has shown itself to cover different areas and kinds of issues well, as shown by their track record.
"Tạo công cụ kiểm thử AI Red Teaming cục bộ với Fuzzer và 280 Payloads để tránh đăng ký hàng tháng. Công cụ này bao gồm Mutation Fuzzer, Payloads thực tế và AI Judge. #AITools #RedTeaming #Fuzzer #AI #AnToànBảoMật #CôngCụKiểmThử"
https://www.reddit.com/r/LocalLLaMA/comments/1p6c21o/i_built_a_local_ai_red_teaming_tool_fuzzer_280/
It turns out that, by running it on an interpreter with ASan enabled, I was the culprit of my fuzzer lafleur using way too much memory.
That even led to me buying some DDR5 to be able to fuzz a bit more comfortably. Running without ASan reduces memory usage to 1/15. So I guess now I'll have some spare RAM, and less money, going forward :)
I'll enhance the JIT fuzzer to run on a different interpreter than the fuzzing scripts, which benefit from ASan.
So someone found a segfault in NumPy and reported it as a security bug: https://huntr.com/bounties/49928a2c-c6bb-4c1c-80ec-5d7bf708bf28. After some back and forth, the NumPy developers agreed it was a security bug (with a low score, but still).
However, since fusil had already found that crash and I had reported it 2 months earlier (https://github.com/numpy/numpy/issues/28829), the report was deemed a duplicate and no CVE was assigned.
We didn't find a CVE, but avoided one :)
Link to fix: https://github.com/numpy/numpy/pull/30071
Fuzzing pyhacl (https://codeberg.org/drlazor8/pyhacl), a package of Cython bindings for HACL* (the High Assurance Cryptographic Library), with fusil we only found one crash.
It turned out to actually be a silly bug in #Cython:
Issue: https://github.com/cython/cython/issues/7263
Fix: https://github.com/cython/cython/pull/7264
Goes to show how fuzzing a C-extension can uncover crashes in many different layers.
Thanks @drlazor8 for taking up the call for C-extensions maintainers to fuzz their code.
After a pause, we're back to running fusil. This time, to fuzz cereggii, a package of very interesting thread synchronization utilities for #Python, made of C-extensions.
We tailored #fusil to target these #cereggii utilities, finding more issues.
Daniele Parmeggiani (dpdani), the maintainer, has been helping in the effort and being very supportive. That's the best welcome fusil has received in any project :)
Here are the issues we found: https://github.com/dpdani/cereggii/issues?q=label%3A%22fusil-fuzzer%22
Been running a fusil campaign for a week, targeting a C-extension. The fuzzer has found 9 issues so far, feels good.
The maintainer is helping with the campaign and eager to fix the issues, which is great.
Some maintainers see us with suspicion and often disregard issues, as if we're after accolades, pointing fingers or complaining about the code.
We fuzz to help, that's all.
Anyway, if you have a C-extension and would like it fuzzed, hit me up :)
I ran the #ELF #parser of #sydbox over 60k #Linux #malware samples from #Virusshare and 40k orcs which are malformed elves generated by the Melkor ELF #Fuzzer and got no crashes. The parser is written in #rustlang. It's free from unsafe code and arithmetic side effects. Syd parses ELF at exec(3) and mmap(2) boundary to perform various restrictions for binaries such as PIE and non-executable stack. #exherbo #security
lafleur, the CPython JIT fuzzer, now can compare timings for running a piece of code with JIT on and off.
The idea is that if the run with JIT on is much slower that with JIT off, we have found a performance bug.
Brandt Bucher suggested this mode. It took a while to get started on it, but it was simple to implement on top of Differential Mode.
So many modes, so little compute available...
![Screen shot of an abridged log of lafleur running, displaying the new messages for the timing fuzzing mode:
[TIMING] Running timed trial with JIT=False.
[TIMING] Running timed trial with JIT=True.
[~] Timing slowdown ratio (JIT/non-JIT) is 0.881.
The screen shot has a dark gray background and the text is a mix of white, green, red, purple and yellow, in a mess of coloring because the text editor identified this content as a Python script.
Full text of the image is:
--- Fuzzing Session #226 ---
[+] Calculating corpus scores for parent selection...
[+] Selected parent for BREADTH session: 523.py (Score: 248.51)
[...]
[TIMING] Running timed trial with JIT=False.
[TIMING] Running timed trial with JIT=True.
[NEW RELATIVE EDGE] '('EXECUTING', '_START_OF_HARNESS_->_SET_IP')' in harness 'f1'
[~] Timing slowdown ratio (JIT/non-JIT) is 0.881.](https://files.mastodon.social/media_attachments/files/115/367/765/525/405/481/original/d552701462e9c291.png)
Differential Mode has just landed in lafleur, the CPython JIT fuzzer. It runs the same code with and without the JIT, compares the result, and flags any discrepancies.
It does this while mutating the code in a feedback-guided loop, so it evolves the fuzzing scripts trying to find one where the JIT gives wrong results.
This is actually the 2nd time this feature is implemented (now better and more robust): it used to work, broke, and now is back.
The ActivityPub Fuzzer. Probably important if you're a fediverse app dev or other feditech person. It seems interesting.
AFL++ v4.34c release - IJON support, UnicornAFL v3, LLVM 22 support, enhanced CMPLOG, bug fixes :-) https://github.com/AFLplusplus/AFLplusplus/releases/tag/v4.34c #afl #fuzzing #fuzzer
Posted a topic about lafleur, the CPython JIT fuzzer I'm developing: https://discuss.python.org/t/introducing-lafleur-a-cpython-jit-fuzzer/103452
It has many details about the project, the results (4 JIT crashes so far), calls for help and ideas for the future.
If any of this interests you, please give a read. Boosts welcome :)
Preparing a post about lafleur, the CPython JIT fuzzer I develop.
It has found 4 JIT crashes so far:
#136996: "JIT: `executor->vm_data.valid` assertion failure in `unlink_executor`".
#137007: "JIT: assertion failure in _PyObject_GC_UNTRACK".
#137728: "Assertion failure or `SystemError` in `_PyEval_EvalFrameDefault` in a JIT build".
#137762: "Assertion failure in `optimize_uops` in a JIT build".
Contributions welcome!
https://github.com/search?q=repo%3Apython%2Fcpython+lafleur&type=issues
Here's how lafleur works:
Starts from a corpus of seed files (generated by fusi)l. It runs each file and parses the JIT debug output to record micro ops (UOPs) and edges between them.
Then it applies mutations to these files and when a mutation results in a new UOP or edge, it adds the mutated file to the corpus.
All the while, it monitors the execution of these files for crashes.
And it's smart about which file to mutate next.
Simple, eh?
One obvious thing from fuzzing the CPython JIT with the lafleur fuzzer is that finding crashes is much lower probability than fuzzing CPython with fusil.
Whenever fusil found a crash, it would find hits for it again and again pretty quickly.
With lafleur, we only got 3 hits (2 issues) in thousands of fuzzing hours.
I'm throwing more compute at it, but maybe we'll need to improve the core ideas to get better results.
Reviewed the 3 PRs we got for lafleur: https://github.com/devdanzin/lafleur/pulls?q=is%3Apr
All 3 marked as "good first issue". They've clear signs of being created by or with help from AI, like tentative code ("# do this in case... ") and removing docstrings.
Merged one and gave feedback on 2, including tips on how to get AI to fix their issues.
I might get unfollows for this, but I welcome AI generated code as long as it's good enough. In fact, AI made this project viable.
Busy day, good day: the new CPython fuzzer lafleur found its first 2 crashes this morning!
Happiness only marred by the bad job I did reducing the testcases and making the reports reproducible. Thanks Brandt Bucher and and Ken Jin for the patience! Next bug reports will be more polished and reliable.
Also, we got 3 new PRs for the fuzzer but I didn't have time to review them, will have to leave that for tomorrow.
