Skip to content

Fix 33 reproduced fuzzing and static-review defects - #8514

Merged
youknowone merged 40 commits into
RustPython:mainfrom
youknowone:fuzzer-issues
Aug 13, 2026
Merged

Fix 33 reproduced fuzzing and static-review defects#8514
youknowone merged 40 commits into
RustPython:mainfrom
youknowone:fuzzer-issues

Conversation

@youknowone

@youknowone youknowone commented Aug 13, 2026

Copy link
Copy Markdown
Member

Fixes 33 reproduced defects from devdanzin's fuzzing + static-review catalogs (rustpython-findings / rustpython-review-findings). Every one is an interpreter abort, panic, unbounded allocation or memory-unsafety reachable from ordinary pure-Python code.

One commit per defect, each independently revertable.
extra_tests/snippets/crash_regressions.py has a case per defect; every expected value in it was
checked against CPython 3.14.

Memory unsafety and unguarded native recursion

id reproducer before after
RPYR-0013 + RPYR-0014 t = (); [t := (t,) for _ in range(300_000)]; hash(t) SIGSEGV RecursionError
RPYR-0015 L = []; L.append(L); list[L] SIGSEGV RecursionError
RPYR-0020 + RUSTPY-0018 _asyncio._enter_task(0, f); _asyncio._enter_task(0, f) SIGSEGV RuntimeError with both tasks' repr
RUSTPY-0008 M = type(re.match('a','a')); M.__new__(M)[0] SIGSEGV TypeError: cannot create 're.Match' instances
RUSTPY-0024 ctypes.CDLL("libc.so.6").strlen(1.5) SIGSEGV TypeError: Don't know how to convert parameter float

PyObject::hash dispatched the hash slot with no with_recursion, unlike the repr and rich-compare
dispatches beside it, so any element-wise __hash__ recursed one native frame per nesting level. The
guard goes on the dispatch, which covers tuple, GenericAlias, slice and code at once.

PyAtomicRef<T> stores a pointer to a Py<T>Deref, load_raw, swap and Drop all read it that
way — but Debug cast it to a bare T and formatted the object header as payload. PyFunction's
code: PyAtomicRef<PyCode> has a pointer-chasing Debug, so {:?} on any Python function dereferenced
header words. The impl now casts to PyObject, which also covers the PyAtomicRef<PyObject> and
PyAtomicRef<Option<T>> instantiations that have no Py<T>.

_ctypes's no-argtypes conversion took its int branch through try_int, which goes through __int__
and so accepted a float; libc.strlen(1.5) passed 1 where a char * was expected. It now does a
PyLong_Check-equivalent downcast, matching ConvParam exactly — verified against CPython 3.13 for
1.5/0.0/1e300/True/5.

The hash guard sits on a hot path, so I measured it: dict/set insert, lookup and bare hash() over
300k keys are unchanged against the pre-guard build (within run-to-run noise, ±3% in both directions).

Missing GC traverse

id reproducer after
RPYR-0010 cycle through deque / defaultdict collected
RPYR-0016 cycle through a classmethod's callable collected
RPYR-0012 cycle through itertools.cycle collected

RPYR-0012 needed a collector fix, not just the traverse opt-in. Traverse for PyIter<O> delegated to
the inherent PyObject::traverse of the object it wraps, so it reported that iterator's referents
instead of the iterator. The iterator's own reference was then never subtracted in the collector's
reference-subtraction pass, its referents kept a non-zero gc_refs, and everything reachable from them
was classified as a root. Every type with a PyIter field leaked as a result — map, filter, zip,
enumerate, reversed and the itertools iterators — while the same cycle through a list, tuple
or list_iterator collected. PyIter now reports the wrapped object, as PyObjectRef, PyRef<T> and
PyStackRef do.

Still leaking after the fix: itertools.tee, whose shared buffer is a PyRc<PyItertoolsTeeData> rather
than a Python object, so the collector cannot see through it. That needs a separate change.

Concurrency

id reproducer before after
RUSTPY-0020 4 threads repr(shared_set) while 4 clear it .expect() panic in a worker the empty repr
RUSTPY-0022 4 threads over one itertools.cycle index-out-of-bounds panic fetch_update advances and wraps atomically

.unwrap() / .expect() on a Python-reachable fallible value

id reproducer before after
RPYR-0001 pickle.dumps(ImportError()) panic in exceptions.rs (ImportError, ())
RPYR-0002 _asyncio._current_tasks = 42; _asyncio.current_task(...) panic returns None, matching the three guarded siblings
RPYR-0003 FutureIter.throw(E) where E.__new__ returns a non-exception panic TypeError
RPYR-0004 mmap.mmap(-1, 10).find(b"x", 5, 2) slice panic not-found
RPYR-0005 mmap.mmap(-1, 10).move(20, 0, 1) index panic ValueError
RPYR-0018 _imp.find_frozen('x', True) unimplemented!() abort TypeError
RUSTPY-0002 pwd.struct_passwd().pw_name index OOB panic TypeError
RUSTPY-0003 import _md5; _md5.md5() "static type has not been initialized" panic works
RUSTPY-0004 _csv.reader([]).__next__() unwrap() on a missing dialect StopIteration
RUSTPY-0005 _typing._idfunc() index OOB panic TypeError
RUSTPY-0006 eval(chr(0xd800)) "PyStr contains surrogates" panic UnicodeEncodeError
RUSTPY-0021 sys.breakpointhook() with an unimportable $PYTHONBREAKPOINT under -W error unwrap() panic the warning propagates

Integer narrowing / arithmetic overflow

id reproducer before after
RPYR-0006 deque([0]) * sys.maxsize allocator capacity-overflow abort MemoryError
RPYR-0007 (1,) * (10**12) Vec::with_capacity abort MemoryError (fallible reservation)
RPYR-0009 itertools.combinations(range(5), 2**64) to_usize().unwrap() panic OverflowError
RPYR-0017 + RUSTPY-0017 ctypes.c_char_p(2**64), p[0] = 2**64 .expect("int too large") abort value masks to the target width

Unbounded eager collection of an iterable

The argument was materialized before being validated, so an infinite iterable exhausted memory. Each is now rejected in O(1).

id reproducer after
RPYR-0011 math.sumprod(count(...), count(...)) streams in lockstep, O(1) memory
RPYR-0019 + RUSTPY-0016 os.posix_spawn('/bin/true', map(str, count()), os.environ) TypeError: posix_spawn: argv must be a tuple or list
RUSTPY-0012 _suggestions._generate_suggestions(count(), 'x') TypeError
RUSTPY-0013 lzma.LZMACompressor(..., filters=<generator>) TypeError
RUSTPY-0014 ExceptionGroup('m', count()) TypeError
RUSTPY-0015 (c_int*3)()[0:3] = count() ValueError

posix_spawn's setsigdef/setsigmask now validate each signal while streaming instead of
collect-then-check, and os.setgroups takes its argument through the sequence protocol.

Also in here

Three things the review turned up alongside the fixes:

  • itertools.combinations/combinations_with_replacement built their index vector with an infallible
    allocation, so an r that passes the ssize_t check but does not fit in memory aborted instead of
    raising MemoryError.
  • The struct sequence constructor discarded its dict argument, so hidden fields (tm_zone,
    st_atime) were always None when constructed directly or restored from a (sequence, dict) pickle,
    and a non-dict second argument was accepted silently. It is now applied, validated, and rejects a key
    that names an already-supplied or non-existent field. os.stat_result and os.statvfs_result did
    not accept a second argument at all. Six expectedFailure markers in Lib/test/test_structseq.py
    became passes.
  • test_code_module.test_unicode_error became an unexpected success once RUSTPY-0006 landed; its
    marker is removed.

Not addressed

Still reproducing, deliberately out of scope: the rest of the concurrency class (RUSTPY-0019/0023) and
itertools.tee's uncollectable shared buffer.

Already fixed on main, no longer reproducing: RPYR-0008, RUSTPY-0001/0009/0010/0011.

Verification

macOS (aarch64) and Linux (aarch64, Debian trixie container), on each platform:

  • cargo clippy --keep-going --workspace --all-targets with the CI feature set and excludes — clean
  • the CI cargo test invocation — pass
  • 73 CPython test modules covering every touched area — 10,059 tests, all pass
  • all 43 catalog reproducers re-run, plus the new regression snippet

Linux matters for several of these. os.setgroups is behind
#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "redox")))], so macOS cannot compile
it at all; on Linux it matches CPython 3.13 exactly (TypeError: setgroups argument must be a sequence),
as do all the posix_spawn paths. (1,) * (10**12) raises MemoryError on Linux for both CPython and
RustPython (on macOS both hang instead). And RUSTPY-0024's reproducer needs libc.so.6.

Two pre-existing conditions worth naming, both reproduced on untouched main: cargo test -p rustpython-capi SIGSEGVs in abstract_::iter::tests::next_item (CI excludes that crate), and
crates/capi/src/pystrcmp.rs trips clippy::unnecessary_cast on aarch64 Linux only, where c_char is
u8.

One difference from CPython remains: posix_spawn(setsigdef=...) reports signal number 0 out of range
where CPython appends the range, [1; 64]. That wording predates this PR and is left alone.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes

    • Improved asyncio task and exception handling, including clearer conflict errors.
    • Fixed mmap range validation and safer oversized sequence allocation.
    • Corrected empty collection representations and exception serialization.
    • Improved ctypes integer conversion, argument validation, and slice assignment checks.
    • Added safer validation for LZMA filters, process arguments, signals, and struct sequences.
    • Improved recursion and memory error reporting for generic aliases and iterators.
  • Compatibility

    • Enhanced hashlib module initialization and standard-library behavior across CSV, math, typing, and operating-system utilities.
  • Tests

    • Added broad regression coverage for the updated behavior, error handling, memory limits, garbage collection, and concurrency.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants