Fix 33 reproduced fuzzing and static-review defects - #8514
Merged
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes 33 reproduced defects from devdanzin's fuzzing + static-review catalogs (
rustpython-findings/rustpython-review-findings). Every one is an interpreter abort, panic, unbounded allocation or memory-unsafety reachable from ordinary pure-Python code.One commit per defect, each independently revertable.
extra_tests/snippets/crash_regressions.pyhas a case per defect; every expected value in it waschecked against CPython 3.14.
Memory unsafety and unguarded native recursion
t = (); [t := (t,) for _ in range(300_000)]; hash(t)RecursionErrorL = []; L.append(L); list[L]RecursionError_asyncio._enter_task(0, f); _asyncio._enter_task(0, f)RuntimeErrorwith both tasks' reprM = type(re.match('a','a')); M.__new__(M)[0]TypeError: cannot create 're.Match' instancesctypes.CDLL("libc.so.6").strlen(1.5)TypeError: Don't know how to convert parameter floatPyObject::hashdispatched the hash slot with nowith_recursion, unlike thereprand rich-comparedispatches beside it, so any element-wise
__hash__recursed one native frame per nesting level. Theguard goes on the dispatch, which covers
tuple,GenericAlias,sliceandcodeat once.PyAtomicRef<T>stores a pointer to aPy<T>—Deref,load_raw,swapandDropall read it thatway — but
Debugcast it to a bareTand formatted the object header as payload.PyFunction'scode: PyAtomicRef<PyCode>has a pointer-chasingDebug, so{:?}on any Python function dereferencedheader words. The impl now casts to
PyObject, which also covers thePyAtomicRef<PyObject>andPyAtomicRef<Option<T>>instantiations that have noPy<T>._ctypes's no-argtypesconversion took its int branch throughtry_int, which goes through__int__and so accepted a float;
libc.strlen(1.5)passed1where achar *was expected. It now does aPyLong_Check-equivalent downcast, matchingConvParamexactly — verified against CPython 3.13 for1.5/0.0/1e300/True/5.The hash guard sits on a hot path, so I measured it: dict/set insert, lookup and bare
hash()over300k keys are unchanged against the pre-guard build (within run-to-run noise, ±3% in both directions).
Missing GC
traversedeque/defaultdictclassmethod's callableitertools.cycleRPYR-0012 needed a collector fix, not just the
traverseopt-in.Traverse for PyIter<O>delegated tothe inherent
PyObject::traverseof the object it wraps, so it reported that iterator's referentsinstead of the iterator. The iterator's own reference was then never subtracted in the collector's
reference-subtraction pass, its referents kept a non-zero
gc_refs, and everything reachable from themwas classified as a root. Every type with a
PyIterfield leaked as a result —map,filter,zip,enumerate,reversedand theitertoolsiterators — while the same cycle through alist,tupleor
list_iteratorcollected.PyIternow reports the wrapped object, asPyObjectRef,PyRef<T>andPyStackRefdo.Still leaking after the fix:
itertools.tee, whose shared buffer is aPyRc<PyItertoolsTeeData>ratherthan a Python object, so the collector cannot see through it. That needs a separate change.
Concurrency
repr(shared_set)while 4 clear it.expect()panic in a workeritertools.cyclefetch_updateadvances and wraps atomically.unwrap()/.expect()on a Python-reachable fallible valuepickle.dumps(ImportError())exceptions.rs(ImportError, ())_asyncio._current_tasks = 42; _asyncio.current_task(...)None, matching the three guarded siblingsFutureIter.throw(E)whereE.__new__returns a non-exceptionTypeErrormmap.mmap(-1, 10).find(b"x", 5, 2)mmap.mmap(-1, 10).move(20, 0, 1)ValueError_imp.find_frozen('x', True)unimplemented!()abortTypeErrorpwd.struct_passwd().pw_nameTypeErrorimport _md5; _md5.md5()_csv.reader([]).__next__()unwrap()on a missing dialectStopIteration_typing._idfunc()TypeErroreval(chr(0xd800))UnicodeEncodeErrorsys.breakpointhook()with an unimportable$PYTHONBREAKPOINTunder-W errorunwrap()panicInteger narrowing / arithmetic overflow
deque([0]) * sys.maxsizeMemoryError(1,) * (10**12)Vec::with_capacityabortMemoryError(fallible reservation)itertools.combinations(range(5), 2**64)to_usize().unwrap()panicOverflowErrorctypes.c_char_p(2**64),p[0] = 2**64.expect("int too large")abortUnbounded eager collection of an iterable
The argument was materialized before being validated, so an infinite iterable exhausted memory. Each is now rejected in O(1).
math.sumprod(count(...), count(...))os.posix_spawn('/bin/true', map(str, count()), os.environ)TypeError: posix_spawn: argv must be a tuple or list_suggestions._generate_suggestions(count(), 'x')TypeErrorlzma.LZMACompressor(..., filters=<generator>)TypeErrorExceptionGroup('m', count())TypeError(c_int*3)()[0:3] = count()ValueErrorposix_spawn'ssetsigdef/setsigmasknow validate each signal while streaming instead ofcollect-then-check, and
os.setgroupstakes its argument through the sequence protocol.Also in here
Three things the review turned up alongside the fixes:
itertools.combinations/combinations_with_replacementbuilt their index vector with an infallibleallocation, so an
rthat passes the ssize_t check but does not fit in memory aborted instead ofraising
MemoryError.dictargument, so hidden fields (tm_zone,st_atime) were always None when constructed directly or restored from a(sequence, dict)pickle,and a non-dict second argument was accepted silently. It is now applied, validated, and rejects a key
that names an already-supplied or non-existent field.
os.stat_resultandos.statvfs_resultdidnot accept a second argument at all. Six
expectedFailuremarkers inLib/test/test_structseq.pybecame passes.
test_code_module.test_unicode_errorbecame an unexpected success once RUSTPY-0006 landed; itsmarker is removed.
Not addressed
Still reproducing, deliberately out of scope: the rest of the concurrency class (RUSTPY-0019/0023) and
itertools.tee's uncollectable shared buffer.Already fixed on main, no longer reproducing: RPYR-0008, RUSTPY-0001/0009/0010/0011.
Verification
macOS (aarch64) and Linux (aarch64, Debian trixie container), on each platform:
cargo clippy --keep-going --workspace --all-targetswith the CI feature set and excludes — cleancargo testinvocation — passLinux matters for several of these.
os.setgroupsis behind#[cfg(not(any(target_os = "ios", target_os = "macos", target_os = "redox")))], so macOS cannot compileit at all; on Linux it matches CPython 3.13 exactly (
TypeError: setgroups argument must be a sequence),as do all the
posix_spawnpaths.(1,) * (10**12)raisesMemoryErroron Linux for both CPython andRustPython (on macOS both hang instead). And RUSTPY-0024's reproducer needs
libc.so.6.Two pre-existing conditions worth naming, both reproduced on untouched
main:cargo test -p rustpython-capiSIGSEGVs inabstract_::iter::tests::next_item(CI excludes that crate), andcrates/capi/src/pystrcmp.rstripsclippy::unnecessary_caston aarch64 Linux only, wherec_charisu8.One difference from CPython remains:
posix_spawn(setsigdef=...)reportssignal number 0 out of rangewhere CPython appends the range,
[1; 64]. That wording predates this PR and is left alone.🤖 Generated with Claude Code
Summary by CodeRabbit
Bug Fixes
Compatibility
Tests