Skip to content

Thread-safe type operations: type lock, QSBR type-cache reclamation, GC stop-the-world, and interpreter optimizations - #7416

Merged
youknowone merged 92 commits into
RustPython:mainfrom
youknowone:typelock
Jul 8, 2026
Merged

Thread-safe type operations: type lock, QSBR type-cache reclamation, GC stop-the-world, and interpreter optimizations#7416
youknowone merged 92 commits into
RustPython:mainfrom
youknowone:typelock

Conversation

@youknowone

@youknowone youknowone commented Mar 13, 2026

Copy link
Copy Markdown
Member

Introduce vm.state.type_mutex to serialize type mutation and version-tag assignment, matching CPython's free-threading model.

  • Add with_type_lock() helper and split modified/assign_version_tag into inner (lock-free) and public (lock-acquiring) variants
  • Add version_for_specialization() and lookup_ref_and_version_interned() for atomic type-cache lookups under the lock
  • Wrap all type-mutating paths (bases, annotations, module, type_params, doc, SetAttr) with the type lock
  • Drop old values outside the lock to prevent deadlock from weakref callbacks that may re-enter specialization
  • Fix _ctypes set_attr calls to route through proper SetAttr path
  • Reinitialize type_mutex after fork

Summary by CodeRabbit

  • Bug Fixes

    • Centralized type-mutation synchronization to reduce races and deadlock risk during bases, annotations, and attribute updates.
  • Performance

    • More consistent type-versioning and cache lookups, enabling faster and more reliable specialization fastpaths and method/attribute retrievals.
  • Chores

    • Added and initialized a global type mutex at VM startup and ensured it is reinitialized after fork to support the new synchronization.

@coderabbitai

coderabbitai Bot commented Mar 13, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Centralizes type-mutation locking under a new global type_mutex, moves type-version and cache mutations into with_type_lock helpers, changes specialization fast-paths to use versioned lookups, and reinitializes the new lock after fork.

Changes

Cohort / File(s) Summary
Type cores & caches
crates/vm/src/builtins/type.rs
Removed per-cache write mutex; added PyType::with_type_lock(vm, f) and internal helpers (assign_version_tag_inner, modified_inner, version_for_specialization, lookup_ref_and_version_interned); moved cache population, invalidation, slot init, and many attribute setters under type_mutex; adjusted getitem version store ordering to Release.
Specialization & frames
crates/vm/src/frame.rs
Replaced manual tp_version_tag loads/assigns with version_for_specialization and lookup_ref_and_version_interned across multiple specializers (getattro, __getitem__, CallAllocAndEnterInit, metaclass guards, ToBool guard), consolidating ref+version retrieval and preserving version==0 backoff behavior.
Global state & fork handling
crates/vm/src/vm/mod.rs, crates/vm/src/vm/interpreter.rs, crates/vm/src/stdlib/posix.rs
Added public type_mutex: PyMutex<()> to PyGlobalState, initialize it at VM startup, and ensure reinit_locks_after_fork reinitializes type_mutex in the child process.

Sequence Diagram(s)

sequenceDiagram
    participant Frame as Frame (specializer)
    participant Type as PyType
    participant VM as VM (type_mutex)
    participant Cache as TYPE_CACHE

    Frame->>Type: lookup_ref_and_version_interned(name, vm)
    Type->>VM: with_type_lock (acquire type_mutex)
    alt TYPE_CACHE hit with non-zero version
        Type->>Cache: read cached entry
        Cache-->>Type: (value, version)
    else cache miss or version==0
        Type->>Type: perform MRO lookup for name
        Type->>Cache: insert/update entry (under lock)
    end
    Type->>VM: release type_mutex
    Type-->>Frame: return (attr_ref_opt, type_version)
    Frame->>Frame: use returned attr_ref_opt and type_version in guard
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Poem

🐰 I hopped into the type-lock glade,
Held mutex snug while caches were made.
Versions counted, MROs aligned,
No more races for bytes I find—
Carrots, locks, and code well-laid.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 78.38% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title captures the main change: adding thread-safe type locking for type operations and related optimizations.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@youknowone
youknowone force-pushed the typelock branch 5 times, most recently from d0cfe84 to 1daba38 Compare March 19, 2026 19:39
@github-actions

github-actions Bot commented Mar 19, 2026

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[ ] test: cpython/Lib/test/test_generators.py (TODO: 10)
[ ] test: cpython/Lib/test/test_genexps.py (TODO: 4)
[x] test: cpython/Lib/test/test_generator_stop.py
[x] test: cpython/Lib/test/test_yield_from.py (TODO: 1)

dependencies:

dependent tests: (no tests depend on generator)

[x] test: cpython/Lib/test/test_frame.py (TODO: 4)

dependencies:

dependent tests: (no tests depend on frame)

[x] test: cpython/Lib/test/test_descr.py (TODO: 32)
[ ] test: cpython/Lib/test/test_descrtut.py (TODO: 3)

dependencies:

dependent tests: (no tests depend on descr)

[x] lib: cpython/Lib/ssl.py
[x] test: cpython/Lib/test/test_ssl.py (TODO: 14)

dependencies:

  • ssl

dependent tests: (53 tests)

  • ssl: test_asyncio test_ftplib test_httplib test_httpservers test_imaplib test_logging test_poplib test_ssl test_urllib test_urllib2_localnet test_venv test_xmlrpc
    • asyncio.selector_events: test_asyncio
    • ftplib: test_urllib2
      • urllib.request: test_http_cookiejar test_pathlib test_pydoc test_sax test_site test_urllib2net test_urllibnet
    • http.client: test_docxmlrpc test_hashlib test_ucn test_unicodedata test_wsgiref
      • logging.handlers: test_concurrent_futures test_pkgutil
    • http.server: test_robotparser
      • pydoc: test_enum
    • smtplib: test_smtplib test_smtpnet
    • urllib.request:
      • pathlib: test_ast test_dbm_sqlite3 test_ensurepip test_importlib test_json test_launcher test_pathlib test_peg_generator test_pyrepl test_runpy test_tarfile test_tempfile test_tomllib test_tools test_traceback test_unparse test_winapi test_zipapp test_zipfile test_zoneinfo test_zstd

[x] lib: cpython/Lib/asyncio
[ ] test: cpython/Lib/test/test_asyncio (TODO: 33)

dependencies:

  • asyncio

dependent tests: (7 tests)

  • asyncio: test_asyncio test_external_inspection test_inspect test_logging test_os test_pdb test_unittest

[ ] test: cpython/Lib/test/test_monitoring.py (TODO: 5)

dependencies:

dependent tests: (no tests depend on monitoring)

[x] lib: cpython/Lib/inspect.py
[ ] test: cpython/Lib/test/test_inspect (TODO: 32)

dependencies:

  • inspect

dependent tests: (96 tests)

  • inspect: test_abc test_argparse test_asyncgen test_buffer test_builtin test_clinic test_code test_collections test_coroutines test_decimal test_enum test_functools test_generators test_grammar test_inspect test_monitoring test_ntpath test_operator test_patma test_posixpath test_pydoc test_signal test_sqlite3 test_traceback test_turtle test_type_annotations test_types test_typing test_unittest test_yield_from test_zipimport test_zipimport_support test_zoneinfo
    • ast: test_ast test_compile test_compiler_codegen test_dis test_fstring test_future_stmt test_peepholer test_peg_generator test_site test_ssl test_type_comments test_ucn test_unparse
      • annotationlib: test_annotationlib test_reprlib test_type_params
      • dbm.dumb: test_dbm_dumb
      • pyclbr: test_pyclbr
      • traceback: test_asyncio test_code_module test_contextlib test_contextlib_async test_dictcomps test_exceptions test_http_cookiejar test_importlib test_iter test_listcomps test_pyexpat test_setcomps test_socket test_subprocess test_sys test_threadedtempfile test_threading test_unittest test_with
    • asyncio: test_asyncio test_external_inspection test_logging test_os test_pdb
    • bdb: test_bdb
    • cmd: test_cmd
      • pstats: test_profile test_pstats
    • dataclasses: test__colorize test_copy test_ctypes test_genericalias test_pprint test_regrtest
      • pprint: test_htmlparser test_sys_setprofile
    • importlib.metadata: test_importlib
    • pkgutil: test_pkgutil test_pyrepl test_runpy
    • pydoc:
      • xmlrpc.server: test_docxmlrpc test_xmlrpc
    • rlcompleter: test_pyrepl test_rlcompleter
    • trace: test_trace

[x] lib: cpython/Lib/pdb.py
[ ] test: cpython/Lib/test/test_pdb.py (TODO: 46)

dependencies:

  • pdb

dependent tests: (1 tests)

  • pdb: test_pdb

[x] lib: cpython/Lib/io.py
[x] lib: cpython/Lib/_pyio.py
[ ] test: cpython/Lib/test/test_io.py (TODO: 13)
[x] test: cpython/Lib/test/test_bufio.py
[x] test: cpython/Lib/test/test_fileio.py (TODO: 1)
[ ] test: cpython/Lib/test/test_memoryio.py (TODO: 27)

dependencies:

  • io

dependent tests: (108 tests)

  • io: test__colorize test_android test_argparse test_ast test_asyncio test_base64 test_buffer test_bufio test_builtin test_bz2 test_calendar test_cmd test_cmd_line_script test_codecs test_compile test_compileall test_compiler_assemble test_concurrent_futures test_configparser test_contextlib test_csv test_dbm_dumb test_descr test_dis test_email test_enum test_file test_fileinput test_fileio test_ftplib test_generated_cases test_getpass test_gzip test_hashlib test_http_cookiejar test_httplib test_httpservers test_importlib test_inspect test_io test_json test_largefile test_logging test_lzma test_mailbox test_marshal test_memoryio test_memoryview test_mimetypes test_minidom test_multibytecodec test_optparse test_pathlib test_pdb test_peg_generator test_pickle test_pickletools test_platform test_plistlib test_pprint test_print test_profile test_pstats test_pty test_pulldom test_pydoc test_pyexpat test_pyrepl test_quopri test_regrtest test_robotparser test_sax test_shlex test_shutil test_site test_smtplib test_socket test_socketserver test_subprocess test_support test_sys test_tarfile test_tempfile test_threadedtempfile test_timeit test_tokenize test_traceback test_types test_typing test_unittest test_univnewlines test_urllib test_urllib2 test_uuid test_wave test_webbrowser test_winconsoleio test_wsgiref test_xml_dom_xmlbuilder test_xml_etree test_xml_etree_c test_xmlrpc test_xpickle test_zipapp test_zipfile test_zipimport test_zoneinfo test_zstd

[x] lib: cpython/Lib/traceback.py
[x] test: cpython/Lib/test/test_traceback.py (TODO: 4)

dependencies:

  • traceback

dependent tests: (161 tests)

  • traceback: test_asyncio test_builtin test_code_module test_contextlib test_contextlib_async test_coroutines test_dictcomps test_exceptions test_http_cookiejar test_importlib test_iter test_listcomps test_pyexpat test_setcomps test_socket test_ssl test_subprocess test_sys test_threadedtempfile test_threading test_traceback test_unittest test_with test_zipimport
    • code:
      • pdb: test_pdb
      • sqlite3.main: test_sqlite3
    • concurrent.futures.process: test_compileall test_concurrent_futures
    • http.cookiejar: test_urllib2
      • urllib.request: test_pathlib test_pydoc test_sax test_site test_urllib test_urllib2_localnet test_urllib2net test_urllibnet
    • logging: test_asyncio test_decimal test_genericalias test_hashlib test_logging test_pkgutil test_support test_unittest
      • hashlib: test_hmac test_smtplib test_tarfile test_unicodedata
      • multiprocessing.util: test_asyncio test_concurrent_futures
      • venv: test_venv
    • multiprocessing: test_fcntl test_memoryview test_multiprocessing_main_handling test_re
    • py_compile: test_argparse test_cmd_line_script test_importlib test_modulefinder test_py_compile test_runpy
      • zipfile: test_shutil test_zipapp test_zipfile test_zipfile64 test_zipimport_support
    • pydoc: test_enum
      • xmlrpc.server: test_docxmlrpc test_xmlrpc
    • socketserver: test_imaplib test_socketserver test_wsgiref
    • threading: test_android test_asyncio test_bytes test_bz2 test_code test_concurrent_futures test_context test_ctypes test_email test_external_inspection test_fork1 test_frame test_ftplib test_functools test_gc test_httplib test_httpservers test_importlib test_inspect test_io test_ioctl test_itertools test_largefile test_linecache test_opcache test_pathlib test_poll test_poplib test_pyrepl test_queue test_robotparser test_sched test_signal test_sqlite3 test_super test_syslog test_termios test_threading_local test_time test_weakref test_winreg test_zstd
      • bdb: test_bdb
      • dummy_threading: test_dummy_threading
      • importlib.util: test_asdl_parser test_ctypes test_doctest test_importlib test_reprlib
      • queue: test_dummy_thread
      • subprocess: test_asyncio test_atexit test_audit test_c_locale_coercion test_cmd_line test_ctypes test_dtrace test_embed test_faulthandler test_file_eintr test_gzip test_json test_launcher test_msvcrt test_ntpath test_os test_osx_env test_peg_generator test_platform test_plistlib test_pyrepl test_quopri test_regrtest test_repl test_script_helper test_select test_sys_settrace test_sysconfig test_tempfile test_unittest test_utf8_mode test_wait3 test_webbrowser test_xpickle
      • sysconfig: test_posix test_tools
      • trace: test_trace
    • timeit: test_timeit

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@youknowone
youknowone force-pushed the typelock branch 5 times, most recently from 1865733 to da1d518 Compare March 23, 2026 05:06
@youknowone
youknowone marked this pull request as ready for review March 23, 2026 09:09

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
extra_tests/custom_text_test_runner.py (1)

391-396: ⚠️ Potential issue | 🟡 Minor

Inconsistent access pattern may cause AttributeError for plain functions.

Line 392-394 still uses the old __func__.__dict__ access pattern while line 395 uses the new helper. For plain functions (without __func__), this will raise AttributeError before the helper is even called.

Proposed fix
         if self.test_types:
-            if "test_type" in getattr(
-                test, test._testMethodName
-            ).__func__.__dict__ and set([s.lower() for s in self.test_types]) == set(
+            if "test_type" in _get_method_dict(test) and set([s.lower() for s in self.test_types]) == set(
                 [s.lower() for s in _get_method_dict(test)["test_type"]]
             ):
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@extra_tests/custom_text_test_runner.py` around lines 391 - 396, The
conditional uses getattr(test, test._testMethodName).__func__.__dict__ which
will raise AttributeError for plain functions; replace that access with the safe
helper used on the next line by calling _get_method_dict(test) consistently:
check for "test_type" in _get_method_dict(test) and compare set([s.lower() for s
in self.test_types]) against set([s.lower() for s in
_get_method_dict(test)["test_type"]]) so you never access __func__ directly
(symbols: self.test_types, test._testMethodName, _get_method_dict, "test_type").
🧹 Nitpick comments (1)
crates/vm/src/frame.rs (1)

7436-7436: Keep the remaining attr specializers on one type snapshot.

These sites now start from version_for_specialization(), but the surrounding specializer still does later MRO/dict/descriptor inspection after that call. That leaves LOAD_ATTR/STORE_ATTR on a looser snapshot than the new __getitem__ / __init__ paths. I’d either switch these to an atomic lookup helper as well or revalidate the version right before publishing the specialized opcode/cache.

Also applies to: 7474-7474, 7694-7694, 7729-7729, 9121-9121

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` at line 7436, The attr specializers call
cls.version_for_specialization() early but then perform additional
MRO/dict/descriptor inspection, leaving LOAD_ATTR/STORE_ATTR snapshots looser
than the new __getitem__/__init__ paths; update the attr-specializer flows (the
sites calling version_for_specialization(), e.g. the code around
version_for_specialization at the lines flagged and the logic that publishes the
specialized opcode/cache for LOAD_ATTR/STORE_ATTR) to either perform the whole
lookup atomically via a helper (similar to the __getitem__/__init__ path) or
revalidate/refresh the class version immediately before publishing the
specialized opcode/cache so the snapshot used for specialization is exact;
locate references to version_for_specialization, the LOAD_ATTR/STORE_ATTR
specialization code path, and the publish/emit-cache logic and ensure the
version is checked/locked at the final publish point.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/vm/src/builtins/type.rs`:
- Around line 2607-2629: The attribute mutation currently invalidates versions
and mutates the attributes map inside with_type_lock but applies slot table
updates (update_slot) outside the same lock; move the call to update_slot() into
the same with_type_lock closure (alongside modified_inner() and the
attributes.write() insert/shift_remove) so the tp_version_tag, inline-cache
invalidation, dict mutation, and slot rewrite occur atomically under the type
lock, then return the previous value and drop it outside the lock as before;
ensure both assign and remove paths in the closure call update_slot on the
appropriate slot and preserve the existing error handling (references:
with_type_lock, modified_inner, attributes.write(), update_slot).
- Around line 1427-1443: Compute and validate the new bases/MROs for zelf and
all descendants before mutating the live structures: inside with_type_lock, do
not immediately assign to zelf.bases; instead build the candidate bases and run
PyType::resolve_mro for zelf and every subclass (the same logic as
update_mro_recursively) into a temporary mapping of PyType -> new_mro
(preserving mro[0] where needed) and return any error if resolution fails; only
after all resolve_mro calls succeed, assign *zelf.bases.write() = bases and then
write each cls.mro from the prepared mapping and proceed (this ensures
update_mro_recursively-like validation happens before any live mutation).
- Around line 1436-1439: The loop over cls.subclasses currently unwraps weak
refs and downcasts (calling upgrade().unwrap() and downcast_ref().unwrap()),
which can panic on stale weakrefs; instead, change the loop to skip entries that
fail to upgrade or fail the downcast by using conditional checks (e.g., if let
Some(strong) = subclass.upgrade() { if let Some(subclass_pytype) =
strong.downcast_ref::<Py<PyType>>() { update_mro_recursively(subclass_pytype,
vm)? } } ) so stale weakrefs are ignored rather than unwrapping and crashing;
apply the same non-unwrapping pattern to the other occurrences around
update_mro_recursively (the similar block at lines ~1451-1459).
- Around line 1427-1449: The MRO update only rewrites bases/mro for zelf and
leaves derived metadata (per-class slot tables and __base__) stale on
subclasses; modify update_mro_recursively (inside with_type_lock) so that after
computing and writing each class's mro you also recompute and write its
base/__base__ (the same logic used for the original class), call that class's
init_slots(&vm.ctx) to rebuild its slots, and call modified_inner() to
invalidate caches for that class before recursing into its subclasses; ensure
you perform these steps for zelf as well (not just at the top level) so every
descendant gets updated slot tables and base-chain info.
- Around line 454-458: with_type_lock currently takes a zero-arg closure so
callers (like assign_version_tag via version_for_specialization /
find_name_in_mro) can end up executing code outside the held lock; change
with_type_lock signature to fn with_type_lock<R>(vm: &VirtualMachine, f: impl
FnOnce(&VirtualMachine) -> R) -> R, acquire the lock into _guard as before and
call f(vm) while the guard is in scope, then update all call sites (notably
version_for_specialization, assign_version_tag, find_name_in_mro and the block
around 490-503) to accept the &VirtualMachine parameter so the work runs while
type_mutex is held, preserving the invariant used by modified_inner.

---

Outside diff comments:
In `@extra_tests/custom_text_test_runner.py`:
- Around line 391-396: The conditional uses getattr(test,
test._testMethodName).__func__.__dict__ which will raise AttributeError for
plain functions; replace that access with the safe helper used on the next line
by calling _get_method_dict(test) consistently: check for "test_type" in
_get_method_dict(test) and compare set([s.lower() for s in self.test_types])
against set([s.lower() for s in _get_method_dict(test)["test_type"]]) so you
never access __func__ directly (symbols: self.test_types, test._testMethodName,
_get_method_dict, "test_type").

---

Nitpick comments:
In `@crates/vm/src/frame.rs`:
- Line 7436: The attr specializers call cls.version_for_specialization() early
but then perform additional MRO/dict/descriptor inspection, leaving
LOAD_ATTR/STORE_ATTR snapshots looser than the new __getitem__/__init__ paths;
update the attr-specializer flows (the sites calling
version_for_specialization(), e.g. the code around version_for_specialization at
the lines flagged and the logic that publishes the specialized opcode/cache for
LOAD_ATTR/STORE_ATTR) to either perform the whole lookup atomically via a helper
(similar to the __getitem__/__init__ path) or revalidate/refresh the class
version immediately before publishing the specialized opcode/cache so the
snapshot used for specialization is exact; locate references to
version_for_specialization, the LOAD_ATTR/STORE_ATTR specialization code path,
and the publish/emit-cache logic and ensure the version is checked/locked at the
final publish point.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: a8a615eb-9d3f-4cd2-a1b8-841c4089a979

📥 Commits

Reviewing files that changed from the base of the PR and between 8c01615 and da1d518.

⛔ Files ignored due to path filters (1)
  • Lib/test/test_class.py is excluded by !Lib/**
📒 Files selected for processing (8)
  • .cspell.json
  • .github/workflows/cron-ci.yaml
  • crates/vm/src/builtins/type.rs
  • crates/vm/src/frame.rs
  • crates/vm/src/stdlib/posix.rs
  • crates/vm/src/vm/interpreter.rs
  • crates/vm/src/vm/mod.rs
  • extra_tests/custom_text_test_runner.py

Comment thread crates/vm/src/builtins/type.rs
Comment thread crates/vm/src/builtins/type.rs Outdated
Comment on lines +1427 to +1443
Self::with_type_lock(vm, || {
*zelf.bases.write() = bases;
// Recursively update the mros of this class and all subclasses
fn update_mro_recursively(cls: &PyType, vm: &VirtualMachine) -> PyResult<()> {
let mut mro =
PyType::resolve_mro(&cls.bases.read()).map_err(|msg| vm.new_type_error(msg))?;
// Preserve self (mro[0]) when updating MRO
mro.insert(0, cls.mro.read()[0].to_owned());
*cls.mro.write() = mro;
for subclass in cls.subclasses.write().iter() {
let subclass = subclass.upgrade().unwrap();
let subclass: &Py<PyType> = subclass.downcast_ref().unwrap();
update_mro_recursively(subclass, vm)?;
}
Ok(())
}
Ok(())
}
update_mro_recursively(zelf, vm)?;
update_mro_recursively(zelf, vm)?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Validate the full __bases__ rewrite before mutating the live hierarchy.

zelf.bases is overwritten before resolve_mro() has succeeded for zelf and every descendant. If any recursive update_mro_recursively() call fails, this setter raises after leaving a partially applied base/MRO graph behind.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/builtins/type.rs` around lines 1427 - 1443, Compute and
validate the new bases/MROs for zelf and all descendants before mutating the
live structures: inside with_type_lock, do not immediately assign to zelf.bases;
instead build the candidate bases and run PyType::resolve_mro for zelf and every
subclass (the same logic as update_mro_recursively) into a temporary mapping of
PyType -> new_mro (preserving mro[0] where needed) and return any error if
resolution fails; only after all resolve_mro calls succeed, assign
*zelf.bases.write() = bases and then write each cls.mro from the prepared
mapping and proceed (this ensures update_mro_recursively-like validation happens
before any live mutation).

Comment thread crates/vm/src/builtins/type.rs Outdated
Comment thread crates/vm/src/builtins/type.rs Outdated
Comment on lines +1436 to +1439
for subclass in cls.subclasses.write().iter() {
let subclass = subclass.upgrade().unwrap();
let subclass: &Py<PyType> = subclass.downcast_ref().unwrap();
update_mro_recursively(subclass, vm)?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Don't unwrap subclass weakrefs during the recursive walk.

subclasses is lazily cleaned, and this branch only appends new weakrefs, so upgrade() can legitimately return None here. Turning that into unwrap() makes a stale weakref crash __bases__ assignment instead of just being skipped.

🔧 Suggested fix
-                for subclass in cls.subclasses.write().iter() {
-                    let subclass = subclass.upgrade().unwrap();
-                    let subclass: &Py<PyType> = subclass.downcast_ref().unwrap();
-                    update_mro_recursively(subclass, vm)?;
-                }
+                let subclasses: Vec<_> = {
+                    let mut subclasses = cls.subclasses.write();
+                    subclasses.retain(|weak| weak.upgrade().is_some());
+                    subclasses.iter().filter_map(|weak| weak.upgrade()).collect()
+                };
+                for subclass in subclasses {
+                    let subclass: &Py<PyType> = subclass.downcast_ref().unwrap();
+                    update_mro_recursively(subclass, vm)?;
+                }

Also applies to: 1451-1459

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/builtins/type.rs` around lines 1436 - 1439, The loop over
cls.subclasses currently unwraps weak refs and downcasts (calling
upgrade().unwrap() and downcast_ref().unwrap()), which can panic on stale
weakrefs; instead, change the loop to skip entries that fail to upgrade or fail
the downcast by using conditional checks (e.g., if let Some(strong) =
subclass.upgrade() { if let Some(subclass_pytype) =
strong.downcast_ref::<Py<PyType>>() { update_mro_recursively(subclass_pytype,
vm)? } } ) so stale weakrefs are ignored rather than unwrapping and crashing;
apply the same non-unwrapping pattern to the other occurrences around
update_mro_recursively (the similar block at lines ~1451-1459).

Comment thread crates/vm/src/builtins/type.rs Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
crates/vm/src/frame.rs (2)

7713-7744: ⚠️ Potential issue | 🔴 Critical

The metaclass descriptor check still has a version-skew window.

mcl_attr is inspected before metaclass_version is captured here. If another thread installs a metaclass data descriptor between those two steps, LoadAttrClass* can still be cached against the post-mutation metaclass version while bypassing the new descriptor precedence.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 7713 - 7744, The metaclass descriptor
check reads mcl_attr before capturing metaclass_version, leaving a race where a
data descriptor could be installed between those steps; to fix, call
mcl.version_for_specialization(_vm) and store metaclass_version (and handle the
zero-version backoff early) before calling mcl.get_attr(attr_name), then only
inspect the attribute's descr_set if metaclass_version is nonzero; reference
mcl.version_for_specialization, metaclass_version, mcl.get_attr, mcl_attr, and
the LoadAttrClass* specialization path when moving the attribute-inspection
after the version capture and preserving the existing adaptive_counter_backoff
behavior when version_for_specialization returns 0.

7433-7439: ⚠️ Potential issue | 🔴 Critical

Capture the type version from the same snapshot as these slot checks.

These blocks still read mutable type behavior (getattro/setattro, __bool__/__len__, tp_new/tp_alloc) before they capture the version they cache against. Under free-threading, another thread can mutate the type in between and make us publish a specialization under the new version while still relying on the old behavior, which can skip a newly-installed __getattribute__, __setattr__, __bool__/__len__, or __new__.

Also applies to: 7477-7490, 8455-8488, 8781-8794, 9106-9112, 9124-9136

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 7433 - 7439, The slot-check code reads
mutable slot pointers (e.g., cls.slots.getattro.load() /
cls.slots.setattro.load(), tp_new/tp_alloc, __bool__/__len__) before capturing
the type version, which can race; fix by first taking a single snapshot of the
type version (read and store the version tag field from cls once into a local,
e.g., version_snapshot) and then read the slot pointers and compute is_default_*
flags, finally publish the specialization tagged with that same
version_snapshot; apply the same change to the equivalent blocks that check
cls.slots.setattro, tp_new/tp_alloc, and __bool__/__len__ (the other occurrences
noted at the listed ranges) so all slot-checks and the cached version come from
the same snapshot.
♻️ Duplicate comments (5)
crates/vm/src/builtins/type.rs (5)

2614-2636: ⚠️ Potential issue | 🔴 Critical

Keep the slot rewrite in the same type-lock transaction.

Lines 2617-2636 publish the dict mutation and version invalidation, but Lines 2638-2643 update the slot table after type_mutex is released. Another thread can observe the new attribute state, assign a fresh version tag, or dispatch through the old slot table in that window.

🔒 Suggested fix
         let _prev_value = Self::with_type_lock(vm, || {
             // Invalidate inline caches before modifying attributes.
             // This ensures other threads see the version invalidation before
             // any attribute changes, preventing use-after-free of cached descriptors.
             zelf.modified_inner();
 
-            if let PySetterValue::Assign(value) = value {
-                Ok(zelf.attributes.write().insert(attr_name, value))
+            let prev_value = if let PySetterValue::Assign(value) = value {
+                zelf.attributes.write().insert(attr_name, value)
             } else {
                 let prev_value = zelf.attributes.write().shift_remove(attr_name); // TODO: swap_remove applicable?
                 if prev_value.is_none() {
                     return Err(vm.new_attribute_error(format!(
                         "type object '{}' has no attribute '{}'",
@@
                         attr_name,
                     )));
                 }
-                Ok(prev_value)
-            }
+                prev_value
+            };
+
+            if attr_name.as_wtf8().starts_with("__") && attr_name.as_wtf8().ends_with("__") {
+                if assign {
+                    zelf.update_slot::<true>(attr_name, &vm.ctx);
+                } else {
+                    zelf.update_slot::<false>(attr_name, &vm.ctx);
+                }
+            }
+
+            Ok(prev_value)
         })?;
-
-        if attr_name.as_wtf8().starts_with("__") && attr_name.as_wtf8().ends_with("__") {
-            if assign {
-                zelf.update_slot::<true>(attr_name, &vm.ctx);
-            } else {
-                zelf.update_slot::<false>(attr_name, &vm.ctx);
-            }
-        }
         Ok(())

Also applies to: 2638-2643

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/builtins/type.rs` around lines 2614 - 2636, The dict mutation,
version invalidation, and the slot-table rewrite must occur inside the same type
lock to avoid a race; move the slot-table update logic into the closure passed
to Self::with_type_lock so modified_inner(), the
attributes.write().insert/shift_remove mutation (handling
PySetterValue::Assign), and the slot table rewrite/update happen before the lock
is released, but keep dropping the previous value outside the lock by returning
it from the closure (as _prev_value) so its destructor runs after with_type_lock
returns.

1434-1450: ⚠️ Potential issue | 🔴 Critical

Validate the full __bases__ rewrite before touching live state.

Line 1435 overwrites zelf.bases before any resolve_mro() call succeeds. If update_mro_recursively() then fails for zelf or any descendant, this setter unwinds after partially mutating the hierarchy.


1437-1456: ⚠️ Potential issue | 🟠 Major

Recompute base and slot tables for every rebased class.

update_mro_recursively() only rewrites mro, and only zelf runs init_slots() on Line 1456. Descendants keep slot tables derived from the old MRO, and self.base / __base__ is never refreshed for any of the rewritten types.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/builtins/type.rs` around lines 1437 - 1456,
update_mro_recursively currently only rewrites each type's mro, but doesn't
refresh per-class base/slot state; update the function so that after computing
and assigning mro it also recomputes and writes the class base/__base__ (e.g.
set cls.base to the appropriate MRO entry) and calls cls.init_slots(&vm.ctx) and
cls.modified_inner() for that class (instead of only calling init_slots/modified
on zelf after recursion). Refer to update_mro_recursively, PyType::resolve_mro,
cls.mro, cls.base, cls.subclasses, modified_inner, and init_slots to locate
where to update base and reinitialize slots for each rewritten subclass.

1443-1446: ⚠️ Potential issue | 🟠 Major

Skip dead subclass weakrefs instead of unwrapping.

subclasses is lazily cleaned. Line 1444's upgrade().unwrap() can legitimately fail and turn __bases__ assignment into a panic instead of just ignoring a stale weakref.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/builtins/type.rs` around lines 1443 - 1446, The loop currently
calls upgrade().unwrap() (and then downcast_ref().unwrap()) on entries in
cls.subclasses, which can panic on dead weakrefs; change it to safely skip stale
weakrefs by checking the Option/Result: replace the unwrap chain with a
conditional that first does if let Some(strong) = subclass.upgrade() { if let
Some(sub_pytype) = strong.downcast_ref::<Py<PyType>>() {
update_mro_recursively(sub_pytype, vm)?; } } so dead weakrefs or unexpected
types are ignored instead of causing a panic.

496-508: ⚠️ Potential issue | 🔴 Critical

Serialize the public version-tag path too.

assign_version_tag() still just forwards to _inner, and Line 1138 still reaches it from find_name_in_mro() without type_mutex. That leaves the base-before-subclass tagging invariant racing with modified_inner(), so a subclass can still end up with a nonzero tp_version_tag after its base was reset to 0.

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/vm/src/builtins/type.rs`:
- Around line 1607-1617: The annotations cache cleanup should run
unconditionally: inside the closure passed to Self::with_type_lock (around
modified_inner, attributes.write(), and the attrs.insert(identifier!(vm,
__annotate_func__), value) call), always remove any existing identifier!(vm,
__annotations_cache__) instead of only calling attrs.swap_remove(...) when
!vm.is_none(&value); i.e., drop the vm.is_none conditional and unconditionally
call attrs.swap_remove(identifier!(vm, __annotations_cache__)) before inserting
the new __annotate_func__ value so assigning None will also clear the cached
__annotations__.

---

Outside diff comments:
In `@crates/vm/src/frame.rs`:
- Around line 7713-7744: The metaclass descriptor check reads mcl_attr before
capturing metaclass_version, leaving a race where a data descriptor could be
installed between those steps; to fix, call mcl.version_for_specialization(_vm)
and store metaclass_version (and handle the zero-version backoff early) before
calling mcl.get_attr(attr_name), then only inspect the attribute's descr_set if
metaclass_version is nonzero; reference mcl.version_for_specialization,
metaclass_version, mcl.get_attr, mcl_attr, and the LoadAttrClass* specialization
path when moving the attribute-inspection after the version capture and
preserving the existing adaptive_counter_backoff behavior when
version_for_specialization returns 0.
- Around line 7433-7439: The slot-check code reads mutable slot pointers (e.g.,
cls.slots.getattro.load() / cls.slots.setattro.load(), tp_new/tp_alloc,
__bool__/__len__) before capturing the type version, which can race; fix by
first taking a single snapshot of the type version (read and store the version
tag field from cls once into a local, e.g., version_snapshot) and then read the
slot pointers and compute is_default_* flags, finally publish the specialization
tagged with that same version_snapshot; apply the same change to the equivalent
blocks that check cls.slots.setattro, tp_new/tp_alloc, and __bool__/__len__ (the
other occurrences noted at the listed ranges) so all slot-checks and the cached
version come from the same snapshot.

---

Duplicate comments:
In `@crates/vm/src/builtins/type.rs`:
- Around line 2614-2636: The dict mutation, version invalidation, and the
slot-table rewrite must occur inside the same type lock to avoid a race; move
the slot-table update logic into the closure passed to Self::with_type_lock so
modified_inner(), the attributes.write().insert/shift_remove mutation (handling
PySetterValue::Assign), and the slot table rewrite/update happen before the lock
is released, but keep dropping the previous value outside the lock by returning
it from the closure (as _prev_value) so its destructor runs after with_type_lock
returns.
- Around line 1437-1456: update_mro_recursively currently only rewrites each
type's mro, but doesn't refresh per-class base/slot state; update the function
so that after computing and assigning mro it also recomputes and writes the
class base/__base__ (e.g. set cls.base to the appropriate MRO entry) and calls
cls.init_slots(&vm.ctx) and cls.modified_inner() for that class (instead of only
calling init_slots/modified on zelf after recursion). Refer to
update_mro_recursively, PyType::resolve_mro, cls.mro, cls.base, cls.subclasses,
modified_inner, and init_slots to locate where to update base and reinitialize
slots for each rewritten subclass.
- Around line 1443-1446: The loop currently calls upgrade().unwrap() (and then
downcast_ref().unwrap()) on entries in cls.subclasses, which can panic on dead
weakrefs; change it to safely skip stale weakrefs by checking the Option/Result:
replace the unwrap chain with a conditional that first does if let Some(strong)
= subclass.upgrade() { if let Some(sub_pytype) =
strong.downcast_ref::<Py<PyType>>() { update_mro_recursively(sub_pytype, vm)?; }
} so dead weakrefs or unexpected types are ignored instead of causing a panic.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: c8c994ce-d646-4364-b6ea-ea561ef96c2b

📥 Commits

Reviewing files that changed from the base of the PR and between da1d518 and f491a45.

📒 Files selected for processing (5)
  • crates/vm/src/builtins/type.rs
  • crates/vm/src/frame.rs
  • crates/vm/src/stdlib/posix.rs
  • crates/vm/src/vm/interpreter.rs
  • crates/vm/src/vm/mod.rs
✅ Files skipped from review due to trivial changes (1)
  • crates/vm/src/stdlib/posix.rs
🚧 Files skipped from review as they are similar to previous changes (2)
  • crates/vm/src/vm/interpreter.rs
  • crates/vm/src/vm/mod.rs

Comment on lines +1607 to +1617
let _prev_values = Self::with_type_lock(vm, || {
self.modified_inner();
let mut attrs = self.attributes.write();
let removed = if !vm.is_none(&value) {
attrs.swap_remove(identifier!(vm, __annotations_cache__))
} else {
None
};
let prev = attrs.insert(identifier!(vm, __annotate_func__), value);
(removed, prev)
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Clear __annotations_cache__ even when __annotate__ becomes None.

Right now the cache is only cleared when the new value is callable. If the class already materialized __annotations__, assigning None here leaves __annotations_cache__ behind, so later reads keep returning the stale dict.

💡 Minimal fix
         let _prev_values = Self::with_type_lock(vm, || {
             self.modified_inner();
             let mut attrs = self.attributes.write();
-            let removed = if !vm.is_none(&value) {
-                attrs.swap_remove(identifier!(vm, __annotations_cache__))
-            } else {
-                None
-            };
+            let removed = attrs.swap_remove(identifier!(vm, __annotations_cache__));
             let prev = attrs.insert(identifier!(vm, __annotate_func__), value);
             (removed, prev)
         });
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/builtins/type.rs` around lines 1607 - 1617, The annotations
cache cleanup should run unconditionally: inside the closure passed to
Self::with_type_lock (around modified_inner, attributes.write(), and the
attrs.insert(identifier!(vm, __annotate_func__), value) call), always remove any
existing identifier!(vm, __annotations_cache__) instead of only calling
attrs.swap_remove(...) when !vm.is_none(&value); i.e., drop the vm.is_none
conditional and unconditionally call attrs.swap_remove(identifier!(vm,
__annotations_cache__)) before inserting the new __annotate_func__ value so
assigning None will also clear the cached __annotations__.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
crates/vm/src/frame.rs (1)

7936-7949: ⚠️ Potential issue | 🔴 Critical

Fix the __getitem__ cache synchronization before expanding this fast path.

Lines 7936-7949 expand BINARY_OP_SUBSCR coverage through cache_getitem_for_specialization(), but the reader side has a memory ordering bug. In crates/vm/src/builtins/type.rs, line 1070 reads getitem_version with Ordering::Relaxed, while the writer at line 1054 stores with Ordering::Release. This breaks the Release-Acquire synchronization protocol—the reader must use Ordering::Acquire to safely pair with the Release store. Without this fix, concurrent type invalidation can surface stale (func, version) pairs, allowing dispatch to obsolete __getitem__ implementations. Change line 1070 to load(Ordering::Acquire) before relying on this cache in the specialization path.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 7936 - 7949, The reader side of the
getitem cache uses a relaxed load and must use acquire semantics to pair with
the release store to avoid seeing stale (func, version) pairs; in the reader
code that consults getitem_version (the load currently using Ordering::Relaxed
in the type lookup path referenced by cache_getitem_for_specialization and the
BINARY_OP_SUBSCR specialization), change the atomic load to use
Ordering::Acquire before relying on the cached (func, version) tuple so it
correctly synchronizes with the writer's store(Ordering::Release).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/vm/src/builtins/type.rs`:
- Around line 1001-1010: The code in the closure passed to Self::with_type_lock
repeats the same tp_version_tag check twice
(self.tp_version_tag.load(Ordering::Acquire) != tp_version); remove the
redundant second if so the version is checked once before calling
ext.specialization_cache.swap_init(Some(init), Some(vm)), keeping the early
return behavior intact and preserving the overall control flow.

---

Outside diff comments:
In `@crates/vm/src/frame.rs`:
- Around line 7936-7949: The reader side of the getitem cache uses a relaxed
load and must use acquire semantics to pair with the release store to avoid
seeing stale (func, version) pairs; in the reader code that consults
getitem_version (the load currently using Ordering::Relaxed in the type lookup
path referenced by cache_getitem_for_specialization and the BINARY_OP_SUBSCR
specialization), change the atomic load to use Ordering::Acquire before relying
on the cached (func, version) tuple so it correctly synchronizes with the
writer's store(Ordering::Release).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: 8a43a5f6-0d1f-49d6-861e-c135fbadbb96

📥 Commits

Reviewing files that changed from the base of the PR and between f491a45 and 84ae152.

📒 Files selected for processing (5)
  • crates/vm/src/builtins/type.rs
  • crates/vm/src/frame.rs
  • crates/vm/src/stdlib/posix.rs
  • crates/vm/src/vm/interpreter.rs
  • crates/vm/src/vm/mod.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • crates/vm/src/vm/interpreter.rs

Comment on lines +1001 to +1010
Self::with_type_lock(vm, || {
if self.tp_version_tag.load(Ordering::Acquire) != tp_version {
return false;
}
if self.tp_version_tag.load(Ordering::Acquire) != tp_version {
return false;
}
ext.specialization_cache.swap_init(Some(init), Some(vm));
true
})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Remove duplicate version check.

Lines 1002-1003 and 1005-1006 perform the identical check twice consecutively. This appears to be a copy-paste error.

🔧 Proposed fix
         Self::with_type_lock(vm, || {
             if self.tp_version_tag.load(Ordering::Acquire) != tp_version {
                 return false;
             }
-            if self.tp_version_tag.load(Ordering::Acquire) != tp_version {
-                return false;
-            }
             ext.specialization_cache.swap_init(Some(init), Some(vm));
             true
         })
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Self::with_type_lock(vm, || {
if self.tp_version_tag.load(Ordering::Acquire) != tp_version {
return false;
}
if self.tp_version_tag.load(Ordering::Acquire) != tp_version {
return false;
}
ext.specialization_cache.swap_init(Some(init), Some(vm));
true
})
Self::with_type_lock(vm, || {
if self.tp_version_tag.load(Ordering::Acquire) != tp_version {
return false;
}
ext.specialization_cache.swap_init(Some(init), Some(vm));
true
})
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/builtins/type.rs` around lines 1001 - 1010, The code in the
closure passed to Self::with_type_lock repeats the same tp_version_tag check
twice (self.tp_version_tag.load(Ordering::Acquire) != tp_version); remove the
redundant second if so the version is checked once before calling
ext.specialization_cache.swap_init(Some(init), Some(vm)), keeping the early
return behavior intact and preserving the overall control flow.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
crates/vm/src/frame.rs (1)

7919-7928: Hoist the cheap gates before lookup_ref_and_version_interned().

This helper now takes the global type_mutex and may allocate a fresh version tag. Since this path can only specialize heap types when eval-frame specialization is enabled, checking those conditions first would avoid paying the lock/version cost on immediate deopts.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/frame.rs` around lines 7919 - 7928, Hoist the cheap guards
before calling cls.lookup_ref_and_version_interned by first checking
cls.slots.flags.has_feature(PyTypeFlags::HEAPTYPE),
!self.specialization_eval_frame_active(vm), and whether the identifier lookup
would be a PyFunction that can_specialize_call(2) (using the same
downcast_ref_if_exact::<PyFunction> and can_specialize_call checks) — only then
call lookup_ref_and_version_interned(identifier!(vm, __getitem__), vm) to obtain
(getitem, type_version); this avoids taking the global lock / allocating a
version tag on fast-failing paths and keep the subsequent logic that uses
type_version and cls.cache_getitem_for_specialization unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@crates/vm/src/frame.rs`:
- Around line 7919-7928: Hoist the cheap guards before calling
cls.lookup_ref_and_version_interned by first checking
cls.slots.flags.has_feature(PyTypeFlags::HEAPTYPE),
!self.specialization_eval_frame_active(vm), and whether the identifier lookup
would be a PyFunction that can_specialize_call(2) (using the same
downcast_ref_if_exact::<PyFunction> and can_specialize_call checks) — only then
call lookup_ref_and_version_interned(identifier!(vm, __getitem__), vm) to obtain
(getitem, type_version); this avoids taking the global lock / allocating a
version tag on fast-failing paths and keep the subsequent logic that uses
type_version and cls.cache_getitem_for_specialization unchanged.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: c9408811-8c8e-494f-9382-cff7f814ef3e

📥 Commits

Reviewing files that changed from the base of the PR and between 84ae152 and 5a29191.

📒 Files selected for processing (5)
  • crates/vm/src/builtins/type.rs
  • crates/vm/src/frame.rs
  • crates/vm/src/stdlib/posix.rs
  • crates/vm/src/vm/interpreter.rs
  • crates/vm/src/vm/mod.rs
🚧 Files skipped from review as they are similar to previous changes (2)
  • crates/vm/src/vm/interpreter.rs
  • crates/vm/src/vm/mod.rs

Comment thread crates/vm/src/builtins/type.rs Outdated
let subclass: &Py<PyType> = subclass.downcast_ref().unwrap();
update_mro_recursively(subclass, vm)?;
Self::with_type_lock(vm, || {
*zelf.bases.write() = bases;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we still need bases lock?

Comment thread crates/vm/src/builtins/type.rs Outdated
PyType::resolve_mro(&cls.bases.read()).map_err(|msg| vm.new_type_error(msg))?;
// Preserve self (mro[0]) when updating MRO
mro.insert(0, cls.mro.read()[0].to_owned());
*cls.mro.write() = mro;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mro?

@youknowone
youknowone force-pushed the typelock branch 2 times, most recently from 37ff5f7 to 8326db9 Compare April 13, 2026 15:54
@youknowone
youknowone force-pushed the typelock branch 5 times, most recently from e637be3 to 6ed555e Compare July 7, 2026 00:00
@youknowone youknowone changed the title Add global type mutex for thread-safe type operations Thread-safe type operations: type lock, QSBR type-cache reclamation, GC stop-the-world, and interpreter optimizations Jul 7, 2026
youknowone added 21 commits July 7, 2026 18:15
Both TestLoadSuperAttr tests now pass; remove their expectedFailure markers.

Assisted-by: Claude
CallAllocAndEnterInit ran __init__ inside a synthetic init-cleanup shim
frame whose code carried the __init__ name. Since the thread frame stack
is derived from the frame chain, that shim frame was visible to
sys._getframe, f_back walks, traceback construction and inspect.stack /
inspect.trace. Its code object has empty co_positions, so a stack walk
that reached it raised StopIteration inside inspect._get_code_position,
cascading into unrelated failures (test_inspect trace/stack/frame,
asyncio source traceback).

Call __init__ directly via run_frame, enforce the __init__() should
return None contract inline, and drop the shim: the init-cleanup code
object, its builder, with_frame_untraced, monitoring_disabled_for_code,
and the extra-frame datastack/recursion budget. Removing the second
frame per construction also cuts the specialization's per-call cost.

Assisted-by: Claude
On unix threading builds, publish each thread's top Python frame in a
single relaxed AtomicPtr store from set_current_frame instead of pushing
onto a parking_lot::Mutex<Vec<FramePtr>> per call. Cross-thread readers
(sys._current_frames, cross-thread f_back, faulthandler.dump_traceback,
the GC unreachable debug-assert) run under stop-the-world and walk the
published top frame down the Frame::previous chain; the owning thread is
then parked at a safepoint, so the pointer and the frames it reaches are
quiescent and alive. The faulthandler watchdog is a plain OS thread that
cannot stop-the-world, so it walks the chain lock-free and best-effort.

Non-unix threading builds have no stop-the-world and keep the existing
mutex-guarded frame stack unchanged.

Assisted-by: Claude
with_frame saves and restores the shared exc_info slot around every
Python call to contain frames that leave it unbalanced. Cache a
has_exc_handling bit on PyCode at creation, set when the bytecode
contains any opcode that calls vm.set_exception (PushExcInfo, PopExcept,
CheckEgMatch, EndAsyncFor, InstrumentedEndAsyncFor). A callee whose code
has none of these cannot mutate the slot, so the save and restore are
skipped for it. Generators go through resume_gen_frame and are
unaffected.

Assisted-by: Claude
Optimized (function) frames now expose `f_locals` as a `FrameLocalsProxy`
implementing PEP 667 semantics instead of a cached snapshot dict:

- reads go live through the fast-local slots; each access mints a fresh
  proxy; keys that do not name a fast local are stored in a per-frame
  `f_extra_locals` side dict and folded into `locals()`.
- writes to a fast-local key store into the slot (or its cell) in place;
  deleting a fast local raises ValueError; extra keys delete normally.
- full mapping protocol: keys/values/items (lists), get/pop/setdefault,
  update (dict or FrameLocalsProxy only), __or__/__ior__/__ror__ (dict
  result), copy (plain dict), __reduce__ blocks pickling/copy, repr with
  recursion guard, mapping-pattern and Mapping ABC support.

Class/module/exec frames keep returning their namespace mapping directly.
Cross-thread access to a frame running on another thread still raises
RuntimeError.

A closed generator now keeps its frame locals when a durable frame
reference escaped (f_locals proxy, sys._getframe, f_back), matching
take_ownership; the escape is tracked with a per-frame flag. The
snapshot-then-fold locals_to_fast/locals_dirty write-back is retired
since proxy writes reach the slots directly.

Assisted-by: Claude
When a frame escapes its execution (referenced through a traceback,
`sys._getframe`, `f_locals`, ...), capture a strong reference to its
caller at release time. `f_back` consults it once the caller has left
the live frame chain, so the Python-visible frame chain survives return.
The retained reference is a GC-traversed edge and is cleared by
`frame.clear()`, so ancestor chains stay collectable.

Assisted-by: Claude
Make check_c_stack_overflow one-sided (trip whenever the stack pointer
is below the soft limit) so a single native frame larger than the margin
cannot step past the danger band undetected.

Raise the debug STACK_MARGIN_BYTES from 4096 to 16384 words so the margin
exceeds a single debug interpreter frame, leaving headroom to raise
RecursionError. Release margin unchanged.

Clamp the soft-limit margin to half the stack so small explicit thread
stacks do not get a soft limit above their stack top.

Assisted-by: Claude
Exception construction went through into_ref_with_type, which eagerly
allocated an empty instance dict for every HAS_DICT type. Add
into_ref_with_type_lazy_dict, which builds the instance with an
unallocated dict slot, and route the four exception construction sites
(PyBaseException, PyOSError, OSErrorBuilder, PyBaseExceptionGroup)
through it. The dict now materializes on first attribute write or
__dict__ access via the existing get_or_insert path.

add_note and PyImportError::slot_init now obtain the dict through
object_get_dict so they materialize it instead of assuming it exists.

A freshly constructed exception no longer reports an empty dict in
gc.get_referents, matching the reference interpreter.

Assisted-by: Claude
Route vm.new_exception() through into_ref_with_type_lazy_dict so
internally raised exceptions (new_type_error, new_value_error, etc.)
start without an instance dict, matching the slot_new path. The dict
is materialized on the first attribute write or __dict__ access.

Assisted-by: Claude
Reject keyword arguments and require exactly one positional argument,
raising TypeError with the "takes no keyword arguments" and "takes
exactly one argument (N given)" messages.

Assisted-by: Claude
- Drop stale vm.frames reference from the release_datastack_frame
  uniqueness argument.
- Assert has_exc_handling when unwinding an Except-typed stack slot,
  documenting the invariant that guards the shared exc_info write.
- Truncate the specialized __init__ return-type name to 200 chars,
  matching the unspecialized wrapper.
- Reword two comments to describe behavior without prose references
  to CPython.

Assisted-by: Claude
Wrap the unix stop-the-world registry walk in a scope with
scopeguard::defer! so start_the_world runs on panic, matching the
f_back and get_all_current_frames sites. Add scopeguard to the stdlib
dependencies. Also correct the restore_exception doc comment to name
with_frame after the rename.

Assisted-by: Claude
Assisted-by: Claude
Match the builtin_* naming convention of extra_tests/snippets.

Assisted-by: Claude
Reformat with rustfmt, fix import spacing in builtin_type_bases.py with
ruff, and add "pointee" to the cspell word list.

Assisted-by: Claude
`online`, `drain_all`, and `reset_after_fork` are called only from unix
code (thread attach/detach, post-fork reset), and `offline` from unix
code plus a unit test. Gate them with matching cfg so `-D dead_code`
does not fire on non-unix targets.

Assisted-by: Claude
PyFunction::traverse visited the cells inside the closure tuple instead
of the tuple object itself, so the tuple's reference from the function
was never subtracted during cycle collection. A closure tuple that
reached back to its function (or, through a frame retained by f_back,
to a Thread) was stranded as a false GC root and never collected,
leaking the whole cycle. Visit the tuple itself, matching clear().

Assisted-by: Claude
test_asyncgen_finalization_by_gc and
test_asyncgen_finalization_by_gc_in_other_thread now pass; GC finalizes
the async generators.

Assisted-by: Claude
The servername-callback reference cycle is now collected by GC.

Assisted-by: Claude
Change `cfg(all(unix, feature = "threading"))` to `cfg(feature = "threading")`
on the stop-the-world machinery so it also compiles and runs on non-unix
threading builds:

- StopTheWorldState, its stats, stw_trace, and the stop_the_world field
- ThreadSlot state/stop_requested/thread fields and their initializers
- wait_while_suspended/attach_thread/detach_thread/suspend_if_needed/do_suspend,
  allow_threads, stop_requested_for_current_thread, and the enter_vm /
  VmBootstrapGuard / attach_current_thread / release_current_thread / cleanup
  attach-state wiring
- eval_breaker_tripped, check_signals, run_scheduled_gc, signal GC_BIT /
  schedule_gc / take_gc_scheduled, and the frame.rs safepoint call
- CollectStopTheWorld and its use in collect_inner
- QSBR::online/offline, now called from attach/detach on all threading builds
- debug_assert_current_thread_attached and its type-cache call sites

maybe_collect defers auto-collection to the bytecode safepoint on every
threading build instead of only unix; non-threading builds keep the inline
collect. stw_trace writes to std stderr on non-unix.

top_frame publishing (CURRENT_TOP_FRAME_SLOT, set_current_frame), the
frame-walk debug assert in collect_inner, and the fork reinit helpers remain
unix-only; non-unix keeps ThreadSlot::frames for introspection.

Assisted-by: Claude
Wrap the blocking Windows wait calls in `vm.allow_threads` so the calling
thread transitions ATTACHED -> DETACHED for the duration of the wait:

- _winapi: WaitForSingleObject, WaitForMultipleObjects,
  BatchedWaitForMultipleObjects, ConnectNamedPipe, ReadFile,
  Overlapped.GetOverlappedResult
- _overlapped: Overlapped.getresult

These previously blocked while ATTACHED, so a stop-the-world requester
could never suspend the thread and spun in its wait loop indefinitely.
…bject

The refcount test asserted exact incref/decref deltas on PyInt's shared
type object. That object's reference count is perturbed by the other capi
tests running in parallel, and is immortal under some interpreter
configurations, so the deltas were not reliably +1/-1. Assert them on a
freshly created, uniquely owned list whose reference count is private to
the test and mortal.

Assisted-by: Claude
The `#[pyexception]` struct macro now forwards a `traverse` option to the
generated `#[pyclass]`, and `ExceptionItemMeta` accepts the `traverse`
key. `PyOSError` is marked `traverse = "manual"`, so `HAS_TRAVERSE` is
true and OSError-family instances are tracked at creation and traversed
by the collector.

`PyOSError::traverse` now visits the underlying `PyBaseException`
(traceback, cause, context, args) instead of `PyException::try_traverse`,
which was a no-op because `PyException` has `HAS_TRAVERSE = false`.
The BlockingIOError reference cycle is now collected by GC.

Assisted-by: Claude
@youknowone
youknowone merged commit c41180d into RustPython:main Jul 8, 2026
27 checks passed
@youknowone
youknowone deleted the typelock branch July 8, 2026 06:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant