[mypyc] Fix cross-group call to inherited __mypyc_defaults_setup#21481
Open
georgesittas wants to merge 2 commits into
Open
[mypyc] Fix cross-group call to inherited __mypyc_defaults_setup#21481georgesittas wants to merge 2 commits into
georgesittas wants to merge 2 commits into
Conversation
With `separate=True` and cross-module inheritance, when a subclass module
is recompiled incrementally without its parent (parent loaded from
mypy's cache, so `ClassDef.defs.body` is empty), `find_attr_initializers`
gathers no defaults from the parent. The subclass therefore has no
`__mypyc_defaults_setup` of its own, and `ClassIR.get_method` returns
the parent's. `emit_attr_defaults_func_call` then emitted a raw
`CPyDef_<parent>___...` call with no cross-group export-table prefix,
producing C that fails to compile:
error: call to undeclared function
'CPyDef_<parent_module>___<Parent>_____mypyc_defaults_setup'
The parent's header only declares the function as a pointer inside
`struct export_table_<group>`, so the symbol isn't reachable as a free
function from the subclass's compilation unit.
Apply `emitter.get_group_prefix(defaults_fn.decl)` at this call site,
matching the pattern already used by `emit_setup_or_dunder_new_call`,
`generate_constructor_for_class`, and the other cross-group call sites
in `emitclass.py`. `get_group_prefix` returns `""` for same-group calls
(intra-group behaviour unchanged) and `"exports_<group>."` when the
target lives in a different group; it also registers the target group
in `context.group_deps` so the right header gets `#include`d.
Reproducer (`base.py` with attribute defaults, `child.py` empty subclass,
`mypycify([...], separate=True)`): cold build succeeds, then touching
only `child.py` and rebuilding previously failed with the
implicit-declaration error. Generated C now correctly emits
`exports_base.CPyDef_base___Parent_____mypyc_defaults_setup(...)` and
`Child().x` returns the inherited default.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reproduces the bug fixed in the parent commit: under TestRunSeparate, the subclass module gets recompiled while the parent module is loaded from mypy's cache (so `ClassDef.defs.body` is empty and the subclass inherits no own `__mypyc_defaults_setup`). The emitted call to the parent's setup function must use the cross-group `exports_<group>.` prefix or the generated C fails to compile. The test passes under TestRun and TestRunMultiFile (which don't exercise cross-group calls) and fails under TestRunSeparate without the fix. Verified by temporarily reverting `emit_attr_defaults_func_call` to the pre-fix form and observing the implicit-declaration error in `__native_other_a.c`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With
separate=Trueand cross-module inheritance, when only the subclass module is recompiled incrementally and the parent is loaded from mypy's incremental cache,find_attr_initializersgathers no defaults from the parent. The subclass therefore has no__mypyc_defaults_setupof its own, andClassIR.get_methodwalks the MRO and returns the parent's.Without this fix,
emit_attr_defaults_func_callemits a rawCPyDef_<parent>___...(...)call. The parent's header only declares that function as a pointer insidestruct export_table_<group>, so the symbol isn't reachable as a free function from the subclass's compilation unit and clang/gcc fail with:A cold build doesn't hit this because the parent's
defs.bodyis populated (everything is freshly parsed), so the subclass gets its own__mypyc_defaults_setupand the call is intra-group. Likewise, an incremental change that propagates through interface-hash deps to the parent makes it get reparsed too, avoiding the trigger. The bug requires an invalidation pattern that touches the subclass but not the parent.Fix
emit_attr_defaults_func_callinmypyc/codegen/emitclass.pynow appliesemitter.get_group_prefix(defaults_fn.decl)when emitting the call, matching the pattern already used by the other cross-group call sites in this file (emit_setup_or_dunder_new_call,generate_constructor_for_class, etc.).get_group_prefixreturns""for same-group calls (so intra-group behaviour is unchanged) and"exports_<group>."when the target lives in a different group. It also registers the target group incontext.group_depsso the right header gets#included.Tests
Added
testIncrementalCrossModuleInheritedAttrDefaultsinmypyc/test-data/run-multimodule.test, a two-step test that reproduces the bug underTestRunSeparate: parent inother_b.pywith attribute defaults, empty subclass inother_a.py, step 2 modifiesother_a.pyto trigger a recompile without touching the parent. Verified to fail underTestRunSeparate(with the implicit-declaration error) on the unpatched tree and to pass under all three modes (TestRun,TestRunMultiFile,TestRunSeparate) with the fix.Local checks:
pre-commit run --all-files— passpython runtests.py self— passTestRunSeparate+TestRunMultiFilerun tests (110) — passReal-world impact
Surfaced first against
sqlglot-mypy==1.20.0.post6(a downstream of mypy used to compile sqlglot) in CI whenbuild/and.mypy_cache/were preserved across GitHub Actions runs and a PR happened to edit only subclass modules. The incremental compile produced malformed C and failed with the implicit-declaration error.Stripped of sqlglot specifics, the bug requires only
separate=True+ cross-module inheritance + parent with attribute defaults + subclass without its own defaults + an invalidation pattern where the subclass is rechecked but the parent is not. These conditions are common in any non-trivial codebase using mypyc withseparate=True, so this should be considered a latent issue affecting incremental builds of such codebases.