tachyon fails to attach with `_ctypes` imported

[mstarodub]

Bug report

Bug description:

test.py:

import os
import polars

print(os.getpid())
while True:
  pass

running it gives
uv run -p 3.15 --with polars test.py:
9586

attach fails (sudo run mode seems to work)
sudo uv run -p 3.15 python -m profiling.sampling attach 9586:

Password:
Can't determine the Python version of the remote process

if I change it to import polars as pl, attach fails with:
Failed to read debug offsets structure from remote process

CPython versions tested on:

3.15

Operating systems tested on:

macOS

Linked PRs

• gh-144563: Fix remote debugging with duplicate libpython mappings from ctypes #144595
• [3.14] gh-144563: Fix remote debugging with duplicate libpython mappings from ctypes (GH-144595) #144655

[johnslavik]

I think this is MacOS-specific -- reproduced it there.

I couldn't reproduce it on Linux:

Details

❯ uv run -p 3.15 --with polars python t.py
78672
^Z
[1]+  Stopped                 uv run -p 3.15 --with polars python t.py

✦ ❯ uv run -p 3.15 python -m profiling.sampling attach -d 1 78672
Captured 1 001 samples in 1,00 seconds
Sample rate: 1 000,99 samples/sec
Error rate: 0,00
Profile Stats:
       nsamples   sample%   tottime (s)    cumul%   cumtime (s)  filename:lineno(function)
      1001/1001     100.0         1.001     100.0         1.001  t.py:6(<module>)

Legend:
  nsamples: Direct/Cumulative samples (direct executing / on call stack)
  sample%: Percentage of total samples this function was directly executing
  tottime: Estimated total time spent directly in this function
  cumul%: Percentage of total samples when this function was on the call stack
  cumtime: Estimated cumulative time (including time in called functions)
  filename:lineno(function): Function location and name

Summary of Interesting Functions:

Functions with Highest Direct/Cumulative Ratio (Hot Spots):
  1.000 direct/cumulative ratio, 100.0% direct samples: t.py:(<module>)

Functions with Highest Call Frequency (Indirect Calls):

Functions with Highest Call Magnification (Cumulative/Direct):

if I change it to import polars as pl, attach fails with:
Failed to read debug offsets structure from remote process

Couldn't reproduce on MacOS.

[pablogsal]

Couldn't reproduce on MacOS.

Sorry then does this means that you cannot reproduce anywhere?

[pablogsal]

Bug report

Bug description:

test.py:

import os
import polars

print(os.getpid())
while True:
  pass

running it gives
uv run -p 3.15 --with polars test.py:
9586

attach fails (sudo run mode seems to work)
sudo uv run -p 3.15 python -m profiling.sampling attach 9586:

Password:
Can't determine the Python version of the remote process

if I change it to import polars as pl, attach fails with:
Failed to read debug offsets structure from remote process

CPython versions tested on:

3.15

Operating systems tested on:

macOS

@mstarodub does it work if you don't import polars?

[pablogsal]

Also can you try with the current main without uv in the mix?

[johnslavik]

Couldn't reproduce on MacOS.

Sorry then does this means that you cannot reproduce anywhere?

No. I couldn't reproduce only the quoted part (if I change it to import polars as pl).

Importing polars does something to the runtime that Tachyon can no longer attach to it, and this happens only on MacOS, in this specific uv setup.
Before import polars is executed, Tachyon can attach. After it is executed, Tachyon no longer recognizes the Python version.

Also can you try with the current main without uv in the mix?

I'll try to find a uv-free repro, but until then uv is currently necessary with how it manages environments.

This won't reproduce for uv run -p ./python.exe but will reproduce for uv run -p 3.15. Something about different environments, but I couldn't figure out what yet.

[johnslavik]

Also can you try with the current main without uv in the mix?

This won't reproduce without uv in the mix, and even with uv but on main.
I'd still investigate to understand what exactly does import polars do that Tachyon can no longer attach, unless you recognize this as a fixed regression? Maybe I'll try to bisect if I can't find any sensible idea on what happens here.

[pablogsal]

CC @zanieb

[johnslavik]

Smaller repro:

# t.py
import os
import _remote_debugging

print(_remote_debugging.RemoteUnwinder(os.getpid()))
import polars
print(_remote_debugging.RemoteUnwinder(os.getpid()))

❯ uv run -p 3.15 --with polars t.py
<_remote_debugging.RemoteUnwinder object at 0x7cdc00000>
Traceback (most recent call last):
  File "/Users/.../Python/cpython/t.py", line 9, in <module>
    print(_remote_debugging.RemoteUnwinder(os.getpid()))
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
RuntimeError: Can't determine the Python version of the remote process

❯ uv run -p ./python.exe --with polars t.py
<_remote_debugging.RemoteUnwinder object at 0xaeb490020>
<_remote_debugging.RemoteUnwinder object at 0xaeb490020>

[pablogsal]

Also can you try with the current main without uv in the mix?

This won't reproduce without uv in the mix, and even with uv but on main.
I'd still investigate to understand what exactly does import polars do that Tachyon can no longer attach, unless you recognize this as a fixed regression? Maybe I'll try to bisect if I can't find any sensible idea on what happens here.

I do not recognise the problem from the top of my head but we have fixed a bunch of stuff around the area where we recognise the offsets and the memory map.

uv python also statically compiles everything so perhaps there is dome weird side effect to that

[pablogsal]

@johnslavik can you try to repro by using the same commit uv is using for the 3.15 version?