[not crashing] runtime error in tokenizer (pointer index expression overflow)

[qexat]

Bug report

Bug description:

Hello! I am not really sure how to qualify this. It is technically a bug but I assume it's only visible with pydebug builds.

I searched in the issue tracker related issues with keywords like tokenizer and runtime error but I could not find anything.

configure flags

--enable-optimizations --with-valgrind --enable-profiling \
    --enable-pystats --with-pydebug --with-trace-refs \
    --with-assertions --with-lto=full --with-undefined-behavior-sanitizer

Steps to reproduce

Assuming that your build was made with the same flags as mine:

1. Open the REPL
2. Type any function name followed by an opening parenthesis and a backslash, e.g. sum(\ and press enter
3. On the next line, type any expression and press enter

A message in the following form has now been printed:

Parser/tokenizer/file_tokenizer.c:255: runtime error: pointer index expression with base <address> overflowed to <address>

with <address> being replaced by some actual addresses.

Few notes:

• it only happens once ; re-doing the steps will not print the message again (until the REPL is exited and re-opened)
• it does not crash the interpreter

Config

Python version: 3.13.0a2+ (heads/main:c2e8298eba)
OS: EndeavourOS 2022.08.28 64-bit (Linux 6.6.9-arch1-1)
Architecture: x86_64

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

[qexat]

Turns out the function is not required: the key seems to be the combination (\ followed by a newline (and an expression afterwards).