Provide improved error message for regex parse failures

[jhance]

Feature or enhancement

Proposal:

I'm porting a really large (multi-million line) application from 3.10 to 3.11. One error that has occurred a few times is from this change:

(?aiLmsux)

    (One or more letters from the set 'a', 'i', 'L', 'm', 's', 'u', 'x'.) The group matches the empty string; the letters set the corresponding flags for the entire regular expression:

    Changed in version 3.11: This construction can only be used at the start of the expression.

Fixing the regexes is not really that hard, but one annoyance is that the error message:

re.error: global flags not at the start of the expression at position 2

Tells me what position the problem is at, but not what the regex is! This isn't a problem if the regex being compiled is some constant, but oftentimes its some parameter to a function and the actual regex is hidden away in a completely different file, and I'm left having to add prints/additional exception handling just to find out what the actual regex was. As such, I think it would be quite useful if the regex pattern was included in the exception error message for debugging.

I think this should be simple to implement and don't see any downsides.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

[serhiy-storchaka]

It may be a part of a larger feature. Currently the traceback module has a special code for SyntaxError to output not only the exception type and message, but the source line where the error occurs, and highlight the position in it. I think that we should generalize this. Add a protocol to allow other exceptions provide the source of the error. It would be useful in JSONDecodeError and other parsing errors. It would be useful even in UnicodeEncodeError and UnicodeDecodeError (the source can be bytes, not only str).

One problem is that while we expect that Python source lines fit on the terminal screen, the source lines of generated RE, JSON, etc can be arbitrary long. So the formatting routine should be more sophisticated: show only important parts (including the head, the error position and the tail) of the long line. Something like what unittest does when reports long strings (although that code needs to be improved too).