Self-documenting expressions conversion + format-spec

[Lordfirespeed]

Bug report

Bug description:

# initialisation
foo = "hello!"
bar = "hello, world!"
long_name = bar
print(f"{foo = }")
print(f"{bar = }")
print()

# these work as I expected
# they default to !r value conversion
print(f"{f"{foo=}" = }")
print(f"{f"{bar=}" = }")
print(f"{(f"{foo=}" == f"{foo=!r}") = }")
print()

# these do not work as I expected
# they default to !s value conversion, instead of !r
print(f"{f"{foo=:}" = }")
print(f"{f"{bar=:}" = }")
print(f"{(f"{foo=:}" == f"{foo=!r}") = }")
print(f"{(f"{foo=:}" == f"{foo=!s}") = }")
print()

# these do not work as I expected
# they apply padding to the expression value
# the expression itself is not taken into account
# as a result, the true string length is dependent on the expression length
print(f"{f"{bar=!r:20}." = }")
print(f"{len(f"{bar=!r:20}.") = }")
print(f"{f"{long_name=!r:20}." = }")
print(f"{len(f"{long_name=!r:20}.") = }")
print()

# these do not work as I expected
# they apply alignment only to the expression value
# the expression itself is always left-aligned
print(f"{f"{bar=!r:>20}" = }")
print(f"{len(f"{bar=!r:>20}") = }")
print(f"{f"{long_name=!r:>20}" = }")
print(f"{len(f"{long_name=!r:>20}") = }")

Expected results

• Default value conversion mode should be consistent regardless of whether : is present
• The whole self-documenting expression should be padded / aligned

import unittest
from secrets import token_urlsafe as random_string


class ExpectedResults(unittest.TestCase):
    def test_consistent_default_value_conversion_mode(self):
        my_value = random_string(16)
        first_way = f"{my_value=}"
        second_way = f"{my_value=:}"
        self.assertEqual(first_way, second_way)

    def test_pad_to_length(self):
        my_value = random_string(16)
        assert len(my_value) == 22
        value_only = f"{my_value!r:35}."
        self_documenting = f"{my_value=!r:35}."
        self.assertEqual(len(value_only), len(self_documenting))

    def test_alignment(self):
        my_value = random_string(16)
        assert len(my_value) == 22
        faux_self_documenting = f"{f"my_value={my_value!r}":>35}"
        assert not faux_self_documenting.startswith("my_value=")
        assert len(faux_self_documenting) == 35
        self_documenting = f"{my_value=!r:>35}"
        self.assertEqual(faux_self_documenting, self_documenting)

    def test_consistent_default_value_conversion_mode_exact(self):
        my_value = "report poet ethics"
        expected = "my_value='report poet ethics'"
        actual = f"{my_value=:}"
        self.assertEqual(expected, actual)

    def test_pad_to_length_exact(self):
        my_value = 'sugar lemon dilemma'
        expected = "my_value='sugar lemon dilemma'      "
        assert len(expected) == 36
        actual = f"{my_value=!r:36}"
        self.assertEqual(expected, actual)
        # or alternatively
        self.assertEqual(36, len(actual))

    def test_alignment_exact(self):
        my_value = 'bonus thought skull'
        expected = "   my_value='bonus thought skull'   "
        assert len(expected) == 36
        actual = f"{my_value=!r:^36}"
        self.assertEqual(expected, actual)

if __name__ == "__main__":
    unittest.main()

CPython versions tested on:

3.13

Operating systems tested on:

Linux

[terryjreedy]

What does the current doc say should happen?

[Lordfirespeed]

What does the current doc say should happen?

In terms of padding/alignment: unspecified.
In terms of the conversion behaviour: it's documented, but I don't think that's sufficient evidence to suggest it is intended to be that way.

The usual f-string format specifiers allow more control over how the result of the expression is displayed
Source

When the equal sign '=' is provided, the output will have the expression text, the '=' and the evaluated value. Spaces after the opening brace '{', within the expression and after the '=' are all retained in the output. By default, the '=' causes the repr() of the expression to be provided, unless there is a format specified. When a format is specified it defaults to the str() of the expression unless a conversion '!r' is declared.
Source

Also relevant is format specification mini-language

[picnixz]

they default to !s value conversion, instead of !r

I think that's what should happen albeit surprising (but let's ask @pablogsal). The reason is also explained in the following sentence:

An empty string is passed when the format specifier is omitted.

Having : but no format specifier causes the formatter to use format(s, '') and the docs for format say

The default format_spec is an empty string which usually gives the same effect as calling str(value).

---

For the padding, I don't think there is really an issue: for instance:

>>> print(f"{long_name=!r:20}.")
long_name='hello, world!'     .
>>> print(f"{bar=!r:20}.")
bar='hello, world!'     .

The padding is applied on the content of long_name or bar, in this case "hello, world" which we put on 20 characters. As a matter of fact, we have 'hello, world!' is exactly 20 characters. Am I missing the point maybe? The long_name= or bar= should not be counted in the padding.

[pablogsal]

@picnixz is right when quoting the current documentation. When we changed the parser in 3.12 we basically focused on ensure we preserved the behaviour and all of these are preserved since before 3.12:

~/github/python/main gh-117174*
❯ cat lel.py
# initialisation
foo = "hello!"
bar = "hello, world!"
long_name = bar
print(f"{foo = }")
print(f"{bar = }")
print()

# these work as I expected
# they default to !r value conversion
print(f"{f"{foo=}" = }")
print(f"{f"{bar=}" = }")
print(f"{(f"{foo=}" == f"{foo=!r}") = }")
print()

# these do not work as I expected
# they default to !s value conversion, instead of !r
print(f"{f"{foo=:}" = }")
print(f"{f"{bar=:}" = }")
print(f"{(f"{foo=:}" == f"{foo=!r}") = }")
print(f"{(f"{foo=:}" == f"{foo=!s}") = }")
print()

# these do not work as I expected
# they apply padding to the expression value
# the expression itself is not taken into account
# as a result, the true string length is dependent on the expression length
print(f"{f"{bar=!r:20}." = }")
print(f"{len(f"{bar=!r:20}.") = }")
print(f"{f"{long_name=!r:20}." = }")
print(f"{len(f"{long_name=!r:20}.") = }")
print()

# these do not work as I expected
# they apply alignment only to the expression value
# the expression itself is always left-aligned
print(f"{f"{bar=!r:>20}" = }")
print(f"{len(f"{bar=!r:>20}") = }")
print(f"{f"{long_name=!r:>20}" = }")
print(f"{len(f"{long_name=!r:>20}") = }")

~/github/python/main gh-117174*
❯ ./python --version
Python 3.11.11+

~/github/python/main gh-117174*
❯ ../main/python --version
Python 3.14.0a5+

~/github/python/main gh-117174*
❯ diff <(./python lel.py) <(../main/python lel.py)

Not sure how much of this will be easy to change technically speaking, but regarding what should and should not happen we should first check what @ericvsmith thinks

[ericvsmith]

It would be easier to understand the issue if the OP would include the exact actual values, and the exact expected values.

As far as a zero length format spec resulting in str(value): that's as expected.

[Lordfirespeed]

@ericvsmith sorry, I was teaching this morning and got sidetracked whilst writing a reply to @picnixz's comment 😅

It would be easier to understand the issue if the OP would include the exact actual values, and the exact expected values.

🫡 I will add exact value testcases to the original post when I am able (I will notify you)

As far as a zero length format spec resulting in str(value): that's as expected.

Having : but no format specifier causes the formatter to use format(s, '')

Agreed, format(s, "") == str(s) should evaluate True always.

Also agreed that the behaviour 'zero length format spec results in str(s)' is expected in the sense that it matches the docs and behaviour of previous Python versions. However, I am not (yet!) convinced that this behaviour was intentional when originally implemented.

Remember that conversion mode is a property of f-string fragments/replacement fields, not format specifiers (See Format String Syntax).
i.e. there is no way to replicate {foo!r} using format(foo, ...) alone; instead, the best translation is format(repr(foo), ...).
Basically, !r has special meaning only to str.format and f-strings.

The conversion mode (either str(), repr(), or ascii()) is supposed to be applied before being passed to format.
(source: see highlighted region, from page source for format string syntax. I interpret the word 'first' in this context to mean 'before applying the format spec')

Wecan verify that this is true by evaluating the literal f"{10.5!s:.2f}": it yields Unknown format code 'f' for object of type 'str' because str() has been called on the float 10.5 before being passed to format().

When no conversion mode is specified, certain special-case types (e.g. int, float etc) are passed to format() without any conversion, and everything else is converted using the default conversion mode.

It makes sense that {foo:} would default to !s (for non-special cases) because {foo} defaults to !s.
The default for {foo=} is !r (for non-special cases), so I think the default for {foo=:} should be too.

It was clearly intentional (at time of initial implementation) that self-documenting expressions default to !r, and we can all agree this is desirable due to their intended purpose. It seems more likely to me that the behavioural difference between omission and inclusion of the format-spec in self-documenting expressions was an oversight that never got amended (Please correct me if I am wrong about this!)

From Format string syntax:

The field_name is optionally followed by a conversion field, which is preceded by an exclamation point '!', and a format_spec, which is preceded by a colon ':'. These specify a non-default format for the replacement value.

[Lordfirespeed]

For the padding, I don't think there is really an issue [...]
The long_name= or bar= should not be counted in the padding.

Padding has to be considered as a complete unit with alignment. They go hand-in-hand; as the docs say, alignment means nothing without specified padding, because aligning a string foo="..." in a space of width len(foo) has no effect.

You haven't mentioned alignment, which leads me to believe you agree the current alignment behaviour should be changed.
Imagine we change the alignment behaviour, without changing the padding behaviour.
Then now

>>> my_variable = "hello"
>>> f"{my_variable!r:^22}"
"       'Hello'        "
>>> len(_)
22
>>> f"{my_variable=!r:^22}"
"       my_variable='Hello'        "
>>> len(_)
34

I asked for my expression to be centred in a space of width 22, and it's been centred in a space of width 34. That's definitely confusing. More importantly, I can't imagine any scenario where that behaviour is actually helpful!
Self-documenting expressions are intended for debugging and simplifying implementation of __repr__ methods. They are intended to support the pattern of rendering an expression and its evaluated result, without needing to reiterate/hard-code the expression.

If someone wishes to pad the value only (and I would love to hear about particular cases where that would be useful /gen), I can only imagine that they must wish to re-use it elsewhere. Therefore, it makes sense to assign the padded expression to its own variable, and this works great in any case:

>>> padded_variable = f"{my_variable!r:^22}"
>>> f"{padded_variable=!s}"
"padded_variable=       'Hello'        "
>>> len(_)
38

However, padding the value taking into account the expression is messy in this scenario. I don't think this use-case should necessitate nesting format-strings (as it currently does).

>>> f"{f"{my_variable=}":^22}"
" my_variable='Hello'  "
>>> len(_)
22

[picnixz]

Padding has to be considered as a complete unit with alignment

Right, I actually only considered the default alignment and haven't thought of the centered one. I need to re-read this tomorrow as it's a bit too late now to reply with a complete answer so I'll come back later.

[Lordfirespeed]

@ericvsmith Exact examples for each concern:

conversion mode

my_value =  "report poet ethics"
f_string = f"{my_value=:}"
expected =  "my_value='report poet ethics'"
actual   =  "my_value=report poet ethics"

padding

my_value =  "sugar lemon dilemma"
f_string = f"{my_value=!r:36}"
expected =  "my_value='sugar lemon dilemma'      "  # length 36
actual   =  "my_value='sugar lemon dilemma'               "  # length 36 + 9

alignment

my_value =  "bonus thought skull"
f_string = f"{my_value=!r:^36}"
expected =  "   my_value='bonus thought skull'   "  # length 36
actual   =  "my_value=       'bonus thought skull'        "  # length 36 + 9