Skip to content

testdir.runpytest_subprocess() fails with UnicodeDecodeError in Windows #7623

Open
@altendky

Description

@altendky

This seems similar to #298.

Original issue code: https://github.com/altendky/qtrio/tree/72863a1e769d9f942e2062904c9b6f3fb077d3b8
CI (GitHub Actions Windows): https://github.com/altendky/qtrio/pull/2/checks?check_run_id=947483753

>       result = testdir.runpytest_subprocess(timeout=4 * timeout)

C:\hostedtoolcache\windows\Python\3.8.5\x64\lib\site-packages\qtrio\_tests\test_pytest.py:44: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <encodings.utf_8.IncrementalDecoder object at 0x0000022A4FF3E520>
input = b'============================= test session starts =============================\r\nplatform win32 -- Python 3.8.5, p... qtrio._exceptions.RunnerTim...\r\n============================== 1 failed in 0.74s ==============================\r\n'
final = True

    def decode(self, input, final=False):
        # decode input (taking the buffer into account)
        data = self.buffer + input
>       (result, consumed) = self._buffer_decode(data, self.errors, final)
E       UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 9091: invalid start byte

c:\hostedtoolcache\windows\python\3.8.5\x64\lib\codecs.py:322: UnicodeDecodeError
>>> b[9050:9150]
b"quest is\r\n           *also* asynchronous \x96 you don't find out until later whether the\r\n           op"

The Trio source being included in the context resulting in the \x96: https://github.com/python-trio/trio/blob/15facd95379a496b234cc762c4ae69bf4fcdd0bf/trio/_core/_traps.py#L120

2020-08-05T02:10:08.7472418Z + python -m pip list
2020-08-05T02:10:09.3155684Z Package             Version
2020-08-05T02:10:09.3156457Z ------------------- ---------
2020-08-05T02:10:09.3156870Z async-generator     1.10
2020-08-05T02:10:09.3157158Z atomicwrites        1.4.0
2020-08-05T02:10:09.3157384Z attrs               19.3.0
2020-08-05T02:10:09.3157830Z cffi                1.14.1
2020-08-05T02:10:09.3158411Z click               7.1.2
2020-08-05T02:10:09.3159025Z colorama            0.4.3
2020-08-05T02:10:09.3159984Z coverage            5.2.1
2020-08-05T02:10:09.3160282Z decorator           4.4.2
2020-08-05T02:10:09.3161003Z idna                2.10
2020-08-05T02:10:09.3161594Z iniconfig           1.0.1
2020-08-05T02:10:09.3162319Z more-itertools      8.4.0
2020-08-05T02:10:09.3162739Z outcome             1.0.1
2020-08-05T02:10:09.3163670Z packaging           20.4
2020-08-05T02:10:09.3164165Z pip                 20.2.1
2020-08-05T02:10:09.3164901Z pluggy              0.13.1
2020-08-05T02:10:09.3165352Z py                  1.9.0
2020-08-05T02:10:09.3166163Z pycparser           2.20
2020-08-05T02:10:09.3166517Z pyparsing           2.4.7
2020-08-05T02:10:09.3167314Z PySide2             5.15.0
2020-08-05T02:10:09.3167907Z pytest              6.0.1
2020-08-05T02:10:09.3168666Z pytest-cov          2.10.0
2020-08-05T02:10:09.3169237Z pytest-faulthandler 2.0.1
2020-08-05T02:10:09.3169938Z pytest-qt           3.3.0
2020-08-05T02:10:09.3170361Z QtPy                1.9.0
2020-08-05T02:10:09.3171850Z qtrio               0.1.0+dev
2020-08-05T02:10:09.3172256Z setuptools          49.2.1
2020-08-05T02:10:09.3172457Z shiboken2           5.15.0
2020-08-05T02:10:09.3172703Z six                 1.15.0
2020-08-05T02:10:09.3172974Z sniffio             1.1.0
2020-08-05T02:10:09.3173542Z sortedcontainers    2.2.2
2020-08-05T02:10:09.3174126Z toml                0.10.1
2020-08-05T02:10:09.3175258Z trio                0.16.0
2020-08-05T02:10:09.3175712Z wheel               0.34.2

So what should be different? I think there are two pieces. First, pytest should probably attempt to control the subprocess' output (and input?) encoding. Second, pytest should not fail on decoding with only the context of the single byte value and location in the otherwise inaccessible output.

Hopefully I'll get to adding a test forcing the issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    plugin: pytesterrelated to the pytester builtin plugintype: bugproblem that needs to be addressed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions