Max recursion during unpickling with a proxy object

[gbrawn]

Bug report

Bug description:

Description of Problem

A wrapper class was needed to wrap an underlying object while exposing the underlying object's attributes. https://stackoverflow.com/questions/68926132/creation-of-a-class-wrapper-in-python

When creating a wrapper object for an object to be placed on a multiprocessing queue, if the wrapper object overrides the getattr method, and the object is retrieved from the queue, a max recursion depth error is observed. The expected behavior was for the wrapper object to be returned and have the underlying object exposed in the process that was handling the object.

Example Snippet

from multiprocessing import Queue

class Foo:
    """
    Any old object
    """
    def __init__(self, var: int):
        self._var = var

    @property
    def var(self) -> int:
        return self._var

class FooWrapper:
    """
    Wrapper class to expose underlying object attrs
    """
    def __init__(self, foo: Foo):
        self.foo = foo

    def __getattr__(self, name):
        return getattr(self.foo, name)

if __name__=="__main__":
    queue = Queue()
    foo = Foo(2)
    foo_wrapper = FooWrapper(foo)
    queue.put(foo_wrapper)
    message_wrapper = queue.get()  # This fails due to max recursion depth reached

>>
    return getattr(self.foo, name)
                   ^^^^^^^^
  [Previous line repeated 994 more times]
RecursionError: maximum recursion depth exceeded

CPython versions tested on:

3.11

Operating systems tested on:

Linux, Windows

[picnixz]

How does it fare with a plain queue.Queue? does it fail as well?

[gbrawn]

Just tested it and it appears to work as intended.

from queue import Queue

class Foo:
    """
    Any old object
    """
    def __init__(self, var: int):
        self._var = var

    @property
    def var(self) -> int:
        return self._var

class FooWrapper:
    """
    Wrapper class to expose underlying object attrs
    """
    def __init__(self, foo: Foo):
        self.foo = foo

    def __getattr__(self, name):
        return getattr(self.foo, name)

if __name__=="__main__":
    queue = Queue()
    foo = Foo(2)
    foo_wrapper = FooWrapper(foo)
    queue.put(foo_wrapper)
    message_wrapper = queue.get()
    print(f"{message_wrapper.var}")
>>>
2

[brianschubert]

This happens during unpickling. Here's a simplified reproducer without multiprocessing.Queue:

import pickle

class Wrapper:
    def __init__(self, x):
        self.x = x

    def __getattr__(self, name):
        return getattr(self.x, name)

b = pickle.dumps(Wrapper("foo"))
pickle.loads(b)  # <-- recursion here

[brianschubert]

The recursion happens during this call of PyObject_GetOptionalAttr, which is checking for the existence of __setstate__ on the instance being built. This calls __getattr__, and since the instance dict hasn't been populated yet, the lookup of self.x inside __getattr__ results in an infinitely recursing __getattr__ call.

[picnixz]

• Does it also affect the Python implementation of pickle?
• Do you want to make a PR for that?