2025 Priorities

[rickeylev]

This issue is to capture some of the topics that we, the rules_python maintainers, are particularly interested in this coming year. These are topics where we'd be particularly excited by PRs or design discussions.

Note that these are not promises of particular work. This is still a community driven project, so people work on what they find interesting or rewarding.

Topics are in no particular order. Discussion for a topic should be in their associated issue. This is not an exclusive list.

• A venv and site-packages layout for dependencies:
  • Implement venv/site-packages based binaries #2156
• Consolidating *_bazel.bzl files, removing semantics.bzl, and general cleanup of the core rule implementations:
  • refactor: consolidate _bazel.bzl and remove semantics.bzl #2522
• Implementing uv-based toolchain
  • Support uv as part of rules_python #1975
• Supporting other lockfile formats (e.g. PEP751)
  • Support PEP751 lockfile format in bzlmod #2787
• Module-based main (python -m <name>)
  • Support python -m invocation style #2539
• Build-phase sdist building
  • Support building from sdist in a build action #2410
• Solution to implicit dependencies on user-space libraries. Related:
  • Support a default hub repo concept in bzlmod #2094
  • Generate one Bazel repo per dependency #2370
• Implementing a C++ based bootstrap (expanding bootstrap to support e.g. compiled languages)
  • Use statically linked loader #2500
• Make bootstrap_impl=script the default
  • Make script bootstrap the default #2521
• Splitting sphinxdocs into its own bzlmod MODULE
  • Split out //sphinxdocs to be a separate bazel submodule and setup publishing to BCR #2511
• Toolchains for local interpreters
  • Allow using a locally installed Python runtime as a full toolchain #2070
• Stabilize the pulling of platform-specific wheels matching target platform (i.e. experimental_index_url):
  • [pypi] Support pulling dependencies using direct url specifiers without using an index #2363
  • Silence download 404 errors when using bazel downloader (experimental_index_url and experimental_extra_index_urls) #2100
  • Enable using experimental_index_url without having hashes in the lock file #2023
  • Avoid merge conflicts in MODULE.bazel.lock #2434
  • "Cross compilation" of py_binary/py_image/py_library targets #260
  • [pypi] Support python micro version in env marker based dependency evaluation #2319
  • Third party dependencies are incorrect when using RBE because host != exec #2241
  • Use bazel's downloader for downloading pip packages #1357
  • Introduce targets for all_requirements, all_whl_requirements, all_data_requirements #2559
• Zip stdlib to reduce file count / runfiles overhead
  • Hermetic toolchains incur a significant overhead to running py_binary and py_test #1653

cc @aignas @groodt

[aignas]

I have modified the list above to include more references and I realized that I want to stabilize the #260 work and make it default at some point.

[aignas]

@arrdem, the above bullet point summarizes all of the related tickets to close #260 and stabilize the feature.

I think the main issues are two:

1. not having PEP508 support in starlark means that the APIs are brittle and we require to pass a lot of things to whl_library as parameters instead of via load to the macro phase. The refactor(pypi): implement PEP508 compliant marker evaluation #2692 and then refactor(pypi): translate wheel METADATA parsing to starlark #2629 should address that. Once that is done I can refactor the passing of experimental_index_platforms and fix [pypi] Support python micro version in env marker based dependency evaluation #2319.
2. Then we should rewrite the experimental_index_url configuration API to be more similar to uv.defaults and uv.configure, so that we have the right level of expressions to configure the target platforms. That should also make things harder to misuse.
3. Doing the above will mean that we have a solid foundation to correctly setup dependencies for any python package given the target platforms (very similar what rules_pycross is doing) and we can turn on the experimental_index_url by default.

Related items but not strictly necessary needed:

1. Separate out the whl_library for building with pip (ticket TBD), just extracting wheels (once PEP508 is done that would be possible without the interpreter) and setting up sources for py_sdist (Support building from sdist in a build action #2410).
2. I think there is something else but I can't remember right now.

That is about it.