From 4be773754d76463285317dc9ad563ee3a76d85df Mon Sep 17 00:00:00 2001
From: Adam Turner <9087854+aa-turner@users.noreply.github.com>
Date: Fri, 18 Apr 2025 13:17:37 +0100
Subject: [PATCH 1/4] Ensure ``make -j`` uses a reasonable argument
---
developer-workflow/extension-modules.rst | 2 +-
getting-started/setup-building.rst | 13 ++++++-------
index.rst | 4 ++--
3 files changed, 9 insertions(+), 10 deletions(-)
diff --git a/developer-workflow/extension-modules.rst b/developer-workflow/extension-modules.rst
index 61c1ff08af..33e203b0a1 100644
--- a/developer-workflow/extension-modules.rst
+++ b/developer-workflow/extension-modules.rst
@@ -547,7 +547,7 @@ Now that the configuration is in place, it remains to compile the project:
.. tip::
- Use ``make -j`` to speed-up compilation by utilizing as many CPU cores
+ Use ``make -j $(nproc)`` to speed-up compilation by utilizing as many CPU cores
as possible or ``make -jN`` to allow at most *N* concurrent jobs.
* ``make regen-configure`` updates the :cpy-file:`configure` script.
diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst
index 64d7f054bd..2b30059e12 100644
--- a/getting-started/setup-building.rst
+++ b/getting-started/setup-building.rst
@@ -203,14 +203,13 @@ do to get a pydebug build of CPython.
Once ``configure`` is done, you can then compile CPython with::
- $ make -s -j2
+ $ make -s -j $(nproc)
This will build CPython with only warnings and errors being printed to
-stderr and utilize up to 2 CPU cores. If you are using a multi-core machine
-with more than 2 cores (or a single-core machine), you can adjust the number
-passed into the ``-j`` flag to match the number of cores you have (or if your
-version of Make supports it, you can use ``-j`` without a number and Make
-will not limit the number of steps that can run simultaneously.).
+stderr. The ``-j`` argument means that :program:`make` will concurrently run
+tasks, limiting the number of parallel jobs to the number of CPU cores in your
+computer. You can adjust the number passed to the ``-j`` flag to change
+the limit on parallel jobs, which can trade RAM usage vs compilation time.
At the end of the build you should see a success message, followed
by a list of extension modules that haven't been built because their
@@ -875,7 +874,7 @@ some of CPython's modules (for example, ``zlib``).
And finally, run ``make``::
- $ make -s -j2
+ $ make -s -j $(nproc)
There will sometimes be optional modules added for a new release which
won't yet be identified in the OS-level build dependencies. In those cases,
diff --git a/index.rst b/index.rst
index 9dbc06908b..138211a037 100644
--- a/index.rst
+++ b/index.rst
@@ -43,13 +43,13 @@ instructions please see the :ref:`setup guide <setup>`.
.. code-block:: shell
- ./configure --with-pydebug && make -j
+ ./configure --with-pydebug && make -j $(nproc)
.. tab:: macOS
.. code-block:: shell
- ./configure --with-pydebug && make -j
+ ./configure --with-pydebug && make -j $(nproc)
.. tab:: Windows
From c2784fa4672dbae1eac6edb474e78ffce459d9cf Mon Sep 17 00:00:00 2001
From: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Date: Fri, 18 Apr 2025 13:23:25 +0100
Subject: [PATCH 2/4] Simplify
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
---
developer-workflow/extension-modules.rst | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/developer-workflow/extension-modules.rst b/developer-workflow/extension-modules.rst
index 33e203b0a1..5ece599b9f 100644
--- a/developer-workflow/extension-modules.rst
+++ b/developer-workflow/extension-modules.rst
@@ -547,7 +547,7 @@ Now that the configuration is in place, it remains to compile the project:
.. tip::
- Use ``make -j $(nproc)`` to speed-up compilation by utilizing as many CPU cores
+ Use ``make -j $(nproc)`` to speed-up compilation by using as many CPU cores
as possible or ``make -jN`` to allow at most *N* concurrent jobs.
* ``make regen-configure`` updates the :cpy-file:`configure` script.
From 1b100130831467f852188505360907d69fe1a501 Mon Sep 17 00:00:00 2001
From: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Date: Fri, 18 Apr 2025 13:23:36 +0100
Subject: [PATCH 3/4] Expand
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
---
getting-started/setup-building.rst | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst
index 2b30059e12..0db68d18b6 100644
--- a/getting-started/setup-building.rst
+++ b/getting-started/setup-building.rst
@@ -209,7 +209,7 @@ This will build CPython with only warnings and errors being printed to
stderr. The ``-j`` argument means that :program:`make` will concurrently run
tasks, limiting the number of parallel jobs to the number of CPU cores in your
computer. You can adjust the number passed to the ``-j`` flag to change
-the limit on parallel jobs, which can trade RAM usage vs compilation time.
+the limit on parallel jobs, which can trade RAM usage versus compilation time.
At the end of the build you should see a success message, followed
by a list of extension modules that haven't been built because their
From 9ce0c3cbe02ab4f2219571cf20fd9f88920b7bf7 Mon Sep 17 00:00:00 2001
From: Thomas Wouters <thomas@python.org>
Date: Fri, 18 Apr 2025 13:13:23 +0100
Subject: [PATCH 4/4] Do not suggest `make -j` for building CPython. `make -j`
means _no_ limit on the number of concurrent jobs, not "a sensible limit
considering the available memory and CPU cores". With LTO it's easy to run
out of memory when too many jobs run at the same time.
---
developer-workflow/extension-modules.rst | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/developer-workflow/extension-modules.rst b/developer-workflow/extension-modules.rst
index 5ece599b9f..d6be888241 100644
--- a/developer-workflow/extension-modules.rst
+++ b/developer-workflow/extension-modules.rst
@@ -547,8 +547,11 @@ Now that the configuration is in place, it remains to compile the project:
.. tip::
- Use ``make -j $(nproc)`` to speed-up compilation by using as many CPU cores
- as possible or ``make -jN`` to allow at most *N* concurrent jobs.
+ Use ``make -jN`` to speed-up compilation by utilizing as many CPU cores
+ as possible, where *N* is as many CPU cores you want to spare (and have
+ memory for). Be careful using ``make -j`` with no argument, as this puts
+ no limit on the number of jobs, and compilation can sometimes use up a
+ lot of memory (like when building with LTO).
* ``make regen-configure`` updates the :cpy-file:`configure` script.