updateCommitMappings: run only one big range-diff

Over the time, GitGitGadget accumulated _quite_ a few open PRs. For all
of them, the commit mappings are updated whenever `pu` changed. But that
can take _quite_ a long time, due to an ancient merge base. See e.g.
gitgitgadget/git#15, which still awaits the time
when this developer can do some fun stuff again.

Even worse: since this task is run on a low-powered Azure VM, it takes
quite a few resources, and quite a long time just to update the commit
mappings.

Let's coalesce all the ranges that need to be compared into a single
one, and run only a single range-diff (which will still take something
like 90 seconds, but that is way better than the 20 minutes the many
range-diffs take right now).

To do so, we construct a throw-away octopus merge of all the base
commits, and another octopus merge of all the head commits, and then use
the range between these octopus merges as the first commit range for the
`git range-diff` command.

To make sure that we're not missing anything (e.g. when one of the PRs
has `pu` as its base commit), we do not use the first octopus merge in
the second commit range. Instead, we use `maint~100..pu`, which should
be plenty enough.

Note: we could save on some running time by using `maint~10` instead
(roughly 20 seconds) or even `maint~1` (about a minute), but we want to
stay safe in case a PR goes directly to `maint` (which _has_ happened
before).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Author: dscho

Diff for: lib/ci-helper.ts

@@ -1,7 +1,7 @@
 import * as fs from "fs";
 import * as util from "util";
 import { ILintError, LintCommit } from "./commit-lint";
-import { commitExists, git } from "./git";
+import { commitExists, git, emptyTreeName } from "./git";
 import { GitNotes } from "./git-notes";
 import { GitGitGadget, IGitGitGadgetOptions } from "./gitgitgadget";
 import { GitHubGlue, IGitHubUser, IPRCommit,
@@ -159,6 +159,14 @@ export class CIHelper {
                        { workDir: this.workDir })).split("\n"),
         );
         let result = false;
+        /*
+         * Both `bases` and `heads` accumulate the `-p<commit-hash>` parameters
+         * for the `git commit-tree` command for the two octopus merges. We
+         * need to make sure that no parent is listed twice, as `git
+         * commit-tree` would error out on that.
+         */
+        const bases = new Set<string>();
+        const heads = new Set<string>();
         for (const pullRequestURL of Object.keys(options.openPRs)) {
             const info = await this.getPRMetadata(pullRequestURL);
             if (info === undefined || info.latestTag === undefined ||
@@ -183,24 +191,44 @@ export class CIHelper {
                 meta.commitInGitGit = undefined;
                 result = true;
             }
+            bases.add(`-p${info.baseCommit}`);
+            heads.add(`-p${info.headCommit}`);
+        }
+
+        if (heads.size > 0) {
+            /*
+             * Generate throw-away octopus merges to combine multiple commit
+             * ranges into a single one.
+             */
+            const octopus = async (set: Set<string>): Promise<string> => {
+                const array = Array.from(set);
+                if (array.length === 1) {
+                    return array[0];
+                }
+                return await
+                    git(["commit-tree", ...array, emptyTreeName, "-m", "()"],
+                        { workDir: this.workDir });
+            };
 
+            const range1 = `${await octopus(bases)}..${await octopus(heads)}`;
+            const range2 =
+                "refs/remotes/upstream/maint~100..refs/remotes/upstream/pu";
             const start = Date.now();
             const out = await git(["-c", "core.abbrev=40", "range-diff", "-s",
-                                   info.baseCommit, info.headCommit,
-                                   "refs/remotes/upstream/pu"],
+                                   range1, range2],
                                   { workDir: this.workDir });
             const duration = Date.now() - start;
             if (duration > 2000)
                 console.log(`warning: \`git range-diff ${
-                    info.baseCommit} ${info.headCommit} upstream/pu\` took ${
+                    range1} ${range2}\` took ${
                     duration / 1000} seconds`);
             for (const line of out.split("\n")) {
                 const match =
                     line.match(/^[^:]*: *([^ ]*) [!=][^:]*: *([^ ]*)/);
                 if (!match) {
                     continue;
                 }
-                const messageID2 = match[1] === info.headCommit ? messageID :
+                const messageID2 =
                     await this.getMessageIdForOriginalCommit(match[1]);
                 if (messageID2 === undefined) {
                     continue;

Diff for: lib/git.ts

@@ -13,6 +13,7 @@ export interface IGitOptions {
 }
 
 export const emptyBlobName = "e69de29bb2d1d6434b8b29ae775ad8c2e48c5391";
+export const emptyTreeName = "4b825dc642cb6eb9a060e54bf8d69288fbee4904";
 
 function trimTrailingNewline(str: string): string {
     return str.replace(/\r?\n$/, "");