--merge-pr reporting "test suite passes: pending... => not eligible for merging!" even though all tests seem to have passed

[migueldiascosta]

further inspection showed that GitHub is reporting two tests, 'https://github.com/apps/dependabot' and 'https://github.com/apps/hound', to have conclusion None instead of Success

this was in a PR to master, might be specific to this branch due to GitHub policies

follow up with GitHub issue?

[migueldiascosta]

fwiw, this is still affecting #3526 and easybuilders/easybuild-easyblocks#2277, at least (again, PRs to master)

[boegel]

For #3526, see https://api.github.com/repos/easybuilders/easybuild-framework/commits/51535c5e131115fdaff06353e144f8228be05349/check-suites .

There's 3 "check suites" there that are stuck in status: queued and conclusion: null, for dependabot, hound and travis-ci (see app.slug).

The latter happens because we've run out of free credits in Travis CI, but we'll get rid of that by simply removing the .travis.yml configuration files going forward (which means we won't be tested with Python 2.6 anymore, but we've deprecated support for that a while ago anyway).

For the other two (dependabot and hound), I think we'll need to actively filter those out and simply ignore the results in --merge-pr for PRs to master, we can't just disable those checks somehow without having unwanted side affects (like not checking other branches, or totally disabling dependabot).

[boegel]

@migueldiascosta Do we need to keep this open, after #3604 and #3607 ?

[migueldiascosta]

@boegel I think this is a separate issue but I´ll check

[migueldiascosta]

@boegel yeah, separate issue, and problem is still there. We may want to add something like

--- a/easybuild/tools/github.py
+++ b/easybuild/tools/github.py
@@ -2274,6 +2274,7 @@ def det_commit_status(account, repo, commit_sha, github_user):
 
         # if any check suite hasn't completed yet, final result is still pending
         if status in ['queued', 'in_progress']:
+            print("%s (%s): %s" % (check_suite_data['app']['name'], check_suite_data['url'], check_suite_data['status']))
             result = STATUS_PENDING
 
         # if check suite is completed, take the conclusion into account

which in the case of #3640 outputted:

Dependabot (https://api.github.com/repos/easybuilders/easybuild-framework/check-suites/2443283009): queued
Hound (https://api.github.com/repos/easybuilders/easybuild-framework/check-suites/2443283011): queued
Travis CI (https://api.github.com/repos/easybuilders/easybuild-framework/check-suites/2443283013): queued

[verdurin]

Seeing this at easybuilders/easybuild-easyconfigs#14950 too

[verdurin]

https://gist.github.com/ce114b85f7457ed17809370c5770b8c6

[verdurin]

And easybuilders/easybuild-easyconfigs#14834
See https://gist.github.com/208c913beb9b4ea9ae4db02c221c0bfd

[branfosj]

I think we need to add a clear warning that status_last_commit being failure means we need a --sync-pr. I don't know why the logic on the GitHub merge button is different from the API with regard to checking if the last test suite run was clean, but it is different. Opening / closing a PR does not clear this status, but syncing the PR does.