unify reduce/reducedim empty case behaviors

[mbauman]

This PR implements a very simple rule for empty dimensional reductions: they should behave exactly the same as their whole-array counterparts. If the whole-array reduction errors, the dimensional flavor should also error. If the whole-array reduction gives a value, then the dimensional version should just return the appropriately-shaped array filled with it.

Previously, Julia tried to guess at initial values for empty dimensional reductions in a slightly different way than whole-array reductions. This change unifies those behaviors, such that mapreduce_empty is called for empty dimensional reductions, just like it is called for empty whole-array reductions.

Beyond the unification (which is valuable in its own right), this change enables some downstream simplifications to dimensional reductions and is likely to be the "most breaking" public behavior in a refactoring targeting more correctness improvements (#55318).

It's a little convoluted to understand the exact motivation, utility, and net effect of this change. Here are two case studies, demonstrating the behaviors prior to this change:

julia> minimum(zeros(0,42))
ERROR: ArgumentError: reducing over an empty collection is not allowed; consider supplying `init` to the reducer

julia> minimum(zeros(0,42), dims=1)
ERROR: ArgumentError: reducing over an empty collection is not allowed; consider supplying `init` to the reducer

julia> minimum(zeros(0,1), dims=2)
0×1 Matrix{Float64}

julia> sum(x->pi, zeros(0,1))
ERROR: ArgumentError: reducing over an empty collection is not allowed; consider supplying `init` to the reducer

julia> sum(x->pi, zeros(0,1), dims=1)
1×1 Matrix{Int64}:
 0

julia> sum(x->pi, zeros(0,1), dims=2)
0×1 Matrix{Int64}

This PR makes all six cases above errors. This is a useful and important change because it's precisely this guessing of eltype and initial array values that lead to the many correctness errors that #55318 fixes. It's also worth noting that the incremental widening approach in #55318 could support the empty-return case, but it would do so at the expense of type stability because the "reducing over an empty collection" error branch adds a possible Array{Union{}} return value.

[mbauman]

Once CI passes (modulo a SparseArrays test), let's run Nanosoldier.

[mbauman]

@nanosoldier runtests()

[nanosoldier]

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

[mbauman]

OK, that went better than I expected. There are three root failures as far as I can see:

• GMT v1.14.2 calls extrema(dest::Matrix{Float64}, dims=1) as it does its precompile workload. Interestingly, the latest release (v.17) does not do this for its precompile work, so this only affects GeophysicalModelGenerator.jl which pins it to the old version somehow. Later versions don't precompile this, but it still causes test failures.
• PeriodicGraphs calls maximum(denominator.(_invmat::Matrix{Rational{Int}}); dims=2) as part of its precompile workflow.
• OnlinePortfolioSelection callsmaximum(p[idx_under_zero, :], dims=2).

[mbauman]

OK, that's fascinating. All of those examples would be fixed by #52004, which was only blocked by precisely the same sparse array failures that this PR faces. I'm not a huge fan of #52004, but perhaps there's a middle ground by just preserving the eltype for minimum/maximum/extrema for the empty return array case.

[JeffBezanson]

Triage is ok with making all of these errors. The result for minimum(zeros(0,1), dims=2) is arguably correct, but that only applies to functions that pick an element from the array rather than applying a more general function, so I see we may not be able to accommodate that.

[mbauman]

OK, let's see what PkgEval sees here now.

@nanosoldier runtests()

[nanosoldier]

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

Report summary

❗ Packages that crashed

1 packages crashed only on the current version.

• An internal error was encountered: 1 packages

3 packages crashed on the previous version too.

✖ Packages that failed

31 packages failed only on the current version.

• Package fails to precompile: 6 packages
• Package has test failures: 4 packages
• Package tests unexpectedly errored: 2 packages
• Test duration exceeded the time limit: 19 packages

1438 packages failed on the previous version too.

✔ Packages that passed tests

21 packages passed tests only on the current version.

• Other: 21 packages

5240 packages passed tests on the previous version too.

~ Packages that at least loaded

11 packages successfully loaded only on the current version.

• Other: 11 packages

2835 packages successfully loaded on the previous version too.

➖ Packages that were skipped altogether

918 packages were skipped on the previous version too.

[mbauman]

Well that new report looks awfully consistent with what we saw last fall — those three packages are still failing. Looks like we additionally snagged C3D this time, with an empty reduction error on (effectively) mapreduce(==, &, Float64[], Float64[]). That was actually an issue in the (now separated) #58185, and that PR additionally has commit 69e4dee which addresses (and tests) it.

I think the periodic graphs example is a perfect motivation. It calls both maximum(x; init) and maximum(y; dims) on sequential lines! Why should they require init for the former but not the latter? From PeriodicGraphs.jl/src/algorithms/dimensionality.jl#L310-L311:

    maxden = maximum(denominator.(_invmat); dims=2)
    invmat::Matrix{Int} = maximum(maxden; init=0) > 1 ? (maxden .* _invmat) : _invmat

[mbauman]

Given that triage gave their 👍 and the limited ecosystem impact here I'm going to swap the labels here to "minor change".

But this isn't ready yet; we need to merge #58185 ✅ and JuliaSparse/SparseArrays.jl#615 first.

[mbauman]

Despite the label here, this is ready to go. It just requires coordination with SparseArrays. I think the best approach would be to merge JuliaSparse/SparseArrays.jl#615 first. It’s failing over there, but those exact same tests pass in this PR’s CI. Then once that’s merged, we can adjust the second commit here to just bump the stdlib normally.

It’d be good to get a final review here in any case, and I don’t have commit rights to SparseArrays myself.