ST_DISTANCE predicate returns true on null distance with Lucene pushdown

[craigtaverner]

With a query like:

FROM airports
| EVAL distance = ST_DISTANCE(location, TO_GEOPOINT("POINT(-28.6359 153.5906)"))
| KEEP distance, location, name, scalerank
| WHERE distance > 500 AND scalerank < 6
| SORT name ASC

The literal geo-point is invalid (y value out of range), and this correctly leads to a null distance value. When run without Lucene pushdown, this query will return no results because the WHERE distance > 500 will evaluate to WHERE null which is effectively WHERE false. However, the Lucene pushdown of the distance predicate is not correctly filtering this, and results in all documents being returned (as if we had WHERE true).

This issue is specific to the Lucene pushdown of the distance predicate, not an issue with WHERE or predicates in general. However, since distance pushdown actually re-writes to ST_INTERSECTS pushdown, we need to verify that the issue is not also for intersects and disjoint cases.

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[craigtaverner]

This problem only occurs for > predicates, because they are translated into DISJOINT queries, while < predicates are translated into INTERSECTS queries:

• INTERSECTS with an invalid shape will give no results (correct answer)
• DISJOINT with an invalid shape will return all documents (which is correct from the definition of DISJOINT, but not correct from the definition of the ES|QL predicate, which means we need to evaluate the validity of the shape before using DISJOINT)

[craigtaverner]

We could either add an extra check for spatial pushdown that the geometries are valid, or add validation at geometry creation. This second option is much easier to implement, and a more comprehensive approach, as it prevents invalid geometries in many more places, hopefully reducing the likelihood of subtle and obscure bugs like this one happening in future. See #128259 for this solution.