Validate any valid spec URL

[ddbeck]

via #78

What I want

• To be able to link to any valid page (or fragment) within a specification known to web-specs
• To not be able to link an invalid page (or fragment) within a specification known to web-specs
• To not be able to link to a specification unknown to web-specs

How it works now

Right now, the tests allow linking only to the single URL known to web-specs. For example https://html.spec.whatwg.org/multipage/ is a valid spec URL, but https://html.spec.whatwg.org/multipage/custom-elements.html#custom-elements is not.

How it could work in the future

We could change the tests to allow any valid derivative URL (i.e., any URL where the scheme and domain are the same and the path starts with the path known to web-specs).

If we wanted to go further, we could periodically scrape spec URLs to fully enumerate all of the pages and anchors and validate against them (I don't think this would be too bad—there's not that many specs, but not exactly zero effort).

[foolip]

@ddbeck do you think we need to do more than we're already doing for this?

[ddbeck]

@foolip We don't need to do anything, but we should probably log the idea somewhere that we wanted to validate the URL fragments (see also: w3c/browser-specs#931). More nice-to-have than need-to-have (at least for now).

[foolip]

Ah, so given that w3c/browser-specs#931 was done we could write some code now.

I think we'd also be well served by aligning exactly with the rules that BCD uses, and there are some minor differences. But how? A new package that both depend on feels like a bit much? Perhaps browser-specs and web-specs itself should have a validation function?

[ddbeck]

Perhaps browser-specs and web-specs itself should have a validation function?

Maybe? I'm not sure how that interacts with web-ref, which knows about the fragments themselves.

But it seems to me that a reverse lookup would make the most sense, given the goals browser- and web-specs: you give it a complete URL and you get back the corresponding spec object. We'd just check for success or failure.

[tidoust]

Perhaps browser-specs and web-specs itself should have a validation function?

Not sure if that's meant to cover fragments. If so, I don't think we'll include them in browser-specs directly. We can look into providing the information in a more directly usable way in Webref though. Webref effectively contains browser-specs, so it should be possible to get all the validation info you need relying on Webref rather than on browser-specs.

[foolip]

Ah, right, it was fixed for pages, not fragment identifiers...

I've filed w3c/webref#1198 for that.

[ddbeck]

This came up at the WebDX call yesterday. The highlights:

• This is a problem we share with BCD. @Elchi3 has started work on solving this in Validate spec_urls based on webref ids mdn/browser-compat-data#23958. It's possible that generic validate spec URL package could be shared between us, or contributed upstream to webref.
• François noted that webref contains some fragment identifiers (for headings and certain definitions), but not all. He's curious to know if we require more "brittle" identifiers to be validated.