Description
In a recent compiler team meeting, the timing of crater runs came up.
Today, we typically do a crater run at the start of the six week release cycle; we run crater against the beta freshly cut from nightly.
The problem with this model is that we then cherry-pick beta backports of individal commits into the beta channel. Sometimes those commits have been run through crater; and sometimes the resulting beta has another crater run done on it. But neither is part of the standard process used by the compiler+release teams (and worse, not everyone on the team knows that there won't be "one last crater run" before release).
Our current model allows us to apply last minute patches that the teams decide are warranted. Such last minute patches are very conservative in nature, more so than typical beta backports.
We should make sure we all understand how much testing, or lack thereof, a backport to beta is going to get. We probably should also include time in the project planning to allow for "one last crater run", and then use very strict backport criteria for anything that lands after that point.