[libfuzzer] FuzzedDataProvider consumeFloatingPointInRange can return potentially invalid results.

[br-lewis]

I've noticed that in certain circumstances consumeFloatingPointInRange can return a float or double that's greater than the given max value. This can be both infinity or a finite value that's greater than max. I'm submitting a bug rather than a fix because I'm not sure if you would consider this an issue.

Steps to reproduce

1. Add these tests to compiler-rt/lib/fuzzer/tests/FuzzedDataProviderUnittest.cpp

TEST(FuzzedDataProvider, ConsumeFloatingPointInRangeInfinityResult) {
  const uint8_t fuzz_data[] = {
      0xff,                   // true for call to ConsumeBool
      0xff, 0xff, 0xff, 0xff, // max int for call to ConsumeIntegral
  };
  FuzzedDataProvider DataProvider(fuzz_data, sizeof(fuzz_data));

  float min = -130363940359280487072189884884913750016.000000;
  float max = 340282346638528859811704183484516925440.000000;

  float result = DataProvider.ConsumeFloatingPointInRange<float>(min, max);

  ASSERT_FLOAT_EQ(result, max);
  ASSERT_EQ(result, max);
}

TEST(FuzzedDataProvider, ConsumeFloatingPointInRangeFiniteResult) {
  const uint8_t fuzz_data[] = {
      0xff,                   // true for call to ConsumeBool
      0xff, 0xff, 0xff, 0xff, // max int for call to ConsumeIntegral
  };

  FuzzedDataProvider DataProvider(fuzz_data, sizeof(fuzz_data));
  float min = -317596856862626935824257237918882463744.00000000000000000;
  float max = 22690334736495996280219465965285408768.00000000000000000;

  float result = DataProvider.ConsumeFloatingPointInRange<float>(min, max);

  ASSERT_FLOAT_EQ(max, result);
  ASSERT_EQ(max, result);
}

2. Build and run the tests

Why this might not be a bug

Both of these tests pass the ASSERT_FLOAT_EQ check which the rest of the unit tests use when dealing with floats and that assert will call gtest's AlmostEquals function which considers both of these cases as correct.

  // Returns true if and only if this number is at most kMaxUlps ULP's away
  // from rhs.  In particular, this function:
  //
  //   - returns false if either number is (or both are) NAN.
  //   - treats really large numbers as almost equal to infinity.
  //   - thinks +0.0 and -0.0 are 0 DLP's apart.

So would you consider it valid for consumeFloatingPointInRange to return numbers greater than max, including infinity in cases with a very large max?