Floating point quantization custom op and datatypes #182
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…on can be found in the `Examples`. `±inf` are clipped to `±max_val`. `±NaN` are mapped to `±NaN`. The zero is always representable. I tested with subnormals (to be intended as subnormals for the output representation) and the quantizer represented the subnormals with no loss (I didn't extensively tested this part though). I tested the function against Brevitas `FloatQuant` implementation: they do not always match. For example I think `0.3125` should be representable (`x == xq`) by a float quantizer with 4bits for mantissa, 4bits for the exponent, 0 bias and 1bit for the sign. Brevitas `FloatQuant` implementation quantize it to `0.25`. Not sure what I should consider correct for this case.
Co-authored-by: Nicolo Ghielmetti <nicolo.ghielmetti@gmail.com>
… provided. Some other tests have been added
… quantization logic. Now QONNX and Brevitas float quantisers match.
…ion. Default exponent bias is now computed if not provided, and tests have been added to compare QONNX and Brevitas float quantization outputs.
A first sample version of `FloatQuant`
… to quant.py and Quant.
…and assumes no denormals.
…oat, added tests.
Merged with float_quant, added exp bias, implemented FloatQuant.infer_node_datatype()
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a merger of PRs #180 #178 and #159 which introduce the following features:
FloatQuantop for arbitrary-precision floating point quantization custom opFloatQuantcustom op and unit tests for its QONNX executionQuanttoIntQuant(with backwards compatibility for now)The spec for
FloatQuantwas put together by myself, Ian Colbert, Nicholas Fraser, Giuseppe Franco and Jakoba Petri-Koenig from AMD. Most of the QONNX-side contributions here are by @nghielme and @ebby-s with additional reviewing by Shane Fleming (AMD) so big kudos to them.