[RFC] Support DistillT5 #652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Draft

idostyle wants to merge 1 commit into leejet:master from idostyle:dt

Contributor

idostyle commented Apr 9, 2025

Add support for https://huggingface.co/LifuWang/DistillT5 https://github.com/LifuWang-66/DistillT5

Can be tested with https://huggingface.co/Eviation/DistillT5 e.g. https://huggingface.co/Eviation/DistillT5/blob/main/DistillT5-F32.safetensors which removes the additional wrapping.

Probably needs to be integrated, and at best automatically detected, alongside T5 XXL
Check if there is a way to use the unmodified LifuWang DistillT5 model with the additional encoder wrapping on the tensor names

I would appreciate any ideas on how best to incorporate this in a less intrusive way.


          Support DistillT5

d525163

Contributor

stduhpf commented Apr 10, 2025

Nice, just as I was thinking of adding it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet