Closed
Description
$ ./generate-coreml-model.sh large-v3-turbo
Torch version 2.6.0 has not been tested with coremltools. You may run into unexpected errors. Torch 2.5.0 is the most recent version that has been tested.
ModelDimensions(n_mels=128, n_audio_ctx=1500, n_audio_state=1280, n_audio_head=20, n_audio_layer=32, n_vocab=51866, n_text_ctx=448, n_text_state=1280, n_text_head=20, n_text_layer=4)
/whisper.cpp/models/convert-whisper-to-coreml.py:146: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert x.shape[1:] == self.positional_embedding.shape[::-1], "incorrect audio shape"
/whisper.cpp/models/.venv/lib/python3.11/site-packages/ane_transformers/reference/layer_norm.py:60: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert inputs.size(1) == self.num_channels
/whisper.cpp/models/convert-whisper-to-coreml.py:88: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
scale = float(dim_per_head)**-0.5
Converting PyTorch Frontend ==> MIL Ops: 100%|████████▉| 6051/6052 [00:00<00:00, 8919.64 ops/s]
Running MIL frontend_pytorch pipeline: 100%|████████████████| 5/5 [00:00<00:00, 21.06 passes/s]
Running MIL default pipeline: 100%|███████████████████████| 87/87 [00:05<00:00, 16.46 passes/s]
Running MIL backend_neuralnetwork pipeline: 100%|███████████| 9/9 [00:00<00:00, 29.21 passes/s]
Translating MIL ==> NeuralNetwork Ops: 100%|█████████████| 5641/5641 [04:14<00:00, 22.18 ops/s]
Traceback (most recent call last):
File "/whisper.cpp/models/convert-whisper-to-coreml.py", line 320, in <module>
encoder = convert_encoder(hparams, encoder, quantize=args.quantize)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/whisper.cpp/models/convert-whisper-to-coreml.py", line 255, in convert_encoder
model = ct.convert(
^^^^^^^^^^^
File "/whisper.cpp/models/.venv/lib/python3.11/site-packages/coremltools/converters/_converters_entry.py", line 635, in convert
mlmodel = mil_convert(
^^^^^^^^^^^^
File "/whisper.cpp/models/.venv/lib/python3.11/site-packages/coremltools/converters/mil/converter.py", line 186, in mil_convert
return _mil_convert(
^^^^^^^^^^^^^
File "/whisper.cpp/models/.venv/lib/python3.11/site-packages/coremltools/converters/mil/converter.py", line 245, in _mil_convert
return modelClass(
^^^^^^^^^^^
File "/whisper.cpp/models/.venv/lib/python3.11/site-packages/coremltools/models/model.py", line 489, in __init__
self.__proxy__, self._spec, self._framework_error = self._get_proxy_and_spec(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/whisper.cpp/models/.venv/lib/python3.11/site-packages/coremltools/models/model.py", line 550, in _get_proxy_and_spec
_MLModelProxy(
ValueError: basic_string
./generate-coreml-model.sh base.en works fine
checkout v1.7.4 works fine
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Milestone
Relationships
Development
No branches or pull requests
Participants
Activity
cssyncing commentedon Apr 18, 2025
Same error here. Did you figure out how to cater?
MushR00m commentedon Apr 18, 2025
checkout v1.7.4
coreml : skip model load in convert-whisper-to-coreml.py
danbev commentedon Apr 19, 2025
@MushR00m @cssyncing Would one of you be able to see if #3060 works for you?
MushR00m commentedon Apr 20, 2025
Thanks for the fix. it looks like working fine.
here is the output log, However, my macos system produces a crash report after the script is allowed to finish, I didn't pay much attention to what it is, it's like a coremlcompiler toolchain error report under xcode.
here are some of the coremlcompiler crash logs.
of course it could be a problem with my xcode environment, nothing to do with whisper, because I also get two crash reports when compiling whisper. Posting this simply to get help to see if it's a problem with my environment😄
compile crash log
danbev commentedon Apr 20, 2025
Sorry about this, I'm getting this error as well. I had been running the python script directly when looking into this and obviously never ran the conversion bash script and therefor did not notice this. I'll take another look and see what the issues and if the current fix is just hiding this error.
Update: I've updated the linked PR with a commit which tries to address this.
coreml : set convert_to="mlprogram" in convert
danbev commentedon Apr 24, 2025
Closing this as #3060 has been merged. Please let us know if this did not fix this issue for you.