Skip to content

[BUG Report]: Keras model.save() throws exception when saving to network path on Windows #1287

Open
@abrguyt

Description

@abrguyt

Description

Using the custom packaging of the latest Tensorflow.NET version from the Github Repo (to overcome the bug in the official NuGet package - see https://github.com/SciSharp/TensorFlow.NET/issues/1246#issuecomment-2630024514 I got Tensorflow.NET / Keras model fitting, saving and loading working well.

Just when saving a Keras model to a Windows network path like '\FS-01\COM-Data$\ML Models' an message-less exception is thrown in

at Tensorflow.Eager.EagerRunner.SetOpAttrList(Context ctx, SafeEagerOpHandle op, String key, Object values, TF_AttrType type, Dictionary`2 attr_list_sizes, Status status) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Eager\EagerRunner.TFE_FastPathExecute.cs:line 326

Looking at the source it seems line 323 in the file 'TensorFlowNET.Core\Eager\EagerRunner.TFE_FastPathExecute.cs' is causing this with

throw new NotImplementedException("");

Saving the model to a local folder on the file system works perfectly well - I suspect somehow the unusual characters in the network file path cause this exception; but I can't dive deep in the Tensorflow.NET source to figure out why this is happening - any suggestions?

Full stack trace:

Source: Tensorflow.Binding
 Strack Trace:    at Tensorflow.Eager.EagerRunner.SetOpAttrList(Context ctx, SafeEagerOpHandle op, String key, Object values, TF_AttrType type, Dictionary`2 attr_list_sizes, Status status) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Eager\EagerRunner.TFE_FastPathExecute.cs:line 326
   at Tensorflow.Eager.EagerRunner.SetOpAttrs(SafeEagerOpHandle op, Object[] attrs) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Eager\EagerRunner.TFE_FastPathExecute.cs:line 241
   at Tensorflow.Eager.EagerRunner.TFE_ExecuteCancelable(Context ctx, String device_name, String op_name, Tensor[] inputs, Object[] attrs, Int32 num_outputs) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Eager\EagerRunner.TFE_Execute.cs:line 63
   at Tensorflow.Eager.EagerRunner.TFE_Execute(Context ctx, String device_name, String op_name, Tensor[] inputs, Object[] attrs, Int32 num_outputs) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Eager\EagerRunner.TFE_Execute.cs:line 36
   at Tensorflow.Eager._execute.quick_execute(String op_name, Int32 num_outputs, Tensor[] inputs, Object[] attrs, Context ctx, String name) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Eager\execute.cs:line 31
   at Tensorflow.io_ops.save_v2_eager_fallback(Tensor prefix, String[] tensor_names, String[] shape_and_slices, Tensor[] tensors, String name, Context ctx) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Operations\io_ops.cs:line 58
   at Tensorflow.io_ops.save_v2(Tensor prefix, String[] tensor_names, String[] shape_and_slices, Tensor[] tensors, String name) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Operations\io_ops.cs:line 40
   at Tensorflow.tensorflow.IoApi.save_v2(Tensor prefix, String[] tensor_names, String[] shape_and_slices, Tensor[] tensors, String name) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\APIs\tf.io.cs:line 45
   at Tensorflow.Checkpoint.SingleDeviceSaver.save(Tensor file_prefix, CheckpointOptions options) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\functional_saver.cs:line 79
   at Tensorflow.Checkpoint.MultiDeviceSaver.<>c__DisplayClass5_2.<save>b__6(ITensorFlowObject _) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\functional_saver.cs:line 285
   at Tensorflow.Binding.tf_with[T](T py, Action`1 action) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Binding.Util.cs:line 199
   at Tensorflow.Checkpoint.MultiDeviceSaver.<>c__DisplayClass5_0.<save>g__save_fn|1() in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\functional_saver.cs:line 283
   at Tensorflow.Checkpoint.MultiDeviceSaver.save(Tensor file_prefix, CheckpointOptions options) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\functional_saver.cs:line 305
   at Tensorflow.Checkpoint.MultiDeviceSaver.save(String file_prefix, CheckpointOptions options) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\functional_saver.cs:line 309
   at Tensorflow.Checkpoint.TrackableSaver.<>c__DisplayClass14_0.<save_cached_when_graph_building>b__0() in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\checkpoint.cs:line 119
   at Tensorflow.Checkpoint.TrackableSaver.save_cached_when_graph_building(String file_prefix, Tensor object_graph_tensor, CheckpointOptions options) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\checkpoint.cs:line 136
   at Tensorflow.Checkpoint.TrackableSaver.save(String file_prefix, Nullable`1 checkpoint_number, Session session, CheckpointOptions options) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Checkpoint\checkpoint.cs:line 180
   at Tensorflow.SavedModelUtils.save_and_return_nodes(Trackable obj, String export_dir, ConcreteFunction signatures, SaveOptions options, Boolean experimental_skip_checkpoint) in D:\temp\TensorFlow.NET-master\src\TensorFlowNET.Core\Training\Saving\SavedModel\save.cs:line 46
   at Tensorflow.Keras.Saving.SavedModel.KerasSavedModelUtils.save_model(Model model, String filepath, Boolean overwrite, Boolean include_optimizer, ConcreteFunction signatures, SaveOptions options, Boolean save_traces)
   at Tensorflow.Keras.Engine.Model.save(String filepath, Boolean overwrite, Boolean include_optimizer, String save_format, SaveOptions options, ConcreteFunction signatures, Boolean save_traces)

Reproduction Steps

No response

Known Workarounds

No response

Configuration and Other Information

Windows Server 2022, Windows 10
Tensorflow.NET v 0.150
Tensorflow.Keras v 0.15

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions