Sample tf_model_garden model cannot be downloaded

[haltyt]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

Windows11

Python Version

3.11

MediaPipe Model Maker version

No response

Task name (e.g. Image classification, Gesture recognition etc.)

Object detection

Describe the actual behavior

tf_model_garden model cannot be downloaded

Describe the expected behaviour

tf_model_garden model should be downloadable

Standalone code/steps you may have used to try to get what you need

https://colab.research.google.com/github/googlesamples/mediapipe/blob/main/examples/customization/object_detector.ipynb

Other info / Complete Logs

model = object_detector.ObjectDetector.create(
    train_data=train_data,
    validation_data=validation_data,
    options=options)

/usr/local/lib/python3.11/dist-packages/keras/src/engine/functional.py:642: UserWarning: Input dict contained keys ['6'] which did not match any model input. They will be ignored by the model.
  inputs = self._flatten_to_reference_inputs(inputs)
WARNING:tensorflow:`tf.keras.layers.experimental.SyncBatchNormalization` endpoint is deprecated and will be removed in a future release. Please use `tf.keras.layers.BatchNormalization` with parameter `synchronized` set to True.
Downloading https://storage.googleapis.com/tf_model_garden/vision/qat/mobilenetv3.5_ssd_coco/mobilenetv3.5_ssd_i256_ckpt.tar.gz to /tmp/model_maker/object_detector/mobilenetmultiavg
---------------------------------------------------------------------------
ReadError                                 Traceback (most recent call last)
/tmp/ipython-input-7-5300388.py in <cell line: 0>()
----> 1 model = object_detector.ObjectDetector.create(
      2     train_data=train_data,
      3     validation_data=validation_data,
      4     options=options)

5 frames
/usr/lib/python3.11/tarfile.py in open(cls, name, mode, fileobj, bufsize, **kwargs)
   1856                     continue
   1857             error_msgs_summary = '\n'.join(error_msgs)
-> 1858             raise ReadError(f"file could not be opened successfully:\n{error_msgs_summary}")
   1859 
   1860         elif ":" in mode:

ReadError: file could not be opened successfully:
- method gz: ReadError('empty file')
- method bz2: ReadError('not a bzip2 file')
- method xz: ReadError('not an lzma file')
- method tar: ReadError('empty file')

[haltyt]

tf_model_garden GCS bucket is returning 403 for all object detector checkpoints. The mediapipe-tasks bucket is still accessible. All 4 model specs in model_spec.py are affected (mobilenetv2_i256, mobilenetv2_i320, mobilenetmultiavg, mobilenetmultiavg_i384). Checkpoints need to be migrated to an accessible bucket, or URLs updated.

[deepzrj]

Are there any alternative GCS buckets or other sources where these models can be manually downloaded from?

[CMDezz]

any workaround or updated information ?

[karthikeyanelumina]

The model trains successfully using TFLite Model Maker, but it is not compatible with MediaPipe.
any one know how to export properly for mediapipe ?

[mptsounds]

Not sure how helpful this is, but I was able to manually download and force the training to start using this older version of MobileNet-V2 from this link: http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_coco_2018_03_29.tar.gz

However, I can see that my model is not working properly after training was done (which may also be because I'm totally new to object detection in general). I can share more details if it helps, but this might have been a terrible/janky "fix".

Would be great if someone with better understanding of Mediapipe & object detection can look at that link first.

[ikanor]

Still happening here. Getting a 403 on all endpoints.

[sustinespacne9-ai]

Not sure how helpful this is, but I was able to manually download and force the training to start using this older version of MobileNet-V2 from this link: http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_coco_2018_03_29.tar.gz

However, I can see that my model is not working properly after training was done (which may also be because I'm totally new to object detection in general). I can share more details if it helps, but this might have been a terrible/janky "fix".

Would be great if someone with better understanding of Mediapipe & object detection can look at that link first.

Hi! Thanks for sharing the MobileNet-V2 link.
Could you share the exact bash commands or Colab cells you used to manually download and point the training to that checkpoint?
I'm encountering errors while trying to override the default checkpoint, and I’d like to compare with your setup.

[mptsounds]

Not sure how helpful this is, but I was able to manually download and force the training to start using this older version of MobileNet-V2 from this link: http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_coco_2018_03_29.tar.gz
However, I can see that my model is not working properly after training was done (which may also be because I'm totally new to object detection in general). I can share more details if it helps, but this might have been a terrible/janky "fix".
Would be great if someone with better understanding of Mediapipe & object detection can look at that link first.

Hi! Thanks for sharing the MobileNet-V2 link. Could you share the exact bash commands or Colab cells you used to manually download and point the training to that checkpoint? I'm encountering errors while trying to override the default checkpoint, and I’d like to compare with your setup.

Hey @sustinespacne9-ai , sure! I did this for a class assignment, and my understanding is school academic policies don't allow me to share the full assignment, so I'll just share the specific steps to bypass the checkpoints.

⚠️ Disclaimer: I had only learned about object detection 1-2 weeks prior to this, so let me know if I used any wrong terms or definitions, or if something doesn't make sense. Also, I honestly have no clue if what I did broke the model in some way, so please use the method at your own risk. I'd really love it if someone chimes in and tell me what I did wrong, and if you know a better fix, please let me know!

I bypassed 4 things in total:

1. Wrong file names & directories in the 2018 version: Some files need to be moved and/or renamed after downloading.
2. Stopping the automatic library download from happening: When the retraining process begins (i.e. when running the cell that starts with model = object_detector.ObjectDetector.create([....]), for some reason, the library will still be re-downloaded from the currently non-working link.
3. Stopping the model creation from crashing: Because the library is older and was trained using Tensorflow 1 (I think?), we need to stop the model creation from crashing. This is the step that I think broke my model somehow.
4. Disabling model.summary(): The library crashes when trying to print the summary, so we need to disable it.

Here's my Colab note with only those 4 steps: https://colab.research.google.com/drive/1xNARiZTrppnbDqBXViO5p5NFYM7AMKzg?usp=sharing

I'm using Python v.3.11 and Tensorflow v2.15.0.

[sustinespacne9-ai]

Not sure how helpful this is, but I was able to manually download and force the training to start using this older version of MobileNet-V2 from this link: http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_coco_2018_03_29.tar.gz
However, I can see that my model is not working properly after training was done (which may also be because I'm totally new to object detection in general). I can share more details if it helps, but this might have been a terrible/janky "fix".
Would be great if someone with better understanding of Mediapipe & object detection can look at that link first.

Hi! Thanks for sharing the MobileNet-V2 link. Could you share the exact bash commands or Colab cells you used to manually download and point the training to that checkpoint? I'm encountering errors while trying to override the default checkpoint, and I’d like to compare with your setup.

Hey @sustinespacne9-ai , sure! I did this for a class assignment, and my understanding is school academic policies don't allow me to share the full assignment, so I'll just share the specific steps to bypass the checkpoints.

⚠️ Disclaimer: I had only learned about object detection 1-2 weeks prior to this, so let me know if I used any wrong terms or definitions, or if something doesn't make sense. Also, I honestly have no clue if what I did broke the model in some way, so please use the method at your own risk. I'd really love it if someone chimes in and tell me what I did wrong, and if you know a better fix, please let me know!

I bypassed 4 things in total:

1. Wrong file names & directories in the 2018 version: Some files need to be moved and/or renamed after downloading.
2. Stopping the automatic library download from happening: When the retraining process begins (i.e. when running the cell that starts with model = object_detector.ObjectDetector.create([....]), for some reason, the library will still be re-downloaded from the currently non-working link.
3. Stopping the model creation from crashing: Because the library is older and was trained using Tensorflow 1 (I think?), we need to stop the model creation from crashing. This is the step that I think broke my model somehow.
4. Disabling model.summary(): The library crashes when trying to print the summary, so we need to disable it.

Here's my Colab note with only those 4 steps: https://colab.research.google.com/drive/1xNARiZTrppnbDqBXViO5p5NFYM7AMKzg?usp=sharing

I'm using Python v.3.11 and Tensorflow v2.15.0.

Hi bro, thank you again for the bypass solution you gave me. It really helped me get the MobileNetV2 model running properly in Colab. I appreciate it a lot 🙏
However, I ran into another issue. While training the model, Colab suddenly disconnected in the middle of the epochs, and when the runtime restarted, all the training progress was wiped out. The export folder didn’t contain any checkpoints, so I couldn’t resume and had to start from zero again.
Do you know a way to prevent Colab from disconnecting during long training? Or maybe a way to force MediaPipe Model Maker to save checkpoints per epoch so I can resume if the runtime crashes? Since we’re using the bypass and patched loading, I’m not sure if that affects checkpoint saving.
I’d really appreciate any advice you have on making the training more stable. Thanks again for your help!

[ikanor]

Still happening here. Getting a 403 on all endpoints.

Have you figured it out? I'm also stuck in this part

No luck so far. I managed to bypass the download and then run the training with the patches @mptsounds posted, but the generated model does not detect a thing. I am using the same dataset I used in previous trainings, so ig there's something wrong with one of the steps. I'm pausing this until they get the official model garden fixed :_)

[sustinespacne9-ai]

Still happening here. Getting a 403 on all endpoints.

Have you figured it out? I'm also stuck in this part

No luck so far. I managed to bypass the download and then run the training with the patches @mptsounds posted, but the generated model does not detect a thing. I am using the same dataset I used in previous trainings, so ig there's something wrong with one of the steps. I'm pausing this until they get the official model garden fixed :_)

Since the TensorFlow Model Maker pipeline in Colab is no longer working properly (GCS bucket issue), I’m planning to train my custom dataset using YOLO instead and then export the trained model to a .tflite file for deployment.

Do you think this workflow is possible and practical? Specifically, I want to confirm if a YOLO-trained model can be converted to TensorFlow Lite and used directly with the TFLite interpreter without major issues.

If you’ve tried this before or have any recommendations, I’d really appreciate your advice.

[haltyt]

I was able to train a TFLite model by directly downloading the EfficientDet checkpoint from Google AutoML. After adding MediaPipe ObjectDetector-compatible metadata, I confirmed it works correctly. Sharing the notebook here.
https://colab.research.google.com/drive/1WxdGtORqeJSQGaqsH_8D1TIWEqlSA3aG?usp=sharing