Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

the exported .pb file folde #13

Open
Enricocloud opened this issue May 30, 2023 · 14 comments
Open

the exported .pb file folde #13

Enricocloud opened this issue May 30, 2023 · 14 comments

Comments

@Enricocloud
Copy link

Sorry, I cannot find this file folder, The Sleap software should generate it, or shoudl create it somehowBecausese you say: "To use the Predict operators, you will need to provide the ModelFileName to the exported .pb file folder containing your pretrained SLEAP model, along with the corresponding PoseConfigFileName to the training_config.json file."
I have the .json file but not the .pb

Thanks

@talmo
Copy link

talmo commented May 30, 2023

Hi @Enricocloud,

You can generate this pb file by using the sleap-export command. Here's more info: https://sleap.ai/guides/cli.html#sleap-export

Cheers,

Talmo

@Enricocloud
Copy link
Author

it works, but now the console gives me this error "

TypeError: Exception encountered when calling layer "bottom_up_inference_layer_1" (type BottomUpInferenceLayer).

'Tensor' object cannot be interpreted as an integer

Call arguments received:
• args=({'image': 'tf.Tensor(shape=(None, 560, 1440, 1), dtype=uint8)'},)
• kwargs=<class 'inspect._empty'>
"
I try installing the TF==2.8.0 but it doesn't work.

Thanks

@talmo
Copy link

talmo commented May 31, 2023

Hi @Enricocloud, yep, so we currently only support export of top-down, ID with top-down, centroid and single instance model's unfortunately. You can try retraining on your labels with a top-down model and giving this another go.

@Enricocloud
Copy link
Author

ok I created the top-down model and the exported file, but now I get this error, I download all the tools and the DLL and put them on the extension folder, so I don't understand what this error is
"
error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice
2023-06-05 14:00:24.181120: W tensorflow/core/framework/op_kernel.cc:1768] UNKNOWN: JIT compilation failed.
"

@bruno-f-cruz
Copy link
Contributor

What is the model of your GPU and what driver version do you have installed? What version of CUDA and cuDNN have you installed? which Tensorflow release did you download?

@Enricocloud
Copy link
Author

I'm working on an Nvidia RTX Quadro 4000, I downloaded the latest driver for Win 11, the CUDA v11.3, and cuDNN 8.2 I downloaded the TensorFlow 2.10 for GPU only.

@bruno-f-cruz
Copy link
Contributor

Is this error coming from Bonsai or Python? Can you make sure that everything you are using is compatible across the board?

e.g. https://transang.me/cuda-cudnn-driver-gcc-tensorflow-python-version-compatibility-charts/

Otherwise, I am not sure that, after following the installation steps I ever came across that error.

@Enricocloud
Copy link
Author

Thanks for all the reply, I check all the compatibility of the environment, I think the error could be by bonsai.
Here it the screen of all the error
image

@bruno-f-cruz
Copy link
Contributor

What do you mean by environment? Bonsai doesn't need to have any Python installed to run. You must install the dependencies as instructed in the tutorial in order for the inference backend to work.

What does your project Extensions folder look like? The error message leads me to believe that some dependency was just not correctly installed. I would probably just try to follow the instructions once again and make sure that all the versions are correctly installed and all the .dll files are in the correct place. Otherwise, I am afraid I won't add much other than just googling for this error and trying to find the solution.

@Enricocloud
Copy link
Author

Hi, just to undestand if was a compatibility problem, I'm now using the CPU tensoflow, i got a top-down-id model, PredictPoseIdentities on bonsai, but i still have an error
image
I used the .pb file exported by bonsai, and as a Traininconfig the training_config.json of the multiclass model.
How can I solve this problem?

@bruno-f-cruz
Copy link
Contributor

I am not sure I follow. Bonsai is not able of exporting any .pb file...

@Enricocloud
Copy link
Author

no sorry, I exported with selap-export, now I want to use the model that I exported to live-track the animals, but it gives me that error. How can I solve it?

@MForsthofer
Copy link

MForsthofer commented Dec 6, 2023

Hey, just wanted to check in if you ever figured out how to solve this? I' running into a similar issue using the PredictPoses node using a top-down model i trained in the GUI. The workflow works nicely with single instance models, but for top-down i get the same 'Object reference not set to an isntance of an object' error.

Also posted on the Sleap discussions about it here
talmolab/sleap#1626

My workflow for testing it is just this

<?xml version="1.0" encoding="utf-8"?>
<WorkflowBuilder Version="2.7.3"
                 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                 xmlns:cv="clr-namespace:Bonsai.Vision;assembly=Bonsai.Vision"
                 xmlns:sleap="clr-namespace:Bonsai.Sleap;assembly=Bonsai.Sleap"
                 xmlns="https://bonsai-rx.org/2018/workflow">
  <Workflow>
    <Nodes>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="cv:FileCapture">
          <cv:FileName>D:\data\training_vids\St46\tadpole_2023-12-05T11_48_42_9.avi</cv:FileName>
          <cv:PlaybackRate>0</cv:PlaybackRate>
          <cv:StartPosition>0.5</cv:StartPosition>
          <cv:PositionUnits>Frames</cv:PositionUnits>
          <cv:Loop>true</cv:Loop>
          <cv:Playing>true</cv:Playing>
        </Combinator>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="sleap:PredictPoses">
          <sleap:ModelFileName>C:\Users\Michael Forsthofer\OneDrive - University of Sussex\Baden Lab\Behavior tracking\TadpoleSwimTracking\SLEAP\models\231206_175613.centered_instance.n=237\frozen_graph.pb</sleap:ModelFileName>
          <sleap:TrainingConfig>C:\Users\Michael Forsthofer\OneDrive - University of Sussex\Baden Lab\Behavior tracking\TadpoleSwimTracking\SLEAP\models\231206_175613.centered_instance.n=237\training_config.json</sleap:TrainingConfig>
          <sleap:CentroidMinConfidence>0.476744175</sleap:CentroidMinConfidence>
          <sleap:PartMinConfidence>0.1744186</sleap:PartMinConfidence>
          <sleap:ScaleFactor xsi:nil="true" />
          <sleap:ColorConversion xsi:nil="true" />
        </Combinator>
      </Expression>
    </Nodes>
    <Edges>
      <Edge From="0" To="1" Label="Source1" />
    </Edges>
  </Workflow>
</WorkflowBuilder>

@talmo
Copy link

talmo commented Dec 8, 2023

Hey guys, just tagging talmolab/sleap#1542 and @ssrinath22 who's working on this issue on the SLEAP side.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants