ReactNative 74, cameraWithTensors fails: Cannot read property 'Type' of undefined

[lucksp]

System information

• iPhone 13Pro, iOS 18

    "@tensorflow/tfjs": "^4.22.0",
    "@tensorflow/tfjs-backend-cpu": "^4.22.0",
    "@tensorflow/tfjs-react-native": "^1.0.0",
    "expo": "^51.0.0",
    "expo-gl": "~14.0.2",
    "react": "18.2.0",
    "react-native": "0.74.5",

Describe the current behavior
I am using a TFJS model from Google Vertex AI, Edge, exported per docs for Object Detection.

Once I have imported the model bin files, and setModel(true), then it is ready to render the TensorCamera component. Unfortunately, the onReady callback from TensorCamera seems to be failing, but not crashing the app. The camera still renders and seems like it's working, but I cannot handle the stream because it's never ready. There are some warnings in the terminal:

Possible Unhandled Promise Rejection (id: 0):
TypeError: Cannot read property 'Type' of undefined

This error goes away when I swap the TensorCamera for the default CameraView, so it feels very certain that something is not compatible with ReactNative 74.

Describe the expected behavior
Based on following the flow from the TFJS example, I would expect newer versions to work as described.

• HOWEVER, I am unsure if the Vertex TFJS model is perhaps incompatible, but rendering the camera should not be related to the model, correct?

Standalone code to reproduce the issue

1. Load the model:

const loadModel: LoadModelType = async (setModel, setIsModelReady) => {
  try {
    await ready();
    const modelJson = require('../../assets/tfjs/model.json');
    const modelWeights1 = require('../../assets/tfjs/1of3.bin');
    const modelWeights2 = require('../../assets/tfjs/2of3.bin');
    const modelWeights3 = require('../../assets/tfjs/3of3.bin');

    const bundle = bundleResourceIO(modelJson, [
      modelWeights1,
      modelWeights2,
      modelWeights3,
    ]);
    const modelConfig = await loadGraphModel(bundle);

    setModel(modelConfig);
    setIsModelReady(true);
  } catch (e) {
      console.error((e as Error).message);
  }
};

export const TFJSProvider = ({ children }) => {
  const [model, setModel] = useState<LayersModel | null>(null);
  const [isModelReady, setIsModelReady] = useState(false);

  const { hasPermission } = useCameraContext();

useEffect(
    function initTFJS() {
      if (hasPermission) {
        (async () => {
          console.log('load model');

          await loadModel(setModel, setIsModelReady);
        })();
      }

    },
    [hasPermission]
  );
}

return (
    <TFJSContext.Provider value={{ model, isModelReady }}>
      {children}
    </TFJSContext.Provider>
  );

2. Create Camera Component

const TensorCamera = cameraWithTensors(CameraView);

export const ObjectDetectionCamera = () => {
  const { model, isModelReady } = useTFJSContext();
  return (
    isModelReady && (
      <TensorCamera
        autorender
        cameraTextureHeight={textureDims.height}
        cameraTextureWidth={textureDims.width}
        onReady={() => console.log('READY!'} // never fires
        resizeDepth={3}
        resizeHeight={TENSOR_HEIGHT}
        resizeWidth={TENSOR_WIDTH}
        style={{ flex: 1 }}
        useCustomShadersToResize={false}
      />
    )
  );
};

Other info / logs

I am unable to find any logs in the console of the device, it seems like the error is being swallowed

I also noticed this logged issue, related to ReactNative 74: #8344