mesh_resource always empty when using mcap file

[ShivKhanna]

Description

I have a CPP app in which I take pose data from a CSV files and embed them into ros2msg's using mcap. My pose and path and marker messages all show up in foxglove. However, my marker's mesh_resource is always an empty string and mesh_use_embedded_matrials is always true. I have serialized the message in my CPP app. I have tried to unserialized it after that, and the data is what I set it to.

• Version: Master
• Platform: Linux x86

Steps To Reproduce

visualization_msgs::msg::Marker poseStampedToMarker(
  const geometry_msgs::msg::PoseStamped& ps,
  const std::string& markerFrame,
  uint32_t id
) {
  visualization_msgs::msg::Marker marker;
  // Header assignment
  marker.header = ps.header;
  // Namespace and unique ID
  marker.ns = "uav_marker";
  marker.id = static_cast<int32_t>(id);
  // Marker type: MESH_RESOURCE
  marker.type = visualization_msgs::msg::Marker::MESH_RESOURCE;
  // Marker action: ADD
  marker.action = visualization_msgs::msg::Marker::ADD;
  // Set the marker's pose equal to the PoseStamped pose.
  marker.pose = ps.pose;
  // Scale: 1.0 means 1:1 scaling.
  marker.scale.x = 1.0;
  marker.scale.y = 1.0;
  marker.scale.z = 1.0;
  // Color (white):
  marker.color.r = 1.0f;
  marker.color.g = 1.0f;
  marker.color.b = 1.0f;
  marker.color.a = 1.0f;
  // Lifetime: 0 means infinite.
  marker.lifetime.sec = 0;
  marker.lifetime.nanosec = 0;
  marker.frame_locked = false;
  marker.mesh_resource = "file:///D:/test.glb";
  marker.mesh_use_embedded_materials = true;

  // Debug: Print the mesh_resource value
  std::cout << "[DEBUG] Marker ID " << id << " mesh_resource: " << marker.mesh_resource << std::endl;
  
  return marker;
}

Expected Behavior

I expect mesh_resource to be populated with my file link as seen above in the code snippet.

[jtbandes]

Could you share the mcap file and the code you used to create it?

[ShivKhanna]

MCAP File: uav_data.zip

Here is the code for how i create an serialize the data

visualization_msgs::msg::Marker poseStampedToMarker(
  const geometry_msgs::msg::PoseStamped& ps,
  const std::string& markerFrame,
  uint32_t id
) {
  visualization_msgs::msg::Marker marker;
  // Header assignment
  marker.header = ps.header;
  // Namespace and unique ID
  marker.ns = "uav_marker";
  marker.id = static_cast<int32_t>(id);
  // Marker type: MESH_RESOURCE
  marker.type = visualization_msgs::msg::Marker::MESH_RESOURCE;
  // Marker action: ADD
  marker.action = visualization_msgs::msg::Marker::ADD;
  // Set the marker's pose equal to the PoseStamped pose.
  marker.pose = ps.pose;
  // Scale: 1.0 means 1:1 scaling.
  marker.scale.x = 1.0;
  marker.scale.y = 1.0;
  marker.scale.z = 1.0;
  // Color (white):
  marker.color.r = 1.0f;
  marker.color.g = 1.0f;
  marker.color.b = 1.0f;
  marker.color.a = 1.0f;
  // Lifetime: 0 means infinite.
  marker.lifetime.sec = 0;
  marker.lifetime.nanosec = 0;
  marker.frame_locked = false;
  marker.mesh_resource = "file:///D:/test.glb";
  marker.mesh_use_embedded_materials = true;

  // Debug: Print the mesh_resource value
  std::cout << "[DEBUG] Marker ID " << id << " mesh_resource: " << marker.mesh_resource << std::endl;
  
  return marker;
}


  //Later in my code I use the function above and write it to mcap

    visualization_msgs::msg::Marker marker = poseStampedToMarker(ps, "map", static_cast<uint32_t>(i));

    // Serialize the Marker into a SerializedMessage
    rclcpp::SerializedMessage markerSer;
    markerSerializer.serialize_message(&marker, &markerSer);

    // Copy the serialized data into a byte vector
    std::vector<std::byte> markerCdr(markerSer.size());
    std::memcpy(markerCdr.data(),
                markerSer.get_rcl_serialized_message().buffer,
                markerSer.size());

    // Immediately deserialize the serialized message to verify the contents.
    visualization_msgs::msg::Marker deserializedMarker;
    rclcpp::Serialization<visualization_msgs::msg::Marker> markerDeser;
    markerDeser.deserialize_message(&markerSer, &deserializedMarker);
    std::cout << "[DEBUG] Deserialized Marker mesh_resource: " 
              << deserializedMarker.mesh_resource << std::endl;
    std::cout << "[DEBUG] Deserialized Marker mesh_use_embedded_materials: " 
              << (deserializedMarker.mesh_use_embedded_materials ? "true" : "false") << std::endl;

    // Create the MCAP message using the serialized data.
    mcap::Message m;
    m.channelId   = markerChannel.id;
    m.logTime     = p.timestamp_ns;
    m.publishTime = p.timestamp_ns;
    m.data        = markerCdr.data();
    m.dataSize    = markerCdr.size();

    // Write the MCAP message.
    status = writer.write(m);

Am I using this marker properly, am i using the wrong type of ros2 message? Since I am not using a ros2 environment to collect the original data, should I not use it as the format to embed the data into mcap? It seems like it is the most supported on the visualization side, so i used it. From the documentation, I believe mesh_resource should be populated with my file link as seen above in the code snippet. Any direction or help would be appreciated. Thanks!

[jtbandes]

Since I am not using a ros2 environment to collect the original data, should I not use it as the format to embed the data into mcap?

Yes, if you aren't actually using ROS, you could avoid a lot of headache by choosing another supported format instead, such as protobuf.

[jtbandes]

I suspect the schema that you put in the mcap schema record doesn't match the actual schema you are using to serialize the data. For example, the text field is present in what the app displays but is not present in your code. Your mesh_resource path is indeed present in the serialized data, but apparently not at the expected offset used by the deserializer:

[ShivKhanna]

Thanks for the info. I was able to find that my schema that I created as a string literal was missing a few fields that existed in the version of the ros2 message. I'll look into switching to protobufs to avoid this issue in the future.