Questions for Gelsight mini Integration

[Marsilea149]

Hi,

Could you please share how do you define and compute the value for "mm_to_pixel" in normal_to_rgb.yaml file? I am using Gelsight mini, but I am not sure what to put for this parameter. Thank you very much!

[Rancho-zhao]

Thank you for your interest. This parameter is the scale ratio between the real world coordinate system and the tactile image pixel coordinate system. To calculate it, you can first press a small ball of a known diameter (in millimeters) using gelsight, and then measure the diameter (in pixels) of the circular region that the ball corresponds to in the tactile image. Hope it can help you.

[Marsilea149]

Hi Rancho,

Thank you very much for your quick reply and very clear explanation. I tried with a rectangular shape object. I found out that for horizontal(length of indentor) and vertical(height of indentor), the resulting mm_to_pixel value is different.

Here is my image:

rectangle shape indentor:

• length:
  real measurement: 7 mm
  pixels: 165 - 83 = 82 pixels
  mm_to_pixels = 82/7 = 11.7
• width:
  real measurement: 5 mm
  pixels: 188 - 134 = 54 pixels
  mm_to_pixels = 54/7 = 10.8

Which value should I take into account? Is it normal that I do not find the same value for length and width?

Thank you very much!

[Rancho-zhao]

Well, due to the curvature of the edges of the indentor you used, it is necessary to measure the positions of four points, as shown in the figure.

My measurement result is:

• length (red points):
  pixels: (162,182) - (83,182) = 79
  mm_to_pixel = 79/7 = 11.3
• width (yellow points):
  pixels: (89,189) - (89,133) = 56
  mm_to_pixel = 56/5 = 11.2

This result is within the allowable range of error.

[Rancho-zhao]

In addition, please don't create new issue to ask similar questions, just propose those questions in this issue, thanks.

[Marsilea149]

Hi Rancho,

Thank you very much for your reply! I will ask all the related questions for Gelsight Mini integration in this issue as you suggested.

In tactile_render.py:

FOTS/tactile_render.py

Line 18 in 5bcde04

ini_bg_mlp = np.load('./utils/utils_data/ini_bg_fots_gelsight.npy')

what is "ini_bg_mlp" and how do you get it? Thanks!

[Rancho-zhao]

Hi Rancho,

Thank you very much for your reply! I will ask all the related questions for Gelsight Mini integration in this issue as you suggested.

In tactile_render.py:

FOTS/tactile_render.py

Line 18 in 5bcde04

ini_bg_mlp = np.load('./utils/utils_data/ini_bg_fots_gelsight.npy')

what is "ini_bg_mlp" and how do you get it? Thanks!

Actually, ini_bg_mlp is the image generated by the mlp model when the object has zero contact depth and no background image is added.

[Marsilea149]

Hi Rancho,

Thanks for your reply!

1. question about "ini_bg_fots_gelsight.py":
   Do you have any existing code to generate './utils/utils_data/ini_bg_fots_gelsight.npy'? I saw in 'fots_render.py', you commented this line: # np.save("ini_bg_fots_gelsight3.npy", sim_img)
   But I don't see where you call this method "generate(self, heightMap, contact_mask, shadow = False)" to generate 'ini_bg_fots_gelsight.npy'. Could you please explain how do you do in practice?

2. question about "light_r", "light_g", "light_b":
   How do you get the values for these 3 parameters in "fots_render.py" at line 103-105 as shown below?
   

   FOTS/utils/fots_render.py

   Line 103 in 5bcde04

   light_r = [-0.01, 0, 0.1]

3. question about "shadow_calib.py":
   In "shadow_calib.py", you are using "pr.pixmm" at line 68:

   FOTS/shadow_calib/shadow_calib.py

   Line 68 in 5bcde04

   ball_r = 3 / pr.pixmm

   
   But, "pr.pixmm" is not defined, what is it exactly? I guess it is pr.pixmm=1.0/pr.mm_to_pixel. Could you please clarify?

4. question about "marker_calib.py"
   In "marker_calib.py", you computed "ball_r" at line 115,

   FOTS/marker_calib/marker_calib.py

   Line 115 in 5bcde04

   ball_r = 3 / (0.0266*2)

   Where does this formula come from? Could you please explain what is it exactly? Is it the ball radius in pixels?

5. question about the experiment for "marker_calib.py":
   Do you have any README for "marker_calib.py", what do you do in practice to collect the data and to perform the marker calibration?

Thank you very much!