how to transform from pixel coordinate to world coordinate?

[kargarisaac]

Hi,

I have some points in the pixel coordinate frame (for example locations and orientations of cars in the scene) and want to transform it into the world frame. I see that you have location_to_pixel() and apply_agent_following_transformation_to_masks() functions to do it in the other direction (world to pixel). Can you help me to do the inverse?

Thanks

[micmarty]

@kargarisaac Hi, sorry for such a huge delay in reponse :(

I rotated the car on purpose to show you that agent's yaw (relative to the map) is important to properly compute the final carla.Location:

Point O is the agent's location on birdview (which is always in the middle of an image), but at the same time we know his carla.Location which is agent_vehicle_loc:

carla-birdeye-view/carla_birdeye_view/__init__.py

Lines 238 to 240 in f666b0f

	rendering_window = RenderingWindow(
	origin=agent_vehicle_loc, area=self.rendering_area
	)

Output array produced by BirdViewProducer has:

• h = DEFAULT_HEIGHT # 336
• w = DEFAULT_WIDTH # 150
• pixels_per_meter = 4
  

  carla-birdeye-view/carla_birdeye_view/__main__.py

  Lines 46 to 52 in f666b0f

  	birdview_producer = BirdViewProducer(
  	client,
  	PixelDimensions(width=DEFAULT_WIDTH, height=DEFAULT_HEIGHT),
  	pixels_per_meter=4,
  	crop_type=BirdViewCropType.FRONT_AND_REAR_AREA,
  	render_lanes_on_junctions=False,
  	)

O = (h/2, w/2) (in pixels)

Let's say you want to know carla.Location for pixel at X = (20, 10).
We know that b = h/2 - 20 and a = w/2 - 10, then c = sqrt(a**2 + b**2) (in pixels).
So X is radius_m = c / pixels_per_meter meters away from point O (in CARLA world).
At this point we know that our X lies somewhere on a circle with radius=radius_m.

The only thing that now we need is to use yaw information (agent_transform.rotation.yaw) to find the final point on a circle. So you'll need to use trigonometry again to compute that.

Angle between lines b and c is alpha = asin(a / c).

gamma = math.radians(agent_transform.rotation.yaw)  + alpha # (as far as I remember that yaw was in degrees)

# distances between agent's car to point X (in carla world coordinates - meters)
x_distance = radius_m * sin(gamma)
y_distance = radius_m * cos(gamma)

# final coordinates of X (in carla coordinates)
X = carla.Location(x=agent_vehicle_loc.x + x_distance, y=agent_vehicle_loc.y + y_distance)

Please double check if there'd no problems with angle signs (when X is on the left/right to the car from birdview perspective).

I hope this will do the job. Once again, sorry for not responding so long. I'm planning to focus a bit more on maintaining my stuff 😛.
If you're still doing something fun with this package, feel free to submit a PR with some new feature.

[micmarty]

I'll commit this utility method at some point