New Release + App Update

Added new functionalities that will be especially useful if you're using the library to control a real manipulator.

Library Update: pip install --upgrade mujoco_ar
Companion App Update: App Store (update required for the new vibration feature)

1. Trigger Device Vibration (Simple Haptics)

You can now trigger a vibration on the device with customized settings for sharpness, intensity, and duration. The goal of this is to add haptic feedback to the user, whether it's on contact with objects or to notify the user of specific events. This can be done as follows:

ar_device.vibrate(sharpness=0.8, intensity=0.4, duration=0.01)

You can also pass in a vibrate_function that automatically triggers a vibration if it returns True while linking the AR data to a certain MuJoCo frame:

def custom_vibrate_condition():
    # Define your condition here
    return some_condition_met

ar_device.link_body(
    name="eef",
    vibrate_fn=custom_vibrate_condition,
    # ... other parameters
)

For a simple demo, try running demos/vibration_demo.py. There’s a box, and when you try to move out of bounds, the device will vibrate.

Note: I think this can be improved upon to provide more localized feedback, such as vibrating different parts of the device continuously based on where the contact occurs. I know this type of feedback is possible on iPhones hardware (check the Ringtone section in iPhone settings), but I’m not certain if Swift allows direct control over this within an app. I will add support for vibrating different areas of the device in a future update if it turned out to be possible, and possibly introducing a profile that provides direct vibration feedback when the received pose deviates from expected compliance (to simulate contact scenarios). I also think implementing continuous vibration would feel more natural than sending different short impulses, but I’ve added this simple version for now as a starting point.

2. Add Pre-pose and Post-pose Transformations

Ability to add pre-pose and post-pose transformation matrices when linking the incoming AR poses to a certain MuJoCo frame. Previously, there was only the option to add a post-pose transformation, requiring manual pre-pose transformations (when necessary).

# Define pre-transform (e.g., rotation of 90 degrees around the Z-axis)
pre_transform = np.array([
    [0, -1, 0, 0],
    [1,  0, 0, 0],
    [0,  0, 1, 0],
    [0,  0, 0, 1]
])

# Define post-transform (e.g., translation along the Y-axis)
post_transform = np.array([
    [1, 0, 0, 0],
    [0, 1, 0, 0.5],
    [0, 0, 1, 0],
    [0, 0, 0, 1]
])

# Link a body with pre and post transformations
ar_device.link_body(
    name="ref",
    pre_transform=pre_transform,
    post_transform=post_transform,
    # ... other parameters
)

3. Position Limits

Added the ability to limit the positions published as the latest AR data (received through get_latest_data()) to ensure that objects do not move beyond the specified ranges. This ensures safety when using MuJoCo AR to control real manipulators and helps prevent potential simulation errors. Limits can be added as shown below:

ar_device.add_position_limit([[-0.1,0.1],[-0.1,0.1],[-0.1,0.1]])

This can also be done at a MuJoCo frame level as follows:

ar_device.link_body(
    name="arm",
    position_limits=np.array([
        [-0.3 0.3],  # X-axis limits
        [-0.8, 0.8],  # Y-axis limits
        [0.0, 1.0]   # Z-axis limits
    ]),
    # ... other parameters
)

If you have any ideas for new features or improvements, feel free to submit your suggestions on the issue tracker.