Awesome-Tactile-Perception

science popularization

• Exploring Tactile Perceptual Dimensions Using Materials Associated with Sensory Vocabulary
  [Frontiers in Psycholog, 2017] [paper]

Tactile Sensor

• Deep Domain Adaptation Regression for Force Calibration of Optical Tactile Sensors
  [paper]

Title	Paper
When Vision Meets Touch: A Contemporary Review for Visuotactile Sensors from the Signal Processing Perspective	https://arxiv.org/abs/2406.12226
Design of a High-Performance Tomographic Tactile Sensor by Manipulating the Detector Conductivity	https://arxiv.org/abs/2406.00978
Optimizing Multi-Touch Textile and Tactile Skin Sensing Through Circuit Parameter Estimation	https://arxiv.org/abs/2404.15131
RoTipBot: Robotic Handling of Thin and Flexible Objects using Rotatable Tactile Sensors	https://arxiv.org/abs/2406.09332
Automatic Calibration for an Open-source Magnetic Tactile Sensor	https://arxiv.org/abs/2405.18582
BeadSight: An Inexpensive Tactile Sensor Using Hydro-Gel Beads	https://arxiv.org/abs/2405.13204
Optimizing BioTac Simulation for Realistic Tactile Perception	https://arxiv.org/abs/2404.10425
A Novel Bioinspired Neuromorphic Vision-based Tactile Sensor for Fast Tactile Perception	https://arxiv.org/abs/2403.10120

Data Collection

• Active Exploration for Real-Time Haptic Training
  [ICRA 2024] [paper]
  Goal: Focusing on only the most informative aspects of an object during data collection.
  Method: We demonstrate our active learning approach using a biomimentic sensor, exploring “tactile scenes” composed of shapes, textures, and objects.

Dataset

• Octopi: Object Property Reasoning with Large Tactile-Language Models
  [RSS 2024] [paper] [code]
  Object Property Reasoning == Physical Property Reasoning. Generally, static physical properties of objects are categorized into geometric (e.g., size), material (e.g., hardness), and affective (e.g., comfort).
  Dataset: contribute the PHYSICLEAR dataset, which comprises GelSight images on a variety of real world objects, along with object labels and part annotations. Three physical property annotations: hardness, roughness, and bumpiness. PHYSICLEAR also includes an training and evaluation suite comprising five reasoning tasks, which can serve as a benchmark for the research community.
  Traing & Evaluation Suite: comprises five physical reasoning tasks.
• Tactile-Augmented Radiance Fields
  [CVPR 2024] [paper]
  Goal: Learning a scene representation

Title	Paper
Learning Visuotactile Skills with Two Multifingered Hands	https://arxiv.org/abs/2404.16823
PseudoTouch: Efficiently Imaging the Surface Feel of Objects for Robotic Manipulation	https://arxiv.org/abs/2403.15107

Simulation

Title	Paper
Soft Contact Simulation and Manipulation Learning of Deformable Objects with Vision-based Tactile Sensor	https://arxiv.org/abs/2405.07237v1

Application

Multimodal Unification

A Case Study on Visual-Audio-Tactile Cross-Modal Retrieval
[IROS2024] [paper] 🌟

Title	Paper
OmniBind: Teach to Build Unequal-Scale Modality Interaction for Omni-Bind of All	https://arxiv.org/abs/2405.16108
Task-Oriented Mulsemedia Communication using Unified Perceiver and Conformal Prediction in 6G Metaverse Systems	https://arxiv.org/abs/2405.08949
Hearing Touch: Audio-Visual Pretraining for Contact-Rich Manipulation	https://arxiv.org/abs/2405.08576
Multi-modal perception for soft robotic interactions using generative models	https://arxiv.org/abs/2404.04220
MultiPLY: A Multisensory Object-Centric Embodied Large Language Model in 3D World	https://arxiv.org/abs/2401.08577

Tactile Task Unification

• Binding Touch to Everything: Learning Unified Multimodal Tactile Representations
  [CVPR 2024] [paper]
  Image Synthesis Tasks with Touch: touch-to-image generation, tactile-driven image stylization. Model: UniTouch representation + text-to-image diffusion model. Metrics: FID, CVTP.
• Generating Visual Scenes from Touch
  [ICCV 2023] [paper]

Cross-Sensor

Title	Paper
Transferable Tactile Transformers for Representation Learning Across Diverse Sensors and Tasks	https://arxiv.org/abs/2406.13640

Fabric Tactile Sensation

• Telextiles: End-to-end Remote Transmission of Fabric Tactile Sensation
  [UIST 2023] [paper]
  Goal: remotely transmit tactile sensations of textiles.
  Method: we propose Telextiles, an interface that can remotely transmit tactile sensations of textiles by creating a latent space that reflects the proximity of textiles through contrastive self-supervised learning.
  Essence: Fabric Recognization.

Title	Paper
Pinching Tactile Display: A Cloth that Changes Tactile Sensation by Electrostatic Adsorption	https://arxiv.org/abs/2405.03358
What Matters for Active Texture Recognition With Vision-Based Tactile Sensors	https://arxiv.org/abs/2403.13701
Exploring Human-AI Perception Alignment in Sensory Experiences: Do LLMs Understand Textile Hand?	https://arxiv.org/abs/2406.06587

Object Recognition

Title	Paper
SonicSense: Object Perception from In-Hand Acoustic Vibration	https://arxiv.org/abs/2406.17932
Imagine2touch: Predictive Tactile Sensing for Robotic Manipulation using Efficient Low-Dimensional Signals	https://arxiv.org/abs/2405.01192
Visuo-Tactile based Predictive Cross Modal Perception for Object Exploration in Robotics	https://arxiv.org/abs/2405.12634

Tactile Texture Recognition

• TEXasGAN: Tactile Texture Exploration and Synthesis System Using Generative Adversarial Network
  [paper]
• An Active Learning Framework with a Class Balancing Strategy for Time Series Classification
  [paper]

Action Classification

Title	Paper
Jointly Modeling Spatio-Temporal Features of Tactile Signals for Action Classification	https://arxiv.org/abs/2404.15279

Stable Object Placing

Title	Paper
Stable Object Placing using Curl and Diff Features of Vision-based Tactile Sensors	https://arxiv.org/abs/2403.19129

Object Compliance

Title	Paper
Learning Object Compliance via Young's Modulus from Single Grasps with Camera-Based Tactile Sensors	https://arxiv.org/abs/2406.15304

Geometry Reconstructions

Title	Paper
Snap-it, Tap-it, Splat-it: Tactile-Informed 3D Gaussian Splatting for Reconstructing Challenging Surfaces	https://arxiv.org/abs/2403.20275

Tactile-based Gesture Recognition

Title	Paper
Advancements in Tactile Hand Gesture Recognition for Enhanced Human-Machine Interaction	https://arxiv.org/abs/2405.17038
Advancements in Tactile Hand Gesture Recognition for Enhanced Human-Machine Interaction	https://arxiv.org/abs/2405.17038
Application of Artificial Intelligence in Hand Gesture Recognition with Virtual Reality: Survey and Analysis of Hand Gesture Hardware Selection	https://arxiv.org/abs/2405.16264

Object Manipulation

• masked Visual-Tactile Pre-training for robot manipulation
  [ICRA 2024] [blog post]
  Task: bottle-cap turning
  Dataset: a bottle-cap turning dataset by visual-tactile acquistion system
  Method: MAE + MDP
  
• Low Fidelity Visuo-Tactile Pretraining Improves Vision-Only Manipulation Performance
  [paper] [code]
  Task: USB cable plugging
  
• RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packing
  [RSS 2024] [paper]
  Task: dense packing

Title	Paper
Low-Cost Teleoperation with Haptic Feedback through Vision-based Tactile Sensors for Rigid and Soft Object Manipulation	https://arxiv.org/abs/2403.16764
Visuo-Tactile Pretraining for Cable Plugging	https://arxiv.org/abs/2403.11898
Visuo-Tactile Keypoint Correspondences for Object Manipulation	https://arxiv.org/abs/2405.14515

Virtual Reality

Title	Paper
Comparing the Effects of Visual, Haptic, and Visuohaptic Encoding on Memory Retention of Digital Objects in Virtual Reality	https://arxiv.org/abs/2406.14139
Haptic Repurposing with GenAI	https://arxiv.org/abs/2406.07228
Towards Robotic Haptic Proxies in Virtual Reality	https://arxiv.org/abs/2406.04491

In-Hand Object Rotation

Title	Paper
AnyRotate: Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touch	https://arxiv.org/abs/2405.07391

Robotic Furniture Assembly

Title	Paper
Generalize by Touching: Tactile Ensemble Skill Transfer for Robotic Furniture Assembly	https://arxiv.org/abs/2404.17684

Tactile Torque Estimation

Title	Paper
An Electromagnetism-Inspired Method for Estimating In-Grasp Torque from Visuotactile Sensors	https://arxiv.org/abs/2404.15626

Haptic Sensory Prostheses

Title	Paper
Spatial Summation of Localized Pressure for Haptic Sensory Prostheses	https://arxiv.org/abs/2404.02565