RSbench

This is the official codebase for "A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts" paper, NeurIPS 2024. This suite provides tools to evaluate and generate datasets focused on Reasoning Shortcuts (RSs) .

Content Overview

• rsscount: Use this module to count RSs in your datasets.

• rsseval: Evaluate the presence and impact of RSs using the rsbench datasets.

• rssgen: Generate datasets designed to study and analyze RSs effectively.

Each component is designed to help you systematically assess and understand RSs in various machine learning models.

Website

For more info, go to the dedicated website: link.