Description

Our codebase primarily relies on pandas for data handling and manipulation tasks. However, we have identified potential performance improvements that could be gained by incorporating support for Polars, a fast DataFrame library implemented in Rust and available in Python.

Polars is designed to outperform pandas in various scenarios and could provide significant speed-ups for our data processing tasks. This can benefit larger datasets and more complex operations, making our toolset more versatile and efficient.

The task would involve reviewing the codebase and integrating the possibility of using Polars as input instead of Pandas. We must ensure the transition is seamless and keeps existing functionalities intact.

This is a substantial task that might require time and careful testing. Any contributors willing to help with this task are welcome. Please feel free to comment below if you'd like to assist or have suggestions on approaching this task.

Use case

No response