Parquet Explorer

A browser-based SQL editor for the blockchain.

📚 Description

Parquet Explorer is a web application that allows users to upload, store, and query Parquet files directly in the browser. It leverages blockchain technology for secure storage and WebAssembly for local-first data processing. This architecture allows for the analysis of surprisingly large datasets, despite the performance limitations of web browsers and the resource constraints inherent to the Internet Computer Protocol. This project is meant to demonstrate the feasibility of adapting familiar data formats and analysis tools such as Parquet and SQL for use in a fully decentralized blockchain application.

✅ Features

• Upload and store Parquet files on-chain using Juno Storage
• Client-side query processing via DuckDB WASM
• Interactive SQL editor for querying Parquet data
• Metadata visualization for uploaded Parquet files
• Secure authentication powered by Internet Identity
• Zero setup required to start exploring your data

🛠️ Tech Stack

• Frontend: Next.js, React, TypeScript
• Styling: Tailwind CSS
• Data Processing: DuckDB WASM
• Storage: Juno (Blockchain-based)
• Authentication: Internet Identity
• Deployment: Juno Hosting

🌟 Live Demo

Experience Parquet Explorer in action:

https://xce32-paaaa-aaaal-arvgq-cai.icp0.io/

🏗️ Local Development

To set up the project locally:

1. Clone the repository

2. Install dependencies:

   pnpm install

3. Start a local Juno satellite:

   docker-compose up -d

4. Start the development server:

   pnpm dev

🚀 Deployment

This project is configured for automatic deployment to Juno hosting. The deployment process is handled by GitHub Actions, as defined in the workflow file.

Ensure that you have set up the JUNO_TOKEN secret in your GitHub repository settings. See Juno Docs for more information.

🚧 Project Status

This project is currently a proof of concept and is not intended for production use.

📄 License

This project is open source and available under the MIT License.

Much of the source code and styles from this project are borrowed from the Juno Next.js example which is also open source and available under an MIT License.

🔗 Links and Resources

• Juno: The hosting and development platform used for this project. Juno provides blockchain-based storage and hosting solutions, enabling decentralized application development.

• DuckDB: An in-process SQL OLAP database management system. This project uses DuckDB's WebAssembly build to enable powerful in-browser SQL querying capabilities.

• Internet Computer: The underlying blockchain infrastructure powering this application. The Internet Computer provides a decentralized cloud platform for building and hosting applications.

• Parquet: An open source, column-oriented data file format designed for efficient data storage and retrieval. This project focuses on exploring and querying Parquet files.

• WebAssembly: A binary instruction format for a stack-based virtual machine, enabling high-performance applications on web pages. This project leverages WebAssembly for efficient data processing in the browser.