Challenge 1 Modeler Starter Kit

Technical Introduction to the 2023 Web3 Credit Scoring Challenge on Spectral

Gitbook | Website | Discord | Slack | Twitter | Announcement

Introduction to the Challenge:

While traditional finance relies on credit scores to gauge the risk of default, decentralized finance (DeFi) has been largely dependent on over-collateralization (i.e. the equivalent of a secured credit card). An effective on-chain credit score would improve capital efficiency and create a more inclusive, efficient DeFi ecosystem that could one day surpass traditional financial institutions.

Your objective is to predict the liquidation (binary classification) of an active borrower on Aave v2 Ethereum and Compound v2 Ethereum using PyTorch. Liquidation here includes both:

• Actual liquidation, when a borrower’s health factor drops below the liquidation threshold, triggering a liquidation event; AND
• Technical liquidation, when a borrower’s health factor drops below 1.2.

For numerical stability, the reciprocal of the health factor (referred to as the risk factor) is used in the features.

Refer to the announcement page for additional details on the significance of this problem and the challenge design.

Our Gitbook contains the most comprehensive information on the tooling required to participate in this challenge, along with an overview of the technical architecture of the Spectral platform.

Goals:

This repository aims to assist participants in understanding the following components of the challenge:

1. Using the Spectral CLI
2. The training dataset
3. The submission format
4. Leveraging zero knowledge machine learning (implicit in this Starter Kit)

You may choose to submit our baseline model, or train your own model.

To submit using our baseline model.

Run the following commands in terminal after cloning the repository.

We're providing you with this code snippet to help you gain a quick understanding of the submission workflow on Spectral.

python3 -m venv env
source env/bin/activate
pip install -r requirements.txt
spectral-cli configure

You will be prompted to enter your API keys and Privy address. You will also need to create your multisig in the Spectral App. See our gitbook for more info

python3 baseline_submission.py

To train your own model, follow the steps below.

Prepare Your Environment

Python 3.11.6 is required to use the Spectral CLI.

It is strongly recommended to interact with the Spectral CLI in an isolated virtual environment.

For managing multiple Python versions, consider using asdf.

First, clone the repository

git clone https://github.com/Spectral-Finance/challenge-1-modeler-starter-kit.git
cd challenge-1-modeler-starter-kit

If you followed Gitbook installation steps 1-4 previously skip to "Install the required packages"

It is not necessary to follow the gitbook installation process if you have not done so already.

Create a new virtual env

python -m venv env

Activate the environment

On macOS and Linux

source env/bin/activate

On Windows (running Git Bash for Windows)

source env/Scripts/activate

Install the required packages.

These packages are required to analyze the dataset in this Starter Kit and to train an example model.

pip install -r requirements.txt

macOS and Linux users ensure you have llvm and libomp installed for PyTorch.

If not already installed, you may install them with homebrew/linuxbrew.

brew install llvm
brew install libomp

If using an application like PyCharm, be sure to set the Python interpreter to the virtual environment you just created.

Download Data, Start Modeling, and Prepare Submissions:

After configuration, you are ready to begin exploring the dataset and training your model!

For a seamless experience, open the notebook in an IDE like PyCharm or run the Jupyter notebook in a browser:

jupyter lab

Refer to this notebook for a walkthrough of the entire process. The notebook will generate example files required for submission, serving as a template for your own submission.

Getting Help:

If you have any questions, consult the Gitbook for answers. For additional assistance, reach out to us on Discord.

Resources:

For context on how Spectral approached Credit Scoring Models in Web3, explore these blog posts:

• Introduction to the MACRO Score
• A Deeper Look at the MACRO Score (Part 1)
• A Deeper Look at the MACRO Score (Part 2)

Acknowledgements

This work is indebted to the contributions from the following teams:

• Alchemy
  • Powerful set of APIs, SDKs, and tools to build and scale your web3 app with ease.
• Cryo
  • Efficient EVM data extraction tools.
• DuckDB
  • In-process SQL OLAP database management system.
• Erigon
  • Implementation of Ethereum (execution layer with embeddable consensus layer).
• ezkl
  • An engine for doing inference for deep learning models and other computational graphs in a zk-snark.
• The Graph
  • A decentralized protocol for indexing and querying blockchain data.
• Messari Subgraphs
  • Core Subgraph Developer for The Graph.
• Reth
  • Modular, contributor-friendly, and blazing-fast implementation of the Ethereum protocol.
• Transpose
  • Indexed real-time blockchain data with powerful integration options.