Gitcoin Grants API Architecture Proposal

[hmrtn]

                                ┌─────────────────────────┐
                                │       Updater Service   │
                                │  ┌───────────────────┐  │◄─────────────────────────────┐
                                │  │                   │  │                              │
                                │  │                   │  │         ┌────────────────┐ ┌─┴──────────────┐
                                │  │                   │  │         │                │ │                │
                                │  │                   │  │         │                │ │                │
                                │  │                   │  │◄────────┤                │ │                │
                                │  │    Event Watcher /│  │         │    Graph       │ │  Infura /      │
                                │  │       Cron        │  │         │    Protocol    │ │    Alchemy     │
                                │  │                   │  │         │                │ │   Event Sub.   │
                                │  │                   │  │         │                │ │                │
                                │  │                   │  │         └─┬──────────────┘ └────────────────┘
                                │  │                   │  │           │
                                │  └───────────────────┘  │           │
                                │                         │           │
                                │                         │           │
                                └────────────┬────────────┘           │
                                             │                        │               ┌──────────────────────────────────────┐
                               ┌─────────────┼────────────────────────┼────────────┐  │             Database Service         │
                               │             │                        │            │  │                                      │
                               │             │             ┌──────────▼───────┐    │  │           ┌──────────────┐           │
          Frontend Apps        │             │             │                  │    │  │       ┌───┘              └────┐      │
┌──────────────────────────┐   │             │             │   ┌───────────┐  │    │  │   ┌───┘                       └─┐    │
│                          │   │             ▼             │   │           │  │    │  │   │                             │    │
│                          │   │     ┌───────────────┐     │   │ API       │  │    │  │   ├──┐                       ┌──┤    │
│                          │   │     │               │     │   │ Container │  │    │  │   │  └─────┐             ┌───┘  │    │
│        Grant Explorer    │◄──┼────►│               │     │   └───────────┘  │    │  │   │        └─────────────┘      │    │
│                          │   │     │               │     │                  │    │  │   │                             │    │
│                          │   │     │               │     │   ┌───────────┐  │◄───┼──┤   │                             │    │
│                          │   │     │    nginx      ├────►│   │           │  │    │  │   │                             │    │
├──────────────────────────┤   │     │  webserver    │     │   │ API       │  │    │  │   ├──┐                       ┌──┤    │
│┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼│   │     │               │     │   │ Container │  │    │  │   │  └───────────────────────┘  │    │
│┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼│   │     │               │     │   └───────────┘  ├────┼─►│   │         PostgreSQL          │    │
│┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼┼│   │     │               │     │                  │    │  │   │                             │    │
├──────────────────────────┤   │     └───────────────┘     │   ┌───────────┐  │    │  │   │                             │    │
│                          │   │            ▲              │   │           │  │    │  │   │                             │    │
│                          │   │            │              │   │ API       │  │    │  │   │                             │    │
│                          │◄──┼────────────┘              │   │ Container │  │    │  │   ├─┐                        ┌──┤    │
│       Round Manager      │   │                           │   └───────────┘  │    │  │   │ └────────────────────────┘  │    │
│                          │   │                           │                  │    │  │   │                             │    │
│                          │   │                           │                  │    │  │   │                             │    │
│                          │   │                           │                  │    │  │   └─┐                         ┌─┘    │
└──────────────────────────┘   │                           └──────────────────┘    │  │     └────┐               ┌────┘      │
                               │                              ▲          ▲         │  │          └───────────────┘           │
                               │                API Service   │          │         │  │                                      │
                               └──────────────────────────────┼──────────┼─────────┘  │                                      │
                                                              │          │            └──────────────────────────────────────┘
                                                              │          │
                                                        ┌─────┴────────┐ └───┬──────────────┐
                                                        │   Passport   │     │   Token      │
                                                        │              │     │   Prices     │
                                                        └──────────────┘     └──────────────┘

In summary - I propose three separate services (they can all exist on the same server - but something something, reliability, redundancy, r-words ...).

• An API service: A server whose entry point is an nginx web server that handles incoming traffic, reverse proxying, and load balancing of N api containers.
• A Database service: A server that contains a PostgreSQL database.
• An Updater service: A server that exists to schedule updates / watch events and makes requests to auth'ed endpoints in the API service to update our DB.

The architectural diagram consists of several components that provide a scalable and reliable API infrastructure.

At the top layer, we have the API service, the entry point for external clients (i.e., round manager, grants explorer) to interact with our system. This is where users/clients would make specific GET requests to access the data.

An Nginx load balancer within the API routes incoming requests to one or more Docker API containers. This load balancer ensures that the containers are balanced and no single container is overwhelmed with requests.

The Docker API containers are where the application code runs. Each container may be identical in functionality, but the load balancer distributes the load across them. These containers can be easily scaled up or down and can be updated independently of each other.

The PostgreSQL database is where the application stores and retrieves data. The containerized database allows for easy scaling, backups, and failover in case of hardware failure.

Lastly, an updater service is included to update the database regularly or when a specific event occurs. The updater may be triggered by a regular CRON job that runs a script or an event subscription. When the process has been initiated, it makes a protected request to the API service to handle and update the database as necessary using the API service endpoint.

Overall, this architecture provides a highly scalable and fault-tolerant system that can handle large amounts of traffic and data.

[vacekj]

Closing as stale