ZenML Implementation Guide

Overview

This guide outlines the step-by-step process for setting up and running the demonstrated ZenML pipeline with Neptune experiment tracking integration. The implementation follows a systematic approach to ensure reproducible machine learning workflows.

Prerequisites

• Python 3.9 or higher
• Access to Neptune.ai account
• ZenML cloud account

Installation and Setup Process

1. Environment Setup

First, create and activate a dedicated virtual environment:

# Create virtual environment
python -m venv .venv

# Activate virtual environment
# For Unix/MacOS
source .venv/bin/activate

2. Dependencies Installation

Install required packages from the requirements file:

pip install -r requirements.txt

3. ZenML Configuration

Initialize and configure ZenML with the following steps:

# Initialize ZenML in your project directory
zenml init
zenml integration install pytorch_lightning neptune

# Connect to ZenML cloud tenant (you can find this command in the overview page of your ZenML cloud tenant)
zenml login 8a462fb6-b...

# Register Neptune experiment tracker
zenml experiment-tracker register neptune_experiment_tracker \
    --flavor=neptune \
    --project="" \
    --api_token=""

# Register and configure stack
zenml stack register neptune_stack \
    -o default \
    -a default \
    -e neptune_experiment_tracker

# Set as active stack
zenml stack set neptune_stack

4. Execute Pipeline

Run the implementation:

python run.py

Troubleshooting

• Ensure all environment variables are properly set
• Verify Neptune.ai credentials are correctly configured
• Check ZenML stack status using zenml stack list