Skip to content

Commit

Permalink
Adds more to docs
Browse files Browse the repository at this point in the history
Adds more structure.
  • Loading branch information
skrawcz committed Sep 20, 2023
1 parent 9867f5e commit 9a5fbaf
Show file tree
Hide file tree
Showing 43 changed files with 182 additions and 681 deletions.
6 changes: 4 additions & 2 deletions contrib/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ Things you need to know:
1. Whether it's a user or official dataflow. If user, what the name of the user is.
2. The name of the dataflow.
```python
```python
from hamilton import driver
# from hamilton.contrib.official import NAME_OF_DATAFLOW
from hamilton.contrib.user.NAME_OF_USER import NAME_OF_DATAFLOW
Expand Down Expand Up @@ -88,7 +87,7 @@ Standards:
- It must not do anything malicious.
- It must be well documented.
- It must work.
- It must follow a standard structure as outlined below.
- It must follow our standard structure as outlined below.


### Checklist for new dataflows:
Expand All @@ -106,3 +105,6 @@ Do you have the following?
- [ ] Added tags.json under my dataflow directory to curate my dataflow.
- [ ] Added valid_configs.jsonl under my dataflow directory to specify the valid configurations.
- [ ] Added a dag.png that shows one possible configuration of my dataflow.

# Got questions?
Join our [slack](https://join.slack.com/t/hamilton-opensource/shared_invite/zt-1bjs72asx-wcUTgH7q7QX1igiQ5bbdcg) community to chat/ask Qs/etc.
12 changes: 0 additions & 12 deletions contrib/docs/blog/2019-05-28-first-blog-post.md

This file was deleted.

44 changes: 0 additions & 44 deletions contrib/docs/blog/2019-05-29-long-blog-post.md

This file was deleted.

20 changes: 0 additions & 20 deletions contrib/docs/blog/2021-08-01-mdx-blog-post.mdx

This file was deleted.

Binary file not shown.
25 changes: 0 additions & 25 deletions contrib/docs/blog/2021-08-26-welcome/index.md

This file was deleted.

17 changes: 17 additions & 0 deletions contrib/docs/blog/2023-09-21-welcome/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
slug: welcome
title: Welcome
authors: [skrawcz]
tags: [hamilton, how-to, dataflows]
---

Welcome to the Hamilton Dataflow Hub!

We're excited to kick off this project and hope that it will be a useful resource for the community.
Hamilton is like a Swiss army knife, it's very versatile, as you can use it for data, ML, LLM, and web work!

We envision this hub, as a place to share the flows you've built with it that can be useful to others!

## How to use

1. Browse the
22 changes: 5 additions & 17 deletions contrib/docs/blog/authors.yml
Original file line number Diff line number Diff line change
@@ -1,17 +1,5 @@
endi:
name: Endilie Yacop Sucipto
title: Maintainer of Docusaurus
url: https://github.com/endiliey
image_url: https://github.com/endiliey.png

yangshun:
name: Yangshun Tay
title: Front End Engineer @ Facebook
url: https://github.com/yangshun
image_url: https://github.com/yangshun.png

slorber:
name: Sébastien Lorber
title: Docusaurus maintainer
url: https://sebastienlorber.com
image_url: https://github.com/slorber.png
skrawcz:
name: Stefan Krawczyk
title: Co-creator of Hamilton and CEO @ DAGWorks Inc.
url: https://github.com/skrawcz
image_url: https://github.com/skrawcz.png
121 changes: 121 additions & 0 deletions contrib/docs/docs/Intro.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
---
title: "Introduction"
sidebar_position: 1
---

# Hamilton Dataflows

Welcome!

Here you'll find a website that curates a collection of Hamilton Dataflows that are
ready to be used in your own projects. They are user-contributed and maintained, with
the goal of making it easier for you to get started with Hamilton.

We expect this collection to grow over time, so check back often! As dataflows become mature we
will move them into the official sub-package of this site and become maintained by the
Hamilton team.

## Usage
There are two methods to get access to dataflows presented here.

Assumptions:

1. You are familiar with Hamilton and have it installed. If not, take
[15 minutes to learn Hamilton in your browser](https://www.tryhamilton.dev/) and then `pip install sf-hamilton` to get started.
Come back here when you're ready to use Hamilton.
2. The assumption is that you have the requisite python dependencies installed on your system.
You'll get import errors if you don't. Don't know what you need? We have convenience functions to help!

For more extensive documentation, please see [Hamilton User Contrib documentation](https://hamilton.dagworks.io).

### Static installation
This approach relies on you installing the package on your system. This is the recommended path for
production purposes as you can version-lock your dependencies.

To install the package, run:

```bash
pip install sf-hamilton-contrib
```

Once installed, you can import the dataflows as follows.

Things you need to know:
1. Whether it's a user or official dataflow. If user, what the name of the user is.
2. The name of the dataflow.
```python
from hamilton import driver
# from hamilton.contrib.official import NAME_OF_DATAFLOW
from hamilton.contrib.user.NAME_OF_USER import NAME_OF_DATAFLOW

dr = (
driver.Builder()
.with_config({}) # replace with configuration as appropriate
.with_modules(NAME_OF_DATAFLOW)
.build()
)
# execute the dataflow, specifying what you want back. Will return a dictionary.
result = dr.execute(
[NAME_OF_DATAFLOW.FUNCTION_NAME, ...], # this specifies what you want back
inputs={...} # pass in inputs as appropriate
)
```

### Dynamic installation
Here we dynamically download the dataflow from the internet and execute it. This is useful for quickly
iterating in a notebook and pulling in just the dataflow you need.

```python
from hamilton import dataflow, driver

# download into ~/.hamilton/dataflows and load the module -- WARNING: ensure you know what code you're importing!
text_summarization = dataflow.import_module(NAME_OF_USER, NAME_OF_DATAFLOW)
dr = (
driver.Builder()
.with_config({}) # replace with configuration as appropriate
.with_modules(NAME_OF_DATAFLOW)
.build()
)
# execute the dataflow, specifying what you want back. Will return a dictionary.
result = dr.execute(
[NAME_OF_DATAFLOW.FUNCTION_NAME, ...], # this specifies what you want back
inputs={...} # pass in inputs as appropriate
)
```

## How to contribute

If you have a dataflow that you would like to share with the community, please submit a pull request
to this repository. We will review your dataflow and if it meets our standards we will add it to the
package. To submit a pull request please use [this link](TODO) as it'll take you to the specific PR template.

### Dataflow standards
We want to ensure that the dataflows in this package are of high quality and are easy to use. To that end,
we have a set of standards that we expect all dataflows to meet. If you have any questions, please reach out.

Standards:
- The dataflow must be a valid Python module.
- It must not do anything malicious.
- It must be well documented.
- It must work.
- It must follow our standard structure as outlined below.


### Checklist for new dataflows:
Do you have the following?
- [ ] Added a directory mapping to my github user name in the contrib/hamilton/contrib/user directory.
- [ ] If my author names contains hyphens I have replaced them with underscores.
- [ ] If my author name starts with a number, I have prefixed it with an underscore.
- [ ] If your author name is a python reserved keyword. Reach out to the maintainers for help.
- [ ] Added an author.md file under my username directory and is filled out.
- [ ] Added an __init__.py file under my username directory.
- [ ] Added a new folder for my dataflow under my username directory.
- [ ] Added a README.md file under my dataflow directory that follows the standard headings and is filled out.
- [ ] Added a __init__.py file under my dataflow directory that contains the Hamilton code.
- [ ] Added a requirements.txt under my dataflow directory that contains the required packages outside of Hamilton.
- [ ] Added tags.json under my dataflow directory to curate my dataflow.
- [ ] Added valid_configs.jsonl under my dataflow directory to specify the valid configurations.
- [ ] Added a dag.png that shows one possible configuration of my dataflow.

# Got questions?
Join our [slack](https://join.slack.com/t/hamilton-opensource/shared_invite/zt-1bjs72asx-wcUTgH7q7QX1igiQ5bbdcg) community to chat/ask Qs/etc.
5 changes: 5 additions & 0 deletions contrib/docs/docs/Official/Intro.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
title: TODO
sidebar_position: 2
---
Check back here later. We're working on it!
8 changes: 8 additions & 0 deletions contrib/docs/docs/Users/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"label": "Users",
"position": 3,
"link": {
"type": "generated-index",
"description": "Find user contributed dataflows here."
}
}
File renamed without changes.
File renamed without changes.
50 changes: 0 additions & 50 deletions contrib/docs/docs/intro.md

This file was deleted.

8 changes: 0 additions & 8 deletions contrib/docs/docs/tutorial-basics/_category_.json

This file was deleted.

23 changes: 0 additions & 23 deletions contrib/docs/docs/tutorial-basics/congratulations.md

This file was deleted.

Loading

0 comments on commit 9a5fbaf

Please sign in to comment.