How to create river type dataset from pandas dataframe

[jacowp357]

I would like to see an example of creating a datasets.Phishing() type dataset from an arbitrary pandas dataframe? Is there such an example somewhere? This is for the purpose of using evaluate.progressive_val_score()

[raphaelsty]

Hi @jacowp357,

The classic structure of datasets with River is a list of dictionary. Here is how to convert a pandas dataframe to a list of dictionary:

import pandas as pd

# Load Iris dataset.
df = pd.read_csv('https://raw.githubusercontent.com/mwaskom/seaborn-data/master/iris.csv')

dataset = df.to_dict(orient = "records")

dataset

[
  {'sepal_length': 5.1,
    'sepal_width': 3.5,
    'petal_length': 1.4,
    'petal_width': 0.2,
    'species': 'setosa'},
   {'sepal_length': 4.9,
    'sepal_width': 3.0,
    'petal_length': 1.4,
    'petal_width': 0.2,
    'species': 'setosa'},
...
]

You can pass the list of dict to evaluate.progressive_val_score() 🙂

[raphaelsty]

Of course, sorry, I answered too quickly without thinking. We have to create a list of tuples. The set of features is defined within a dictionary and the target variable is defined outside the dictionary.

import pandas as pd

# Load Iris dataset.
df = pd.read_csv('https://raw.githubusercontent.com/mwaskom/seaborn-data/master/iris.csv')

# "target" instead of "species" for your dataset
dataset = [(x, x.pop("species")) for x in  df.to_dict(orient = "records")]

[jacowp357]

@raphaelsty thanks! That did the trick!

[MaxHalford]

We also have a stream.iter_pandas method :)

[Divya216]

Explain with an example stream.iter_pandas

[MaxHalford]

https://riverml.xyz/0.14.0/api/stream/iter-pandas/#examples