Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor process parquet #124

Open
wants to merge 14 commits into
base: croptype
Choose a base branch
from
3 changes: 2 additions & 1 deletion presto/dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,14 +143,15 @@ def row_to_arrays(
timestep_positions = cls.get_timestep_positions(row_d, augment=augment, is_ssl=is_ssl)

if cls.NUM_TIMESTEPS == 12:
initial_start_date_position = pd.to_datetime(row_d["start_date"]).month
initial_start_date_position = datetime.strptime(row_d["start_date"], "%Y-%m-%d").month
elif cls.NUM_TIMESTEPS > 12:
# get the correct index of the start_date based on NUM_TIMESTEPS`
# e.g. if NUM_TIMESTEPS is 36 (dekadal setup), we should take the correct
# 10-day interval that the start_date falls into
# TODO: 1) this needs to go into a separate function
# 2) definition of valid_position and timestep_ind
# should also be changed accordingly

year = pd.to_datetime(row_d["start_date"]).year
year_dates = pd.date_range(start=f"{year}-01-01", end=f"{year}-12-31")
bins = pd.cut(year_dates, bins=cls.NUM_TIMESTEPS, labels=False)
Expand Down
Loading
Loading