Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement expected analysis #236

Merged
merged 90 commits into from
Dec 20, 2024
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
15eaff3
init expected power analysis
paulf81 Nov 6, 2024
59960b3
Rename energy ratio input to analysis input
paulf81 Nov 6, 2024
bd0d792
Update total uplift function name
paulf81 Nov 6, 2024
ec083b6
rename total_uplift module
paulf81 Nov 6, 2024
7c9a502
refactoring
paulf81 Nov 6, 2024
9655a1a
Add output module
paulf81 Nov 6, 2024
4ac90ac
Add skeleton structure
paulf81 Nov 6, 2024
8a6a295
Add grouping function and test
paulf81 Nov 7, 2024
b91712d
Add initial work
paulf81 Nov 7, 2024
f9a83e5
Updates
paulf81 Nov 12, 2024
d064255
Add covariance tests
paulf81 Nov 12, 2024
2987a55
Next steps
paulf81 Nov 13, 2024
e7d724b
v1 completion
paulf81 Nov 14, 2024
a9bcf4e
fix mistakes in equations
paulf81 Nov 14, 2024
bd05656
start output object
paulf81 Nov 14, 2024
6a62889
Complete and test public function
paulf81 Nov 14, 2024
94a0be7
Improve docstrings and type hints
paulf81 Nov 14, 2024
e8688d5
formatting
paulf81 Nov 14, 2024
4af9093
Update ruff precommit
paulf81 Nov 14, 2024
8305f87
Run pre-commit over files
paulf81 Nov 14, 2024
16069ca
Split to utilities
paulf81 Nov 14, 2024
c1e9dbd
pre_commit
paulf81 Nov 14, 2024
e89d387
fix too long line
paulf81 Nov 14, 2024
cce9d78
missing docstring
paulf81 Nov 14, 2024
e481d81
fix long lines
paulf81 Nov 14, 2024
0294ec2
Merge branch 'develop' into feature/expected_power
paulf81 Nov 15, 2024
7545706
fix long line
paulf81 Nov 15, 2024
1017ed9
Rough in full cov null
paulf81 Nov 19, 2024
1522fbe
Start adding fill
paulf81 Nov 19, 2024
aee52f7
Add options for filling or striking cov
paulf81 Nov 20, 2024
d075534
remove null std bins gone
paulf81 Nov 22, 2024
315ad86
add check on both variance_only and fill_cov_with_var
paulf81 Nov 22, 2024
2a88411
remove refs in output
paulf81 Nov 22, 2024
7e7c5af
Remove null option
paulf81 Nov 22, 2024
6b28fd3
Add in _synchronize_mean_power_cov_nulls
paulf81 Nov 28, 2024
8aaaf54
clearer name for num points function
paulf81 Dec 2, 2024
bb64b7d
update var cov fill
paulf81 Dec 2, 2024
097e44c
Better name for set cov to zero
paulf81 Dec 2, 2024
0a8b1f6
change 1 to 2
paulf81 Dec 2, 2024
dfe8daa
Update analysis
paulf81 Dec 2, 2024
0534199
fix zero test
paulf81 Dec 2, 2024
f6de7ab
clean up cov nulls back to mean
paulf81 Dec 2, 2024
0bc0028
Update analysis
paulf81 Dec 2, 2024
49aae88
remove deprecated count calls
paulf81 Dec 2, 2024
2981777
match uplifts
paulf81 Dec 2, 2024
d4ec6db
Merge branch 'develop' into feature/expected_power
paulf81 Dec 2, 2024
4547f6f
bugfix
paulf81 Dec 2, 2024
85f0e04
Change cov filling
paulf81 Dec 3, 2024
4bc905a
add to public
paulf81 Dec 3, 2024
b5a8737
Re-allow using cov directly
paulf81 Dec 3, 2024
cb5f2ad
Update tests
paulf81 Dec 3, 2024
5d51d38
Update output object
paulf81 Dec 3, 2024
63e37ec
bug fixes
paulf81 Dec 3, 2024
616b84c
Change to single parameter
paulf81 Dec 4, 2024
45283d2
drop un-used var calculation
paulf81 Dec 11, 2024
568dee9
Code 0s to None earlier
paulf81 Dec 11, 2024
9ac4d1a
remove null_sync_cov
paulf81 Dec 11, 2024
b197007
Update tests
paulf81 Dec 11, 2024
68062c5
allow count to be zero
paulf81 Dec 11, 2024
81ffd70
Remove remaining printouts
paulf81 Dec 11, 2024
8d489b0
Add uplift by
paulf81 Dec 11, 2024
db62c73
First pass analysis
paulf81 Dec 11, 2024
b973f7e
switch to percent change
paulf81 Dec 18, 2024
c1b0eb2
update example
paulf81 Dec 18, 2024
cfbecff
Update example
paulf81 Dec 18, 2024
6c46fd4
re-add pc results
paulf81 Dec 18, 2024
07e15ca
re-add pc calculation
paulf81 Dec 18, 2024
e9541f9
make sure no empty dataframes
paulf81 Dec 18, 2024
f552b69
Add some output printouts
paulf81 Dec 18, 2024
1177759
Add some checks for emptiness
paulf81 Dec 18, 2024
e990696
update example
paulf81 Dec 18, 2024
589c843
allow for schema override
paulf81 Dec 19, 2024
cc77222
re-use schema
paulf81 Dec 19, 2024
6f0ae90
Add distribution plots
paulf81 Dec 19, 2024
db93a30
Save the counts
paulf81 Dec 19, 2024
b30f60a
Report the counts
paulf81 Dec 19, 2024
837514c
Update example
paulf81 Dec 19, 2024
3889a44
Add sweep function
paulf81 Dec 19, 2024
8b0d4d9
Add shifting plot
paulf81 Dec 19, 2024
ff62c7c
Update example
paulf81 Dec 19, 2024
39a9c7b
Show sign
paulf81 Dec 19, 2024
582492d
Finish example 10
paulf81 Dec 20, 2024
a1fba8b
Update docs
paulf81 Dec 20, 2024
862399c
Update example 9
paulf81 Dec 20, 2024
b9bb378
Update example 6
paulf81 Dec 20, 2024
a95ce71
Merge branch 'develop' into feature/expected_power
paulf81 Dec 20, 2024
42a79f3
updating expected power docstrings
ejsimley Dec 20, 2024
741d729
cleaning up commented out code
ejsimley Dec 20, 2024
295d711
adding overview of TNO method to analysis module file headers
ejsimley Dec 20, 2024
449be83
updating energy change documentation and text of expected power uplif…
ejsimley Dec 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
"\n",
"from flasc import FlascDataFrame\n",
"from flasc.analysis import energy_ratio as erp\n",
"from flasc.analysis.energy_ratio_input import EnergyRatioInput"
"from flasc.analysis.analysis_input import AnalysisInput"
]
},
{
Expand Down Expand Up @@ -220,7 +220,7 @@
"source": [
"# Energy Ratio Input\n",
"\n",
"In the new syntax, the first step in computing an energy is building an EnergyRatioInput object. This is analagous to the construction of an energy ratio suite object in prior versions. \n",
"In the new syntax, the first step in computing an energy is building an AnalysisInput object. This is analagous to the construction of an energy ratio suite object in prior versions. \n",
"\n",
"The inputs to provide are the list of dataframes, and the names of those dataframes. An optional 3rd input is how many blocks to divide the data into, this will be used in block bootstrapping and is done here to save time. Setting the number of blocks equal to the number of rows in the dataframe with the smallest number of rows approximates non-block boostreapping/"
]
Expand All @@ -231,7 +231,7 @@
"metadata": {},
"outputs": [],
"source": [
"er_in = EnergyRatioInput(\n",
"a_in = AnalysisInput(\n",
" [df_baseline_noisy_pd, df_wakesteering_noisy_pd], [\"baseline\", \"wakesteering\"], num_blocks=10\n",
")"
]
Expand All @@ -242,7 +242,7 @@
"source": [
"# Computing the energy ratio\n",
"\n",
"Computing the energy ratio is now done by passing the above er_in into the compute_energy_ratio function. The energy ratio expects to be instructed which sets of turbines to average to produce the wind speed, wind direction, reference power and test power. However, there is also the option to use pre-defined values, in which case the columns 'wd', 'ws' and/or 'pow_ref' need to be already defined in the dataframes.\n",
"Computing the energy ratio is now done by passing the above a_in into the compute_energy_ratio function. The energy ratio expects to be instructed which sets of turbines to average to produce the wind speed, wind direction, reference power and test power. However, there is also the option to use pre-defined values, in which case the columns 'wd', 'ws' and/or 'pow_ref' need to be already defined in the dataframes.\n",
"\n",
"In this example since wd,ws,and pow_ref are defined, we'll use that option here"
]
Expand Down Expand Up @@ -270,7 +270,7 @@
"outputs": [],
"source": [
"er_out = erp.compute_energy_ratio(\n",
" er_in,\n",
" a_in,\n",
" test_turbines=[2],\n",
" use_predefined_ref=True,\n",
" use_predefined_wd=True,\n",
Expand Down Expand Up @@ -374,7 +374,7 @@
"outputs": [],
"source": [
"er_out = erp.compute_energy_ratio(\n",
" er_in,\n",
" a_in\n",
" test_turbines=[2],\n",
" use_predefined_ref=True,\n",
" use_predefined_wd=True,\n",
Expand Down Expand Up @@ -434,7 +434,7 @@
"outputs": [],
"source": [
"er_out = erp.compute_energy_ratio(\n",
" er_in,\n",
" a_in\n",
" test_turbines=[2],\n",
" use_predefined_ref=True,\n",
" use_predefined_wd=True,\n",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
"\n",
"from flasc import FlascDataFrame\n",
"from flasc.analysis import energy_ratio as erp\n",
"from flasc.analysis.energy_ratio_input import EnergyRatioInput\n",
"from flasc.analysis.analysis_input import AnalysisInput\n",
"from flasc.data_processing import dataframe_manipulations as dfm"
]
},
Expand Down Expand Up @@ -206,7 +206,7 @@
"outputs": [],
"source": [
"# Build the energy ratio input\n",
"er_in = EnergyRatioInput([df], [\"baseline\"], num_blocks=10)"
"a_in = AnalysisInput([df], [\"baseline\"], num_blocks=10)"
]
},
{
Expand Down Expand Up @@ -242,7 +242,7 @@
"# Calculate and plot the energy ratio of turbine 2 with respect to\n",
"# turbine 0, using turbine 0's measurements of wind speed and wind direction\n",
"er_out = erp.compute_energy_ratio(\n",
" er_in, test_turbines=[2], ref_turbines=[0], ws_turbines=[0], wd_turbines=[0], N=50\n",
" a_in, test_turbines=[2], ref_turbines=[0], ws_turbines=[0], wd_turbines=[0], N=50\n",
")\n",
"er_out.plot_energy_ratios()"
]
Expand Down Expand Up @@ -280,7 +280,7 @@
"# Reverse the above calculation showing the energy ratio of T0 / T2,\n",
"# letting T1 supply wind speed and direction\n",
"er_out = erp.compute_energy_ratio(\n",
" er_in, test_turbines=[0], ref_turbines=[2], ws_turbines=[1], wd_turbines=[1], N=50\n",
" a_inest_turbines=[0], ref_turbines=[2], ws_turbines=[1], wd_turbines=[1], N=50\n",
")\n",
"er_out.plot_energy_ratios()"
]
Expand Down Expand Up @@ -314,11 +314,11 @@
"source": [
"# Overplot the energy ratios of turbine 2 and 3, with respect to the averages of turbines 0 and 1\n",
"er_out_2 = erp.compute_energy_ratio(\n",
" er_in, test_turbines=[2], ref_turbines=[0, 1], ws_turbines=[0, 1], wd_turbines=[0, 1], N=50\n",
" a_in, test_turbines=[2], ref_turbines=[0, 1], ws_turbines=[0, 1], wd_turbines=[0, 1], N=50\n",
")\n",
"\n",
"er_out_3 = erp.compute_energy_ratio(\n",
" er_in, test_turbines=[3], ref_turbines=[0, 1], ws_turbines=[0, 1], wd_turbines=[0, 1], N=50\n",
" a_in, test_turbines=[3], ref_turbines=[0, 1], ws_turbines=[0, 1], wd_turbines=[0, 1], N=50\n",
")\n",
"\n",
"fig, axarr = plt.subplots(3, 1, sharex=True, figsize=(8, 11))\n",
Expand Down Expand Up @@ -536,10 +536,10 @@
"source": [
"# Now use the predefined values in the calculation of the average of turbines 2 and 3\n",
"\n",
"er_in = EnergyRatioInput([df], [\"baseline\"], num_blocks=10)\n",
"a_in = AnalysisInput([df], [\"baseline\"], num_blocks=10)\n",
"\n",
"er_out = erp.compute_energy_ratio(\n",
" er_in,\n",
" a_in\n",
" test_turbines=[2, 3],\n",
" use_predefined_ref=True,\n",
" use_predefined_wd=True,\n",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
import pandas as pd

from flasc.analysis import energy_ratio as er
from flasc.analysis.energy_ratio_input import EnergyRatioInput
from flasc.analysis.analysis_input import AnalysisInput
from flasc.data_processing import dataframe_manipulations as dfm
from flasc.utilities import floris_tools as fsatools
from flasc.utilities.utilities_examples import load_floris_artificial as load_floris
Expand Down Expand Up @@ -67,11 +67,11 @@ def load_data():
df = dfm.set_pow_ref_by_turbines(df, turbine_numbers=[0, 6])

# # Initialize energy ratio object for the dataframe
er_in = EnergyRatioInput([df], ["baseline"])
a_in = AnalysisInput([df], ["baseline"])

# Get energy ratio without uncertainty quantification
er_out = er.compute_energy_ratio(
er_in,
a_in,
test_turbines=[1],
use_predefined_ref=True,
use_predefined_wd=True,
Expand All @@ -91,7 +91,7 @@ def load_data():
# Get energy ratio with uncertainty quantification
# using N=20 bootstrap samples and 5-95 percent conf. bounds.
er_out = er.compute_energy_ratio(
er_in,
a_in,
test_turbines=[1],
use_predefined_ref=True,
use_predefined_wd=True,
Expand All @@ -107,9 +107,9 @@ def load_data():

# Get energy ratio with uncertainty quantification
# using N=20 bootstrap samples and without block bootstrapping.
er_in_noblocks = EnergyRatioInput([df], ["baseline"], num_blocks=len(df))
a_in_noblocks = AnalysisInput([df], ["baseline"], num_blocks=len(df))
er_out = er.compute_energy_ratio(
er_in_noblocks,
a_in_noblocks,
test_turbines=[1],
use_predefined_ref=True,
use_predefined_wd=True,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
from floris.utilities import wrap_360

from flasc.analysis import energy_ratio as er
from flasc.analysis.energy_ratio_input import EnergyRatioInput
from flasc.analysis.analysis_input import AnalysisInput
from flasc.data_processing import dataframe_manipulations as dfm
from flasc.utilities import floris_tools as fsatools
from flasc.utilities.utilities_examples import load_floris_artificial as load_floris
Expand Down Expand Up @@ -69,13 +69,13 @@ def load_data():
# Initialize the energy ratio input object and add dataframes
# separately. We will add the original data and the manipulated
# dataset.
er_in = EnergyRatioInput([df, df2], ["Original data", "Data with wd bias of 7.5 degrees"])
a_in = AnalysisInput([df, df2], ["Original data", "Data with wd bias of 7.5 degrees"])

# Calculate the energy ratios for test_turbines = [1] for a subset of
# wind directions with uncertainty quantification using 50 bootstrap
# samples
er_out = er.compute_energy_ratio(
er_in,
a_in,
test_turbines=[1],
use_predefined_ref=True,
use_predefined_wd=True,
Expand All @@ -91,7 +91,7 @@ def load_data():

# Look at another test turbine with the same masked datasets
er_out = er.compute_energy_ratio(
er_in,
a_in,
test_turbines=[3],
use_predefined_ref=True,
use_predefined_wd=True,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from floris.utilities import wrap_360

from flasc.analysis import energy_ratio as er
from flasc.analysis.energy_ratio_input import EnergyRatioInput
from flasc.analysis.analysis_input import AnalysisInput
from flasc.data_processing import dataframe_manipulations as dfm
from flasc.utilities.utilities_examples import load_floris_artificial as load_floris

Expand Down Expand Up @@ -69,7 +69,7 @@ def _calculate_energy_ratios(df, test_turbines, aligned_wd, N=1):
df = dfm.filter_df_by_ws(df, [6, 10])

# Finally, construct the energy ratio input with the dataframe
er_in = EnergyRatioInput([df], ["baseline"])
a_in = AnalysisInput([df], ["baseline"])

# Now, we calculate the energy ratio for each turbine for the one wind
# direction and wind speed bin. We save those values to
Expand All @@ -78,7 +78,7 @@ def _calculate_energy_ratios(df, test_turbines, aligned_wd, N=1):
for ti in test_turbines:
# Get energy ratios
er_out = er.compute_energy_ratio(
er_in,
a_in,
test_turbines=[ti],
use_predefined_ref=True,
use_predefined_wd=True,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

from flasc import FlascDataFrame
from flasc.analysis import energy_ratio as er
from flasc.analysis.energy_ratio_input import EnergyRatioInput
from flasc.analysis.analysis_input import AnalysisInput
from flasc.utilities.utilities_examples import load_floris_artificial as load_floris
from flasc.visualization import plot_binned_mean_and_ci

Expand Down Expand Up @@ -136,7 +136,7 @@
color_palette = sns.color_palette("Paired", 4)[::-1]

# Initialize the energy ratio input object
er_in = EnergyRatioInput(
a_in = AnalysisInput(
[df_baseline, df_wakesteering, df_baseline_noisy, df_wakesteering_noisy],
["Baseline", "WakeSteering", "Baseline (Noisy)", "WakeSteering (Noisy)"],
)
Expand All @@ -145,7 +145,7 @@
# With respect to reference turbine [0]
# datasets with uncertainty quantification using 50 bootstrap samples
er_out = er.compute_energy_ratio(
er_in,
a_in,
test_turbines=[2],
use_predefined_ref=True,
use_predefined_wd=True,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
import numpy as np

from flasc import FlascDataFrame
from flasc.analysis import total_uplift as tup
from flasc.analysis.energy_ratio_input import EnergyRatioInput
from flasc.analysis import total_uplift_power_ratio as tup
from flasc.analysis.analysis_input import AnalysisInput
from flasc.utilities.utilities_examples import load_floris_artificial as load_floris
from flasc.visualization import plot_binned_mean_and_ci

Expand Down Expand Up @@ -155,12 +155,12 @@
)

# Calculate the uplift on the non-noisy data
er_in = EnergyRatioInput(
a_in = AnalysisInput(
[df_baseline, df_wakesteering], ["baseline", "wake_steering"], num_blocks=1
)

total_uplift_result = tup.compute_total_uplift(
er_in,
total_uplift_result = tup.total_uplift_power_ratio(
a_in,
ref_turbines=[0],
test_turbines=[2],
use_predefined_wd=True,
Expand All @@ -173,12 +173,12 @@
uplift_non_noisy = total_uplift_result["uplift"]["energy_uplift_ctr_pc"]

# Calculate the uplift on the noisy data
er_in = EnergyRatioInput(
a_in = AnalysisInput(
[df_baseline_noisy, df_wakesteering_noisy], ["baseline", "wake_steering"], num_blocks=1
)

total_uplift_result_noisy = tup.compute_total_uplift(
er_in,
total_uplift_result_noisy = tup.total_uplift_power_ratio(
a_in,
ref_turbines=[0],
test_turbines=[2],
use_predefined_wd=True,
Expand All @@ -191,21 +191,21 @@
uplift_noisy = total_uplift_result_noisy["uplift"]["energy_uplift_ctr_pc"]
print("=======Total Uplift======")
print(
f"The uplift calculated using the compute_total_uplift module "
f"The uplift calculated using the total_uplift_power_ratio module "
f" is {uplift_non_noisy:.3}% in the"
f" non-noisy data and {uplift_noisy:.3}% in the noisy data"
)

# Recompute using bootstrapping to understand uncertainty bounds
# Calculate the uplift on the non-noisy data
er_in = EnergyRatioInput(
a_in = AnalysisInput(
[df_baseline, df_wakesteering],
["baseline", "wake_steering"],
num_blocks=df_baseline.shape[0], # Use N blocks to do non-block boostrapping
)

total_uplift_result = tup.compute_total_uplift(
er_in,
total_uplift_result = tup.total_uplift_power_ratio(
a_in,
ref_turbines=[0],
test_turbines=[2],
use_predefined_wd=True,
Expand All @@ -220,14 +220,14 @@
uplift_non_noisy_ub = total_uplift_result["uplift"]["energy_uplift_ub_pc"]

# Calculate the uplift on the noisy data
er_in = EnergyRatioInput(
a_in = AnalysisInput(
[df_baseline_noisy, df_wakesteering_noisy],
["baseline", "wake_steering"],
num_blocks=df_baseline.shape[0], # Use N blocks to do non-block boostrapping
)

total_uplift_result_noisy = tup.compute_total_uplift(
er_in,
total_uplift_result_noisy = tup.total_uplift_power_ratio(
a_in,
ref_turbines=[0],
test_turbines=[2],
use_predefined_wd=True,
Expand Down
8 changes: 4 additions & 4 deletions examples_smarteole/05_baseline_energy_ratio_analysis.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
"import pandas as pd\n",
"\n",
"from flasc.analysis import energy_ratio as er\n",
"from flasc.analysis.energy_ratio_input import EnergyRatioInput\n",
"from flasc.analysis.analysis_input import AnalysisInput\n",
"from flasc.data_processing import dataframe_manipulations as dfm\n",
"from flasc.utilities import floris_tools as ftools\n",
"from flasc.utilities.utilities_examples import load_floris_smarteole as load_floris"
Expand Down Expand Up @@ -684,7 +684,7 @@
"outputs": [],
"source": [
"# Calculate and plot energy ratios\n",
"er_in = EnergyRatioInput(\n",
"a_in = AnalysisInput(\n",
" df_fm_list + [df_scada], [\"FLORIS: \" + wm for wm in wake_models] + [\"SCADA data\"]\n",
")"
]
Expand Down Expand Up @@ -729,7 +729,7 @@
"print(\"This may take a couple seconds...\")\n",
"np.random.seed(0)\n",
"er_out = er.compute_energy_ratio(\n",
" er_in,\n",
" a_in,\n",
" test_turbines=[4],\n",
" use_predefined_ref=True,\n",
" use_predefined_wd=True,\n",
Expand Down Expand Up @@ -817,7 +817,7 @@
"print(\"This may take a couple seconds...\")\n",
"np.random.seed(0)\n",
"er_out = er.compute_energy_ratio(\n",
" er_in,\n",
" a_in,\n",
" test_turbines=[4],\n",
" use_predefined_ref=True,\n",
" use_predefined_wd=True,\n",
Expand Down
Loading
Loading