Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Features/schedule for flow #666

Open
wants to merge 53 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
c34d5e7
Add schedule for flows
Dec 9, 2019
c49a010
Adapted existing test files
Dec 9, 2019
0887645
Adapted existing tests for scheduled flows
Dec 9, 2019
b760396
added constraint and variables to docstring
Dec 10, 2019
ea2e2ff
Added flexible schedule
Dec 10, 2019
78fceff
Added new `whatsnew` version file
Dec 10, 2019
8bd7dda
Fixed if statement according to Style Guide
Dec 10, 2019
650f9ec
Fixed boolean logic mistake
Dec 10, 2019
e264e5a
Clearified docstring for penalty parameters
Dec 10, 2019
a9ca25a
Clarified doc for _pos _neg parameters/variables
Dec 10, 2019
46cc201
Improved code to pep8 standards
Dec 10, 2019
c7882a9
Improved code to pep8 standards
Dec 10, 2019
d5e5f6c
Added constraint test for scheduled flows
Dec 10, 2019
2d89775
Added flow_scheduled.lp file for testing
Dec 10, 2019
151576d
Improve test for flow schedule
Dec 10, 2019
97d94b0
Merge branch 'dev' into features/schedule_for_flow
c-koehl Dec 10, 2019
31397a1
Rename slack_neg and slack_pos to schedule_slack
Jan 13, 2020
bd43f3e
Deleted unnecessary print statement
Jan 13, 2020
a7709af
Deleted unnecessary else statement
Jan 13, 2020
cbabaa7
Replaced lp file due to new variable names
Jan 13, 2020
f71ae4a
Replaced lp file due to new variable names
Jan 13, 2020
54fa08e
Merge branch
Jan 13, 2020
4a7106f
Renamed added parameters
Feb 10, 2020
c65349c
add csv
Feb 12, 2020
fab3209
renamed csv file
Feb 12, 2020
810ea70
Merge upstream dev into schedule_for_flows
uvchik Apr 7, 2020
ecf0c67
Fix merge in blocks
uvchik Apr 7, 2020
0b07a2c
Merge branch 'dev' into features/schedule_for_flow
uvchik Apr 7, 2020
36d7ad0
Fix pep8 issues
uvchik Apr 7, 2020
2a66fc2
Fix codacy issue
uvchik Apr 7, 2020
abab4ea
Fix another codacy issue
uvchik Apr 7, 2020
5f63483
Remove default value dor schedule penalty costs
Apr 10, 2020
3048635
mind the dynamic property of sequence
Apr 10, 2020
5b5d328
fixed logical bug in schedule attribute validation
Apr 10, 2020
15d32d1
fixed logical bug in schedule attribute validation
Apr 10, 2020
0cd492b
Adapt tests to removed schedulecosts default value
Apr 10, 2020
564c7e7
added test for schedule attributes validation
Apr 10, 2020
9a2b0fd
Merge branch 'features/schedule_for_flow' of https://github.com/c-koe…
Apr 10, 2020
a31aed0
Merge branch 'dev' into features/schedule_for_flow
Jun 10, 2020
c51772b
Added whatsnew file
Jun 10, 2020
2c3f8f1
Delete old whatsnew file
Jun 10, 2020
0436ab1
Added missing line at end of file
Jun 10, 2020
134ec9a
Added schedule attributes to Flow class
Jun 10, 2020
4ba1c2c
Deleted files added by mistake
Jun 10, 2020
c948e6d
Renamed penalty_costs to schedule_costs
Jun 10, 2020
346e687
Added ValueError tests for schedule attributes
Jun 10, 2020
b6cefa2
Fixed merge conflict
Jun 10, 2020
5ec2d0d
Fixed merge conflict
Jun 10, 2020
40e0a9f
Merge pull request #4 from oemof/dev
Aug 24, 2020
f37c523
Added newfeatures to whatsnew file
Aug 31, 2020
309ba2e
Merge branch 'dev' into features/schedule_for_flow
Aug 31, 2020
5a7a4de
Merge branch 'dev' into features/schedule_for_flow
Sep 2, 2020
589d230
Deleted todo from older version
Sep 2, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions doc/whatsnew/v0-3-3.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
v0.3.3 (??, ??)
+++++++++++++++++++++++++++


API changes
###########

* something

New features
############

* It is now possible to determine a schedule for a flow. Flow will be pushed
to the schedule, if possible.

New components
##############

* something

Documentation
#############

* something

Known issues
############

* something

Bug fixes
#########

* something

Testing
#######

* something

Other changes
#############

* something

Contributors
############

* Caterina Köhl
65 changes: 63 additions & 2 deletions oemof/solph/blocks.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,14 @@ class Flow(SimpleBlock):
Difference of a flow in consecutive timesteps if flow is increased
indexed by NEGATIVE_GRADIENT_FLOWS, TIMESTEPS.

slack_pos :
Difference of a flow to schedule in consecutive timesteps if flow
has deficit to schedule. Indexed by SCHEDULE_FLOWS, TIMESTEPS.

slack_neg :
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe these could be named to schedule_slack_pos, but I am not sure...It's also fine for me like this. Naming is always so controversial ;-)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like your idea. It is always better to be a little bit more specific and if we put "schedule" into the variable names, everybody knows where they belong to.

Excees of flow compared to schedule. Indexed by SCHEDULE_FLOWS,
TIMESTEPS.

**The following sets are created:** (-> see basic sets at
:class:`.Model` )

Expand All @@ -42,8 +50,11 @@ class Flow(SimpleBlock):
A set of flows with the attribute :attr:`positive_gradient` being not
None
INTEGER_FLOWS
A set of flows wher the attribute :attr:`integer` is True (forces flow
A set of flows where the attribute :attr:`integer` is True (forces flow
to only take integer values)
SCHEDULE_FLOWS
A set of flows with :attr:`schedule` not None (forces flow to follow
schedule)

**The following constraints are build:**

Expand Down Expand Up @@ -71,6 +82,12 @@ class Flow(SimpleBlock):
\forall (i, o) \in \textrm{POSITIVE\_GRADIENT\_FLOWS}, \\
\forall t \in \textrm{TIMESTEPS}.

Schedule constraint :attr:`om.Flow.positive_gradient_constr[i, o]`:
.. math:: flow(i, o, t) + slack_pos(i, o, t) - \
slack_neg(i, o, t) = schedule(i, o, t) \\
\forall (i, o) \in \textrm{SCHEDULE\_FLOWS}, \\
\forall t \in \textrm{TIMESTEPS}.

**The following parts of the objective function are created:**

If :attr:`variable_costs` are set by the user:
Expand All @@ -80,7 +97,13 @@ class Flow(SimpleBlock):
The expression can be accessed by :attr:`om.Flow.variable_costs` and
their value after optimization by :meth:`om.Flow.variable_costs()` .

If :attr:`schedule`, :attr:`penalty_pos` and :attr:`penalty_neg` are
set by the user:
.. math:: \sum_{(i,o)} \sum_t penalty_pos(i, o, t) \cdot \
slack_pos(i, o, t) + penalty_neg(i, o, t) \cdot \
slack_neg(i, o, t)
"""

def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)

Expand Down Expand Up @@ -120,6 +143,12 @@ def _create(self, group=None):
self.INTEGER_FLOWS = Set(
initialize=[(g[0], g[1]) for g in group
if g[2].integer])

self.SCHEDULE_FLOWS = Set(
initialize=[(g[0], g[1]) for g in group if (
len(g[2].schedule) != 0 or
(len(g[2].schedule) == 0 and
g[2].schedule[0] is not None))])
# ######################### Variables ################################

self.positive_gradient = Var(self.POSITIVE_GRADIENT_FLOWS,
Expand All @@ -130,6 +159,13 @@ def _create(self, group=None):

self.integer_flow = Var(self.INTEGER_FLOWS,
m.TIMESTEPS, within=NonNegativeIntegers)

self.slack_pos = Var(self.SCHEDULE_FLOWS,
m.TIMESTEPS, within=NonNegativeReals)

self.slack_neg = Var(self.SCHEDULE_FLOWS,
m.TIMESTEPS, within=NonNegativeReals)

# set upper bound of gradient variable
for i, o, f in group:
if m.flows[i, o].positive_gradient['ub'][0] is not None:
Expand Down Expand Up @@ -209,6 +245,21 @@ def _integer_flow_rule(block, i, o, t):
self.integer_flow_constr = Constraint(self.INTEGER_FLOWS, m.TIMESTEPS,
rule=_integer_flow_rule)

def _schedule_rule(model):
for inp, out in self.SCHEDULE_FLOWS:
for ts in m.TIMESTEPS:
if m.flows[inp, out].schedule[ts] is not None:
lhs = (m.flow[inp, out, ts] +
self.slack_pos[inp, out, ts] -
self.slack_neg[inp, out, ts])
rhs = m.flows[inp, out].schedule[ts]
self.schedule_constr.add((inp, out, ts),
lhs == rhs)
self.schedule_constr = Constraint(
self.SCHEDULE_FLOWS, m.TIMESTEPS, noruleinit=True)
self.schedule_build = BuildAction(
rule=_schedule_rule)

def _objective_expression(self):
r""" Objective expression for all standard flows with fixed costs
and variable costs.
Expand All @@ -217,6 +268,7 @@ def _objective_expression(self):

variable_costs = 0
gradient_costs = 0
penalty_costs = 0

for i, o in m.FLOWS:
if m.flows[i, o].variable_costs[0] is not None:
Expand All @@ -236,7 +288,16 @@ def _objective_expression(self):
m.flows[i, o].negative_gradient[
'costs'])

return variable_costs + gradient_costs
schedule = m.flows[i, o].schedule
if (len(schedule) > 1 or
(len(schedule) == 0 and
schedule[0] is not None)):
for t in m.TIMESTEPS:
penalty_costs += (self.slack_pos[i, o, t] *
m.flows[i, o].penalty_pos[t])
penalty_costs += (self.slack_neg[i, o, t] *
m.flows[i, o].penalty_neg[t])
return variable_costs + gradient_costs + penalty_costs


class InvestmentFlow(SimpleBlock):
Expand Down
28 changes: 27 additions & 1 deletion oemof/solph/network.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,23 @@ class Flow(on.Edge):
:class:`Flow <oemof.solph.blocks.Flow>`.
Note: at the moment this does not work if the investment attribute is
set .
schedule : numeric (sequence or scalar)
Schedule for the flow. Flow has to follow the schedule, otherwise a
penalty term will be activated. If array-like, values can be None
for flexible, non-fixed flow during the certain timestep. Used in
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work actually, i.e. setting someting like schedule=[1, 2, None, 3]. Has this been tested. I thought pyomo would give an error if None is provided in the equation. Please check if to be sure.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested it and it worked. But I'm confused now, because you're actually right. Usually pyomo is not amused if None is provided. I will add a "return Constraint.Skip" for completeness

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I think it is possible, because of BuildAction no return value has to be provided and the constraints will be added by add() function.
I remember pyomo did expect a return value (and throws an error if you provide None) if you build constraints without BuildAction.
Maybe with BuildAction it does not expect anything and is fine with only executing the add() function. But I am honestly just guessing, because the documentary of pyomo's BuildAction is not very detailed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I tried adding the return value Constraints.Skip, but as soon as it will be returned once, pyomo quits the constraint completely and continues with the next constraint. So in your case (schedule = [1, 2, None, 3]), the flow will only follow the schedule until the first None: [1, 2, None]. Everything following the first None will not be handled.
Interesting, Constraint.Skip behaves differently when building constraints with BuildAction compared to without.

combination with :attr:`penalty_pos` and :attr:`penalty_neg`.
penalty_pos : numeric (sequence or scalar)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be a high default value of something like 10e9 force the schedule kind of by default?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was also thinking about adding a default value. It's actually more user friendly. I think it depends on what kind of users are working with oemof. Do you think it will be necessary? I can easily add one :-)

A penalty parameter of the penalty term which describes the costs
associated with one unit of deficit of the flow compared to the
schedule. If this is set the costs will be added to the objective
expression of the optimization problem. Used in combination with the
:attr:`schedule` and :attr:`penalty_neg`
penalty_neg: numeric (sequence or scalar)
A penalty parameter of the penalty term which describes the costs
associated with one unit of the exceeded flow when flow exceeds the
schedule. If this is set the costs will be added to the objective
expression of the optimization problem. Used in combination with
the :attr:`schedule` and :attr:`penalty_pos`

Notes
-----
Expand Down Expand Up @@ -157,11 +174,13 @@ def __init__(self, **kwargs):

scalars = ['nominal_value', 'summed_max', 'summed_min',
'investment', 'nonconvex', 'integer', 'fixed']
sequences = ['actual_value', 'variable_costs', 'min', 'max']
sequences = ['actual_value', 'variable_costs', 'min', 'max',
'schedule', 'penalty_neg', 'penalty_pos']
dictionaries = ['positive_gradient', 'negative_gradient']
defaults = {'fixed': False, 'min': 0, 'max': 1, 'variable_costs': 0,
'positive_gradient': {'ub': None, 'costs': 0},
'negative_gradient': {'ub': None, 'costs': 0},
'penalty_neg': 0, 'penalty_pos': 0, 'schedule': None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking if this could not rather be implemented like the gradient cost stuff as a dictionary to reduced number of attributes. It seems better to me, but again-> there are always multiple ways. Maybe ask @gnn for this

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for taking so long to review this one... :-/

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm I do not know about this one. I personally think it can be helpful, if the user can determine during what timesteps he will punish the system harder or being more "relaxed" if flow can not follow the schedule.

No worries, we are all busy and even holidays were in between :-)

}
keys = [k for k in kwargs if k != 'label']

Expand All @@ -188,6 +207,13 @@ def __init__(self, **kwargs):
if self.investment and self.nonconvex:
raise ValueError("Investment flows cannot be combined with " +
"nonconvex flows!")
if (
len(self.schedule) != 0 and
((len(self.penalty_pos) == 0 and not self.penalty_pos[0]) or
(len(self.penalty_neg) == 0 and not self.penalty_neg[0]))):
raise ValueError("The penalty and schedule attribute need "
"to be used in combination. \n Please set "
"the schedule attribute of the flow.")


class Bus(on.Bus):
Expand Down
24 changes: 21 additions & 3 deletions tests/constraint_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -341,7 +341,7 @@ def test_storage_minimum_invest(self):
ep_costs=145, minimum=100, maximum=200))

self.compare_lp_files('storage_invest_minimum.lp')

def test_storage_unbalanced(self):
"""Testing a unbalanced storage (e.g. battery)."""
bel = solph.Bus(label='electricityBus')
Expand All @@ -354,9 +354,9 @@ def test_storage_unbalanced(self):
initial_storage_level=None,
balanced=False,
invest_relation_input_capacity=1,
invest_relation_output_capacity=1)
invest_relation_output_capacity=1)
self.compare_lp_files('storage_unbalanced.lp')

def test_storage_invest_unbalanced(self):
"""Testing a unbalanced storage (e.g. battery)."""
bel = solph.Bus(label='electricityBus')
Expand Down Expand Up @@ -680,3 +680,21 @@ def test_dsm_module_interval(self):
cost_dsm_down=2,
)
self.compare_lp_files('dsm_module_interval.lp')

def test_flow_schedule(self):
"""Contraint test of scheduled flows
"""
b_gas = solph.Bus(label='bus_gas')
b_th = solph.Bus(label='bus_th_penalty')

schedule = [None, 300, 50]
solph.Transformer(
label="boiler_penalty",
inputs={b_gas: solph.Flow()},
outputs={b_th: solph.Flow(nominal_value=200, variable_costs=0,
penalty_pos=[0, 800, 900],
penalty_neg=999,
schedule=schedule)},
conversion_factors={b_th: 1}
)
self.compare_lp_files('flow_schedule.lp')
79 changes: 79 additions & 0 deletions tests/lp_files/flow_schedule.lp
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
\* Source Pyomo model name=Model *\

min
objective:
+999 Flow_slack_neg(boiler_penalty_bus_th_penalty_0)
+999 Flow_slack_neg(boiler_penalty_bus_th_penalty_1)
+999 Flow_slack_neg(boiler_penalty_bus_th_penalty_2)
+800 Flow_slack_pos(boiler_penalty_bus_th_penalty_1)
+900 Flow_slack_pos(boiler_penalty_bus_th_penalty_2)

s.t.

c_e_Bus_balance(bus_gas_0)_:
+1 flow(bus_gas_boiler_penalty_0)
= 0

c_e_Bus_balance(bus_gas_1)_:
+1 flow(bus_gas_boiler_penalty_1)
= 0

c_e_Bus_balance(bus_gas_2)_:
+1 flow(bus_gas_boiler_penalty_2)
= 0

c_e_Bus_balance(bus_th_penalty_0)_:
+1 flow(boiler_penalty_bus_th_penalty_0)
= 0

c_e_Bus_balance(bus_th_penalty_1)_:
+1 flow(boiler_penalty_bus_th_penalty_1)
= 0

c_e_Bus_balance(bus_th_penalty_2)_:
+1 flow(boiler_penalty_bus_th_penalty_2)
= 0

c_e_Transformer_relation(boiler_penalty_bus_gas_bus_th_penalty_0)_:
-1 flow(boiler_penalty_bus_th_penalty_0)
+1 flow(bus_gas_boiler_penalty_0)
= 0

c_e_Transformer_relation(boiler_penalty_bus_gas_bus_th_penalty_1)_:
-1 flow(boiler_penalty_bus_th_penalty_1)
+1 flow(bus_gas_boiler_penalty_1)
= 0

c_e_Transformer_relation(boiler_penalty_bus_gas_bus_th_penalty_2)_:
-1 flow(boiler_penalty_bus_th_penalty_2)
+1 flow(bus_gas_boiler_penalty_2)
= 0

c_e_Flow_schedule_constr(boiler_penalty_bus_th_penalty_1)_:
-1 Flow_slack_neg(boiler_penalty_bus_th_penalty_1)
+1 Flow_slack_pos(boiler_penalty_bus_th_penalty_1)
+1 flow(boiler_penalty_bus_th_penalty_1)
= 300

c_e_Flow_schedule_constr(boiler_penalty_bus_th_penalty_2)_:
-1 Flow_slack_neg(boiler_penalty_bus_th_penalty_2)
+1 Flow_slack_pos(boiler_penalty_bus_th_penalty_2)
+1 flow(boiler_penalty_bus_th_penalty_2)
= 50

c_e_ONE_VAR_CONSTANT:
ONE_VAR_CONSTANT = 1.0

bounds
0 <= flow(boiler_penalty_bus_th_penalty_0) <= 200
0 <= flow(boiler_penalty_bus_th_penalty_1) <= 200
0 <= flow(boiler_penalty_bus_th_penalty_2) <= 200
0 <= flow(bus_gas_boiler_penalty_0) <= +inf
0 <= flow(bus_gas_boiler_penalty_1) <= +inf
0 <= flow(bus_gas_boiler_penalty_2) <= +inf
0 <= Flow_slack_pos(boiler_penalty_bus_th_penalty_1) <= +inf
0 <= Flow_slack_pos(boiler_penalty_bus_th_penalty_2) <= +inf
0 <= Flow_slack_neg(boiler_penalty_bus_th_penalty_0) <= +inf
0 <= Flow_slack_neg(boiler_penalty_bus_th_penalty_1) <= +inf
0 <= Flow_slack_neg(boiler_penalty_bus_th_penalty_2) <= +inf
end
7 changes: 6 additions & 1 deletion tests/test_processing.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,8 @@ def test_flows_with_none_exclusion(self):
'positive_gradient_costs': 0,
'variable_costs': 0,
'label': str(b_el2.outputs[demand].label),
'penalty_pos': 0,
'penalty_neg': 0,
}
).sort_index()
)
Expand Down Expand Up @@ -132,6 +134,9 @@ def test_flows_without_none_exclusion(self):
'flow': None,
'values': None,
'label': str(b_el2.outputs[demand].label),
'penalty_pos': 0,
'penalty_neg': 0,
'schedule': None,
}
assert_series_equal(
param_results[(b_el2, demand)]['scalars'].sort_index(),
Expand All @@ -141,7 +146,7 @@ def test_flows_without_none_exclusion(self):
'actual_value': self.demand_values,
}
default_sequences = [
'actual_value'
'actual_value',
]
for attr in default_sequences:
if attr not in sequences_attributes:
Expand Down