Features/schedule for flow #666

c-koehl · 2019-12-10T14:51:29Z

Added a schedule for Flow

Functionality
Sometimes it´s important to set a schedule for a flow (e.g. CHP electrical output), e.g. for fulfilling contracts. During optimization flow will be pushed to fulfill the schedule, if possible. If it´s not possible a penalty term will be activated and added to the objective expression for each unit of the difference flow-schedule.
Changes made in oemof.solph.network

Parameter added to class Flow:
- schedule
  Flow has to follow the schedule, but LP won´t be infeasible, if flow can´t fulfill schedule: slack variables will be added (see Changes made in solph.blocks)
- penalty_pos
  Describes the costs associated with one unit of the deficit of the flow (:= slack_pos) compared to the schedule. Costs will be added to the objective expression.
- penalty_neg
  Describes the costs associated with one unit of the excessed flow (:= slack_neg). Will be added to the objective expression.
Changes made in oemof.solph.blocks

Added Set and Variables to class Flow:
- SCHEDULE_FLOWS (pyomo.Set)
  contains all flows with schedules
- slack_neg (pyomo.Var, within=NonNegativeReals)
  slack variable for representing excess of flow compared to schedule (see Added constraint to class Flow)
- slack_pos (pyomo.Var, within=NonNegativeReals)
  slack variable for representing deficit of flow compared to schedule
Added constraint to class Flow
- flow(i, o, t) - slack_neg(i, o, t) + slack_pos(i, o, t) = schedule(i, o, t)
Term added to objective expression:
- penalty_pos(i, o, t) * slack_pos(i, o, t) + penalty_neg(i, o, t) * slack_neg(i, o, t)
Changes made in oemof.tests.test_processing
New Parameters of class Flow had to be added to the test file as well.
Changes made in oemof.tests.constraint_tests
Test for scheduled flow was added.
Changes made in oemof.tests.lp_files.flow_schedule.lp
Created file.
Changes made in doc.whatsnew.v0-3-3.rst
Created file.
Possible Improvements
- Default value for penalty_pos and penalty_neg is 0. This could be improved by setting a reasonable default value, but this acquires mathematical research on how to choose appropriate penalties.
- If it's not possible for the flow to follow the schedule, "non-real" costs will be added to the objective expression. You might not want to interfere the artificial costs (added by the slack variables) with the real costs of the solved LP.

It can be important to determine the output of e.g. a CHP by a given schedule. In practice it´s used to fulfill contracts.

New parameters for class `Flow` has been added. They had to be added to the existing test files too.

Now it`s possible to fix the flow to a schedule with possible None values. If schedule is None for the certain timestep, flow will freely be calculated without any penalty constraints.

The implementation for penalty terms for scheduled flows is finished. * Maybe set a reasonable default value for penalty terms (penalty_pos and penalty_neg), but this acquires mathematical research.

pep8speaks · 2019-12-10T14:51:33Z

Hello @c-koehl! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file src/oemof/solph/blocks.py:

Line 163:46: W504 line break after binary operator
Line 267:53: W504 line break after binary operator
Line 268:70: W504 line break after binary operator
Line 308:35: W504 line break after binary operator
Line 309:37: W504 line break after binary operator
Line 312:73: W504 line break after binary operator
Line 314:73: W504 line break after binary operator

In the file src/oemof/solph/network.py:

Line 241:37: W504 line break after binary operator
Line 242:48: W504 line break after binary operator

Comment last updated at 2022-06-24 14:12:31 UTC

coveralls · 2019-12-10T14:54:04Z

Coverage increased (+0.06%) to 94.894% when pulling 4a7106f on c-koehl:features/schedule_for_flow into 83cfc0b on oemof:dev.

c-koehl · 2019-12-11T11:23:33Z

If you're interested, I could also add an example to oemof-examples.

gplssm · 2019-12-12T10:46:30Z

Once you're done with the implementation, select a reviewer who cross-checks the newly added code. Preferably, select someone who already has some context about this

c-koehl · 2019-12-12T11:15:06Z

@gplssm I'm done with the implementation, but it seems like I don't have the rights to select a reviewer or labels. I don't know what I'm doing wrong? Also I cannot tell, which reviewer is the best to choose

gplssm · 2019-12-12T11:53:13Z

@gplssm I'm done with the implementation, but it seems like I don't have the rights to select a reviewer or labels. I don't know what I'm doing wrong? Also I cannot tell, which reviewer is the best to choose

I'm not too familiar with permissions granted to individual devs. Maybe @uvchik can help out here.

Meanwhile, I can request a review for you. Did you talk to anyone before/during your implementation of "schedule for flow"? I would prefer to select this person then.

c-koehl · 2019-12-12T12:05:07Z

@gplssm I'm done with the implementation, but it seems like I don't have the rights to select a reviewer or labels. I don't know what I'm doing wrong? Also I cannot tell, which reviewer is the best to choose

I'm not too familiar with permissions granted to individual devs. Maybe @uvchik can help out here.

Meanwhile, I can request a review for you. Did you talk to anyone before/during your implementation of "schedule for flow"? I would prefer to select this person then.

@gplssm I actually didn't talk to anyone. I would suggest a reviewer with mathematical optimization knowledge, but I think my procedure is also comprehensible for non mathematicians.

gplssm · 2019-12-12T14:05:16Z

I've requested a review by @simnh because he's quite deep into mathematical modeling in oemof.solph.
@simnh please re-request a review by someone else if you can't do it.

simnh

Thanks for the new feature. Looks great, I just added some minor comments regarding naming etc. But from my side, this looks quite good.

simnh · 2020-01-10T16:09:15Z

oemof/solph/blocks.py

+    slack_pos :
+        Difference of a flow to schedule in consecutive timesteps if flow
+        has deficit to schedule. Indexed by SCHEDULE_FLOWS, TIMESTEPS.
+
+    slack_neg :


Maybe these could be named to schedule_slack_pos, but I am not sure...It's also fine for me like this. Naming is always so controversial ;-)

I like your idea. It is always better to be a little bit more specific and if we put "schedule" into the variable names, everybody knows where they belong to.

simnh · 2020-01-10T16:13:29Z

oemof/solph/network.py

+        penalty term will be activated. If array-like, values can be None
+        for flexible, non-fixed flow during the certain timestep.  Used in


Does this work actually, i.e. setting someting like schedule=[1, 2, None, 3]. Has this been tested. I thought pyomo would give an error if None is provided in the equation. Please check if to be sure.

I tested it and it worked. But I'm confused now, because you're actually right. Usually pyomo is not amused if None is provided. I will add a "return Constraint.Skip" for completeness

Ah I think it is possible, because of BuildAction no return value has to be provided and the constraints will be added by add() function.
I remember pyomo did expect a return value (and throws an error if you provide None) if you build constraints without BuildAction.
Maybe with BuildAction it does not expect anything and is fine with only executing the add() function. But I am honestly just guessing, because the documentary of pyomo's BuildAction is not very detailed.

Ok, I tried adding the return value Constraints.Skip, but as soon as it will be returned once, pyomo quits the constraint completely and continues with the next constraint. So in your case (schedule = [1, 2, None, 3]), the flow will only follow the schedule until the first None: [1, 2, None]. Everything following the first None will not be handled.
Interesting, Constraint.Skip behaves differently when building constraints with BuildAction compared to without.

simnh · 2020-01-10T16:14:14Z

oemof/solph/network.py

+        penalty term will be activated. If array-like, values can be None
+        for flexible, non-fixed flow during the certain timestep.  Used in
+        combination with :attr:`penalty_pos` and :attr:`penalty_neg`.
+    penalty_pos : numeric (sequence or scalar)


Should there be a high default value of something like 10e9 force the schedule kind of by default?

I was also thinking about adding a default value. It's actually more user friendly. I think it depends on what kind of users are working with oemof. Do you think it will be necessary? I can easily add one :-)

simnh · 2020-01-10T16:15:59Z

oemof/solph/network.py

-        sequences = ['actual_value', 'variable_costs', 'min', 'max']
+        sequences = ['actual_value', 'variable_costs', 'min', 'max',
+                     'schedule', 'penalty_neg', 'penalty_pos']
        dictionaries = ['positive_gradient', 'negative_gradient']
        defaults = {'fixed': False, 'min': 0, 'max': 1, 'variable_costs': 0,
                    'positive_gradient': {'ub': None, 'costs': 0},
                    'negative_gradient': {'ub': None, 'costs': 0},
+                    'penalty_neg': 0, 'penalty_pos': 0, 'schedule': None


I was thinking if this could not rather be implemented like the gradient cost stuff as a dictionary to reduced number of attributes. It seems better to me, but again-> there are always multiple ways. Maybe ask @gnn for this

Sorry for taking so long to review this one... :-/

Hmmm I do not know about this one. I personally think it can be helpful, if the user can determine during what timesteps he will punish the system harder or being more "relaxed" if flow can not follow the schedule.

No worries, we are all busy and even holidays were in between :-)

Update upstream dev branch

Deleted line with todo from older oemof version

c-koehl · 2020-11-24T09:00:02Z

I do not know how to un - fail these tests. I am not sure if they were caused by me... Please correct me if I am wrong

Consolinno added 10 commits December 9, 2019 17:53

Add schedule for flows

c34d5e7

It can be important to determine the output of e.g. a CHP by a given schedule. In practice it´s used to fulfill contracts.

Adapted existing test files

c49a010

New parameters for class `Flow` has been added. They had to be added to the existing test files too.

Adapted existing tests for scheduled flows

0887645

added constraint and variables to docstring

b760396

Added flexible schedule

ea2e2ff

Now it`s possible to fix the flow to a schedule with possible None values. If schedule is None for the certain timestep, flow will freely be calculated without any penalty constraints.

Added new whatsnew version file

78fceff

The implementation for penalty terms for scheduled flows is finished. * Maybe set a reasonable default value for penalty terms (penalty_pos and penalty_neg), but this acquires mathematical research.

Fixed if statement according to Style Guide

8bd7dda

Fixed boolean logic mistake

650f9ec

Clearified docstring for penalty parameters

e264e5a

Clarified doc for _pos _neg parameters/variables

a9ca25a

Consolinno and others added 6 commits December 10, 2019 16:45

Improved code to pep8 standards

46cc201

Improved code to pep8 standards

c7882a9

Added constraint test for scheduled flows

d5e5f6c

Added flow_scheduled.lp file for testing

2d89775

Improve test for flow schedule

151576d

Merge branch 'dev' into features/schedule_for_flow

97d94b0

gplssm requested a review from simnh December 12, 2019 14:03

gplssm assigned c-koehl Dec 12, 2019

gplssm added this to the v0.3.3 milestone Dec 12, 2019

gplssm added enhancement oemof-solph labels Dec 12, 2019

simnh approved these changes Jan 10, 2020

View reviewed changes

uvchik modified the milestones: v0.3.3, v0.4.1 May 8, 2020

c.koehl added 15 commits June 10, 2020 19:30

Merge branch 'dev' into features/schedule_for_flow

a31aed0

Added whatsnew file

c51772b

Delete old whatsnew file

2c3f8f1

Added missing line at end of file

0436ab1

Added schedule attributes to Flow class

134ec9a

Deleted files added by mistake

4ba1c2c

Renamed penalty_costs to schedule_costs

c948e6d

Added ValueError tests for schedule attributes

346e687

Fixed merge conflict

b6cefa2

Fixed merge conflict

5ec2d0d

Merge pull request #4 from oemof/dev

40e0a9f

Update upstream dev branch

Added newfeatures to whatsnew file

f37c523

Merge branch 'dev' into features/schedule_for_flow

309ba2e

Merge branch 'dev' into features/schedule_for_flow

5a7a4de

Deleted todo from older version

589d230

Deleted line with todo from older oemof version

uvchik mentioned this pull request Apr 26, 2021

[meeting] 2021 oemof Spring Meeting oemof/oemof#93

Closed

p-snft modified the milestones: v0.4.3, v0.4.4 May 26, 2021

uvchik changed the base branch from dev to v0.5 December 21, 2021 21:21

uvchik modified the milestones: v0.4.5, v0.5.1 May 20, 2022

uvchik deleted the branch oemof:dev June 24, 2022 10:01

uvchik closed this Jun 24, 2022

p-snft reopened this Jun 24, 2022

p-snft changed the base branch from v0.5 to dev June 24, 2022 14:28

p-snft modified the milestones: v0.5.1, v0.5.2 Aug 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Features/schedule for flow #666

Features/schedule for flow #666

c-koehl commented Dec 10, 2019 •

edited

Loading

pep8speaks commented Dec 10, 2019 •

edited

Loading

coveralls commented Dec 10, 2019 •

edited

Loading

c-koehl commented Dec 11, 2019

gplssm commented Dec 12, 2019

c-koehl commented Dec 12, 2019

gplssm commented Dec 12, 2019

c-koehl commented Dec 12, 2019

gplssm commented Dec 12, 2019

simnh left a comment

simnh Jan 10, 2020

c-koehl Jan 10, 2020

simnh Jan 10, 2020

c-koehl Jan 10, 2020

c-koehl Jan 10, 2020

c-koehl Jan 10, 2020

simnh Jan 10, 2020

c-koehl Jan 10, 2020

simnh Jan 10, 2020

simnh Jan 10, 2020

c-koehl Jan 10, 2020

c-koehl commented Nov 24, 2020

		penalty term will be activated. If array-like, values can be None
		for flexible, non-fixed flow during the certain timestep. Used in

Features/schedule for flow #666

Are you sure you want to change the base?

Features/schedule for flow #666

Conversation

c-koehl commented Dec 10, 2019 • edited Loading

pep8speaks commented Dec 10, 2019 • edited Loading

Comment last updated at 2022-06-24 14:12:31 UTC

coveralls commented Dec 10, 2019 • edited Loading

c-koehl commented Dec 11, 2019

gplssm commented Dec 12, 2019

c-koehl commented Dec 12, 2019

gplssm commented Dec 12, 2019

c-koehl commented Dec 12, 2019

gplssm commented Dec 12, 2019

simnh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

c-koehl commented Nov 24, 2020

c-koehl commented Dec 10, 2019 •

edited

Loading

pep8speaks commented Dec 10, 2019 •

edited

Loading

coveralls commented Dec 10, 2019 •

edited

Loading