Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmarks: add Temoa benchmarks and instructions #89

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

danielelerede-oet
Copy link
Contributor

Hi @siddharth-krishna , I'm adding here the working TEMOA benchmarks with their datasets and metadata

Copy link

vercel bot commented Jan 28, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
solver-benchmark ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 14, 2025 2:59pm

Copy link
Contributor

@siddharth-krishna siddharth-krishna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@danielelerede-oet when you have a moment, could you please amend this PR by removing the .sqlite files, and adding a README to the benchmarks/temoa/ directory with some details/instructions on how these benchmarks were generated? I think you said in the call that the .sqlite files can be obtained from temoa's repo, so linking to them in the README should be sufficient. Thank you!

@danielelerede-oet
Copy link
Contributor Author

Done! @siddharth-krishna

Copy link
Contributor

@siddharth-krishna siddharth-krishna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Daniele. I'll also modify this PR to add the changes from #93 and merge the metadata. If you have some time though, I was wondering if it might be nice to add some more details to the instructions. (Though this can also be saved for another PR if today is busy.) Thanks

- Clone the [TEMOA v2.0](https://github.com/TemoaProject/temoa/tree/v2.0) repository (release v2.0 was preferred over the latest v3.0 due to modifications on input .sql files to be performed for using v3.0)
- Follow the instructions to create a TEMOA environment at [TEMOA v2.0](https://github.com/TemoaProject/temoa/tree/v2.0)
- Copy the .sqlite files at [TemoaProject/oeo](https://github.com/TemoaProject/oeo) in the local folder `temoa/data_files`
- Modify `temoa/temoa_model/config_sample` providing the name of the `--input` and `--output` .sqlite dataset you wish to run (i.e. `--input=data_files/temoa_utopia.sqlite` and `--output=data_files/temoa_utopia.sqlite` if you wish to run `temoa_utopia.sqlite)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps it's useful either here or in the metadate file to add the command you used to generate each benchmark? For me it's not obvious which input file corresponds to which benchmark (unless I'm missing something?) Thanks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Sid, I expanded the instructions and added the reference to --keep_pyomo_lp_files, but do you have anything additional you want me to add?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I'm still not able to figure out what config and command you would use to generate the LP file for e.g. temoa-US_9R_TS_SP... I don't see a .sqlite file of that name in https://github.com/TemoaProject/temoa/tree/v2.0/data_files

Perhaps it's helpful to add a list or table here with the mapping of benchmark names to .sqlite files? Or if it's not too much work, adding the config files for each benchmark to this PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I get your point. I can make the explanation more explicit

- Follow the instructions to create a TEMOA environment at [TEMOA v2.0](https://github.com/TemoaProject/temoa/tree/v2.0)
- Copy the .sqlite files at [TemoaProject/oeo](https://github.com/TemoaProject/oeo) in the local folder `temoa/data_files`
- Modify `temoa/temoa_model/config_sample` providing the name of the `--input` and `--output` .sqlite dataset you wish to run (i.e. `--input=data_files/temoa_utopia.sqlite` and `--output=data_files/temoa_utopia.sqlite` if you wish to run `temoa_utopia.sqlite)
- Run TEMOA using *Option 1* at [TEMOA v2.0](https://github.com/TemoaProject/temoa/tree/v2.0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you also need to use the --keep_pyomo_lp_file option to generate the LP files?

@siddharth-krishna siddharth-krishna changed the title Daniele/temoa Benchmarks: add Temoa benchmarks and instructions Feb 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants