Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Attribute error when trying to run HiGHS #783

Closed
2 tasks done
ekatef opened this issue Jul 13, 2023 · 5 comments
Closed
2 tasks done

Attribute error when trying to run HiGHS #783

ekatef opened this issue Jul 13, 2023 · 5 comments
Labels
bug Something isn't working

Comments

@ekatef
Copy link
Member

ekatef commented Jul 13, 2023

Checklist

  • I am using the current main branch or the latest release. Please indicate.
  • I am running on an up-to-date pypsa-earth environment. Update via conda env update -f envs/environment.yaml.

Describe the Bug

An attempt to use HiGHS leads to an error in network_lopf. The solver options look like this:

solving:
  options:
    formulation: kirchhoff
    load_shedding: true
    noisy_costs: true
    min_iterations: 4
    max_iterations: 6
    clip_p_max_pu: 0.01
    skip_iterations: true
    track_iterations: false
  solver:
    name: highs

Simulation with cbc runs.

HiGHS has been installed just recently, not sure if the problem is caused by some issues with setup. However, the installation tests were successful and highs --version happily results in HiGHS version 1.5.3.

Error Message

rule solve_network:
    input: networks/MY/elec_s_10_ec_lcopt_Co2L-2H.nc
    output: results/MY/networks/elec_s_10_ec_lcopt_Co2L-2H.nc
    log: logs/MY/solve_network/elec_s_10_ec_lcopt_Co2L-2H_solver.log, logs/MY/solve_network/elec_s_10_ec_lcopt_Co2L-2H_python.log, logs/MY/solve_network/elec_s_10_ec_lcopt_Co2L-2H_memory.log
    jobid: 1
    benchmark: benchmarks/MY/solve_network/elec_s_10_ec_lcopt_Co2L-2H
    reason: Missing output files: results/MY/networks/elec_s_10_ec_lcopt_Co2L-2H.nc
    wildcards: simpl=, clusters=10, ll=copt, opts=Co2L-2H
    resources: tmpdir=/tmp, mem=17925
INFO:snakemake.logging:rule solve_network:
    input: networks/MY/elec_s_10_ec_lcopt_Co2L-2H.nc
    output: results/MY/networks/elec_s_10_ec_lcopt_Co2L-2H.nc
    log: logs/MY/solve_network/elec_s_10_ec_lcopt_Co2L-2H_solver.log, logs/MY/solve_network/elec_s_10_ec_lcopt_Co2L-2H_python.log, logs/MY/solve_network/elec_s_10_ec_lcopt_Co2L-2H_memory.log
    jobid: 1
    benchmark: benchmarks/MY/solve_network/elec_s_10_ec_lcopt_Co2L-2H
    reason: Missing output files: results/MY/networks/elec_s_10_ec_lcopt_Co2L-2H.nc
    wildcards: simpl=, clusters=10, ll=copt, opts=Co2L-2H
    resources: tmpdir=/tmp, mem=17925

INFO:snakemake.logging:
Restricted license - for non-production use only - expires 2024-10-28
No parameters matching '_test' found
~/miniconda3/envs/pypsa-earth-upd/lib/python3.10/site-packages/pypsa/networkclustering.py:16: UserWarning: The namespace `pypsa.networkclustering` is deprecated and will be removed in PyPSA v0.24. Please use `pypsa.clustering.spatial instead`. 
  warnings.warn(
Building DAG of jobs...
WARNING:snakemake.logging:Building DAG of jobs...
Using shell: /usr/bin/bash
WARNING:snakemake.logging:Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
WARNING:snakemake.logging:Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
WARNING:snakemake.logging:Rules claiming more threads will be scaled down.
Provided resources: mem=17925
WARNING:snakemake.logging:Provided resources: mem=17925
Select jobs to execute...
WARNING:snakemake.logging:Select jobs to execute...
Changing to shadow directory: ~/github/pypsa-earth/.snakemake/shadow/tmpjegfs7fi
WARNING:snakemake.logging:Changing to shadow directory: ~/github/pypsa-earth/.snakemake/shadow/tmpjegfs7fi
~/miniconda3/envs/pypsa-earth-upd/lib/python3.10/site-packages/pypsa/networkclustering.py:16: UserWarning: The namespace `pypsa.networkclustering` is deprecated and will be removed in PyPSA v0.24. Please use `pypsa.clustering.spatial instead`. 
  warnings.warn(
INFO:pypsa.io:Imported network elec_s_10_ec_lcopt_Co2L-2H.nc has buses, carriers, generators, global_constraints, lines, links, loads, storage_units, stores
INFO:pypsa.linopf:Prepare linear problem
INFO:pypsa.linopf:Total preparation time: 8.2s
INFO:pypsa.linopf:Solve linear problem using Highs solver
WARNING:pypsa.linopt:The HiGHS solver can potentially solve towards variables that slightly deviate from Gurobi, cbc, glpk
INFO:pypsa.linopt:Options: "{'primal_feasibility_tolerance': 0.0001, 'dual_feasibility_tolerance': 1e-05, 'ipm_optimality_tolerance': 1e-06, 'presolve': 'on', 'run_crossover': True, 'parallel': 'off', 'threads': 4, 'solution_file': '/tmp/pypsa-solvevk4cvru9.sol', 'write_solution_to_file': True, 'write_solution_style': 1, 'log_to_console': True}". List of options: https://www.maths.ed.ac.uk/hall/HiGHS/HighsOptions.set
INFO:pypsa.linopt:Solver command: "highs --model_file /tmp/pypsa-problem-_8w2v5e3.lp --solver ipm --options_file highs_options.txt"
Traceback (most recent call last):
  File "~/github/pypsa-earth/.snakemake/shadow/tmpjegfs7fi/.snakemake/scripts/tmp2l8htsdw.solve_network.py", line 441, in <module>
    n = solve_network(
  File "~/github/pypsa-earth/.snakemake/shadow/tmpjegfs7fi/.snakemake/scripts/tmp2l8htsdw.solve_network.py", line 392, in solve_network
    network_lopf(
  File "~/miniconda3/envs/pypsa-earth-upd/lib/python3.10/site-packages/pypsa/linopf.py", line 1576, in network_lopf
    res = solve(
  File "~/miniconda3/envs/pypsa-earth-upd/lib/python3.10/site-packages/pypsa/linopt.py", line 776, in run_and_read_highs
    log.to_csv(solver_logfile, sep="\t")
AttributeError: 'str' object has no attribute 'to_csv'
Not cleaning up ~/github/pypsa-earth/.snakemake/shadow/tmpjegfs7fi/.snakemake/scripts/tmp2l8htsdw.solve_network.py
CRITICAL:snakemake.logging:Not cleaning up ~/github/pypsa-earth/.snakemake/shadow/tmpjegfs7fi/.snakemake/scripts/tmp2l8htsdw.solve_network.py
[Thu Jul 13 15:46:56 2023]
INFO:snakemake.logging:[Thu Jul 13 15:46:56 2023]
Error in rule solve_network:
    jobid: 0
    input: networks/MY/elec_s_10_ec_lcopt_Co2L-2H.nc
    output: results/MY/networks/elec_s_10_ec_lcopt_Co2L-2H.nc
    log: logs/MY/solve_network/elec_s_10_ec_lcopt_Co2L-2H_solver.log, logs/MY/solve_network/elec_s_10_ec_lcopt_Co2L-2H_python.log, logs/MY/solve_network/elec_s_10_ec_lcopt_Co2L-2H_memory.log (check log file(s) for error details)

ERROR:snakemake.logging:Error in rule solve_network:
    jobid: 0
    input: networks/MY/elec_s_10_ec_lcopt_Co2L-2H.nc
    output: results/MY/networks/elec_s_10_ec_lcopt_Co2L-2H.nc
    log: logs/MY/solve_network/elec_s_10_ec_lcopt_Co2L-2H_solver.log, logs/MY/solve_network/elec_s_10_ec_lcopt_Co2L-2H_python.log, logs/MY/solve_network/elec_s_10_ec_lcopt_Co2L-2H_memory.log (check log file(s) for error details)

Shutting down, this might take some time.
WARNING:snakemake.logging:Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
ERROR:snakemake.logging:Exiting because a job execution failed. Look above for error message
@ekatef ekatef added the bug Something isn't working label Jul 13, 2023
@davide-f
Copy link
Member

Interesting @ekatef , I feel like this may also be a pypsa issue, but needs to be investigated.
Have you tried a different solver?
I'm wondering if this happens when the problem is infeasible

@ekatef
Copy link
Member Author

ekatef commented Jul 13, 2023

@davide-f thanks for looking into it! Good to have a checklist on case of solver troubles :)

cbc solver is working properly for this problem and exactly the solution has been found

@ekatef
Copy link
Member Author

ekatef commented Jul 13, 2023 •
edited
Loading

Just putting here outputs of weekly meeting on solver interfaces (@davide-f, @pz-max, @Tomkourou thanks a lot for the amazing discussion!):

HiGHS interface is currently not maintained in PyPSA-Earth. For being able to support HiGHS, we need to move to linopy, meaning quite some re-implementation work. This needs to be coordinated with the sector-coupled model as it currently heavily relies on another interface (pyomo? -- ek). However, we need to start investigating that because soon PyPSA 1.0 will drop support to functions we are currently using, so good to be prepared

@davide-f
Copy link
Member

davide-f commented Jul 13, 2023 •
edited
Loading

I think we can close this issue in favor of #494
Feel free to reopen if you disagree :)

@ekatef ekatef closed this as completed Jul 13, 2023
@ekatef
Copy link
Member Author

ekatef commented Jul 13, 2023 •
edited
Loading

@davide-f I think you are absolutely right :)
Thanks a lot for linking to #494!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ekatef @davide-f