Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ANT-583] Improve retrieval of simulation results (option -z) #1738

Open
laurent-laporte-pro opened this issue Sep 18, 2023 · 1 comment · May be fixed by #1774
Open

[ANT-583] Improve retrieval of simulation results (option -z) #1738

laurent-laporte-pro opened this issue Sep 18, 2023 · 1 comment · May be fixed by #1774

Comments

@laurent-laporte-pro
Copy link
Contributor

Problem Statement

Currently, Antares Web is able to handle scenarios where Antares Launcher provides the entire study archive, including the results in the study's output folder.
Antares Web is able to take into account the case where results are stored in a sub-folder, e.g.
output/20230914-1918eco, but also the case where results are provided in a ZIP archive, e.g.
output/20230914-1918eco.zip.

However, Antares Web does not have the capacity to handle the situation where the archive does not contain the entire study, but only the simulation results in the form of a compressed archive.
This is particularly the case when using the -z option in Antares Solver.

Proposed Solution

The aim of this feature request is to modify Antares Web to take account of this new scenario in addition to the existing scenarios.

Specifically, the SlurmLauncher._import_study_output method (which is an implementation detail) needs to be modified to take into account the case where the directory path self.local_workspace / STUDIES_OUTPUT_DIR_NAME / job_id represents a compressed archive of results instead of a entire study.

The old behavior should also be retained, i.e.
where the directory path self.local_workspace / STUDIES_OUTPUT_DIR_NAME / job_id is a complete study containing an output sub-folder.

Developer Notes

  • In order to test this evolution, we need to use a version of the launchAntares.sh script that exports only the results, not the entire study.
@laurent-laporte-pro laurent-laporte-pro added this to the v2.16 milestone Sep 18, 2023
@laurent-laporte-pro laurent-laporte-pro changed the title Improve retrieval of simulation results [ANT-583] Improve retrieval of simulation results Sep 18, 2023
@laurent-laporte-pro laurent-laporte-pro changed the title [ANT-583] Improve retrieval of simulation results [ANT-583] Improve retrieval of simulation results (option -z) Sep 21, 2023
@MartinBelthle
Copy link
Contributor

MartinBelthle commented Oct 17, 2023

Processus de rapatriement des résultats de Calin vers AntaresWeb:

Quand l'étude est finie et revient de Calin, on reçoit un zip dans le slurm workspace.
Pour exporter l’output vers AntaresWeb, il faut:

  • Si le zip reçu est une étude (étude zippée contenant un output non zippé): Cas actuel avec antares-solver et xpansion avec analyse de sensitivité

      1. On la dézippe
      1. On zippe l’output
      1. On l’exporte du slurm workspace vers AntaresWeb
  • Si le zip reçu est une étude (étude zippée contenant un output zippé): Cas actuel avec xpansion sans analyse de sensi

      1. On la dézippe
      1. On dézippe l’output et on le rezippe (totalement stupide mais c'est pour ajouter le "updated_links" qui provient du dossier input je crois. Ça pourrait être remplacé par un zipFile.write(... , "a") mais ça serait pour un autre dev).
      1. On l’exporte du slurm workspace vers AntaresWeb
  • Si le zip reçu est un output (output zippé): Cas de l’option -z du solver

    • On devrait juste faire le c. et changer son nom (enlever le «finished_» au début et le "job_id" à la fin) : Nécessite des modifications dans antares-launcher et dans AntaREST

Si l’option auto_unzip est activée on le dézippe sur AntaresWeb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment