Add regenerate option for applications

[philross]

This PR adds the regenerate option for applications.
The regenerate key can be set in the config file and defines whether you want to regenerate the application
instance for every repetition. When this key is missing, the default value is True. This option can be handy if you
are using random seeds in the creation of the application instance and you want to have the same instance for every
repetition. If you want to change the default behavior for your application, you have to add the regenerate key to
your get_parameter_options function.
Example:

application:
  config:
    regenerate:
    - False
    nodes:
    - 3
  name: TSP
  submodules:
  - config: {}
    name: GreedyClassicalTSP
    submodules:
    - config: {}
      name: Local
      submodules: []
repetitions: 3

[philross]

Hey @jusschwitalla , this is a feature we talked about in one of the open calls. While the default behavior will regenerate the application instance for every repetition, there will be now the option to keep the same instance.

[jusschwitalla]

Hi,

I think this is not quite the same logic that was implemented in the old QUARK (and which is still available in quark2_adapter.adapters.Application). Let me try to explain:
The loop in the benchmark manager can thought of being composed from three parts:

1. the iteration loop (index 'i_loop' in the following)
2. the loop over different problem configurations (index 'i_problem' in the following)
3. the loop over different non-problem configurations (e.g. different solvers) (index 'i_solver' in the following)

In the old implementation a new problem instance is created for every i_loop, i_problem pair.
For fixed i_loop and i_problem but different i_solver the same instance is used.

The idea behind that is that you can run different solver configurations on some statistical ensemble of problems.

A second difference is that in the old implementation the user does not directly decide which problem creation mode is to be used but this is decided by the application. This is because it is in the nature of the problem whether it is a probabilistic or a deterministic one. For example in my MaxCut application the user can decide whether to use a probabilitc graph (Erdos Renyi - Graph) or a circular graph. In the first case the problem will be recreated in the second it will not be recreated.

May be it is an option to implement differen recreation strategies as abstract aplication classes and every one how writes a new application can choose which one he inherits from.

Hope this helps to clarify things.
Best wishes,
Jürgen

[philross]

Hello @jusschwitalla, thanks for the extensive explanation!
I agree that if this "regenerate" is set to false, an application/problem instance should stay exactly the same for the whole QUARK run across iterations, solvers, mappings or other submodules.
The implementation currently allows the developer of the application to the decide the default for an application, but enables the experienced user to override this default in the config (and not in the question/answer mode) for whatever reason. I would like to keep this aspect of the feature.

I will come up with an updated implementation and will update this PR.

[jusschwitalla]

Hi Philipp, I’m not quite sure but it seems to me that you missed the main point of my argument. May be my sentence “For fixed i_loop and i_problem but different i_solver the same instance is used.” is not clear. What I wanted to say is that even in the regenerate mode the instance should not be regenerated every time. But should stay the same if i_loop and i_problem are the same. Best wishes, Jürgen Von: Philipp ***@***.***> Gesendet: Mittwoch, 31. Januar 2024 09:07 An: QUARK-framework/QUARK ***@***.***> Cc: jusschwitalla ***@***.***>; Mention ***@***.***> Betreff: Re: [QUARK-framework/QUARK] Add regenerate option for applications (PR #106) Hello @jusschwitalla<https://github.com/jusschwitalla>, thanks for the extensive explanation! I agree that if this "regenerate" is set to false, an application/problem instance should stay exactly the same for the whole QUARK run across iterations, solvers, mappings or other submodules. The implementation currently allows the developer of the application to the decide the default for an application, but enables the experienced user to override this default in the config (and not in the question/answer mode) for whatever reason. I would like to keep this aspect of the feature. I will come up with an updated implementation and will update this PR. — Reply to this email directly, view it on GitHub<#106 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AGX2QFAEAVRHVKUOG4EDQF3YRH3SLAVCNFSM6AAAAABB6JJBEWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJYGU4DMNRUGU>. You are receiving this because you were mentioned.Message ID: ***@***.***>