Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add random state for reproducibility #86

Closed
wants to merge 5 commits into from
Closed

Conversation

fnattino
Copy link
Collaborator

@fnattino fnattino commented Nov 11, 2021

Addressing #46: In order to have reproducible simulation sampling in the store, we should have the possibility of seeding the random number generation. My impression is that the seeding is probably best introduced within Bound, so that one could do:

bound = UnitCubeBound(n_parameters)
bound.set_seed(1234)
store.add(100, prior, bound)
store.add(100, prior, bound) # add more points following the previous stream of random numbers

The disadvantage is that one needs to instantiate the Bound object outside the store.

Tasks:

  • code
  • tests (?)
  • tutorial notebooks

@fnattino
Copy link
Collaborator Author

Do you agree @bkmi ?

@bkmi
Copy link
Collaborator

bkmi commented Nov 30, 2021

I like what you propose, but I haven't groked how this may affect the new store definition. I suppose we should try to merge the new store specification first then incorporate this request?

I'm still thinking about instantiating the bound outside the store... I think it's okay because the "deepest" function which will generate the bounds is most likely this one...

Perhaps we can provide an option to give a current rng, then "split" it... I'm not sure about that yet. I'm not super comfortable with these new randomness generators.

@rogerkuou
Copy link
Collaborator

@bkmi Hi Ben, thanks for the feedback. I have adapted the current modifications to the `simplify branch as much as possible. and added some minimum tests.

Indeed we can incorporate this one later. I did not update the notebook with the bound seeding method, since the tutorial notebooks are updated in master.

@bkmi bkmi assigned bkmi and unassigned bkmi Nov 30, 2021
@bkmi bkmi added the enhancement New feature or request label Nov 30, 2021
@bkmi
Copy link
Collaborator

bkmi commented Nov 30, 2021

the notebooks from master have been brought into simplify now.

@fnattino
Copy link
Collaborator Author

fnattino commented Dec 1, 2021

(Thanks a lot @rogerkuou for continuing with this work which I had left halfway!)

Perhaps we can provide an option to give a current rng, then "split" it...

@bkmi do you mean to provide the possibility to set a seeded rng in Store and then propagate this to all the calls to pdf.sample()? We could avoid storing rng as an attribute into the bound, but do something like:

class Bound:
  ...
  def sample(self, n_samples, rng=None):
    rng = rng or np.random.default_rng()
    ...
  ...

In this way one would not need to seed the bound outside the store - one would seed the store instead. Also, the seeded rng in the store could be used to draw samples from the Poissonian here - which I had overlooked earlier - and which would still not be covered if we were to store rng in the bound..

Base automatically changed from simplify to master December 9, 2021 22:04
@cweniger
Copy link
Member

This is a pull request for v0.3, which is not actively developed. Closed for now, although the topic of reproducibility is relevant for v0.4 as well.

@cweniger cweniger closed this May 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

introduce new numpy random_state, especially in store, for reproducibility
4 participants