Add padding_type parameter to FrameSkipObservation and change default to "reset" to preserve backwards comp

[jamartinh]

Description

FrameStacking as initially (from long time ago) designed filled the buffer with the initial observation from env.reset, I.e. repeated many times as the stacking number.

In the re-implementation the assumption changed to fill it with zeroes, but this not only breaks many codes, it breaks "what is expected" and its proper documentation design.

I have fixed this by using its proper definition:

Note:
        - After :meth:`reset` is called, the frame buffer will be filled with the initial observation.
          I.e. the observation returned by :meth:`reset` will consist of `num_stack` many identical frames.

Fixes #829

Also, I have modified the call to super() with self.env as this is the right way for working with wrappers.
If super() used then unexpected errors can arise when using multiple wrappers.

Type of change

Please delete options that are not relevant.

• [X ] Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Screenshots

Please attach before and after screenshots of the change if applicable.

Before:

Original env:
[ 1.25 -0.   -0.   -0.    0.   -0.    0.    0.   -0.   -0.    0.  ]

Stacked env:
[[ 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.  ]
 [ 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.  ]
 [ 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.  ]
 [ 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.  ]
 [ 1.25 -0.    0.    0.    0.    0.    0.   -0.   -0.    0.    0.  ]]

After:

Original env:
[ 1.25  0.   -0.   -0.    0.   -0.    0.   -0.   -0.   -0.    0.  ]

Stacked env:
[[ 1.25 -0.   -0.   -0.    0.    0.    0.    0.    0.   -0.   -0.  ]
 [ 1.25 -0.   -0.   -0.    0.    0.    0.    0.    0.   -0.   -0.  ]
 [ 1.25 -0.   -0.   -0.    0.    0.    0.    0.    0.   -0.   -0.  ]
 [ 1.25 -0.   -0.   -0.    0.    0.    0.    0.    0.   -0.   -0.  ]
 [ 1.25 -0.   -0.   -0.    0.    0.    0.    0.    0.   -0.   -0.  ]]

Checklist:

• I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[pseudo-rnd-thoughts]

Thanks for pointing this out.
I implemented with zero as the padding observation due to this is the standard atari implementation / usage.
However I agree that this breaks the prior implementation.

Therefore, I would propose the user having the ability to select the pad observation in __init__ with the options being "zero", "initial", and custom

[pseudo-rnd-thoughts]

@jamartinh I have made some changes to your implementation, before I merge, are you happy with the changes and new tests

[jamartinh]

Good, thanks!