GPU-accelerated environment

[cpwan]

Hi there,
Does tianshou support GPU-accelerated environments (such as Issac Gym )?
By support, I mean whether the GPU-accelerated environment can interact with the neural network model all on GPU, without converting the states/ action back and forth between devices.
I have been using other RL libraries but I found that at least 30% of the time was used for moving the data across CPU/GPU. I read from the docs that the ReplayBuffer stores the state as numpy.ndarray or dict. Is it possible for it to store tensors on GPU? It would be great if tianshou can deal with the overhead or if there are some workarounds.

[Trinkle23897]

Theoretically it can support. Buffer is implemented on top of Batch, and Batch can naively support torch.Tensor.