-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The performance reproducing #26
Comments
Hi Jaesik, I haven't touched the v1 code in a while, will take a look when I get more time. In the meanwhile, could you try the v1.5 code on main branch with transformer? |
Okay I will do. Thank you for your quick reply!
…________________________________
보낸 사람: Boyuan Chen ***@***.***>
보낸 날짜: Tuesday, November 5, 2024 5:30:07 AM
받는 사람: buoyancy99/diffusion-forcing ***@***.***>
참조: Jaesik Yoon ***@***.***>; Author ***@***.***>
제목: Re: [buoyancy99/diffusion-forcing] The performance reproducing (Issue #26)
Hi Jaesik,
I haven't touched the v1 code in a while, will take a look when I get more time. In the meanwhile, could you try the v1.5 code on main branch with transformer?
—
Reply to this email directly, view it on GitHub<#26 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFLDXI7ANDNVGNSTSZHXD4DZ67DL7AVCNFSM6AAAAABREIQNC6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJVGY2DAMRUHE>.
You are receiving this because you authored the thread.Message ID: ***@***.***>
|
I believe increasing experiment.validation.limit_batch could help achieve more accurate results. In my case, I was able to reproduce the performance when I set it to 10. |
Hi @buoyancy99 , while investigating this issue, I found the setting suggested in README is only evaluating very few samples in the validation set. Thus, I tried to evaluate every sample in the set, but it requires very long time due to the roll-out based planning. So, I have two questions,
Thank you in advance. |
We do evaluate a good amount of data points and yes, it's very slow in the v1 RNN code, which is why I rewrote it into the v1.5 code on main branch. |
Thank you for your quick reply. Currently my concern is resolved enough. If you want to close this issue, I am okay to close. May another issue will be raised while investigating your model, I will reopen this.
Again, thank you for your contribution on planning domain and sharing your source codes! |
Hi @buoyancy99 ,
Thank you for sharing the source code of your project.
I tried to reproduce the performance on Maze2D medium and large in your paper with the configuration suggested in your paper, but I couldn't get the performance as reported in the paper.
Could you check what I missed?
paper
branchWhen using the above configurations with your commend shared in README.md file, I got about 117 episode reward for Maze2D-Large.
Bests, Jaesik.
The text was updated successfully, but these errors were encountered: