[Feature] DataLoadingPrimer handling of dataloader with batch-size > 0

[vmoens]

Stack from ghstack (oldest at bottom):

• [Feature] DensifyReward postproc #2823
• [Feature] DataLoadingPrimer.repeat #2822
• -> [Feature] DataLoadingPrimer handling of dataloader with batch-size > 0 #2821
• [Lint] pyupgrade #2819
• [Feature] LLMEnv and DataLoadingPrimer #2818
• [BugFix] Fix batch_locked check in check_env_specs + error message callable #2817
• [Feature] NonTensor batched arg #2816
• [BugFix] Fix env.full_done_spec~s~ #2815
• [Feature] TensorDictPrimer with single default_value callable #2732

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2821

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 8 Pending

As of commit 1541dd5 with merge base 8c9dc05 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Workflow failed! Cannot read properties of undefined (reading 'ops')
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t d88e38e912a3f11520c04f1c22c4047d3636606205cf89a5f98e74b2c686c126 /exec failed with exit code 1
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t 265648f1cda3ac7d5eb93dadef921260bdd90f34bdc3f8645c843dc21d9fc915 /exec failed with exit code 2

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}29$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_simple	0.6460s	0.5358s	1.8663 Ops/s	1.8945 Ops/s	$\color{#d91a1a}-1.49\%$
test_transformed	1.1342s	1.0259s	0.9748 Ops/s	0.9775 Ops/s	$\color{#d91a1a}-0.28\%$
test_serial	1.5222s	1.5086s	0.6629 Ops/s	0.6648 Ops/s	$\color{#d91a1a}-0.29\%$
test_parallel	1.4506s	1.3403s	0.7461 Ops/s	0.7486 Ops/s	$\color{#d91a1a}-0.34\%$
test_step_mdp_speed[True-True-True-True-True]	0.1649ms	29.9062μs	33.4379 KOps/s	33.5429 KOps/s	$\color{#d91a1a}-0.31\%$
test_step_mdp_speed[True-True-True-True-False]	79.7200μs	17.9224μs	55.7961 KOps/s	57.2521 KOps/s	$\color{#d91a1a}-2.54\%$
test_step_mdp_speed[True-True-True-False-True]	49.3130μs	16.9133μs	59.1252 KOps/s	59.5101 KOps/s	$\color{#d91a1a}-0.65\%$
test_step_mdp_speed[True-True-True-False-False]	84.4290μs	10.0350μs	99.6513 KOps/s	101.1316 KOps/s	$\color{#d91a1a}-1.46\%$
test_step_mdp_speed[True-True-False-True-True]	88.3260μs	32.7106μs	30.5712 KOps/s	31.6087 KOps/s	$\color{#d91a1a}-3.28\%$
test_step_mdp_speed[True-True-False-True-False]	81.7240μs	19.8480μs	50.3829 KOps/s	51.6405 KOps/s	$\color{#d91a1a}-2.44\%$
test_step_mdp_speed[True-True-False-False-True]	95.3090μs	18.9285μs	52.8305 KOps/s	53.2275 KOps/s	$\color{#d91a1a}-0.75\%$
test_step_mdp_speed[True-True-False-False-False]	59.4010μs	11.9576μs	83.6292 KOps/s	84.7855 KOps/s	$\color{#d91a1a}-1.36\%$
test_step_mdp_speed[True-False-True-True-True]	99.3670μs	34.0660μs	29.3548 KOps/s	29.6278 KOps/s	$\color{#d91a1a}-0.92\%$
test_step_mdp_speed[True-False-True-True-False]	53.0700μs	21.8245μs	45.8201 KOps/s	46.6893 KOps/s	$\color{#d91a1a}-1.86\%$
test_step_mdp_speed[True-False-True-False-True]	0.1009ms	18.8811μs	52.9629 KOps/s	53.4466 KOps/s	$\color{#d91a1a}-0.90\%$
test_step_mdp_speed[True-False-True-False-False]	40.9170μs	12.1140μs	82.5493 KOps/s	84.9220 KOps/s	$\color{#d91a1a}-2.79\%$
test_step_mdp_speed[True-False-False-True-True]	0.1183ms	35.4067μs	28.2433 KOps/s	28.2966 KOps/s	$\color{#d91a1a}-0.19\%$
test_step_mdp_speed[True-False-False-True-False]	80.9320μs	23.6223μs	42.3329 KOps/s	43.3686 KOps/s	$\color{#d91a1a}-2.39\%$
test_step_mdp_speed[True-False-False-False-True]	91.3530μs	20.6460μs	48.4355 KOps/s	48.6786 KOps/s	$\color{#d91a1a}-0.50\%$
test_step_mdp_speed[True-False-False-False-False]	71.7950μs	13.6802μs	73.0985 KOps/s	74.2801 KOps/s	$\color{#d91a1a}-1.59\%$
test_step_mdp_speed[False-True-True-True-True]	78.7180μs	34.1538μs	29.2793 KOps/s	29.6466 KOps/s	$\color{#d91a1a}-1.24\%$
test_step_mdp_speed[False-True-True-True-False]	79.7100μs	21.8930μs	45.6768 KOps/s	46.6580 KOps/s	$\color{#d91a1a}-2.10\%$
test_step_mdp_speed[False-True-True-False-True]	68.8400μs	21.7404μs	45.9972 KOps/s	46.3726 KOps/s	$\color{#d91a1a}-0.81\%$
test_step_mdp_speed[False-True-True-False-False]	73.7790μs	13.2894μs	75.2480 KOps/s	76.5728 KOps/s	$\color{#d91a1a}-1.73\%$
test_step_mdp_speed[False-True-False-True-True]	78.0870μs	35.8704μs	27.8781 KOps/s	28.2985 KOps/s	$\color{#d91a1a}-1.49\%$
test_step_mdp_speed[False-True-False-True-False]	95.6000μs	23.6370μs	42.3066 KOps/s	43.2065 KOps/s	$\color{#d91a1a}-2.08\%$
test_step_mdp_speed[False-True-False-False-True]	2.7405ms	23.4489μs	42.6460 KOps/s	43.7136 KOps/s	$\color{#d91a1a}-2.44\%$
test_step_mdp_speed[False-True-False-False-False]	49.3130μs	15.4209μs	64.8470 KOps/s	67.2918 KOps/s	$\color{#d91a1a}-3.63\%$
test_step_mdp_speed[False-False-True-True-True]	0.1036ms	37.7476μs	26.4917 KOps/s	26.8397 KOps/s	$\color{#d91a1a}-1.30\%$
test_step_mdp_speed[False-False-True-True-False]	76.1230μs	25.9937μs	38.4709 KOps/s	39.8532 KOps/s	$\color{#d91a1a}-3.47\%$
test_step_mdp_speed[False-False-True-False-True]	90.5710μs	23.3122μs	42.8960 KOps/s	43.5116 KOps/s	$\color{#d91a1a}-1.41\%$
test_step_mdp_speed[False-False-True-False-False]	54.6330μs	15.4208μs	64.8473 KOps/s	67.0176 KOps/s	$\color{#d91a1a}-3.24\%$
test_step_mdp_speed[False-False-False-True-True]	0.1128ms	39.4888μs	25.3236 KOps/s	25.1253 KOps/s	$\color{#35bf28}+0.79\%$
test_step_mdp_speed[False-False-False-True-False]	70.8140μs	27.2591μs	36.6850 KOps/s	35.1860 KOps/s	$\color{#35bf28}+4.26\%$
test_step_mdp_speed[False-False-False-False-True]	91.4220μs	24.7581μs	40.3908 KOps/s	41.0737 KOps/s	$\color{#d91a1a}-1.66\%$
test_step_mdp_speed[False-False-False-False-False]	46.5680μs	16.7843μs	59.5795 KOps/s	61.0581 KOps/s	$\color{#d91a1a}-2.42\%$
test_values[generalized_advantage_estimate-True-True]	10.7569ms	10.0489ms	99.5131 Ops/s	99.0823 Ops/s	$\color{#35bf28}+0.43\%$
test_values[vec_generalized_advantage_estimate-True-True]	27.8510ms	24.8680ms	40.2124 Ops/s	37.4111 Ops/s	$\textbf{\color{#35bf28}+7.49\%}$
test_values[td0_return_estimate-False-False]	0.2940ms	0.2005ms	4.9887 KOps/s	5.1198 KOps/s	$\color{#d91a1a}-2.56\%$
test_values[td1_return_estimate-False-False]	26.3490ms	24.6798ms	40.5190 Ops/s	40.1030 Ops/s	$\color{#35bf28}+1.04\%$
test_values[vec_td1_return_estimate-False-False]	26.2430ms	24.8508ms	40.2402 Ops/s	36.8648 Ops/s	$\textbf{\color{#35bf28}+9.16\%}$
test_values[td_lambda_return_estimate-True-False]	36.3572ms	35.1008ms	28.4894 Ops/s	28.3168 Ops/s	$\color{#35bf28}+0.61\%$
test_values[vec_td_lambda_return_estimate-True-False]	27.8024ms	24.9016ms	40.1581 Ops/s	37.4250 Ops/s	$\textbf{\color{#35bf28}+7.30\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	11.9999ms	8.5775ms	116.5847 Ops/s	114.8182 Ops/s	$\color{#35bf28}+1.54\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.4067ms	1.7432ms	573.6664 Ops/s	542.2026 Ops/s	$\textbf{\color{#35bf28}+5.80\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.5159ms	0.3697ms	2.7047 KOps/s	2.7070 KOps/s	$\color{#d91a1a}-0.08\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	44.1061ms	40.0574ms	24.9642 Ops/s	22.5893 Ops/s	$\textbf{\color{#35bf28}+10.51\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	4.6850ms	3.6108ms	276.9447 Ops/s	281.3384 Ops/s	$\color{#d91a1a}-1.56\%$
test_dqn_speed[False-None]	1.9629ms	1.4239ms	702.2885 Ops/s	710.1292 Ops/s	$\color{#d91a1a}-1.10\%$
test_dqn_speed[False-backward]	1.9721ms	1.9124ms	522.9137 Ops/s	529.1352 Ops/s	$\color{#d91a1a}-1.18\%$
test_dqn_speed[True-None]	0.7557ms	0.4799ms	2.0837 KOps/s	2.0258 KOps/s	$\color{#35bf28}+2.86\%$
test_dqn_speed[True-backward]	1.1578ms	0.9551ms	1.0470 KOps/s	1.0912 KOps/s	$\color{#d91a1a}-4.05\%$
test_dqn_speed[reduce-overhead-None]	0.7313ms	0.4855ms	2.0596 KOps/s	2.0625 KOps/s	$\color{#d91a1a}-0.14\%$
test_dqn_speed[reduce-overhead-backward]	1.0654ms	0.9246ms	1.0815 KOps/s	1.0075 KOps/s	$\textbf{\color{#35bf28}+7.35\%}$
test_ddpg_speed[False-None]	3.4053ms	2.9605ms	337.7793 Ops/s	336.3500 Ops/s	$\color{#35bf28}+0.42\%$
test_ddpg_speed[False-backward]	5.3050ms	4.2352ms	236.1184 Ops/s	241.3378 Ops/s	$\color{#d91a1a}-2.16\%$
test_ddpg_speed[True-None]	1.4082ms	1.2393ms	806.9199 Ops/s	790.6051 Ops/s	$\color{#35bf28}+2.06\%$
test_ddpg_speed[True-backward]	2.3891ms	2.2061ms	453.2873 Ops/s	448.2229 Ops/s	$\color{#35bf28}+1.13\%$
test_ddpg_speed[reduce-overhead-None]	1.7405ms	1.2457ms	802.7318 Ops/s	797.0864 Ops/s	$\color{#35bf28}+0.71\%$
test_ddpg_speed[reduce-overhead-backward]	3.3637ms	2.1907ms	456.4743 Ops/s	449.6637 Ops/s	$\color{#35bf28}+1.51\%$
test_sac_speed[False-None]	10.4466ms	8.5129ms	117.4693 Ops/s	116.6924 Ops/s	$\color{#35bf28}+0.67\%$
test_sac_speed[False-backward]	13.7023ms	11.7167ms	85.3486 Ops/s	85.4735 Ops/s	$\color{#d91a1a}-0.15\%$
test_sac_speed[True-None]	2.6793ms	2.2486ms	444.7213 Ops/s	442.3315 Ops/s	$\color{#35bf28}+0.54\%$
test_sac_speed[True-backward]	4.8047ms	4.0389ms	247.5952 Ops/s	216.5682 Ops/s	$\textbf{\color{#35bf28}+14.33\%}$
test_sac_speed[reduce-overhead-None]	2.6882ms	2.1514ms	464.8170 Ops/s	386.3794 Ops/s	$\textbf{\color{#35bf28}+20.30\%}$
test_sac_speed[reduce-overhead-backward]	5.3821ms	4.1489ms	241.0293 Ops/s	206.8565 Ops/s	$\textbf{\color{#35bf28}+16.52\%}$
test_redq_speed[False-None]	15.9269ms	13.6681ms	73.1630 Ops/s	65.7553 Ops/s	$\textbf{\color{#35bf28}+11.27\%}$
test_redq_speed[False-backward]	25.1705ms	23.4222ms	42.6945 Ops/s	39.9344 Ops/s	$\textbf{\color{#35bf28}+6.91\%}$
test_redq_speed[True-None]	7.1319ms	6.5462ms	152.7596 Ops/s	154.7802 Ops/s	$\color{#d91a1a}-1.31\%$
test_redq_speed[True-backward]	14.2133ms	13.8277ms	72.3185 Ops/s	68.6842 Ops/s	$\textbf{\color{#35bf28}+5.29\%}$
test_redq_speed[reduce-overhead-None]	7.1937ms	6.2123ms	160.9699 Ops/s	149.5781 Ops/s	$\textbf{\color{#35bf28}+7.62\%}$
test_redq_speed[reduce-overhead-backward]	16.3051ms	13.9601ms	71.6329 Ops/s	67.0696 Ops/s	$\textbf{\color{#35bf28}+6.80\%}$
test_redq_deprec_speed[False-None]	16.7512ms	14.1916ms	70.4642 Ops/s	66.2359 Ops/s	$\textbf{\color{#35bf28}+6.38\%}$
test_redq_deprec_speed[False-backward]	22.1543ms	20.3870ms	49.0508 Ops/s	46.8194 Ops/s	$\color{#35bf28}+4.77\%$
test_redq_deprec_speed[True-None]	5.2488ms	4.5734ms	218.6573 Ops/s	203.9210 Ops/s	$\textbf{\color{#35bf28}+7.23\%}$
test_redq_deprec_speed[True-backward]	11.4747ms	10.0466ms	99.5366 Ops/s	97.9819 Ops/s	$\color{#35bf28}+1.59\%$
test_redq_deprec_speed[reduce-overhead-None]	5.2700ms	4.5870ms	218.0056 Ops/s	208.9419 Ops/s	$\color{#35bf28}+4.34\%$
test_redq_deprec_speed[reduce-overhead-backward]	10.5016ms	9.9063ms	100.9456 Ops/s	98.4946 Ops/s	$\color{#35bf28}+2.49\%$
test_td3_speed[False-None]	9.3327ms	8.8541ms	112.9417 Ops/s	113.5376 Ops/s	$\color{#d91a1a}-0.52\%$
test_td3_speed[False-backward]	14.4135ms	11.8847ms	84.1416 Ops/s	85.7305 Ops/s	$\color{#d91a1a}-1.85\%$
test_td3_speed[True-None]	2.2628ms	1.8911ms	528.8043 Ops/s	515.0162 Ops/s	$\color{#35bf28}+2.68\%$
test_td3_speed[True-backward]	4.7063ms	3.9606ms	252.4847 Ops/s	236.4460 Ops/s	$\textbf{\color{#35bf28}+6.78\%}$
test_td3_speed[reduce-overhead-None]	2.2808ms	1.9490ms	513.0839 Ops/s	448.7434 Ops/s	$\textbf{\color{#35bf28}+14.34\%}$
test_td3_speed[reduce-overhead-backward]	4.2052ms	3.6477ms	274.1479 Ops/s	224.4347 Ops/s	$\textbf{\color{#35bf28}+22.15\%}$
test_cql_speed[False-None]	42.7955ms	39.0221ms	25.6265 Ops/s	24.7643 Ops/s	$\color{#35bf28}+3.48\%$
test_cql_speed[False-backward]	52.0522ms	49.1637ms	20.3402 Ops/s	19.9744 Ops/s	$\color{#35bf28}+1.83\%$
test_cql_speed[True-None]	20.0385ms	16.9416ms	59.0263 Ops/s	58.0237 Ops/s	$\color{#35bf28}+1.73\%$
test_cql_speed[True-backward]	25.9697ms	24.4774ms	40.8540 Ops/s	39.5390 Ops/s	$\color{#35bf28}+3.33\%$
test_cql_speed[reduce-overhead-None]	18.8921ms	17.1794ms	58.2091 Ops/s	57.8676 Ops/s	$\color{#35bf28}+0.59\%$
test_cql_speed[reduce-overhead-backward]	26.8767ms	24.6263ms	40.6071 Ops/s	40.4335 Ops/s	$\color{#35bf28}+0.43\%$
test_a2c_speed[False-None]	8.5107ms	7.7499ms	129.0333 Ops/s	126.3306 Ops/s	$\color{#35bf28}+2.14\%$
test_a2c_speed[False-backward]	17.2168ms	15.9939ms	62.5239 Ops/s	62.9216 Ops/s	$\color{#d91a1a}-0.63\%$
test_a2c_speed[True-None]	4.7278ms	4.0928ms	244.3337 Ops/s	241.5938 Ops/s	$\color{#35bf28}+1.13\%$
test_a2c_speed[True-backward]	11.8533ms	11.1962ms	89.3158 Ops/s	85.1431 Ops/s	$\color{#35bf28}+4.90\%$
test_a2c_speed[reduce-overhead-None]	6.1293ms	3.9545ms	252.8754 Ops/s	225.8467 Ops/s	$\textbf{\color{#35bf28}+11.97\%}$
test_a2c_speed[reduce-overhead-backward]	12.4329ms	11.3145ms	88.3822 Ops/s	83.8040 Ops/s	$\textbf{\color{#35bf28}+5.46\%}$
test_ppo_speed[False-None]	9.6420ms	8.0488ms	124.2422 Ops/s	115.8039 Ops/s	$\textbf{\color{#35bf28}+7.29\%}$
test_ppo_speed[False-backward]	18.3605ms	16.2966ms	61.3624 Ops/s	59.7239 Ops/s	$\color{#35bf28}+2.74\%$
test_ppo_speed[True-None]	4.9522ms	4.4254ms	225.9708 Ops/s	220.3072 Ops/s	$\color{#35bf28}+2.57\%$
test_ppo_speed[True-backward]	11.8662ms	11.1441ms	89.7335 Ops/s	88.5093 Ops/s	$\color{#35bf28}+1.38\%$
test_ppo_speed[reduce-overhead-None]	6.2041ms	4.4802ms	223.2059 Ops/s	205.7597 Ops/s	$\textbf{\color{#35bf28}+8.48\%}$
test_ppo_speed[reduce-overhead-backward]	11.6070ms	10.8909ms	91.8198 Ops/s	90.2747 Ops/s	$\color{#35bf28}+1.71\%$
test_reinforce_speed[False-None]	8.3030ms	6.8676ms	145.6106 Ops/s	143.1763 Ops/s	$\color{#35bf28}+1.70\%$
test_reinforce_speed[False-backward]	12.6380ms	10.4474ms	95.7180 Ops/s	94.4739 Ops/s	$\color{#35bf28}+1.32\%$
test_reinforce_speed[True-None]	3.9531ms	3.2766ms	305.1980 Ops/s	283.6935 Ops/s	$\textbf{\color{#35bf28}+7.58\%}$
test_reinforce_speed[True-backward]	11.0595ms	9.9375ms	100.6293 Ops/s	95.8309 Ops/s	$\textbf{\color{#35bf28}+5.01\%}$
test_reinforce_speed[reduce-overhead-None]	4.0733ms	3.3294ms	300.3523 Ops/s	286.9206 Ops/s	$\color{#35bf28}+4.68\%$
test_reinforce_speed[reduce-overhead-backward]	10.4665ms	9.9281ms	100.7240 Ops/s	98.7550 Ops/s	$\color{#35bf28}+1.99\%$
test_iql_speed[False-None]	35.9105ms	33.4531ms	29.8926 Ops/s	29.3646 Ops/s	$\color{#35bf28}+1.80\%$
test_iql_speed[False-backward]	49.5744ms	46.9059ms	21.3193 Ops/s	20.9615 Ops/s	$\color{#35bf28}+1.71\%$
test_iql_speed[True-None]	13.7293ms	12.0794ms	82.7857 Ops/s	79.2216 Ops/s	$\color{#35bf28}+4.50\%$
test_iql_speed[True-backward]	25.6593ms	23.9957ms	41.6741 Ops/s	41.1441 Ops/s	$\color{#35bf28}+1.29\%$
test_iql_speed[reduce-overhead-None]	13.2535ms	12.1439ms	82.3458 Ops/s	80.6130 Ops/s	$\color{#35bf28}+2.15\%$
test_iql_speed[reduce-overhead-backward]	25.5916ms	24.0292ms	41.6160 Ops/s	41.1332 Ops/s	$\color{#35bf28}+1.17\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	8.2664ms	5.3859ms	185.6704 Ops/s	187.2406 Ops/s	$\color{#d91a1a}-0.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	2.6275ms	0.5695ms	1.7558 KOps/s	1.7765 KOps/s	$\color{#d91a1a}-1.16\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.8892ms	0.5370ms	1.8620 KOps/s	1.8003 KOps/s	$\color{#35bf28}+3.43\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.7549ms	5.0785ms	196.9071 Ops/s	200.0462 Ops/s	$\color{#d91a1a}-1.57\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.0556ms	0.5539ms	1.8054 KOps/s	1.8320 KOps/s	$\color{#d91a1a}-1.45\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.9206ms	0.5298ms	1.8876 KOps/s	1.8652 KOps/s	$\color{#35bf28}+1.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.7022ms	1.7240ms	580.0308 Ops/s	574.6960 Ops/s	$\color{#35bf28}+0.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.4935ms	1.6442ms	608.1906 Ops/s	600.3335 Ops/s	$\color{#35bf28}+1.31\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	7.8303ms	5.1605ms	193.7815 Ops/s	191.1795 Ops/s	$\color{#35bf28}+1.36\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1054ms	0.7019ms	1.4247 KOps/s	1.4365 KOps/s	$\color{#d91a1a}-0.82\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	1.0440ms	0.6807ms	1.4691 KOps/s	1.4965 KOps/s	$\color{#d91a1a}-1.83\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.3796ms	5.0025ms	199.8992 Ops/s	187.9726 Ops/s	$\textbf{\color{#35bf28}+6.34\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.1587ms	0.5637ms	1.7739 KOps/s	1.7988 KOps/s	$\color{#d91a1a}-1.39\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.8035ms	0.5380ms	1.8588 KOps/s	1.8678 KOps/s	$\color{#d91a1a}-0.48\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	7.6380ms	5.1435ms	194.4192 Ops/s	195.4520 Ops/s	$\color{#d91a1a}-0.53\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.4069ms	0.5567ms	1.7964 KOps/s	1.8049 KOps/s	$\color{#d91a1a}-0.47\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.8111ms	0.5292ms	1.8895 KOps/s	1.8959 KOps/s	$\color{#d91a1a}-0.34\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.0325ms	5.3804ms	185.8581 Ops/s	185.2455 Ops/s	$\color{#35bf28}+0.33\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1092ms	0.7012ms	1.4261 KOps/s	1.4153 KOps/s	$\color{#35bf28}+0.76\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	1.1862ms	0.6759ms	1.4796 KOps/s	1.4941 KOps/s	$\color{#d91a1a}-0.97\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	6.8886ms	4.6870ms	213.3543 Ops/s	197.1562 Ops/s	$\textbf{\color{#35bf28}+8.22\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	5.5756ms	2.3484ms	425.8160 Ops/s	408.0266 Ops/s	$\color{#35bf28}+4.36\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	6.7734ms	1.5044ms	664.7275 Ops/s	631.7180 Ops/s	$\textbf{\color{#35bf28}+5.23\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.5574s	15.5773ms	64.1962 Ops/s	204.8372 Ops/s	$\textbf{\color{#d91a1a}-68.66\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	7.6891ms	2.3532ms	424.9490 Ops/s	412.7845 Ops/s	$\color{#35bf28}+2.95\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	9.0399ms	1.6152ms	619.1102 Ops/s	686.3551 Ops/s	$\textbf{\color{#d91a1a}-9.80\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	6.8144ms	4.7404ms	210.9524 Ops/s	209.4248 Ops/s	$\color{#35bf28}+0.73\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	6.8842ms	2.5249ms	396.0616 Ops/s	389.0223 Ops/s	$\color{#35bf28}+1.81\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	5.6383ms	1.5519ms	644.3859 Ops/s	564.0658 Ops/s	$\textbf{\color{#35bf28}+14.24\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	13.0913ms	12.6357ms	79.1409 Ops/s	77.4449 Ops/s	$\color{#35bf28}+2.19\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	23.2034ms	14.8797ms	67.2055 Ops/s	67.5793 Ops/s	$\color{#d91a1a}-0.55\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	23.5215ms	21.4861ms	46.5417 Ops/s	46.2994 Ops/s	$\color{#35bf28}+0.52\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	16.5271ms	14.9575ms	66.8560 Ops/s	66.5782 Ops/s	$\color{#35bf28}+0.42\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	24.0220ms	21.4052ms	46.7177 Ops/s	45.9848 Ops/s	$\color{#35bf28}+1.59\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	22.4228ms	16.3922ms	61.0047 Ops/s	60.8537 Ops/s	$\color{#35bf28}+0.25\%$