Wrong results for LastFM-Collaborative indexing

[mgulcin]

I'm trying to replicate the results from the paper (on LastFm dataset).

Even if I managed to replicate results for 'random' and 'sequential' indexing, I couldn't replicate results for 'collaborative' (OpenP5-T5-C (un/seen)).

Here is an example result I obtained, using same command and model (from checkpoints provided):
OpenP5-T5-C (unseen)

testing filtered LastFM dataset on sequential task
hit@5: 0.0027522935779816515
hit@10: 0.0027522935779816515
ndcg@5: 0.001392461064959632
ndcg@10: 0.001392461064959632

whereas results in the paper:

hit@5: 0.0504 
ndcg@5: 0.0332 
hit@10: 0.0724 
ndcg@10:0.0420

Can you please guide me with right command or right model (checkpoint)?

[xiaxin1998]

same problem here. I used the command in the repo.