Fix todo: avoid relying on logits_all == true in perplexity_v2

Fix todo: avoid relying on logits_all == true in perplexity_v2 #3352