[Bug]: KV Cache exploded

[rakkit]

Describe the bug

In the case of using softmax attention or any other attention with window_size=None, the KV cache update falls into this branch. This logic concatenates all historical sequence states with the new states (attn_state[0] and attn_state[1]), causing exponential growth in the KV cache.

Steps to reproduce the bug

Inference with attention with window_size=None

Expected behavior

KV-cache exploded

Environment info

[yzhangcs]

@rakkit Hello, could you provide a minimal script for reproduction. I didn't meet the errors you reported by setting window_size=None