Skip to content

Commit dbf3c37

Browse files
authored
Support case when cache_len == 0
Differential Revision: D90526985 Pull Request resolved: #16546
1 parent 064ef7c commit dbf3c37

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

examples/models/llama/static_attention.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -632,6 +632,8 @@ def _run_once(
632632
return y, attn_updates
633633

634634
def _update_states(self, attn_updates, update_pos, update_len):
635+
if attn_updates["out_cache_state"] is None:
636+
return
635637
for mask in self._masks.values():
636638
mask.unmask(update_len)
637639
k_cache_updates, v_cache_updates = attn_updates["out_cache_state"]

0 commit comments

Comments
 (0)