Skip to content

MNN:Refractor: Don't copy kvcache when meta is not the same#4102

Open
jxt1234 wants to merge 1 commit intoalibaba:masterfrom
jxt1234:feature/clone_opt
Open

MNN:Refractor: Don't copy kvcache when meta is not the same#4102
jxt1234 wants to merge 1 commit intoalibaba:masterfrom
jxt1234:feature/clone_opt

Conversation

@jxt1234
Copy link
Collaborator

@jxt1234 jxt1234 commented Jan 7, 2026

  1. 根据 KVMeta 是否一致,来判断是否在 clone attention 时共享 kvcache
  2. 支持用户在 clone module 时传入 runtimemanager

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant