[model] add support for mixtral moe model #128

Open

936187425 wants to merge 14 commits intovectorch-ai:mainfrom

936187425:Mixtral

Commits on Apr 16, 2024

[feat]add mixtral.h
936187425
committed

Commits on Apr 18, 2024

[feat] add some classes in mixtral.h
936187425
committed

Commits on Apr 19, 2024

[feat] construct modules from mixtral model except Mixtral moe impl
936187425
committed

Commits on Apr 26, 2024

Commits on May 13, 2024

Commits on May 14, 2024

[feat] add the load_state_dict in fused_moe.cpp
936187425
committed

Commits on May 22, 2024

Merge branch 'vectorch-ai:main' into Mixtral
936187425
authored