Are Qwen1.5 MOE models supported?

[l3utterfly]

I tried to convert the Qwen 1.5 MOE: https://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B

It gives me an error message:

 python convert-hf-to-gguf.py /home/layla/src/text-generation-webui/models/Qwen1.5-MoE-A2.7B
Loading model: Qwen1.5-MoE-A2.7B
Traceback (most recent call last):
  File "/home/layla/src/llama.cpp/convert-hf-to-gguf.py", line 2296, in <module>
    main()
  File "/home/layla/src/llama.cpp/convert-hf-to-gguf.py", line 2276, in main
    model_class = Model.from_model_architecture(hparams["architectures"][0])
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/layla/src/llama.cpp/convert-hf-to-gguf.py", line 215, in from_model_architecture
    raise NotImplementedError(f'Architecture {arch!r} not supported!') from None
NotImplementedError: Architecture 'Qwen2MoeForCausalLM' not supported!

I see others have uploaded GGUFs of the Qwen MOE models on HF, so it leads me to think I'm doing something wrong

[l3utterfly]

I see this issue: huggingface/transformers#29377

Which adds support to Qwen2, do I need that PR to convert Qwen1.5 moe?

[gswsqffsapd3056]

Same needs, waiting to be solved

[Jeximo]

I see there's only 2 people making Qwen MOE 1.5 GGUFs on Huggingface(not sure how). llama.cpp shows support for Qwen, but doesn't specify Qwen Moe(yet).

Here's a PR discussing converting/running it, though it appears there's more work to be done before it'll run.

[Lyzin]

I don't know when qwen1.5-MoE will support llama.cpp, I can only wait for qwen official update!

[maziyarpanahi]

Speaking of MoE and Qwen, #6453

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.