Cannot get execute_services to work no matter which model I've tried, only conversation is possible

[zachfeldman]

I cannot seem to get this extension to work no matter which local LLM model I've tried - with "Use Tools" on, off and many different prompt variations (though mostly using the default prompt).

The models I've tried include:
Mistral-7B-Instruct-v0.2-GGUF
Qwen2.5-7B-Instruct-Q6_K_L
Qwen3-30B-A3B-Q8_0
gemma-2-27B-it-function-calling-Q6_K
Hermes-3-Llama-3.1-8B.Q8_0

Pretty much all as .gguf files.

This is the invocation I am using for llama-server for the last model:
LLAMA_KV_OVERALLOC=2.0 LLAMA_CHAT_TEMPLATE=qwen:tool_use ./bin/llama-server --host 0.0.0.0 --port 8000 --jinja --chat-template-file <(python ../scripts/get_chat_template.py NousResearch/Hermes-3-Llama-3.1-8B tool_use) -m ~/dev/hermes/Hermes-3-Llama-3.1-8B.Q8_0.gguf -v --log-timestamps

I always get a response, sometimes in an XML-like format, sometimes as plain conversation, and I've even confirmed with ChatGPT that some of my responses include tool calls, but no matter what, my lights do not turn off, my scenes do not trigger, etc.

Appreciate any help or advice anyone has!

[alucryd]

I'm in the same situation, I am unable to get my local llama.cpp server to work with this extension.

When Use tools is off, I can see the service calls in the answer:

And when Use tools is on, I only see the answer. In either case, nothing happens though.

Also tried multiple models, not sure what I'm missing. I got the Home LLM extension to work but wanted to give this one a try as it seems I could get more out of it.

[zachfeldman]

I eventually gave up on this library and switched to Home LLM like you said. Worth a try though! Kudos to the maintainer, tough problem to solve.

[jenko1080]

This was a bit tedious to find but it was the LLM simplifying the function call incorrectly.
What was missing was {"list": [ ... ]} wrapping the request.

The fix was to add a simple extra instruction to the prompt template:

When you want to take action, you MUST use the execute_services function. You must wrap your commands in a list array.
Example Correct Format: {"list": [{"domain": "light", "service": "turn_off", "service_data": {"entity_id": "light.desk_1_light"}}]}

This was with Qwen2.5:14b for below:

A bad tool call:

{
    "id": "call_4jwx3re3",
    "function": {
        "arguments": '{"domain": "light", "service": "turn_on", "service_data": {"entity_id": "light.desk_1_light"}}',
        "name": "execute_services",
    },
    "type": "function",
    "index": 0,
},

A good tool call:

{
    "id": "call_051zs7pe",
    "function": {
        "arguments": '{"list": [{"domain": "light", "service": "turn_on", "service_data": {"entity_id": "light.desk_1_light"}}, {"domain": "light", "service": "turn_on", "service_data": {"entity_id": "light.desk_2_light"}}]}',
        "name": "execute_services",
    },
    "type": "function",
    "index": 0,
}

Hope that helps someone else 🎊

[Lzryde]

This was a bit tedious to find but it was the LLM simplifying the function call incorrectly. What was missing was {"list": [ ... ]} wrapping the request.

The fix was to add a simple extra instruction to the prompt template:

When you want to take action, you MUST use the execute_services function. You must wrap your commands in a list array.
Example Correct Format: {"list": [{"domain": "light", "service": "turn_off", "service_data": {"entity_id": "light.desk_1_light"}}]}

This was with Qwen2.5:14b for below:

A bad tool call:

{
    "id": "call_4jwx3re3",
    "function": {
        "arguments": '{"domain": "light", "service": "turn_on", "service_data": {"entity_id": "light.desk_1_light"}}',
        "name": "execute_services",
    },
    "type": "function",
    "index": 0,
},

A good tool call:

{
    "id": "call_051zs7pe",
    "function": {
        "arguments": '{"list": [{"domain": "light", "service": "turn_on", "service_data": {"entity_id": "light.desk_1_light"}}, {"domain": "light", "service": "turn_on", "service_data": {"entity_id": "light.desk_2_light"}}]}',
        "name": "execute_services",
    },
    "type": "function",
    "index": 0,
}

Hope that helps someone else 🎊

Worked for me thank you.