Chat Template error "Trying to access property '0' on null! at row 74"
#2
by espadrine - opened
I got the following error while running Devstral. My setup:
docker run --gpus all -v /data/ml/models/gguf:/models -p 11432:8080 ghcr.io/ggml-org/llama.cpp:full-cuda -s --host 0.0.0.0 -m "/models/Devstral-Small-2507-Q5_K_M.gguf" --jinja -ub 2048 -b 2048 --n-gpu-layers 99 -fa -c 36864OPENAI_BASE_URL=http://127.1:11432 qwenusing Qwen code.
My error:
✕ [API Error: 500 Trying to access property '0' on null! at row 74, column 16:
{%- else %}
{{- message['content'][0]['text'] }}
^
{%- endif %}
at row 74, column 16:
{%- else %}
{{- message['content'][0]['text'] }}
^
{%- endif %}
at row 74, column 13:
{%- else %}
{{- message['content'][0]['text'] }}
^
{%- endif %}
at row 73, column 20:
{{- message['content'] }}
{%- else %}
^
{{- message['content'][0]['text'] }}
at row 71, column 9:
{%- elif message['role'] == 'assistant' %}
{%- if message['content'] is string %}
^
{{- message['content'] }}
at row 70, column 47:
{%- elif message['role'] == 'assistant' %}
^
{%- if message['content'] is string %}
at row 37, column 5:
{%- for message in loop_messages %}
{%- if message['role'] == 'user' %}
^
at row 36, column 36:
{%- for message in loop_messages %}
^
{%- if message['role'] == 'user' %}
at row 36, column 1:
{%- for message in loop_messages %}
^
{%- if message['role'] == 'user' %}
at row 1, column 69:
{#- Copyright 2025-present the Unsloth team. All rights reserved. #}
^
{#- Licensed under the Apache License, Version 2.0 (the "License") #}
]
Just hit this same error from llama.cpp inside llama-swap.
Same issue for us, also trying to run via llama.cpp using the --jinja flag. It would be great to get an official fix or suggestions, because doctoring the chat template to make it work seems risky in the long run without a complete understanding of what everything does.