When a Tool Schema Is Visible but the Model Still Calls bash

I debugged tool calling across a locally served DeepSeek-V4-Flash checkpoint, SGLang, and the Responses-to-Chat-Completions compatibility proxy used by the Codex ecosystem.

At first, this looked like an ordinary parser failure. Instead of a structured tool call, the model sometimes emitted JSON, a Markdown code block, a shell command, or prose. The parser was not the common cause. The tool schemas and DSML instructions reached the model correctly, but it sometimes skipped the call and fabricated results. On other requests, it generated valid DSML around invented tools such as bash, shell, read, and str_replace_editor.

Once an invented call entered the conversation history, it became a strong but incorrect in-context example for later turns. A deliberately redundant prompt footer, containing the exact allowed names and a concrete exec_command(cmd) example, restored the tested shell workflow in thinking and non-thinking modes.

The mitigation worked, though it remained a prompt-level patch for weak dynamic tool-schema grounding.

System under test

The test setup had these components:

  • A local DeepSeek-V4-Flash checkpoint described by its model card as mixed precision: MoE expert parameters use FP4, while most other parameters use FP8.
  • The expert format is MXFP4 rather than NVFP4.
  • SGLang provides the OpenAI-compatible /v1/chat/completions endpoint.
  • A compatibility proxy translates Codex/Responses-style requests into Chat Completions messages and tools.
  • SGLang's custom encoding_dsv4.py converts the messages and OpenAI tool schemas into DeepSeek's DSML prompt format.

The commercial DeepSeek provider worked with the same proxy, while the local SGLang instance was unreliable before mitigation. The comparison does not isolate checkpoint quality, post-training, quantization, hidden serving prompts, or constrained decoding. It does make the proxy a less likely sole cause.

The DSML format

The SGLang encoder instructs the model to invoke tools using blocks of this form:

<｜DSML｜tool_calls>
<｜DSML｜invoke name="$TOOL_NAME">
<｜DSML｜parameter name="$PARAMETER_NAME" string="true|false">$PARAMETER_VALUE</｜DSML｜parameter>
</｜DSML｜invoke>
</｜DSML｜tool_calls>

String parameters use string="true"; numbers, booleans, arrays, and objects use JSON and string="false".

The real command-execution tool supplied by the Codex request was:

{
  "name": "exec_command",
  "description": "Runs a command in a PTY, returning output or a session ID for ongoing interaction.",
  "parameters": {
    "type": "object",
    "properties": {
      "cmd": {
        "type": "string",
        "description": "Shell command to execute."
      },
      "workdir": {
        "type": "string"
      },
      "yield_time_ms": {
        "type": "number"
      },
      "sandbox_permissions": {
        "type": "string",
        "enum": ["use_default", "require_escalated"]
      },
      "justification": {
        "type": "string"
      }
    },
    "required": ["cmd"]
  }
}

The exact identifiers were:

tool name:      exec_command
required field: cmd

The request contained no bash tool.

First instrumentation: log the constructed DSV4 prompt

The first patch added best-effort logging at the end of encode_messages() in encoding_dsv4.py:

log_path = f"/tmp/encoding_dsv4_{os.getpid()}.log"

Each entry records:

  • The tools received by the encoder.
  • The constructed prompt character count.
  • The complete prompt after tool injection and message serialization.

The log confirmed that the constructed prompt contained the DSML instructions and schemas. The compatibility proxy had not silently dropped the tool definitions.

Small requests worked

A small weather request produced a prompt of roughly 1,282 characters and correctly called the supplied get_weather tool. Its prompt contained:

## Tools

You have access to a set of tools...

### Available Tool Schemas

{"description":"Get the weather","name":"get_weather",...}

The model emitted valid DSML, and the call parsed correctly.

The encoder and parser therefore worked on at least a small request.

Larger Codex prompts failed differently

Codex requests contained a long system instruction, permissions, environment metadata, and 13 verbose tool schemas. Constructed prompts commonly measured approximately 50,000 to 95,000 characters.

Character counts made these prompts look larger than their tokenized form. Later usage records measured about 11,800 to 15,900 prompt tokens, far below a 256K context limit and a claimed one-million-token window.

Simple context exhaustion did not fit the measurements.

An early mitigation appended a footer after all schemas:

You MUST strictly follow the above defined tool name and parameter schemas to invoke tool calls.

When invoking a tool, you MUST output only the DSML format shown above.
Never output a tool invocation as Markdown, a code block, raw JSON, a shell command, or explanatory prose.

27.flow included this footer, yet a fresh list files in pwd request produced no tool call:

"tool_calls": null,
"finish_reason": "stop"

The model hallucinated filesystem information or claimed it lacked filesystem access. A serialization reminder could not help when the model never decided to use a tool.

29.flow: duplicated metadata amplified the failure

29.flow contained:

  • 8 messages.
  • 13 tools.
  • tool_choice: "auto".
  • Permissions and environment metadata duplicated three times.
  • A constructed prompt of 72,349 characters.

The model reasoned that there was no explicit tool list and fabricated a directory listing, including plausible entries inferred from permission metadata such as .git.

The response ended normally:

"tool_calls": null,
"finish_reason": "stop"

The duplication added proxy-side stress, but the next flow ruled it out as the root cause.

30.flow: a clean request still hallucinated a tool ontology

30.flow provided a clean case before the later bash investigation:

  • Only four messages: system, permissions, environment, and user.
  • 13 tools.
  • tool_choice: "auto".
  • No duplicated permissions or environment messages.
  • Constructed prompt length: 52,659 characters.

The exec_command schema and DSML footer were present. The user asked:

list files in cwd

The model still reasoned about nonexistent tools named view_source_code and ExecutePty. It chose ExecutePty but emitted only a Markdown shell block:

ls -la

The response again ended with:

"tool_calls": null,
"finish_reason": "stop"

A parser cannot recover DSML that was never generated. The failure occurred in model-level schema grounding.

31.flow: the clean first-turn root cause

31.flow captured the first request of a new session, before incorrect calls entered the history.

Request structure

The request contained exactly four messages:

0 system
1 user: permissions, collaboration mode, skills
2 user: environment context
3 user: list files in cwd

It supplied these 13 tool names:

exec_command
write_stdin
update_plan
request_user_input
view_image
multi_agent_v1__close_agent
multi_agent_v1__resume_agent
multi_agent_v1__send_input
multi_agent_v1__spawn_agent
multi_agent_v1__wait_agent
get_goal
create_goal
update_goal

The prompt used only 11,843 tokens.

What the model knew

The model's reasoning explicitly recalled current tools, including exec_command and the multi-agent names. It had not simply forgotten the schema.

It then said:

We can use `bash` tool to run `ls -la`...

What the model emitted

It produced a structured tool call that the gateway parsed as:

{
  "name": "bash",
  "arguments": {
    "command": "ls -la",
    "description": "List all files in current working directory with details"
  }
}

The DSML syntax was valid. The failure was semantic grounding:

requested schema: exec_command(cmd, ...)
generated schema: bash(command, description)

This was more than a misspelling. The model produced an entire alternative shell-tool interface.

Was bash hidden in the request?

The complete HTTP request was searched structurally. The result was:

Occurrences of "bash" in the request: 0
Tools whose serialized schema contains "bash": 0

The encoder prompt for the same request was then searched. Before generation:

Occurrences of bash:                  0
Occurrences of "cmd":                2
Occurrences of "command":            1
Occurrences of "description":       70
Occurrences of exec_command:          1
Occurrences of Available Tool Schemas: 1

The only "command" occurrence was inside a Codex instruction example:

Use the `apply_patch` tool to edit files...
{"command":["apply_patch","*** Begin Patch..."]}

This command key belonged to another tool-call envelope, not the business parameter of the DSML exec_command tool. The compatibility proxy placed the Codex instructions in the system message, and the DSV4 encoder preserved them as ordinary text.

That cross-protocol example may have increased interference, but it cannot explain the complete invented signature bash(command, description). Neither bash nor that full schema appeared in the prompt. A tool ontology learned during pretraining or agent-oriented post-training is the most likely source.

The exact training source cannot be proven from output alone.

36.flow: incorrect tool calls poisoned the session

36.flow contained five assistant tool calls followed by five tool errors:

bash               -> unsupported call: bash
bash               -> unsupported call: bash
shell              -> unsupported call: shell
read               -> unsupported call: read
str_replace_editor -> unsupported call: str_replace_editor

The currently supplied tools still contained only exec_command, write_stdin, and the other valid names listed earlier.

The DSV4 encoder converted the historical OpenAI assistant calls back into DSML. Near the end of the constructed prompt, the model saw several syntactically correct examples such as:

<｜DSML｜invoke name="bash">
...
</｜DSML｜invoke>
<｜end▁of▁sentence｜>
<｜User｜><tool_result>unsupported call: bash</tool_result>

The model concluded that all tool calls were unsupported and stopped trying. It fabricated or inferred directory contents from environment metadata, then suggested that the user run a shell command manually.

36.flow used 15,890 prompt tokens, approximately 6.2% of a 256K context window.

The session poisoned itself through this loop:

model invents tool
    -> gateway preserves invented call in history
    -> tool runtime returns generic unsupported error
    -> encoder renders invented call as valid DSML ICL
    -> recent incorrect ICL outweighs distant correct schema
    -> model invents more tools or gives up

Given the corrupted history, the later failure was understandable. The clean first-turn invention in 31.flow remained the root cause.

Second instrumentation: log the exact scheduler input

The encoding_dsv4.py log captured the template builder's text, not the scheduler input after tokenization.

A second hook was added in TokenizerManager, immediately before:

self.send_to_scheduler.send_pyobj(tokenized_obj)

The log path is:

f"/tmp/tokenizer_manager_{os.getpid()}.log"

Each entry records:

  • Request ID.
  • Exact token count.
  • input_text.
  • Complete input_ids.
  • tokenizer.decode(input_ids, skip_special_tokens=False).

The hook catches exceptions so prompt inspection cannot interrupt request dispatch.

At this observation point, the object has already been tokenized and the next action sends it to the scheduler. It is therefore the authoritative input for this API/tokenizer-manager architecture.

Prompt-synthesis mitigation

The initial footer enforced DSML serialization but did not stop the model from placing an invented name inside valid DSML.

I changed the encoder to synthesize a final constraint block from the tools in each request.

For a request containing exec_command, the generated block resembles:

### Final Tool Constraints

Allowed tool names (exactly): exec_command, write_stdin, update_plan, request_user_input, view_image, multi_agent_v1__close_agent, multi_agent_v1__resume_agent, multi_agent_v1__send_input, multi_agent_v1__spawn_agent, multi_agent_v1__wait_agent, get_goal, create_goal, update_goal.

Never invent, rename, abbreviate, or substitute a tool name. Before emitting DSML, verify that every invoke name appears in this exact list and that every parameter name appears in that tool's schema.

For every shell command, use `exec_command` with the `cmd` parameter. Never use `bash`, `shell`, `terminal`, `execute`, or `command_runner` as a tool name, and never substitute `command` for the `cmd` parameter.

Correct example for running `ls -la`:
<｜DSML｜tool_calls>
<｜DSML｜invoke name="exec_command">
<｜DSML｜parameter name="cmd" string="true">ls -la</｜DSML｜parameter>
</｜DSML｜invoke>
</｜DSML｜tool_calls>

The current request determines the tool list. The encoder adds the shell-specific example only when exec_command exists.

This repeats a schema that already appeared a few lines earlier. The repetition was necessary in this test because seeing and even reasoning about the correct schema had not constrained the model's action.

Post-patch validation: thinking mode

37.flow and 38.flow form a successful two-request tool interaction with thinking explicitly enabled.

37.flow request

"chat_template_kwargs": {
  "thinking": true
}

This clean four-message request had no prior tool call. The scheduler-input log recorded:

tokens=12087
...
list files in cwd<｜Assistant｜><think>

The model reasoned:

We need to list files in the current working directory. The command is `ls -la` or similar. The user asked "list files in cwd". We can use `exec_command` with `cmd` parameter.

It emitted:

{
  "name": "exec_command",
  "arguments": {
    "cmd": "ls -la"
  }
}

The response ended with:

"finish_reason": "tool_calls"

38.flow continuation

The tool executed successfully and returned the real ls -la output. The second scheduler prompt used 12,546 tokens and contained:

list files in cwd<｜Assistant｜><think>
We need to list files...
</think>
<｜DSML｜tool_calls>
<｜DSML｜invoke name="exec_command">
...
</｜DSML｜tool_calls>
<｜end▁of▁sentence｜>
<｜User｜><tool_result>Chunk ID: ...
...
</tool_result><｜Assistant｜><think>

The model then summarized the actual directory listing and ended with finish_reason: "stop".

The full interaction succeeded: schema selection, DSML generation, parsing, execution, tool-result serialization, and final response.

Post-patch validation: non-thinking mode

1.flow and 2.flow captured the corresponding non-thinking interaction.

1.flow clean first turn

The request contained:

"chat_template_kwargs": null

It had four messages and no prior ICL. The scheduler-input log recorded exactly 12,087 tokens and ended with:

list files in cwd<｜Assistant｜></think>

The model emitted no reasoning content and directly returned:

{
  "name": "exec_command",
  "arguments": {
    "cmd": "ls -la"
  }
}

The response ended with:

"finish_reason": "tool_calls"

2.flow continuation

The scheduler-input log recorded 12,482 tokens. It contained the historical call and real result:

<｜DSML｜invoke name="exec_command">
...
</｜DSML｜tool_calls>
<｜end▁of▁sentence｜>
<｜User｜><tool_result>Chunk ID: ...
...
</tool_result><｜Assistant｜></think>

The model produced the final answer without reasoning content.

The tested repair did not depend on thinking mode:

37/38: thinking=true  -> success
1/2:   thinking off   -> success

The common change was the synthesized final tool constraint and concrete exec_command(cmd) DSML example.

3.flow: switching thinking mode mid-session

3.flow continued the successful non-thinking session but explicitly set:

"chat_template_kwargs": {
  "thinking": true
}

The scheduler-input log recorded 12,553 tokens. The current generation position was:

<｜User｜>good good<｜Assistant｜><think>

The model generated reasoning and then a friendly final response.

The tokenizer log also exposed an encoder behavior. Switching the current request to thinking mode re-rendered earlier assistant messages without reasoning content as empty thinking blocks:

<｜Assistant｜><think></think>
<｜DSML｜tool_calls>...

The previous final answer similarly became:

<｜Assistant｜><think></think>The current directory contains: ...

The short test still worked. Long resumed sessions that switch thinking modes may accumulate these synthetic blocks, which needs separate testing.

What the successful patch does and does not prove

The patch fixed the tested list files in cwd workflow, not general tool grounding.

The prompt now gives the model almost the exact desired answer:

For shell commands use exec_command(cmd)
Never use bash
Here is exec_command running ls -la

Success on the same command may reflect strong local copying rather than robust schema induction.

A broader evaluation should include:

  1. Tools without dedicated positive examples, such as view_image and update_plan.
  2. Randomly generated tool names that cannot be recalled from training.
  3. Random parameter names.
  4. Multiple similar tools requiring semantic selection.
  5. Tool schemas placed at different distances from the user request.
  6. Fresh sessions versus sessions containing one invalid historical call.
  7. Thinking and non-thinking modes.
  8. Tool lists of different sizes.

Why asking the model to repeat the tool list in thinking is insufficient

One proposed mitigation asked the model to repeat the allowed tool names in its reasoning before making a call.

31.flow approximated this test naturally. The model named the supplied tools correctly in its reasoning, then called bash anyway.

This separates declarative recall from action grounding:

The model can state the current schema correctly.
The model can still generate an action from a different learned schema.

Repeating the list in thinking may consume tokens without constraining the action. More direct controls include a compact manifest immediately before the user turn, a positive example, runtime validation, and constrained decoding.

Gateway defenses

The gateway should also prevent one hallucinated call from corrupting the session.

Reject unknown names with corrective information

Instead of returning only:

unsupported call: bash

return:

Invalid tool name "bash".
Allowed tool names: exec_command, write_stdin, update_plan, ...
For shell commands retry with exec_command({"cmd":"..."}).

Do not preserve repeated invalid calls as positive assistant history

An invalid assistant tool call followed by a tool error becomes a valid-looking DSML example after re-encoding. The gateway should consider stripping, replacing, or separately annotating invalid call pairs before the next model request.

Optional alias repair

For a tightly controlled deployment, obvious mappings could be repaired:

bash(command=X)  -> exec_command(cmd=X)
shell(command=X) -> exec_command(cmd=X)

Because names or semantics can be ambiguous, validation and retry are safer than silent rewriting.

Force a specific tool when the application knows the required action

For benchmark cases such as list files in cwd, an exact tool choice separates tool selection from parameter generation. It does not replace autonomous selection in general use.

Quantization and checkpoint uncertainty

The public local checkpoint is described as:

FP4 + FP8 Mixed: MoE expert parameters use FP4 precision; most other parameters use FP8.

The code indicates that DSV4 MXFP4 support primarily wraps MoE expert execution. Generic serialized MXFP4 attention is explicitly unsupported in the general MXFP4 quantization class:

Mxfp4 attention layer is not implemented

The likely layout is:

MoE experts:             MXFP4
most attention/dense:    FP8
KV cache:                independently configured

The KV cache can be explicitly requested as:

--kv-cache-dtype fp8_e4m3

That flag does not change model weights or convert MXFP4 experts to FP8.

There is no clean public quantization A/B checkpoint. A separate -base checkpoint is reportedly pure FP8, but is likely not instruction-tuned equivalently. Comparing it with the mixed-precision Flash checkpoint changes at least two variables:

post-training/instruction tuning
quantization

The observed grounding failure therefore cannot be attributed rigorously to MXFP4 alone.

The commercial API model reportedly carries a W8A8 suffix and works through the same compatibility proxy. Possible checkpoint revisions, proprietary post-training, hidden tool prompts, validation retries, and constrained decoding still confound that comparison.

Final interpretation

The evidence does not support the broad claim that "the model cannot do tool calling."

The local checkpoint can:

  • Parse a long Codex-style prompt.
  • Recall the current tool list in reasoning.
  • Generate syntactically valid DSML.
  • Use tools correctly when given a strong, recent, concrete example.
  • Complete an end-to-end tool-result conversation in thinking and non-thinking modes.

It can also:

  • Ignore a visible, nearby schema.
  • Replace it with a complete learned alternative interface.
  • Hallucinate filesystem results rather than call a tool.
  • Treat recent invalid historical tool calls as stronger ICL than the current schema.

The observed failure is weak dynamic tool-schema grounding under a long, heterogeneous agent prompt. The token counts rule out ordinary context-window exhaustion, and the valid DSML examples rule out the parser as the primary failure.

The footer is operationally useful but awkward. After providing a formal schema, the system repeats the exact tool names, forbids several nonexistent aliases, restates the required parameter, and demonstrates the exact command the model should produce.

The patch is sufficient for experimentation. A fully autonomous security-audit agent still needs stronger runtime guarantees:

  • Validate every generated tool name and parameter against the active schema.
  • Prevent invalid calls from poisoning history.
  • Retry with explicit corrective constraints.
  • Prefer constrained decoding where available.
  • Benchmark dynamic random-name tools rather than only familiar shell actions.
  • Consider using this model as an analysis worker while a more reliable model controls the agent loop.

Raw evidence summary

Capture Prompt size Thinking Outcome
Small weather test 1,282 characters varies Correct DSML call
27.flow not central not central No call; hallucinated/declined
29.flow 72,349 characters not central No call; duplicated metadata and fabricated listing
30.flow 52,659 characters thinking visible Invented ExecutePty; emitted Markdown shell block
31.flow 11,843 tokens thinking Valid DSML but invented bash(command, description)
36.flow 15,890 tokens thinking Five invalid historical tools caused tool abandonment
37.flow 12,087 tokens enabled Correct exec_command({"cmd":"ls -la"})
38.flow 12,546 tokens enabled Correct final response from real tool result
1.flow 12,087 tokens disabled Correct exec_command({"cmd":"ls -la"})
2.flow 12,482 tokens disabled Correct final response from real tool result
3.flow 12,553 tokens enabled mid-session Correct response; earlier turns gained empty <think></think> blocks

Files and logs used

Traffic captures:

~/srv/sglang-kt/1.flow
~/srv/sglang-kt/2.flow
~/srv/sglang-kt/3.flow
~/srv/sglang-kt/23.flow
~/srv/sglang-kt/25.flow
~/srv/sglang-kt/27.flow
~/srv/sglang-kt/29.flow
~/srv/sglang-kt/30.flow
~/srv/sglang-kt/31.flow
~/srv/sglang-kt/36.flow
~/srv/sglang-kt/37.flow
~/srv/sglang-kt/38.flow

Constructed prompt logs:

/tmp/encoding_dsv4_<pid>.log

Final tokenized scheduler-input logs:

/tmp/tokenizer_manager_<pid>.log

Patched encoder:

~/srv/sglang-kt/.venv/lib/python3.12/site-packages/sglang/srt/entrypoints/openai/encoding_dsv4.py

Patched scheduler-dispatch observation point:

~/srv/sglang-kt/.venv/lib/python3.12/site-packages/sglang/srt/managers/tokenizer_manager.py
An unhandled error has occurred. Reload ✕

Rejoining the server...

Rejoin failed... trying again in seconds.

Failed to rejoin.
Please retry or reload the page.

The session has been paused by the server.

Failed to resume the session.
Please retry or reload the page.