When a Tool Schema Is Visible but the Model Still Calls bash

This post documents a debugging session involving a locally served DeepSeek-V4-Flash checkpoint, SGLang, and a Responses-to-Chat-Completions compatibility proxy used by the Codex ecosystem.

The initial symptom looked like an ordinary tool parser failure: instead of returning a structured tool call, the model sometimes emitted JSON, Markdown code blocks, shell commands, or prose. The deeper investigation showed several distinct failure modes:

1. The tool schemas and DSML instructions were injected correctly.
2. The model sometimes chose not to call a tool at all and fabricated results.
3. In other cases, it emitted syntactically valid DSML but invented tool names such as bash, shell, read, and str_replace_editor.
4. A failed invented tool call, once added to conversation history, became a powerful incorrect in-context example and poisoned subsequent turns.
5. A deliberately redundant prompt footer containing the exact allowed tool names and a concrete exec_command(cmd) example restored the tested shell-tool workflow in both thinking and non-thinking modes.

The final mitigation works, but it is difficult to regard it as anything other than a prompt-level bandage for weak dynamic tool-schema grounding.

System under test

The relevant components were:

• A local DeepSeek-V4-Flash checkpoint described by its model card as mixed precision: MoE expert parameters use FP4, while most other parameters use FP8.
• The expert format is MXFP4 rather than NVFP4.
• SGLang provides the OpenAI-compatible /v1/chat/completions endpoint.
• A compatibility proxy translates Codex/Responses-style requests into Chat Completions messages and tools.
• SGLang's custom encoding_dsv4.py converts the messages and OpenAI tool schemas into DeepSeek's DSML prompt format.

The commercial DeepSeek provider worked with the same proxy. The local SGLang instance did not reliably work before mitigation. This comparison does not isolate checkpoint quality, post-training, quantization, hidden serving prompts, or constrained decoding, but it substantially reduces the likelihood that the proxy alone explains the behavior.

The DSML format

The SGLang encoder instructs the model to invoke tools using blocks of this form:

<｜DSML｜tool_calls>
<｜DSML｜invoke name="$TOOL_NAME">
<｜DSML｜parameter name="$PARAMETER_NAME" string="true|false">$PARAMETER_VALUE</｜DSML｜parameter>
</｜DSML｜invoke>
</｜DSML｜tool_calls>

String parameters use string="true"; numbers, booleans, arrays, and objects use JSON and string="false".

The real command-execution tool supplied by the Codex request was:

{
  "name": "exec_command",
  "description": "Runs a command in a PTY, returning output or a session ID for ongoing interaction.",
  "parameters": {
    "type": "object",
    "properties": {
      "cmd": {
        "type": "string",
        "description": "Shell command to execute."
      },
      "workdir": {
        "type": "string"
      },
      "yield_time_ms": {
        "type": "number"
      },
      "sandbox_permissions": {
        "type": "string",
        "enum": ["use_default", "require_escalated"]
      },
      "justification": {
        "type": "string"
      }
    },
    "required": ["cmd"]
  }
}

The important exact identifiers are therefore:

tool name:      exec_command
required field: cmd

There was no bash tool in the request.

First instrumentation: log the constructed DSV4 prompt

The first patch added best-effort logging at the end of encode_messages() in encoding_dsv4.py:

log_path = f"/tmp/encoding_dsv4_{os.getpid()}.log"

Each entry records:

• The tools received by the encoder.
• The constructed prompt character count.
• The complete prompt after tool injection and message serialization.

This proved that the DSML instructions and schemas existed in the constructed prompt. It also disproved an early hypothesis that the compatibility proxy had silently dropped the tool definitions.

Small requests worked

A small weather request produced a prompt of roughly 1,282 characters and correctly called the supplied get_weather tool. Its prompt contained:

## Tools

You have access to a set of tools...

### Available Tool Schemas

{"description":"Get the weather","name":"get_weather",...}

The model emitted valid DSML, and the call parsed correctly.

This showed that the encoder and parser could work in principle.

Larger Codex prompts failed differently

Codex requests contained a long system instruction, permissions, environment metadata, and 13 verbose tool schemas. Constructed prompts commonly measured approximately 50,000 to 95,000 characters.

Character count initially made the prompts look larger than they were in tokens. Later usage records provided exact measurements around 11,800 to 15,900 prompt tokens. This is nowhere near a 256K context limit and is especially far from a claimed one-million-token context window.

The issue was therefore not simple context exhaustion.

27.flow: the stronger DSML footer was present, but no tool call occurred

An early mitigation appended a footer after all schemas:

You MUST strictly follow the above defined tool name and parameter schemas to invoke tool calls.

When invoking a tool, you MUST output only the DSML format shown above.
Never output a tool invocation as Markdown, a code block, raw JSON, a shell command, or explanatory prose.

In 27.flow this footer was loaded, but a fresh list files in pwd request still produced no tool call:

"tool_calls": null,
"finish_reason": "stop"

The model hallucinated filesystem information or claimed it lacked filesystem access. This indicated that a serialization-only reminder cannot help when the model never makes the tool-use decision.

29.flow: duplicated metadata amplified the failure

29.flow contained:

• 8 messages.
• 13 tools.
• tool_choice: "auto".
• Permissions and environment metadata duplicated three times.
• A constructed prompt of 72,349 characters.

The model reasoned that there was no explicit tool list and fabricated a directory listing, including plausible entries inferred from permission metadata such as .git.

The response ended normally:

"tool_calls": null,
"finish_reason": "stop"

The duplication was a real proxy-side stress amplifier, but it was not the root cause, as the next flow demonstrated.

30.flow: a clean request still hallucinated a tool ontology

30.flow was the decisive clean case before the later bash investigation:

• Only four messages: system, permissions, environment, and user.
• 13 tools.
• tool_choice: "auto".
• No duplicated permissions or environment messages.
• Constructed prompt length: 52,659 characters.

The exec_command schema and DSML footer were present. The user asked:

list files in cwd

Nevertheless, the model reasoned about nonexistent tools named view_source_code and ExecutePty. It chose ExecutePty, but emitted only a Markdown shell block:

ls -la

The response again ended with:

"tool_calls": null,
"finish_reason": "stop"

No parser can recover a DSML call that the model never generated. This moved attention away from parsing and toward model-level schema grounding.

31.flow: the clean first-turn root cause

31.flow captured the first request of a new session before incorrect tool calls could poison the history.

Request structure

The request contained exactly four messages:

0 system
1 user: permissions, collaboration mode, skills
2 user: environment context
3 user: list files in cwd

It supplied these 13 tool names:

exec_command
write_stdin
update_plan
request_user_input
view_image
multi_agent_v1__close_agent
multi_agent_v1__resume_agent
multi_agent_v1__send_input
multi_agent_v1__spawn_agent
multi_agent_v1__wait_agent
get_goal
create_goal
update_goal

The prompt used only 11,843 tokens.

What the model knew

The model's reasoning explicitly recalled actual current tools, including exec_command and the multi-agent tool names. It therefore had not simply forgotten the schema.

It then said:

We can use `bash` tool to run `ls -la`...

What the model emitted

It produced a structured tool call that the gateway parsed as:

{
  "name": "bash",
  "arguments": {
    "command": "ls -la",
    "description": "List all files in current working directory with details"
  }
}

The DSML format itself was successful. The failure was semantic grounding:

requested schema: exec_command(cmd, ...)
generated schema: bash(command, description)

The model did not merely misspell a name. It recalled an entire alternative shell-tool interface.

Was bash hidden in the request?

The complete HTTP request was searched structurally. The result was:

Occurrences of "bash" in the request: 0
Tools whose serialized schema contains "bash": 0

The encoder prompt for the same request was then searched. Before generation:

Occurrences of bash:                  0
Occurrences of "cmd":                2
Occurrences of "command":            1
Occurrences of "description":       70
Occurrences of exec_command:          1
Occurrences of Available Tool Schemas: 1

The only "command" occurrence was inside a Codex instruction example:

Use the `apply_patch` tool to edit files...
{"command":["apply_patch","*** Begin Patch..."]}

This command key is best understood as part of another tool-call envelope, not the business parameter of the DSML exec_command tool. The compatibility proxy placed the Codex instructions into the system message, and the DSV4 encoder preserved them as ordinary text.

That cross-protocol example may have increased interference, but it does not explain the complete invented signature bash(command, description). Neither bash nor that full schema appeared in the prompt. Its most likely source is a tool ontology learned during pretraining or agent-oriented post-training.

The exact training source cannot be proven from output alone.

36.flow: incorrect tool calls poisoned the session

36.flow initially appeared even stranger. Its history contained five assistant tool calls followed by five tool errors:

bash               -> unsupported call: bash
bash               -> unsupported call: bash
shell              -> unsupported call: shell
read               -> unsupported call: read
str_replace_editor -> unsupported call: str_replace_editor

The currently supplied tools still contained only exec_command, write_stdin, and the other valid names listed earlier.

The DSV4 encoder faithfully converted the historical OpenAI assistant tool calls back into DSML. Near the end of the constructed prompt, the model therefore saw multiple syntactically correct DSML examples such as:

<｜DSML｜invoke name="bash">
...
</｜DSML｜invoke>
<｜end▁of▁sentence｜>
<｜User｜><tool_result>unsupported call: bash</tool_result>

The model then reasoned that all tool calls were unsupported and stopped trying. It fabricated or inferred directory contents from environment metadata and suggested that the user manually run a shell command.

36.flow used 15,890 prompt tokens, approximately 6.2% of a 256K context window.

This sequence illustrates a self-poisoning agent loop:

model invents tool
    -> gateway preserves invented call in history
    -> tool runtime returns generic unsupported error
    -> encoder renders invented call as valid DSML ICL
    -> recent incorrect ICL outweighs distant correct schema
    -> model invents more tools or gives up

The later failure was rational relative to the corrupted history. The root cause remained the clean first-turn invention in 31.flow.

Second instrumentation: log the exact scheduler input

Logging at the end of encoding_dsv4.py proves what text the template builder constructed, but it does not prove what the scheduler received after tokenization.

A second hook was added in TokenizerManager, immediately before:

self.send_to_scheduler.send_pyobj(tokenized_obj)

The log path is:

f"/tmp/tokenizer_manager_{os.getpid()}.log"

Each entry records:

• Request ID.
• Exact token count.
• input_text.
• Complete input_ids.
• tokenizer.decode(input_ids, skip_special_tokens=False).

The hook is best-effort and catches exceptions so prompt inspection cannot interrupt request dispatch.

This is the authoritative observation point for the API/tokenizer-manager architecture: the object has already been tokenized, and the next action sends it to the scheduler.

Prompt-synthesis mitigation

The initial footer only insisted on DSML serialization. It did not prevent the model from inserting an invented name into otherwise valid DSML.

The encoder was therefore changed to synthesize a final constraint block dynamically from the actual tools in each request.

For a request containing exec_command, the generated block resembles:

### Final Tool Constraints

Allowed tool names (exactly): exec_command, write_stdin, update_plan, request_user_input, view_image, multi_agent_v1__close_agent, multi_agent_v1__resume_agent, multi_agent_v1__send_input, multi_agent_v1__spawn_agent, multi_agent_v1__wait_agent, get_goal, create_goal, update_goal.

Never invent, rename, abbreviate, or substitute a tool name. Before emitting DSML, verify that every invoke name appears in this exact list and that every parameter name appears in that tool's schema.

For every shell command, use `exec_command` with the `cmd` parameter. Never use `bash`, `shell`, `terminal`, `execute`, or `command_runner` as a tool name, and never substitute `command` for the `cmd` parameter.

Correct example for running `ls -la`:
<｜DSML｜tool_calls>
<｜DSML｜invoke name="exec_command">
<｜DSML｜parameter name="cmd" string="true">ls -la</｜DSML｜parameter>
</｜DSML｜invoke>
</｜DSML｜tool_calls>

The tool list is not hard-coded. It is generated from the current request. The shell-specific example is included only when exec_command exists.

This mitigation is intentionally redundant. The correct schema already appeared a few lines earlier. The redundancy exists solely because the model had demonstrated that seeing and even reasoning about the correct schema was insufficient.

Post-patch validation: thinking mode

37.flow and 38.flow form a successful two-request tool interaction with thinking explicitly enabled.

37.flow request

"chat_template_kwargs": {
  "thinking": true
}

It was a clean four-message request with no prior tool call. The scheduler-input log recorded:

tokens=12087
...
list files in cwd<｜Assistant｜><think>

The model reasoned:

We need to list files in the current working directory. The command is `ls -la` or similar. The user asked "list files in cwd". We can use `exec_command` with `cmd` parameter.

It emitted:

{
  "name": "exec_command",
  "arguments": {
    "cmd": "ls -la"
  }
}

The response ended with:

"finish_reason": "tool_calls"

38.flow continuation

The tool executed successfully and returned the real ls -la output. The second scheduler prompt used 12,546 tokens and contained:

list files in cwd<｜Assistant｜><think>
We need to list files...
</think>
<｜DSML｜tool_calls>
<｜DSML｜invoke name="exec_command">
...
</｜DSML｜tool_calls>
<｜end▁of▁sentence｜>
<｜User｜><tool_result>Chunk ID: ...
...
</tool_result><｜Assistant｜><think>

The model then summarized the actual directory listing and ended with finish_reason: "stop".

This was an end-to-end success: schema selection, DSML generation, parsing, execution, tool-result serialization, and final response.

Post-patch validation: non-thinking mode

1.flow and 2.flow form the corresponding successful non-thinking interaction.

1.flow clean first turn

The request contained:

"chat_template_kwargs": null

It had four messages and no prior ICL. The scheduler-input log recorded exactly 12,087 tokens and ended with:

list files in cwd<｜Assistant｜></think>

The model emitted no reasoning content and directly returned:

{
  "name": "exec_command",
  "arguments": {
    "cmd": "ls -la"
  }
}

The response ended with:

"finish_reason": "tool_calls"

2.flow continuation

The scheduler-input log recorded 12,482 tokens. It contained the historical call and real result:

<｜DSML｜invoke name="exec_command">
...
</｜DSML｜tool_calls>
<｜end▁of▁sentence｜>
<｜User｜><tool_result>Chunk ID: ...
...
</tool_result><｜Assistant｜></think>

The model produced the final answer without reasoning content.

This comparison shows that thinking mode was not necessary for the tested repair:

37/38: thinking=true  -> success
1/2:   thinking off   -> success

The common change was the synthesized final tool constraint and concrete exec_command(cmd) DSML example.

3.flow: switching thinking mode mid-session

3.flow continued the successful non-thinking session but explicitly set:

"chat_template_kwargs": {
  "thinking": true
}

The scheduler-input log recorded 12,553 tokens. The current generation position was:

<｜User｜>good good<｜Assistant｜><think>

The model generated reasoning and then a friendly final response.

The tokenizer log exposed a subtle encoder behavior: changing the current request to thinking mode causes earlier assistant messages without reasoning content to be re-rendered with empty thinking blocks:

<｜Assistant｜><think></think>
<｜DSML｜tool_calls>...

The previous final answer similarly became:

<｜Assistant｜><think></think>The current directory contains: ...

This worked in the short test, but long resumed sessions that switch thinking modes may accumulate synthetic empty thinking blocks. That behavior deserves separate testing.

What the successful patch does and does not prove

The patch clearly fixed the tested list files in cwd workflow. It does not yet prove general tool grounding.

The prompt now gives the model almost the exact desired answer:

For shell commands use exec_command(cmd)
Never use bash
Here is exec_command running ls -la

Success on that same command may demonstrate strong local copying rather than robust schema induction.

A more meaningful evaluation should include:

1. Tools without dedicated positive examples, such as view_image and update_plan.
2. Randomly generated tool names that cannot be recalled from training.
3. Random parameter names.
4. Multiple similar tools requiring semantic selection.
5. Tool schemas placed at different distances from the user request.
6. Fresh sessions versus sessions containing one invalid historical call.
7. Thinking and non-thinking modes.
8. Tool lists of different sizes.

Why asking the model to repeat the tool list in thinking is insufficient

One proposed mitigation was to instruct the model to repeat the allowed tool names in its reasoning before calling a tool.

31.flow already approximated this test naturally: the model correctly named actual supplied tools in its reasoning and then called bash anyway.

This separates declarative recall from action grounding:

The model can state the current schema correctly.
The model can still generate an action from a different learned schema.

Repetition in thinking may consume more tokens without constraining the final action. A compact manifest immediately before the user turn, a positive example, runtime validation, and constrained decoding are more direct interventions.

Gateway defenses

Prompt mitigation should not be the only protection. The gateway can prevent one hallucinated call from corrupting the entire session.

Reject unknown names with corrective information

Instead of returning only:

unsupported call: bash

return:

Invalid tool name "bash".
Allowed tool names: exec_command, write_stdin, update_plan, ...
For shell commands retry with exec_command({"cmd":"..."}).

Do not preserve repeated invalid calls as positive assistant history

An invalid assistant tool call followed by a tool error becomes a valid-looking DSML example after re-encoding. The gateway should consider stripping, replacing, or separately annotating invalid call pairs before the next model request.

Optional alias repair

For a tightly controlled deployment, obvious mappings could be repaired:

bash(command=X)  -> exec_command(cmd=X)
shell(command=X) -> exec_command(cmd=X)

This is risky when names or semantics are ambiguous, so validation and retry are preferable to silent rewriting.

Force a specific tool when the application knows the required action

For benchmark cases such as list files in cwd, setting an exact tool choice can separate tool-selection ability from parameter-generation ability. It is not a general replacement for autonomous selection.

Quantization and checkpoint uncertainty

The public local checkpoint is described as:

FP4 + FP8 Mixed: MoE expert parameters use FP4 precision; most other parameters use FP8.

The code indicates that DSV4 MXFP4 support primarily wraps MoE expert execution. Generic serialized MXFP4 attention is explicitly unsupported in the general MXFP4 quantization class:

Mxfp4 attention layer is not implemented

This suggests a likely layout:

MoE experts:             MXFP4
most attention/dense:    FP8
KV cache:                independently configured

The KV cache can be explicitly requested as:

--kv-cache-dtype fp8_e4m3

That flag does not change model weights or convert MXFP4 experts to FP8.

There is no clean public quantization A/B checkpoint. A separate -base checkpoint is reportedly pure FP8, but is likely not instruction-tuned equivalently. Comparing it with the mixed-precision Flash checkpoint changes at least two variables:

post-training/instruction tuning
quantization

Therefore the observed grounding failure cannot be rigorously attributed to MXFP4 alone.

The commercial API model reportedly carries a W8A8 suffix and works through the same compatibility proxy. That difference is suggestive but still confounded by possible checkpoint revisions, proprietary post-training, hidden tool prompts, validation retries, and constrained decoding.

Final interpretation

The strongest conclusion supported by the data is narrower than “the model cannot do tool calling.”

The local checkpoint can:

• Parse a long Codex-style prompt.
• Recall the current tool list in reasoning.
• Generate syntactically valid DSML.
• Use tools correctly when given a strong, recent, concrete example.
• Complete an end-to-end tool-result conversation in thinking and non-thinking modes.

It can also:

• Ignore a visible, nearby schema.
• Replace it with a complete learned alternative interface.
• Hallucinate filesystem results rather than call a tool.
• Treat recent invalid historical tool calls as stronger ICL than the current schema.

The failure is best described as weak dynamic tool-schema grounding under a long, heterogeneous agent prompt. It is not ordinary context-window exhaustion, and it is not primarily a DSML parser failure.

The successful footer is operationally useful but conceptually embarrassing: after providing a formal schema, the system must repeat the exact tool names, forbid several nonexistent aliases, restate the required parameter, and demonstrate the exact command the model should produce.

That is prompt engineering in its most literal form.

For experimentation, the patch is sufficient. For a fully autonomous security-audit agent, stronger runtime guarantees remain necessary:

• Validate every generated tool name and parameter against the active schema.
• Prevent invalid calls from poisoning history.
• Retry with explicit corrective constraints.
• Prefer constrained decoding where available.
• Benchmark dynamic random-name tools rather than only familiar shell actions.
• Consider using this model as an analysis worker while a more reliable model controls the agent loop.

Raw evidence summary

Capture	Prompt size	Thinking	Outcome
Small weather test	1,282 characters	varies	Correct DSML call
27.flow	not central	not central	No call; hallucinated/declined
29.flow	72,349 characters	not central	No call; duplicated metadata and fabricated listing
30.flow	52,659 characters	thinking visible	Invented ExecutePty; emitted Markdown shell block
31.flow	11,843 tokens	thinking	Valid DSML but invented bash(command, description)
36.flow	15,890 tokens	thinking	Five invalid historical tools caused tool abandonment
37.flow	12,087 tokens	enabled	Correct exec_command({"cmd":"ls -la"})
38.flow	12,546 tokens	enabled	Correct final response from real tool result
1.flow	12,087 tokens	disabled	Correct exec_command({"cmd":"ls -la"})
2.flow	12,482 tokens	disabled	Correct final response from real tool result
3.flow	12,553 tokens	enabled mid-session	Correct response; earlier turns gained empty <think></think> blocks

Files and logs used

Traffic captures:

~/srv/sglang-kt/1.flow
~/srv/sglang-kt/2.flow
~/srv/sglang-kt/3.flow
~/srv/sglang-kt/23.flow
~/srv/sglang-kt/25.flow
~/srv/sglang-kt/27.flow
~/srv/sglang-kt/29.flow
~/srv/sglang-kt/30.flow
~/srv/sglang-kt/31.flow
~/srv/sglang-kt/36.flow
~/srv/sglang-kt/37.flow
~/srv/sglang-kt/38.flow

Constructed prompt logs:

/tmp/encoding_dsv4_<pid>.log

Final tokenized scheduler-input logs:

/tmp/tokenizer_manager_<pid>.log

Patched encoder:

~/srv/sglang-kt/.venv/lib/python3.12/site-packages/sglang/srt/entrypoints/openai/encoding_dsv4.py

Patched scheduler-dispatch observation point:

~/srv/sglang-kt/.venv/lib/python3.12/site-packages/sglang/srt/managers/tokenizer_manager.py