agent-hints (Dify plugin)

OpenAI-compatible model provider for Dify that attaches
agentic inference hints to every model request, enabling hint-aware
inference gateways to do priority routing, KV-cache affinity and capacity
planning.

The contract: X-Agent-Hint (full schema)

Every request carries an HTTP header — the AgentHint object
serialized as UTF-8 JSON and encoded as base64url — with the complete
agreed cross-harness schema:

Field-by-field reference and data-source matrix:
docs/SCHEMA.md. Lifecycle states () are reported on a separate status-event channel,
not inside model requests.

The compatibility projection: nvext.agent_hints (NVIDIA Dynamo)

The same data is additionally projected into the request body as
, compatible with
NVIDIA Dynamo 1.0 agentic inference:

is a content hash of the prompt's static prefix (KV-cache
affinity); is converted from this plugin's "higher = more urgent"
scale to Dynamo's inverted one. Endpoints that ignore unknown fields are
unaffected.

Data sources (low → high precedence)

1. Auto defaults — identity + .
2. Credential — a static partial AgentHint JSON.
3. Node completion params —
   tunable per LLM node in the Dify workflow UI.
4. Hint service (optional) — per-request dynamic enrichment; see below.

A fresh install is fully functional from tiers 1–3 (the scheduling basics +
); the optional hint service enriches the remaining fields
(, , measured statistics, derivation) without any
plugin change. Outages degrade silently to a plain request.

Quick start

1. Install the plugin (Dify → Plugins → GitHub: this repo, or Marketplace, or
   the release asset).
2. Add a model from the Agent Hints provider: set the OpenAI-compatible
   (e.g. a Dynamo frontend) and optionally an .
3. Bind the model in any workflow LLM node and tune the scheduling parameters
   in the node's parameter panel.

Optional hint service

→
with any partial AgentHint JSON (unknown fields ignored, highest merge
precedence). A zero-dependency reference implementation ships in this
repository under :

Producers: your scheduling layer, CI pipelines, or compile-time tooling that
registers per-node metadata keyed by the same prefix hash the plugin derives.
Unreachable or malformed responses count as "no hints this time".

Output switches & privacy

See PRIVACY.md. The plugin is stateless: it contacts only the
configured inference endpoint and (optionally) the configured hint service,
sending scheduling metadata (content hashes, integer estimates) — never
prompt content to the hint service.

Development

Field naming and wire encoding are pinned by a cross-language golden vector;
see docs/SCHEMA.md for the schema contract.