LiteLLM

Overview

Use a LiteLLM Proxy as the model provider for Dify. The proxy gives one OpenAI-compatible endpoint for 100+ model providers, with virtual keys, spend tracking, budgets, and fallbacks.

The plugin adds your proxy models to Dify as customizable models. It supports:

LLM (chat and completion), including streaming and tool calls
Text embedding

This plugin is for the LiteLLM Proxy server. It does not use the LiteLLM Python library.

Configure

Start a LiteLLM Proxy. See the quick start. The default address is .
Optional: create a virtual key for Dify.
Install LiteLLM from the Dify Marketplace.
Open Settings → Model Provider → LiteLLM and select Add Model.
Choose the model type and enter these values:
- Model Name: the alias from your proxy config, for example for an LLM or for an embedding model.
- LiteLLM Proxy URL: for example . A trailing is optional.
- API Key: a virtual key or the master key. Leave it empty if the proxy has no authentication.

Example proxy config:

The model names come from your own proxy config. The plugin does not include a model list. For more about the proxy, see LiteLLM Proxy as a provider.

If Dify runs in Docker, points to the Dify container. Use or the address of the proxy host.

Usage

Set Function Call Type to Tool Call for models that support tools. Agent nodes and tool calling need this setting.
Set Vision Support for models that accept images.
Set Model context size and Upper bound for max tokens to the limits of the model.
For embedding models, set Model max chunks per batch to the batch size that the model accepts.

Credential validation sends a small request () to or . The model must exist in the proxy config for the check to pass.

When a model answers with tool calls only, the proxy can leave out . The plugin treats it as an empty string.

Support

Open an issue in dify-official-plugins and include the Dify version, the plugin version, and the model alias. Never include API keys or prompt content.

License

Apache License 2.0. See the repository LICENSE.