thinwrap_llm (Erlang)

Unified, SDK-free, zero-dependency Erlang wrapper for chat completions (with streaming) and embeddings across 18 LLM providers. Switch vendor by changing the config's provider + the input's model; the input and output shapes stay identical. Synchronous, stateless, OTP-only — bring your own HTTP client if you want hackney, gun or Finch. Works the same from Elixir and Gleam.

%% rebar.config
{deps, [thinwrap_llm]}.

# mix.exs
{:thinwrap_llm, "~> 1.1"}

Requires Erlang/OTP ≥ 27. No runtime dependencies — Bedrock's SigV4 is hand-rolled on crypto, SSE is parsed by the library, JSON is OTP json.

Two-minute chat

Config = #{provider => openai, api_key => <<"...">>},
case thinwrap_llm:complete(Config, #{
    model => <<"gpt-4o-mini">>,
    messages => [#{role => user, content => <<"Say hi in one word.">>}]
}) of
    {ok, #{message := #{content := Text}, usage := #{total_tokens := N}}} ->
        io:format("~ts (~B tokens)~n", [Text, N]);
    {error, #{code := Code, message := Msg}} ->
        io:format("~s: ~ts~n", [Code, Msg])
end.

{:ok, %{message: %{content: text}}} =
  :thinwrap_llm.complete(%{provider: :anthropic, api_key: key}, %{
    model: "claude-opus-5-5",
    messages: [%{role: :user, content: "Say hi in one word."}]
  })

Streaming

stream/4 folds your fun over the normalized deltas in the calling process as they arrive and leaves nothing in your mailbox. The library starts no long-lived processes; the default httpc transport runs each streaming call through one helper process that ends with the call (a transport of your own is unaffected):

{ok, _} = thinwrap_llm:stream(Config, Input,
    fun(#{content_delta := Chunk}, Acc) -> io:put_chars(Chunk), Acc;
       (_Delta, Acc) -> Acc
    end, ok).

A stream that is cut off before the provider's terminal event, or a 2xx answer that is not text/event-stream, is {error, #{code := provider_unavailable}} rather than ending as if it were complete; deltas already passed to your fun stay delivered. If your fun raises, the request is cancelled and the exception propagates unchanged.

Usage

usage counts every billed token: input_tokens includes cache reads and writes (Anthropic, Bedrock), output_tokens includes thinking tokens (Gemini), and total_tokens is the vendor's total, or input + output when it reports none. When streaming, usage arrives once, on the terminal delta.

Tool loops and `provider_metadata`

Some providers need opaque state from a response sent back on the next turn: Gemini 3 rejects a replayed function call without its thought signature, and Anthropic keeps its thinking blocks only if they are replayed verbatim. provider_metadata on each tool call and on the assistant message carries that state, keyed by provider (#{<<"gemini">> => ...}, #{<<"anthropic">> => ...}). Treat it as opaque and copy it into the next request together with content and tool_calls:

{ok, #{message := Msg}} = thinwrap_llm:complete(Config, Input#{tools => Tools}),
Next = Messages ++ [maps:with([role, content, tool_calls, provider_metadata], Msg)].

When streaming, the state rides on tool_call_delta's provider_metadata (copy it onto the tool call you assemble) and on the terminal delta. A connector reads only its own key, so a history survives a provider switch.

Providers

The config's provider selects the connector; inputs and results are identical across all of them. thinwrap_llm:supports(Provider, complete | stream | embed) answers at runtime.

Chat (18): 15 OpenAI-compatible — openai, 'azure-openai', openrouter, groq, together, fireworks, deepseek, xai, mistral, perplexity, deepinfra, cloudflare, vllm, ollama, lmstudio (config: api_key, base_url — required for 'azure-openai' and cloudflare — and headers); plus 3 natives: anthropic (api_key), bedrock (region, access_key_id, secret_access_key, optional session_token), gemini (api_key).

Embeddings (11): openai, 'azure-openai', openrouter, together, fireworks, mistral, deepinfra, cloudflare, vllm, ollama, lmstudio.

{ok, #{embeddings := Vectors}} =
    thinwrap_llm:embed(#{provider => openai, api_key => Key},
                       #{model => <<"text-embedding-3-small">>, input => [<<"hello">>, <<"world">>]}).
%% Vectors is a list of float lists, in input order.

Any credential may be a binary or a zero-arity fun returning one, which keeps the secret out of crash reports that print the config. Per-provider details live in docs/providers/.

The ≥90% baseline & passthrough

Only fields ≥90% of providers support normalizably are first-class on the input and result; everything else rides through passthrough (request) / raw (response) and is never emulated:

{ok, #{raw := Raw}} = thinwrap_llm:complete(Config, Input#{
    passthrough => #{body => #{logprobs => true}}
}).

Errors

Every operation failure is {error, Error}, a plain map #{code, message, status?, provider_message?, cause?}. code is one of rate_limited, auth_failed, provider_unavailable, invalid_request, context_length_exceeded, content_filtered, unknown. The raw Retry-After rides in cause as <<"retryAfter">> (+ parsed <<"retryAfterSeconds">>) — there is no top-level retry key.

By HTTP status: 401/403 → auth_failed; 402 (out of credit) / 429 → rate_limited; 408 / 5xx → provider_unavailable; every other 4xx → invalid_request, unless the vendor's context-overflow or content-filter signal matches. An error inside a 2xx body or mid-stream keeps the HTTP status it arrived with and is classified by its own code or type.

Misuse — an unknown or missing config/input key, a wrong type, an atom outside its set, a config for another provider, embeddings on a chat-only provider — raises error({badarg, Description}) before any request. The description never includes config values.

Bring your own transport

Config = #{provider => openai, api_key => Key, transport => {my_transport, Opts}}.

my_transport implements the thinwrap_llm_transport behaviour: request/2 for one request/response, and optionally stream/4 for true incremental streaming (without it a buffered fallback is used). The default, thinwrap_llm_httpc, does both on OTP httpc, never follows redirects, names itself in User-Agent, and is HTTPS-only: the http:// defaults of local servers (vLLM, Ollama, LM Studio) get an invalid_request telling you to bring your own transport. Its timeout is 600 s (whole request; idle time between chunks when streaming), so a long non-streaming reasoning call is not cut off; pass {thinwrap_llm_httpc, #{timeout => Ms}} for a different budget. The wrapper is stateless — no caching, retries, or conversation memory.

Relationship to the siblings

One of five thinwrap llm libraries sharing a normalized surface and byte-identical provider ids / error codes: TypeScript (@thinwrap/llm), PHP (thinwrap/llm), Go (thinwrap/llm-go), Python (thinwrap-llm), Erlang (this package).

Security

Report vulnerabilities privately — please do not open a public issue. Preferred: a private security advisory on this repository. Alternatively, email security@thinwrap.dev. Include the affected versions and a minimal reproduction if you have one.

A vulnerability in a provider's own API or service belongs to that vendor rather than to this wrapper — please report those upstream.

License

Next Page → Changelog