thinwrap_llm_transport behaviour (thinwrap_llm v1.1.0)
View SourceThe bring-your-own HTTP seam.
The library never depends on a third-party HTTP client. It defines this
behaviour and ships thinwrap_llm_httpc (OTP httpc) as the default. To use
hackney, gun, Finch, … implement the behaviour and pass it in the
config:
Config = #{provider => openai, api_key => Key,
transport => {my_hackney_transport, #{pool => llm}}}.request/2 (required)
Sends one request and returns one buffered response. Rules:
- A non-2xx answer is a response, never an error: return
{ok, #{status := 429, ...}}. The connector owns status-to-error mapping. - Any
{error, Reason}is reported asprovider_unavailable. Only the shape of the reason (its atom, or the atom tag of a tuple) is kept; the reason is never returned or printed, because a client's error may embed the request URL and its credentials. {error, {connector_error, Error}}surfaces a ready-madethinwrap_llm:connector_error/0unchanged (the default transport uses it to refuse non-httpsURLs).- Never follow redirects: a 3xx must come back as a response, or the provider's credentials would be re-sent to wherever it points.
stream/4 (optional)
Sends one request and delivers the response incrementally, in the calling
process, by folding Fun over these events:
{status, Status, Headers}— exactly once, first (headers lower-cased);{data, Chunk}— zero or more raw body chunks, split anywhere.
Fun(Event, Acc) returns {cont, Acc1} to keep reading or {halt, Acc1}
to stop (the provider's terminal event arrived); stop reading and release the
connection either way. Return {ok, AccFinal} once the body ended or the fold
halted, and {error, Reason, AccSoFar} when the request or a read failed
(the accumulator lets the connector still classify a non-2xx whose body read
broke off). Fun may raise; release the connection and let it propagate. The
non-2xx and redirect rules above apply unchanged: a non-2xx arrives as a status
event plus its body.
A transport without stream/4 still streams: the connector sends the request
through request/2 and parses the buffered body, which is functionally the
same but not token-by-token.
Summary
Types
-type method() :: get | post.
-type stream_fun() :: fun((stream_event(), Acc :: term()) -> {cont, term()} | {halt, term()}).