thinwrap_llm_httpc (thinwrap_llm v1.1.0)
View SourceDefault transport, built on OTP httpc. Implements both request/2
(buffered) and stream/4 (incremental, used by thinwrap_llm:stream/4).
Options (all optional), passed as transport => {thinwrap_llm_httpc, Opts}:
timeout— milliseconds (default600000, ten minutes). Forrequest/2it is the whole-request budget: a non-streaming reasoning call sends nothing until the model has finished, so a shorter default would cut long answers off (600 s matches the official OpenAI and Anthropic SDK defaults). Forstream/4it is the idle budget between two chunks, so a long stream that keeps producing tokens is never cut off.httpcitself defaults toinfinity, so a blackholed connection would otherwise hang the calling process forever.connect_timeout— milliseconds (default:timeout).profile— anhttpcprofile you started yourself (default:default). The library never starts a profile or any long-lived process (seestream/4below for the one short-lived helper).ssl— a complete list ofsslclient options. Leave it out: since OTP 26httpcverifies the peer and its host name against the OS trust store by default, and passing anyssllist replaces those defaults rather than adding to them.
The adapter is HTTPS-only: an http:// (or any other) URL is refused with an
invalid_request error instead of being sent in cleartext. The http://
defaults of the local servers (vLLM, Ollama, LM Studio) therefore need your
own transport. Redirects are never followed.
stream/4 runs httpc in asynchronous streaming mode ({sync, false},
{stream, self}) inside one helper process per call, spawned and monitored
by the caller, which forwards each event to the caller; the fold itself runs
in the calling process. On every exit path — end of body, a halted fold, an
idle timeout, a failure, or an exception raised by the fold — the helper
cancels the request and the caller waits for the helper's 'DOWN', dropping
whatever it forwarded meanwhile. Because messages between two processes
arrive in send order, nothing can follow that 'DOWN': the caller's mailbox
is left exactly as it was. The helper also monitors the caller and cancels
the request if the caller dies, so it never outlives the call. An exception
from the fold propagates unchanged after that cleanup.
First body bytes may be held back (an OTP httpc limitation). When the
first body bytes arrive in the same socket read as the response headers,
httpc decodes them but does not stream them: in OTP's httpc_handler
module, handle_http_body keeps an incomplete chunked body (the
{Module, Function, Args} continuation returned by http_chunk's decode)
without streaming what it already decoded, and the "steal decoded chunks"
path that does stream partial bodies runs only for later reads (the
do_handle_info clause for socket data with a pending continuation). Checked
in inets 9.2 (OTP 27.0) and 9.8.1 (OTP 29.1); the {stream, {self, once}}
mode shares the same path. In practice the first SSE
event of a response can therefore reach the fold only together with the next
read, and the idle timeout keeps running meanwhile. A server that sends its
headers, one event, and then stays silent longer than timeout ends in
{error, timeout, _} even though that event was sent. Use your own transport
if this matters.
Summary
Functions
-spec request(thinwrap_llm_transport:request(), map()) -> {ok, thinwrap_llm_transport:response()} | {error, term()}.
Send one request with httpc. See the module doc for Opts.
-spec stream(thinwrap_llm_transport:request(), map(), thinwrap_llm_transport:stream_fun(), term()) -> {ok, term()} | {error, term(), term()}.
Send one request and fold Fun over the response as it arrives. See
thinwrap_llm_transport for the event protocol and the module doc for Opts.