ReqLLM - Release v1.27.0

ReqLLM – Composable LLM client built on Req

ReqLLM v1.27.0

I’m especially excited about this release because it brings the OpenAI Decisions API into ReqLLM through the new ReqLLM.evaluate/4 function.

I also want to highlight ReqLLM’s callback-based Model Routing API. It lets an application select a model at request time without putting a fixed routing policy inside ReqLLM.

OPENAI DECISIONS API

Use an evaluation-capable model to ask named boolean, choice, and score questions about text or JSON-compatible state.

questions = %{
  queue: %{
    type: :choice,
    instructions: "Which team should handle this?",
    criteria: %{
      billing: "Billing and refunds",
      support: "Technical help"
    }
  },
  urgent: %{
    type: :boolean,
    instructions: "Does this need immediate action?"
  }
}

{:ok, decision} =
  ReqLLM.evaluate(
    "openai:gpt-6-luna",
    %{ticket: "Refund me today"},
    questions,
    provider_options: [openai: [safety_identifier: "tenant-123"]]
  )

decision.object["queue"]["choice"]
decision.object["urgent"]["probability"]

The safety_identifier option is optional. Answers remain keyed by question name. The full provider response remains available in decision.provider_meta.raw_response.

LLMDB is the source for model eligibility. ReqLLM.evaluation_models/0 lists the evaluation models that ReqLLM can call.

CALLBACK MODEL ROUTING

Pass a router struct to a normal ReqLLM generation function. ReqLLM normalizes the request, calls resolve/2 once, and continues through the selected model’s standard provider path.

defmodule MyApp.ModelRouter do
  @behaviour ReqLLM.Router

  defstruct fast: "openai:gpt-6-luna",
            deep: "openai:gpt-6-sol"

  @impl true
  def resolve(router, %ReqLLM.Router.Request{routing_context: context}) do
    model = if context[:depth] == :deep, do: router.deep, else: router.fast
    ReqLLM.model(model)
  end
end

router = struct(MyApp.ModelRouter)

{:ok, response} =
  ReqLLM.generate_text(
    router,
    "Explain OTP supervision",
    routing_context: %{depth: :deep}
  )

ReqLLM.Response.text(response)

The application owns the routing policy. The callback can use local rules, an evaluation model, or another service. ReqLLM passes routing_context through unchanged.

The same routing contract works with generate_text, stream_text, generate_object, and stream_object.

OTHER CHANGES

• Azure Chat Completions log-probability support
• Gemini 3 object output when tools are present
• A bounded JSON Schema validator cache with FIFO by default and optional LRU or custom eviction
• Safer redaction of unknown provider reasoning payloads
• Correct OpenAI anonymous authentication and exact streaming model identity
• Responses API refusals decoded as text with a content_filter finish reason
• A fix for non-streaming connection-pool timeouts
• Contract hardening for streaming, tools, telemetry, usage, billing, URL selection, and timeout cleanup

Thanks to Arton Ragsdale, mvanhorn, Wu Shuwen, Justin Smestad, and Nickjgniklu for their work in this release.

LINKS

• ReqLLM v1.27.0 on Hex
• Release notes
• Documentation
• GitHub repository