LLM Profiles for Custom LLM Providers and API Keys

Laminar is an open-source, OpenTelemetry-native observability platform for AI agents. An LLM profile bundles a provider, the credentials for it, and the list of models you want to use into one workspace-level entry. Profiles are what the playground runs on, and on self-hosted deployments they are what Signals run on. Add a profile for a first-party provider (OpenAI, Anthropic, Google Gemini, Groq, Mistral, AWS Bedrock, Azure AI Foundry) or point one at any OpenAI-compatible gateway, including a model you host yourself.

Workspace settings: three LLM profiles, each with the models it exposes

Profiles are workspace-scoped: create one once and every project in the workspace can use it. Any workspace member can add, edit, and delete profiles.

Create a profile

Open LLM profiles

Go to Settings > LLM profiles, under Workspace settings, and click Profile.

Name it and pick a provider

The name is how the profile appears in the playground and Signal pickers, and it must be unique in the workspace. Pick the provider, then fill in the fields it asks for. See Providers below for what each one needs.

The profile form: provider credentials on top, the model list below, Test connection on the left

List the models

Add every model you want this profile to offer, written exactly as the provider expects it (gpt-5-mini, claude-sonnet-4-5-20250929, anthropic.claude-sonnet-4-5-20250929-v1:0). Laminar does not fetch a model catalog: the list you type is the list the playground and Signal pickers show. One model minimum, 64 maximum per profile.

Test the connection

Test connection sends one small request per listed model with the credentials in the form, and reports the round trip per model. A model that the provider rejects (wrong name, no access, bad key) shows the provider’s own error, so you can fix it before saving.

Save

Click Create. The profile is immediately selectable in every project in the workspace.

Providers

One profile is one provider and one set of credentials. For OpenAI and Azure AI Foundry, also pick the API shape, which is the endpoint your deployments speak.

Provider	API shape	Fields
OpenAI	Chat Completions or Responses API	API key
Anthropic	-	API key
Google Gemini	-	API key
Groq	-	API key
Mistral	-	API key
AWS Bedrock	-	AWS region, then either an access key ID with its secret access key, or a Bedrock API key (bearer token)
Azure AI Foundry	Chat Completions, Responses API, or Anthropic Messages	API key, plus either the resource name (the `<name>` in `https://<name>.services.ai.azure.com`) or a full base URL. Optional API version
Custom (OpenAI-compatible)	-	Base URL, API key, and any number of custom headers

A few notes on the less obvious ones:

API shape is not the same as the model. One Azure AI Foundry resource can serve all three shapes; pick the one the deployment you are calling speaks. Switching shape within a provider keeps the stored credentials, since the key is the same.
Bedrock takes long-form model ids, including the inference-profile prefix when you use one (us.anthropic.claude-sonnet-4-5-20250929-v1:0).
Custom speaks OpenAI Chat Completions only. Give it the API root; Laminar appends /chat/completions. Header names are stored with the profile, header values are stored as secrets. This is the option for vLLM, Ollama behind a proxy, LiteLLM, OpenRouter, an internal gateway, or any other OpenAI-compatible endpoint.

How credentials are stored

Every secret in a profile (API keys, AWS secret access keys, bearer tokens, custom header values) is encrypted at rest and write-only. Reads return a masked hint instead of the value: sk-***********kEA, or fully starred for a short value, plus the names of any custom headers. That is enough to tell which key a profile holds, and never enough to reuse it. Because secrets are write-only, editing a profile does not require retyping them. Leave a secret field blank and the stored value is kept; type a new value to rotate it. The same rule applies over the API: an omitted secret keeps what is stored.

On self-hosted deployments, profile secrets are encrypted with your own AEAD_SECRET_KEY, the same key that encrypts project API keys. See Self-hosting configuration.

Run the playground on a profile

The playground’s model picker is built from the workspace’s profiles: one submenu per profile, listing that profile’s models. Pick a profile and model, and the run uses that profile’s credentials.

The playground model picker: one submenu per profile, models listed as you entered them

A workspace with no profiles gets a No LLM profiles configured notice in place of the picker, with a link to settings. Open in playground on a traced LLM span binds the playground to a profile only when one of your profiles lists that span’s model verbatim; otherwise the rest of the span (messages, tools, parameters) is copied over and you pick the model. Deleting a profile, or removing a model from it, clears that selection on any playground that used it. The playground reopens with Select model and everything else intact.

Route Signals through a profile

Selecting a profile for a Signal is available on self-hosted deployments. On Laminar Cloud, Signals run on Laminar’s own model credentials and no profile is involved.

On a self-hosted deployment, each Signal runs on one profile and one of its models. The LLM section of the Signal form holds the pair:

The Signal form's LLM section: profile on the left, one of its models on the right

New Signals must pick a profile and a model. If the workspace has exactly one profile, it is preselected with its first model. With no profiles at all, the picker reads No profiles yet and the Signal cannot be created until you add one.
Signals created before you added profiles keep running on the server’s LLM_PROVIDER environment configuration. The form flags them so you can move them onto a profile; there is no way back to environment credentials once a Signal points at a profile.
A bad profile fails the run, visibly. A revoked key, a model you lost access to, or a deleted profile fails the Signal run with the provider’s error and the profile name, so the Runs tab tells you what to fix instead of retrying forever.

Deleting a profile, or removing a model, is refused while a Signal uses it. Re-point or delete those Signals first. This is the opposite of the playground, where the same delete just clears the selection. Other AI features on a self-hosted deployment (chat with trace, SQL with AI) still use the server’s LLM_PROVIDER and LLM_API_KEY. Profiles do not replace those. See Self-hosting configuration.

Manage profiles from the CLI

lmnr-cli llm-profile is full CRUD over the same profiles the settings page edits, so you can script provisioning or hand the surface to a coding agent. It works on Laminar Cloud and on self-hosted deployments; only pinning a Signal to a profile is self-hosted only. The CLI authenticates as you and resolves the workspace from the linked project, so lmnr-cli login plus a linked directory (or --project-id <uuid>) is all you need. See CLI.

lmnr-cli llm-profile list

ID                                    Name               Provider          Models
05cc437c-5e4a-4c95-8207-3b295619645c  Anthropic (prod)   anthropic         claude-haiku-4-5-20251001, claude-sonnet-4-5-20250929
2cc210c3-3f6f-4655-a3fc-a12cf4dea5e5  Internal gateway   custom            llama-3.3-70b
23ea6106-6ec8-4d1b-b2bc-5423272244b3  Production OpenAI  openai_responses  gpt-4.1-mini, gpt-5-mini, gpt-5

The ID column is what signal create --llm-profile-id takes. llm-profile get <profile-id> shows one profile in full, with secrets as masks:

lmnr-cli llm-profile get 2cc210c3-3f6f-4655-a3fc-a12cf4dea5e5

Internal gateway (2cc210c3-3f6f-4655-a3fc-a12cf4dea5e5)
  provider:     custom
  auth:         api_key
  base url:     https://gateway.internal.example.com/v1
  models:       llama-3.3-70b
  api key:      gw-**************def
  headers:      X-Team
  created:      2026-09-14T17:26:46.903960Z
  updated:      2026-09-14T17:26:46.903960Z

Create a profile from the CLI

create takes the name as its argument, the provider in --provider, one --model per model, and the credential flags that provider needs:

# OpenAI on the Responses API
lmnr-cli llm-profile create "Staging OpenAI" \
  --provider openai_responses \
  --api-key sk-... \
  --model gpt-5-mini --model gpt-4.1-mini

# Bedrock with AWS keys
lmnr-cli llm-profile create "Bedrock west" \
  --provider bedrock \
  --access-key-id AKIA... --secret-access-key ... --region us-west-2 \
  --model anthropic.claude-sonnet-4-5-20250929-v1:0

# An OpenAI-compatible gateway with a custom header
lmnr-cli llm-profile create "Internal gateway" \
  --provider custom \
  --provider-base-url https://gateway.internal.example.com/v1 \
  --api-key gw-... --header X-Team=ml-platform \
  --model llama-3.3-70b

--provider-base-url is the provider’s endpoint. --base-url is the Laminar API URL (the one you set for a self-hosted deployment). They are different flags.

Which credential flags a provider needs matches the Providers table:

Provider	Flags
`openai_completions`, `openai_responses`, `anthropic`, `gemini`, `groq`, `mistral`	`--api-key`
`bedrock`	`--region`, plus either `--access-key-id` with `--secret-access-key`, or `--token`
`azure_chat_completions`, `azure_responses`, `azure_anthropic`	`--api-key`, exactly one of `--resource-id` or `--provider-base-url`, optional `--api-version`
`custom`	`--api-key` and `--provider-base-url`, optional `--header <name=value>` (repeatable)

The auth type is inferred from the flags you pass: --access-key-id selects AWS keys, --token selects a bearer token, otherwise an API key. Nothing is validated locally, so the server’s own message is what you see:

$ lmnr-cli llm-profile create "Groq (prod)" --provider groq --api-key sk-...
ERROR: 400 Add at least one model

Update and delete from the CLI

update is a partial patch: only the flags you pass change. Two rules to keep in mind, both server-side:

--model replaces the whole model list, it does not append.
Any config flag (--region, --provider-base-url, --resource-id, --api-version, --header) replaces the whole config object, so re-pass every config field the provider needs.

Secrets are the exception: omit them and the stored credentials are kept, pass one to rotate it.

# Rename, credentials and models untouched
lmnr-cli llm-profile update 1ba7f4d8-a75f-4188-8372-871f1b5493f7 --name "Staging OpenAI (responses)"

# Replace the model list with one model
lmnr-cli llm-profile update 1ba7f4d8-a75f-4188-8372-871f1b5493f7 --model gpt-5

# Rotate the key, everything else untouched
lmnr-cli llm-profile update 1ba7f4d8-a75f-4188-8372-871f1b5493f7 --api-key sk-new...

lmnr-cli llm-profile delete 1ba7f4d8-a75f-4188-8372-871f1b5493f7

Deleted LLM profile "Staging OpenAI (responses)" (1ba7f4d8-a75f-4188-8372-871f1b5493f7).

An update with no flags is an error rather than a silent no-op, and a delete is refused while a Signal uses the profile (409). Every subcommand takes --json for machine-readable output on stdout:

lmnr-cli llm-profile list --json | jq '.[] | {name, provider, models}'
lmnr-cli llm-profile get <profile-id> --json

The group is also aliased to llm-profiles, and every subcommand has examples in its help text:

lmnr-cli llm-profile --help
lmnr-cli llm-profile create --help
lmnr-cli llm-profile update --help

Manage profiles over the API

Profiles are also full CRUD over HTTP, authenticated with a project API key from Settings > Project API keys. A project key manages the profiles of the workspace that project belongs to. Every field, status code, and the masked secrets object are in the API reference: list, create, get, update, and delete.

Self-hosting: replace the base URL with your app-server HTTP origin, including the port. For example, http://localhost:8000/v1/llm-profiles.

config carries the plaintext, provider-specific settings and secrets the credentials, split exactly as the form does it:

curl -X POST https://api.lmnr.ai/v1/llm-profiles \
  -H "Authorization: Bearer $LMNR_PROJECT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Bedrock EU",
    "provider": "bedrock",
    "config": { "region": "eu-west-1", "auth": { "type": "aws_keys", "accessKeyId": "AKIA..." } },
    "secrets": { "secretAccessKey": "..." },
    "models": ["eu.anthropic.claude-sonnet-4-5-20250929-v1:0"]
  }'

An OpenAI-compatible gateway keeps its header names in config and their values in secrets:

{
  "name": "Internal gateway",
  "provider": "custom",
  "config": {
    "baseUrl": "https://gateway.internal.example.com/v1",
    "headerNames": ["X-Team"]
  },
  "secrets": { "apiKey": "gw-...", "headers": { "X-Team": "ml-platform" } },
  "models": ["llama-3.3-70b"]
}

PATCH is a partial update: every field is optional, omitted secrets keep their stored values, and models replaces the whole list. Three things to watch:

Changing provider requires sending config with it (400 config is required when provider changes).
Sending any config replaces the whole object, so include every field the provider needs, not just the one you are changing.
Dropping a model a Signal runs on is refused with 409, same as deleting the profile itself.

Next steps

Playground - iterate on prompts, tools, and models on top of a profile.
Signals - extract structured outcomes and failures from your traces.
Self-hosting configuration - the environment-level LLM settings that the rest of the AI features use.
Custom model costs - price the models a custom or self-hosted profile calls.

Overview

Tracing

Signals

Debugger

Evaluations

Datasets

Platform

LLM Profiles for Custom LLM Providers and API Keys

Create a profile

Providers

How credentials are stored

Run the playground on a profile

Route Signals through a profile

Manage profiles from the CLI

Create a profile from the CLI

Update and delete from the CLI

Manage profiles over the API

Next steps

​Create a profile

​Providers

​How credentials are stored

​Run the playground on a profile

​Route Signals through a profile

​Manage profiles from the CLI

​Create a profile from the CLI

​Update and delete from the CLI

​Manage profiles over the API

​Next steps

Create a profile

Providers

How credentials are stored

Run the playground on a profile

Route Signals through a profile

Manage profiles from the CLI

Create a profile from the CLI

Update and delete from the CLI

Manage profiles over the API

Next steps