AI Gateway
Configure AI providers, assistants, knowledge retrieval, stored conversations, usage, and budgets through one public API.
AI Gateway API
The AI Gateway API lets your server configure provider connections, prompts, optional knowledge sources, assistants, and conversational inference. Donutwork provides a stable assistant identifier while the provider and model can be changed independently in your configuration.
Call these endpoints from a trusted backend. Never embed a Donutwork bearer token, provider credential, vector store API key, or private CA certificate in browser or mobile application code. Credentials are write-only and are never returned by the API.
Recommended setup
- Discover providers and create a
generationconnection. - Create and publish a prompt.
- Optionally create a lexical knowledge base, or configure Qdrant plus an
embeddingconnection before creating a vector knowledge base. - Add approved text documents.
- Create an assistant that links the generation connection, prompt, and permitted knowledge bases.
- Test with the native assistant endpoint, Chat Completions, or the stored Conversations API.
- Monitor usage and configure a monthly budget.
All examples use API version 2026-02-01. Send JSON with Content-Type: application/json and authenticate with Authorization: Bearer YOUR_DONUTWORK_TOKEN.
Retrieval modes
Inference endpoints accept an optional retrieval object:
assistant: use the knowledge bases saved on the assistant;lexical: use only linked lexical knowledge bases;vector: use only linked vector knowledge bases;none: answer without document retrieval.
knowledge_base_ids may only contain knowledge bases already linked to the assistant. limit accepts values from 1 to 10. Responses include retrieval metadata and citations when relevant source passages were used.
Providers and connections
List AI providers
ai_connections:readQuery Parameters
No query parameters required.
Responses
Available provider adapters and their public capabilities.
{
"elements": [
{
"key": "openai",
"name": "OpenAI",
"capabilities": [
"chat",
"streaming",
"embeddings"
]
},
{
"key": "anthropic",
"name": "Anthropic",
"capabilities": [
"chat",
"streaming"
]
},
{
"key": "openai_compatible",
"name": "OpenAI-compatible",
"capabilities": [
"chat",
"streaming"
]
},
{
"key": "openai_embeddings_compatible",
"name": "OpenAI-compatible embeddings",
"capabilities": [
"embeddings"
]
},
{
"key": "voyage",
"name": "Voyage AI",
"capabilities": [
"embeddings"
]
},
{
"key": "ollama",
"name": "Ollama",
"capabilities": [
"chat",
"streaming",
"embeddings"
]
}
]
}List connections
ai_connections:readQuery Parameters
No query parameters required.
Responses
Provider connections visible to the authenticated organization.
{
"elements": [
{
"id": "connection_123",
"name": "Production answers",
"provider": "openai",
"purpose": "generation",
"model": "MODEL_ID",
"status": "active",
"credential_configured": true,
"last_test_status": "succeeded"
}
]
}Create a connection
Create a generation connection for assistants or an embedding connection for vector knowledge bases. Use the exact model identifier enabled by your provider account.
ai_connections:writeQuery Parameters
No query parameters required.
Request Body
{
"name": "Production answers",
"provider": "openai",
"purpose": "generation",
"model": "MODEL_ID",
"credential": "YOUR_PROVIDER_API_KEY",
"input_cost_per_million": 0.15,
"output_cost_per_million": 0.6
}namestringRequiredproviderstringRequiredpurposestringmodelstringRequiredbase_urlstringcredentialstringinput_cost_per_millionnumberoutput_cost_per_millionnumberResponses
Connection created. The credential is not included in the response.
{
"id": "connection_123",
"name": "Production answers",
"provider": "openai",
"purpose": "generation",
"model": "MODEL_ID",
"status": "active",
"credential_configured": true
}Unsupported provider, purpose, model configuration, URL, or credential.
{
"error": "Invalid connection configuration"
}Update a connection
Provider and purpose cannot be changed. For an embedding connection already used by a vector knowledge base, create a new connection instead of changing the model.
ai_connections:writeQuery Parameters
connectionIdstringRequiredRequest Body
{
"name": "Production answers",
"status": "active",
"input_cost_per_million": 0.15,
"output_cost_per_million": 0.6
}namestringmodelstringbase_urlstringinput_cost_per_millionnumberoutput_cost_per_millionnumberstatusstringResponses
Connection updated.
{
"id": "connection_123",
"name": "Production answers",
"status": "active"
}Connection not found.
{
"error": "AI connection not found"
}The requested model change would invalidate a linked vector knowledge base.
{
"error": "Create a new embedding connection and reindex instead"
}Rotate a connection credential
ai_connections:writeQuery Parameters
connectionIdstringRequiredRequest Body
{
"credential": "YOUR_NEW_PROVIDER_API_KEY"
}credentialstringRequiredResponses
Credential rotated. Secret material is never returned.
{
"id": "connection_123",
"credential_configured": true
}Connection not found.
{
"error": "AI connection not found"
}Test a connection
This performs a real provider request and may consume tokens or incur provider cost.
ai_connections:writeQuery Parameters
connectionIdstringRequiredResponses
Provider request succeeded.
{
"status": "succeeded",
"connection_id": "connection_123",
"duration_ms": 420,
"usage": {
"input_tokens": 8,
"output_tokens": 1,
"total_tokens": 9
},
"estimated_cost_usd": 0.000002,
"request_id": "request_123"
}Connection is disabled.
{
"error": "The AI connection is disabled"
}Provider rejected or could not complete the test.
{
"error": "Provider request failed"
}Delete a connection
Deletion is blocked while an assistant or vector knowledge base depends on the connection.
ai_connections:writeQuery Parameters
connectionIdstringRequiredResponses
Connection deleted.
{
"deleted": true,
"connection_id": "connection_123"
}Connection is still used by another resource.
{
"error": "Remove linked resources before deleting this connection"
}Create embeddings
This endpoint uses an active embedding connection owned by the authenticated organization. It returns vectors but never exposes the provider credential or base URL. The required scope is ai_embeddings:write.
input_type is available for providers that distinguish query and document embeddings, such as Voyage AI. dimensions is accepted only when the selected adapter supports an explicit output dimension.
ai_embeddings:writeQuery Parameters
No query parameters required.
Request Body
{
"connection_id": "connection_embeddings_123",
"model": "voyage-4",
"input": [
"Return policy",
"Shipping policy"
],
"input_type": "document",
"dimensions": 1024
}connection_idstringRequiredmodelstringinputstring | string[]Requiredinput_typestringdimensionsintegerResponses
Embedding vectors plus normalized usage and Donutwork ledger metadata.
{
"object": "list",
"data": [
{
"object": "embedding",
"index": 0,
"embedding": [
0.012,
-0.034
]
}
],
"model": "voyage-4",
"usage": {
"prompt_tokens": 8,
"total_tokens": 8
},
"donutwork": {
"request_id": "request_123",
"run_id": "run_123",
"connection_id": "connection_embeddings_123",
"provider": "voyage",
"input_items": 2,
"dimensions": 1024,
"estimated_cost_usd": 0.000001
}
}The configured hard budget would be exceeded.
{
"error": "AI Gateway monthly budget would be exceeded"
}Input, connection purpose, input type, or dimension is invalid.
{
"error": "Invalid embedding request"
}Embeddings JSON alias
This alias has the same connection, limits, provider options, ledger behavior, and ai_embeddings:write scope as /ai-gateway/embeddings.
ai_embeddings:writeQuery Parameters
No query parameters required.
Request Body
{
"connection_id": "connection_embeddings_123",
"model": "voyage-4",
"input": [
"Return policy"
],
"input_type": "document",
"dimensions": 1024
}connection_idstringRequiredmodelstringinputstring | string[]Requiredinput_typestringdimensionsintegerResponses
Embedding vectors with normalized usage and ledger metadata.
{
"object": "list",
"data": [
{
"object": "embedding",
"index": 0,
"embedding": [
0.012,
-0.034
]
}
],
"model": "voyage-4",
"usage": {
"prompt_tokens": 4,
"total_tokens": 4
},
"donutwork": {
"run_id": "run_123",
"connection_id": "connection_embeddings_123",
"provider": "voyage",
"input_items": 1,
"dimensions": 1024
}
}Input or embedding connection contract is invalid.
{
"error": "Invalid embedding request"
}Vector stores
Vector stores are optional. They are needed only by knowledge bases configured with retrieval_mode: vector. The public API currently supports Qdrant over HTTPS.
List vector store providers
ai_vector_stores:readQuery Parameters
No query parameters required.
Responses
Supported vector store providers.
{
"elements": [
{
"key": "qdrant",
"name": "Qdrant",
"capabilities": [
"dense_vectors",
"metadata_filters"
]
}
]
}List vector stores
ai_vector_stores:readQuery Parameters
No query parameters required.
Responses
Configured vector stores without secret material.
{
"elements": [
{
"id": "vector_store_123",
"name": "Knowledge search",
"provider": "qdrant",
"endpoint": "https://qdrant.example.com:6333",
"status": "active",
"credential_configured": true,
"tls_ca_configured": false,
"last_test_status": "succeeded"
}
]
}Create a vector store
ai_vector_stores:writeQuery Parameters
No query parameters required.
Request Body
{
"name": "Knowledge search",
"provider": "qdrant",
"endpoint": "https://qdrant.example.com:6333",
"api_key": "YOUR_QDRANT_API_KEY",
"collection_prefix": "product_docs"
}namestringRequiredproviderstringRequiredendpointstringRequiredapi_keystringRequiredca_certificatestringcollection_prefixstringResponses
Vector store configuration created without returning its API key or certificate.
{
"id": "vector_store_123",
"name": "Knowledge search",
"provider": "qdrant",
"endpoint": "https://qdrant.example.com:6333",
"status": "active",
"credential_configured": true
}Endpoint, provider, key, certificate, or prefix is invalid.
{
"error": "Invalid vector store configuration"
}Update a vector store
ai_vector_stores:writeQuery Parameters
vectorStoreIdstringRequiredRequest Body
{
"name": "Knowledge search",
"status": "active"
}namestringendpointstringcollection_prefixstringstatusstringResponses
Vector store updated.
{
"id": "vector_store_123",
"name": "Knowledge search",
"status": "active"
}Vector store not found.
{
"error": "Vector store not found"
}Rotate vector store access
Send the CA certificate again when the endpoint still requires it. Omitting it switches the configuration to the public system trust bundle.
ai_vector_stores:writeQuery Parameters
vectorStoreIdstringRequiredRequest Body
{
"api_key": "YOUR_NEW_QDRANT_API_KEY"
}api_keystringRequiredca_certificatestringResponses
Access material rotated without returning it.
{
"id": "vector_store_123",
"credential_configured": true
}Vector store not found.
{
"error": "Vector store not found"
}Test a vector store
ai_vector_stores:writeQuery Parameters
vectorStoreIdstringRequiredResponses
Connectivity and authentication succeeded.
{
"status": "succeeded",
"vector_store_id": "vector_store_123",
"duration_ms": 86,
"collections_count": 4,
"request_id": "request_123"
}Vector store is disabled.
{
"error": "The vector store is disabled"
}The remote service could not be reached or rejected the request.
{
"error": "Vector store request failed"
}Delete a vector store
ai_vector_stores:writeQuery Parameters
vectorStoreIdstringRequiredResponses
Vector store configuration deleted.
{
"deleted": true,
"vector_store_id": "vector_store_123"
}A knowledge base still uses this vector store.
{
"error": "Remove linked knowledge bases before deletion"
}Prompts
A prompt contains reusable system instructions. Do not place credentials or unneeded personal data in prompt content.
List prompts
ai_prompts:readQuery Parameters
No query parameters required.
Responses
Published prompt configurations.
{
"elements": [
{
"id": "prompt_123",
"name": "Product assistant",
"content": "Answer accurately and state when information is unavailable.",
"version": 1,
"status": "published"
}
]
}Create a prompt
ai_prompts:writeQuery Parameters
No query parameters required.
Request Body
{
"name": "Product assistant",
"content": "Answer accurately using approved information. If the answer is unavailable, say so clearly."
}namestringRequiredcontentstringRequiredResponses
Prompt version 1 created and published.
{
"id": "prompt_123",
"name": "Product assistant",
"version": 1,
"status": "published"
}Name or content is missing or too large.
{
"error": "Invalid prompt"
}Knowledge bases and documents
A lexical knowledge base searches matching words and needs no vector store. A vector knowledge base searches by meaning and requires an active Qdrant configuration plus an active embedding connection. Documents are split into searchable passages automatically. Each passage can carry page, section, language, access principals, and a chunking strategy version so retrieval remains traceable across reindexing changes.
List knowledge bases
ai_knowledge_bases:readQuery Parameters
No query parameters required.
Responses
Knowledge bases and readiness metadata.
{
"elements": [
{
"id": "kb_123",
"name": "Product policies",
"description": "Approved customer-facing policies",
"retrieval_mode": "vector",
"index_status": "ready",
"status": "ready",
"documents_count": 3,
"chunks_count": 24
}
]
}Create a knowledge base
For retrieval_mode: vector, both vector_store_id and embedding_connection_id are required. The embedding connection must use purpose: embedding.
ai_knowledge_bases:writeQuery Parameters
No query parameters required.
Request Body
{
"name": "Product policies",
"description": "Approved customer-facing policies",
"retrieval_mode": "vector",
"content_storage_mode": "vector_store_payload",
"vector_store_id": "vector_store_123",
"embedding_connection_id": "connection_embeddings_123"
}namestringRequireddescriptionstringretrieval_modestringcontent_storage_modestringvector_store_idstringembedding_connection_idstringResponses
Knowledge base created and ready for documents.
{
"id": "kb_123",
"name": "Product policies",
"retrieval_mode": "vector",
"index_status": "ready",
"status": "empty",
"documents_count": 0,
"chunks_count": 0
}Retrieval mode or required linked resources are invalid.
{
"error": "Invalid knowledge base configuration"
}A required external embedding or vector service did not complete setup.
{
"error": "Knowledge base setup failed"
}Add a text document
Vector knowledge bases create embeddings and index the resulting passages during this request. Use only current, approved source text.
ai_knowledge_bases:writeQuery Parameters
knowledgeBaseIdstringRequiredRequest Body
{
"name": "Returns policy",
"content": "Unused products may be returned within 30 days of delivery when they meet the published return conditions.",
"metadata": {
"page": 4,
"section": "Returns",
"language": "en",
"acl": [
"group:customer-care"
]
}
}namestringRequiredcontentstringRequiredmetadataobjectResponses
Document stored, split into passages, and indexed when vector retrieval is enabled.
{
"document_id": "document_123",
"chunks_count": 4,
"retrieval_mode": "vector",
"index_status": "ready",
"indexed_in_vector_store": true
}Knowledge base not found.
{
"error": "Knowledge base not found"
}The document was accepted but vector indexing could not be completed. Resolve the external connection issue before retrying.
{
"error": "Vector indexing failed"
}Enqueue a remote document
For a vector knowledge base configured with content_storage_mode: vector_store_payload, this endpoint creates a durable ingestion manifest and returns immediately. Donutwork stores the source reference encrypted; the worker fetches and parses the source, then stores searchable text with its vectors in the configured Qdrant payload instead of duplicating chunks in Donutwork content storage.
Sources may be an HTTPS URL or an object in Google Drive, SharePoint, or S3 linked through a managed source connector. The manifest stores only the protected source reference and lifecycle state. Redirects to unsafe destinations, embedded URL credentials, content over 10 MB, and normalized text over 5 MB are rejected.
ai_knowledge_bases:writeQuery Parameters
knowledgeBaseIdstringRequiredRequest Body
{
"name": "Returns policy",
"source": {
"type": "remote_url",
"url": "https://storage.example/policies/returns.md",
"content_type": "text/markdown",
"sha256": "OPTIONAL_LOWERCASE_SHA256"
},
"metadata": {
"department": "legal",
"locale": "en"
}
}namestringRequiredsourceobjectRequiredmetadataobjectResponses
Durable manifest queued for ingestion.
{
"id": "document_123",
"knowledge_base_id": "kb_123",
"name": "Returns policy",
"source_type": "remote_url",
"content_storage_mode": "vector_store_payload",
"status": "queued",
"index_status": "queued",
"ingestion_attempts": 0
}Source, content type, checksum, metadata, or knowledge base storage mode is invalid.
{
"error": "Invalid ingestion manifest"
}Get document ingestion status
ai_knowledge_bases:readQuery Parameters
knowledgeBaseIdstringRequireddocumentIdstringRequiredResponses
Sanitized document and ingestion state. The encrypted source reference is never returned.
{
"id": "document_123",
"knowledge_base_id": "kb_123",
"status": "ready",
"index_status": "ready",
"bytes": 4200,
"chunks_count": 8,
"content_type": "text/markdown",
"ingestion_attempts": 1
}The document is not in this organization and knowledge base.
{
"error": "Document not found"
}Replace an expiring remote source
Use this endpoint to rotate a signed URL or replace the source manifest. The new descriptor is encrypted, ingestion attempts are reset, and the document is queued again without exposing either the previous or replacement URL.
ai_knowledge_bases:writeQuery Parameters
knowledgeBaseIdstringRequireddocumentIdstringRequiredRequest Body
{
"source": {
"type": "remote_url",
"url": "https://storage.example/new-signed-url",
"content_type": "text/markdown",
"sha256": "OPTIONAL_LOWERCASE_SHA256"
}
}Responses
Source version replaced and document queued.
{
"id": "document_123",
"source_version": 2,
"status": "queued",
"index_status": "queued",
"ingestion_attempts": 0
}The document is not a remote ingestion or the new source is invalid.
{
"error": "Invalid remote source"
}Search a knowledge base
The response contains text passages that your application may use directly or provide to another search workflow. Vector queries may consume embedding tokens.
ai_knowledge_bases:readQuery Parameters
knowledgeBaseIdstringRequiredRequest Body
{
"query": "How long do I have to return a product?",
"limit": 5,
"principals": [
"group:customer-care"
]
}querystringRequiredlimitintegerprincipalsarrayResponses
Matching authorized passages returned.
{
"knowledge_base": {
"id": "kb_123",
"name": "Product policies",
"retrieval_mode": "vector",
"index_status": "ready"
},
"count": 1,
"results": [
{
"chunk_id": "chunk_123",
"document_id": "document_123",
"document_name": "Returns policy",
"position": 0,
"score": 0.91,
"content": "Unused products may be returned within 30 days of delivery.",
"page": 4,
"section": "Returns",
"language": "en",
"chunking_strategy_version": "words-v1"
}
]
}The knowledge base or one of its required connections is not ready.
{
"error": "Knowledge base is not ready"
}Retrieval evaluation
Use a reference dataset before adding hybrid search or reranking. Each case supplies a query, expected document or passage identifiers, and optional access principals. Donutwork reports hit rate, recall, precision, mean reciprocal rank, and normalized discounted cumulative gain at the selected cutoff. A recommendation is produced only for datasets with at least ten cases.
List retrieval evaluations
ai_knowledge_bases:readQuery Parameters
knowledgeBaseIdstringRequiredlimitintegerResponses
Recent sanitized evaluation summaries.
{
"elements": [
{
"id": "evaluation_123",
"knowledge_base_id": "kb_123",
"name": "Returns baseline",
"cases_count": 20,
"limit": 5,
"metrics": {
"recall_at_5": 0.94,
"mrr_at_5": 0.88,
"ndcg_at_5": 0.9
},
"decision": {
"status": "sufficient",
"recommendation": "keep_current_retrieval"
}
}
]
}Knowledge base not found.
{
"error": "Knowledge base not found"
}Run a retrieval evaluation
ai_knowledge_bases:writeQuery Parameters
knowledgeBaseIdstringRequiredRequest Body
{
"name": "Returns baseline",
"limit": 5,
"cases": [
{
"query": "return period",
"relevant_document_ids": [
"document_123"
],
"principals": [
"group:customer-care"
]
}
]
}namestringRequiredlimitintegercasesarrayRequiredResponses
Evaluation completed and retained as a baseline.
{
"id": "evaluation_123",
"status": "completed",
"cases_count": 20,
"limit": 5,
"metrics": {
"hit_rate_at_5": 1,
"recall_at_5": 0.94,
"precision_at_5": 0.31,
"mrr_at_5": 0.88,
"ndcg_at_5": 0.9
},
"decision": {
"status": "sufficient",
"recommendation": "keep_current_retrieval"
}
}Dataset, expected identifiers, principals, or cutoff is invalid.
{
"error": "Invalid evaluation dataset"
}Rerun a retrieval baseline
ai_knowledge_bases:writeQuery Parameters
knowledgeBaseIdstringRequiredevaluationIdstringRequiredRequest Body
{
"name": "Returns after reindex",
"limit": 5
}Responses
The protected baseline dataset was executed again.
{
"id": "evaluation_456",
"baseline_evaluation_id": "evaluation_123",
"status": "completed",
"cases_count": 20,
"metrics": {
"recall_at_5": 0.97,
"mrr_at_5": 0.91
}
}Knowledge base or baseline evaluation not found.
{
"error": "Retrieval evaluation not found"
}Managed source connectors
Source connectors keep reusable authentication separate from document manifests. Configuration is write-only. A document ingestion references a connector and a remote object identifier, while searchable text can remain in the external vector payload.
List source connector types
ai_knowledge_bases:readQuery Parameters
No query parameters required.
Responses
Supported connector and authentication contracts.
{
"elements": [
{
"type": "google_drive",
"label": "Google Drive",
"authentication": "oauth_refresh_token",
"object_fields": [
"file_id"
]
},
{
"type": "sharepoint",
"label": "SharePoint",
"authentication": "client_credentials",
"object_fields": [
"drive_id",
"item_id"
]
},
{
"type": "s3",
"label": "Amazon S3 compatible",
"authentication": "aws_signature_v4",
"object_fields": [
"key",
"version_id"
]
}
]
}List source connectors
ai_knowledge_bases:readQuery Parameters
No query parameters required.
Responses
Sanitized source connector configurations.
{
"elements": [
{
"id": "connector_123",
"name": "Corporate archive",
"type": "s3",
"status": "active",
"config_version": 1
}
]
}Create a source connector
ai_knowledge_bases:writeQuery Parameters
No query parameters required.
Request Body
{
"name": "Corporate archive",
"type": "s3",
"config": {
"region": "eu-west-1",
"bucket": "approved-documents",
"access_key_id": "YOUR_ACCESS_KEY_ID",
"secret_access_key": "YOUR_SECRET_ACCESS_KEY"
}
}namestringRequiredtypestringRequiredconfigobjectRequiredResponses
Active connector created without returning its protected configuration.
{
"id": "connector_123",
"name": "Corporate archive",
"type": "s3",
"status": "active",
"config_version": 1
}Connector type or configuration is invalid.
{
"error": "Invalid source connector configuration"
}Rotate source connector authentication
ai_knowledge_bases:writeQuery Parameters
connectorIdstringRequiredRequest Body
{
"config": {
"region": "eu-west-1",
"bucket": "approved-documents",
"access_key_id": "YOUR_NEW_ACCESS_KEY_ID",
"secret_access_key": "YOUR_NEW_SECRET_ACCESS_KEY"
}
}Responses
Authentication rotated and configuration version incremented.
{
"id": "connector_123",
"name": "Corporate archive",
"type": "s3",
"status": "active",
"config_version": 2
}Source connector not found.
{
"error": "Source connector not found"
}Delete a source connector
ai_knowledge_bases:writeQuery Parameters
connectorIdstringRequiredResponses
Unused connector deleted.
{
"deleted": true,
"source_connector_id": "connector_123"
}One or more document manifests still reference the connector.
{
"error": "Connector is still in use"
}Assistants and inference
An assistant links one generation connection, one published prompt, optional knowledge bases, and answer limits. The assistant ID or slug is the stable identifier used by calling applications.
List assistants
ai_assistants:readQuery Parameters
No query parameters required.
Responses
Configured assistants.
{
"elements": [
{
"id": "assistant_123",
"name": "Product helper",
"slug": "product-helper",
"connection_id": "connection_123",
"prompt_id": "prompt_123",
"knowledge_base_ids": [
"kb_123"
],
"temperature": 0.2,
"max_tokens": 800,
"retrieval_limit": 5,
"status": "active"
}
]
}Create an assistant
ai_assistants:writeQuery Parameters
No query parameters required.
Request Body
{
"name": "Product helper",
"slug": "product-helper",
"connection_id": "connection_123",
"prompt_id": "prompt_123",
"knowledge_base_ids": [
"kb_123"
],
"temperature": 0.2,
"max_tokens": 800,
"retrieval_limit": 5
}namestringRequiredslugstringconnection_idstringRequiredprompt_idstringRequiredknowledge_base_idsarraytemperaturenumbermax_tokensintegerretrieval_limitintegerResponses
Active assistant created.
{
"id": "assistant_123",
"name": "Product helper",
"slug": "product-helper",
"connection_id": "connection_123",
"prompt_id": "prompt_123",
"knowledge_base_ids": [
"kb_123"
],
"status": "active"
}A linked connection, prompt, or knowledge base was not found.
{
"error": "Linked resource not found"
}The connection purpose, slug, limits, or linked resources are invalid.
{
"error": "Invalid assistant configuration"
}Invoke an assistant
Use either a single message or a messages array. The final message must have role user. Supplying an Idempotency-Key header is recommended for safe retries.
ai_inference:writeQuery Parameters
assistantIdstringRequiredIdempotency-KeystringRequest Body
{
"messages": [
{
"role": "user",
"content": "What is the return period?"
}
],
"variables": {
"locale": "en"
},
"retrieval": {
"mode": "assistant",
"limit": 5
}
}messagestringmessagesarrayvariablesobjectretrievalobjectidempotency_keystringResponses
Assistant response with citations and usage.
{
"id": "run_123",
"request_id": "request_123",
"assistant_id": "assistant_123",
"answer": "The return period is 30 days.",
"citations": [
{
"document_id": "document_123",
"document_name": "Returns policy",
"source": 1
}
],
"usage": {
"input_tokens": 120,
"output_tokens": 18,
"total_tokens": 138
},
"retrieval": {
"mode": "assistant",
"knowledge_base_ids": [
"kb_123"
],
"results_count": 1
}
}A configured hard monthly budget would be exceeded.
{
"error": "AI Gateway monthly budget would be exceeded"
}A dependency is unavailable or the idempotency key cannot be replayed.
{
"error": "Assistant dependency unavailable"
}The selected AI provider failed the request.
{
"error": "Provider request failed"
}Chat Completions
This OpenAI-compatible endpoint is stateless: send the conversation messages required for every request. Set model to assistant/{id-or-slug}. With stream: true, the response uses Server-Sent Events and ends with data: [DONE]. Set stream_options.include_usage to receive a final usage chunk.
ai_inference:writeQuery Parameters
Idempotency-KeystringRequest Body
{
"model": "assistant/product-helper",
"messages": [
{
"role": "user",
"content": "What is the return period?"
}
],
"retrieval": {
"mode": "assistant",
"limit": 5
},
"stream": false
}modelstringRequiredmessagesarrayRequiredvariablesobjectretrievalobjectstreambooleanstream_optionsobjectidempotency_keystringResponses
Buffered Chat Completions response when stream is false.
{
"id": "chatcmpl-run_123",
"object": "chat.completion",
"model": "assistant/product-helper",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The return period is 30 days."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 120,
"completion_tokens": 18,
"total_tokens": 138
},
"donutwork": {
"request_id": "request_123",
"assistant_id": "assistant_123",
"citations": [
{
"document_id": "document_123",
"document_name": "Returns policy",
"source": 1
}
],
"retrieval": {
"mode": "assistant",
"results_count": 1
},
"idempotent_replay": false
}
}SSE chunks when stream is true. Each data event contains a chat.completion.chunk object and the stream ends with the documented terminal marker.
Server-Sent Events containing chat.completion.chunk objects, followed by the terminal marker.Model, messages, retrieval, streaming, or idempotency settings are invalid.
{
"error": "Invalid completion request"
}Chat Completions JSON alias
This alias has the same request, response, streaming behavior, and scope as /ai-gateway/chat/completions.
ai_inference:writeQuery Parameters
Idempotency-KeystringRequest Body
{
"model": "assistant/product-helper",
"messages": [
{
"role": "user",
"content": "What is the return period?"
}
],
"stream": false
}modelstringRequiredmessagesarrayRequiredvariablesobjectretrievalobjectstreambooleanstream_optionsobjectidempotency_keystringResponses
OpenAI-compatible completion response.
{
"id": "chatcmpl-run_123",
"object": "chat.completion",
"model": "assistant/product-helper",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The return period is 30 days."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 120,
"completion_tokens": 18,
"total_tokens": 138
}
}Invalid completion request.
{
"error": "Invalid completion request"
}Stored conversations
The Conversations API stores conversation history and automatically sends recent context with each new turn. Use Chat Completions instead when your application manages history itself.
Create a conversation
ai_conversations:writeQuery Parameters
No query parameters required.
Request Body
{
"assistant_id": "product-helper",
"external_user_id": "customer-42",
"variables": {
"locale": "en"
},
"retention_days": 30
}assistant_idstringRequiredexternal_user_idstringvariablesobjectretention_daysintegerResponses
Empty active conversation created.
{
"id": "conversation_123",
"assistant_id": "assistant_123",
"status": "active",
"messages_count": 0,
"external_user_id_hash": "opaque_user_reference",
"retention_days": 30,
"created_at": "2026-08-08T10:00:00+00:00",
"expires_at": "2026-09-07T10:00:00+00:00",
"messages": []
}Assistant, variables, external user reference, or retention is invalid.
{
"error": "Invalid conversation configuration"
}Get a conversation
ai_conversations:readQuery Parameters
conversationIdstringRequiredResponses
Conversation metadata and ordered message history.
{
"id": "conversation_123",
"assistant_id": "assistant_123",
"status": "active",
"messages_count": 2,
"retention_days": 30,
"expires_at": "2026-09-07T10:00:00+00:00",
"messages": [
{
"id": "message_user_1",
"role": "user",
"content": "What is the return period?",
"status": "succeeded",
"sequence": 1
},
{
"id": "message_assistant_1",
"role": "assistant",
"content": "The return period is 30 days.",
"status": "succeeded",
"sequence": 2,
"citations": [
{
"document_name": "Returns policy",
"source": 1
}
],
"usage": {
"total_tokens": 138
},
"retrieval": {
"mode": "assistant",
"results_count": 1
}
}
]
}Conversation not found.
{
"error": "Conversation not found"
}Conversation retention period has ended.
{
"error": "The conversation has expired"
}Add a conversation turn
Every turn requires a new Idempotency-Key. A successful retry with the same key and identical message returns the saved turn without a second provider request. Do not reuse a key for different text or retrieval settings.
ai_conversations:writeQuery Parameters
conversationIdstringRequiredIdempotency-KeystringRequiredRequest Body
{
"message": "And what if the product is damaged?",
"retrieval": {
"mode": "assistant",
"limit": 5
},
"stream": false
}messagestringRequiredretrievalobjectstreambooleanidempotency_keystringResponses
User message and assistant response appended.
{
"conversation": {
"id": "conversation_123",
"assistant_id": "assistant_123",
"status": "active",
"messages_count": 4
},
"user_message": {
"id": "message_user_2",
"role": "user",
"content": "And what if the product is damaged?",
"status": "succeeded",
"sequence": 3
},
"assistant_message": {
"id": "message_assistant_2",
"role": "assistant",
"content": "Contact support before returning a damaged product.",
"status": "succeeded",
"sequence": 4,
"citations": [
{
"document_name": "Returns policy",
"source": 1
}
],
"usage": {
"total_tokens": 164
},
"retrieval": {
"mode": "assistant",
"results_count": 1
}
},
"idempotent_replay": false
}SSE start, delta, completion, and terminal events when stream is true.
Server-Sent Events for the durable conversation turn.Another turn is processing, the key was reused incorrectly, a previous attempt failed, or the conversation reached its message limit.
{
"error": "Conversation turn conflict"
}Conversation retention period has ended.
{
"error": "The conversation has expired"
}Message, retrieval, or Idempotency-Key is missing or invalid.
{
"error": "Invalid conversation turn"
}With stream: true, adding a turn returns Server-Sent Events while preserving the same durable user and assistant messages. Long active conversations can be summarized automatically according to the organization settings; the rolling memory is used as context and is not returned as a separate message.
Close a conversation
Closing is idempotent and prevents new turns while retaining the history until its expiry date.
ai_conversations:writeQuery Parameters
conversationIdstringRequiredResponses
Conversation closed.
{
"id": "conversation_123",
"assistant_id": "assistant_123",
"status": "closed",
"messages_count": 4,
"retention_days": 30
}A turn is still being processed.
{
"error": "Conversation cannot be closed during an active turn"
}Update conversation retention
The new expiry is calculated from the update time and is applied to the conversation and all its messages. It cannot exceed the organization maximum.
ai_conversations:writeQuery Parameters
conversationIdstringRequiredRequest Body
{
"retention_days": 45
}retention_daysintegerRequiredResponses
Conversation and message expiry updated.
{
"id": "conversation_123",
"status": "active",
"retention_days": 45,
"expires_at": "2026-10-04T10:00:00+00:00"
}Retention is missing or exceeds the allowed maximum.
{
"error": "Invalid retention period"
}Delete a conversation
Deletion removes the conversation and its stored messages. It is rejected while a turn is being processed.
ai_conversations:writeQuery Parameters
conversationIdstringRequiredResponses
Conversation and messages deleted.
{
"deleted": true,
"conversation_id": "conversation_123"
}A turn is still being processed.
{
"error": "Conversation cannot be deleted during an active turn"
}Get conversation settings
ai_conversations:readQuery Parameters
No query parameters required.
Responses
Organization retention and rolling-summary policy.
{
"default_retention_days": 30,
"max_retention_days": 90,
"summary_enabled": true,
"summary_trigger_messages": 40,
"summary_keep_recent_messages": 12
}Update conversation settings
ai_conversations:writeQuery Parameters
No query parameters required.
Request Body
{
"default_retention_days": 30,
"max_retention_days": 90,
"summary_enabled": true,
"summary_trigger_messages": 40,
"summary_keep_recent_messages": 12
}default_retention_daysintegerRequiredmax_retention_daysintegerRequiredsummary_enabledbooleanRequiredsummary_trigger_messagesintegerRequiredsummary_keep_recent_messagesintegerRequiredResponses
Organization conversation policy updated.
{
"default_retention_days": 30,
"max_retention_days": 90,
"summary_enabled": true,
"summary_trigger_messages": 40,
"summary_keep_recent_messages": 12
}Retention or summary thresholds are invalid.
{
"error": "Invalid conversation settings"
}Usage and budgets
Usage responses contain operational metadata, token counts, latency, estimated cost, and sanitized error information. They do not return request messages, assistant answers, provider credentials, or retrieved passage content.
Get usage
ai_usage:readQuery Parameters
limitintegerResponses
Usage summary, budget status, and recent run metadata.
{
"summary": {
"requests": 12,
"succeeded": 11,
"failed": 1,
"input_tokens": 4200,
"output_tokens": 900,
"total_tokens": 5100,
"estimated_cost_usd": 0.021,
"costed_requests": 12
},
"budget": {
"monthly_budget_usd": 50,
"hard_limit": true,
"spent_usd": 0.021,
"remaining_usd": 49.979,
"period": "2026-08"
},
"runs": [
{
"id": "run_123",
"assistant_id": "assistant_123",
"provider": "openai",
"model": "MODEL_ID",
"request_id": "request_123",
"status": "succeeded",
"duration_ms": 680,
"usage": {
"input_tokens": 120,
"output_tokens": 18,
"total_tokens": 138
},
"estimated_cost_usd": 0.00003
}
]
}Get budget settings
ai_usage:readQuery Parameters
No query parameters required.
Responses
Current monthly budget status.
{
"monthly_budget_usd": 50,
"hard_limit": true,
"spent_usd": 0.021,
"remaining_usd": 49.979,
"period": "2026-08"
}Update budget settings
A hard limit requires a positive monthly budget. Costs are estimates based on prices configured on the provider connections, not provider invoices.
ai_usage:writeQuery Parameters
No query parameters required.
Request Body
{
"monthly_budget_usd": 50,
"hard_limit": true
}monthly_budget_usdnumberhard_limitbooleanResponses
Budget settings updated.
{
"monthly_budget_usd": 50,
"hard_limit": true,
"spent_usd": 0.021,
"remaining_usd": 49.979,
"period": "2026-08"
}Budget value or hard-limit combination is invalid.
{
"error": "Invalid budget configuration"
}Common errors
| Status | Meaning | Recommended action |
|---|---|---|
400 | The request body is not a JSON object. | Send valid JSON and Content-Type: application/json. |
401 | Bearer token is missing, invalid, or revoked. | Replace the token on your backend. |
403 | The token does not have the required scope. | Grant the scope shown on the endpoint. |
402 | A hard AI budget would be exceeded. | Review usage, pricing, and budget settings. |
404 | The resource does not exist or is not available to this organization. | Verify the public resource identifier. |
409 | A dependency, lifecycle rule, or idempotency claim prevents the operation. | Resolve the dependency or retry with the correct key. |
410 | A stored conversation expired. | Create a new conversation. |
413 | Message or conversation input is too large. | Reduce message count or content size. |
422 | Payload fields or linked resources are invalid. | Correct the request using the endpoint field descriptions. |
502 | An external AI or vector service failed. | Test the related connection and retry with a new idempotency key when appropriate. |
Error responses include a request identifier when available. Provide that identifier to support; never send bearer tokens or provider credentials in support messages.