Custom Connectors

For: Platform operators configuring custom AI provider connectors on Axemere Gateway, self-hosted or managed.

Configuration Overview | Credentials | Workloads | Policies | Custom Connectors

Custom connectors let you route AI traffic through any OpenAI-compatible or Anthropic-compatible endpoint (private model servers, fine-tuned deployments, third-party providers, or internal proxies) without waiting for Axemere to add built-in support.

Managed gateway users: Register custom connectors from the Providers page in the console instead of editing YAML directly; the console writes the same underlying connector config described below. Everything in this guide about how the gateway resolves and proxies a request to a custom connector applies the same way regardless of how the connector was registered.

Table of Contents

• How it works
• Compat types
• Auth modes
• Configuration methods
  • Method 1: Bundle YAML section
  • Method 2: Standalone YAML file
• Field reference
• Quirks
• Referencing a custom connector in policy
• Client request URL
• Reload without restart
• Lookup priority
• See also

How it works

Custom connectors are registered under the custom: connector prefix. When the gateway resolves a policy effect like connector: custom:my-connector, it looks up the connector config and proxies the request to the configured base_url using the selected auth mode.

The {connector_id} you register (custom:my-connector) does not appear in the client's request URL. A client authenticates with an ordinary gateway key, and the gateway resolves which custom connector to dispatch to from that key's associated credential, the same way it resolves any other provider. See Client request URL below.

The connector is pass-through: the gateway does not translate request formats. The client must send requests in the format expected by the upstream (OpenAI-compatible JSON for openai_compat, Anthropic Messages API JSON for anthropic_compat).

Compat types

Choose openai_compat for providers that implement the OpenAI API (Groq, Together AI, Mistral, local Ollama, LM Studio, etc.).

Choose anthropic_compat for providers that implement the Anthropic Messages API.

Auth modes

For bearer and header modes, set credential_id to a credential registered in credentials.yaml (or via the admin API). The credential must have mode: alias or mode: inline.

For header mode, also set auth_header to the exact header name the upstream expects (e.g. x-api-key).

Configuration methods

Method 1: Bundle YAML section

Add a custom_connectors: section to your policy bundle YAML. Connectors defined here are loaded when the bundle loads and have no TTL; they remain active until the bundle is replaced.

# configs/policies/bundle.yaml

custom_connectors:
  - connector_id: my-provider
    display_name: "My Private Model"
    description: "Internal fine-tuned model on GPU cluster"
    compat_type: openai_compat
    base_url: https://models.internal.example.com/v1
    auth_mode: bearer
    credential_id: cred-my-provider

  - connector_id: local-ollama
    display_name: "Local Ollama"
    compat_type: openai_compat
    base_url: http://localhost:11434/v1
    auth_mode: passthrough

  - connector_id: together-ai
    display_name: "Together AI"
    compat_type: openai_compat
    base_url: https://api.together.xyz/v1
    auth_mode: header
    auth_header: Authorization
    credential_id: cred-together-ai
    quirks:
      model_prefix: "together/"

Note: http:// base URLs are only accepted when MVGC_DEV_MODE=true. Production deployments require https://.

Method 2: Standalone YAML file

For deployments where you want to manage custom connectors separately from the policy bundle, use a standalone file with the same custom_connectors: structure:

# /etc/mvgc/custom-connectors.yaml

custom_connectors:
  - connector_id: groq
    display_name: "Groq"
    description: "Groq LPU inference API"
    compat_type: openai_compat
    base_url: https://api.groq.com/openai/v1
    auth_mode: bearer
    credential_id: cred-groq

Point the gateway to the file with:

MVGC_CUSTOM_CONNECTORS_FILE=/etc/mvgc/custom-connectors.yaml

Or in mvgc.yaml:

custom_connectors_file: /etc/mvgc/custom-connectors.yaml

The file is loaded at startup. Send SIGHUP to reload it without restarting (see Reload without restart).

Field reference

Quirks

The quirks block handles provider-specific behavior that falls outside the standard request format.

Example:

quirks:
  strip_think_tags: true
  model_prefix: "together/"
  extra_body_fields: '{"repetition_penalty": 1.1}'

Referencing a custom connector in policy

Use the custom:<connector_id> prefix in policy effects:

effects:
  - connector: custom:my-provider
    credential_id: cred-my-provider
    decision: allow

To allow all workloads to use a connector, add a rule in the connectors: layer of your bundle:

connectors:
  rules:
    - id: allow-my-provider
      conditions: []
      effects:
        - connector: custom:my-provider
          credential_id: cred-my-provider
          decision: allow

Client request URL

Custom connectors use the shared custom path segment, through an ordinary Transparent Proxy request:

POST /proxy/custom/k/<your-axemere-key>/w/<workload-id>/v1/chat/completions

custom in the path doesn't identify which custom connector; every custom connector shares this one path segment. What determines the upstream destination is the credential tied to <your-axemere-key>. Register a credential with connector_id: custom:<connector_id> (see Referencing a custom connector in policy above) and requests authenticated with that credential are dispatched to that specific connector's base_url.

The gateway strips the matched provider/path prefix and appends the remaining path to base_url. For example, if base_url is https://api.groq.com/openai/v1 and the remaining request path is /v1/chat/completions, the upstream request becomes POST https://api.groq.com/openai/v1/chat/completions.

Reload without restart

Bundle YAML: Replace the bundle file and push a new bundle version through the admin API or the console. The gateway picks up the new custom_connectors: section automatically.

Standalone YAML file: Send SIGHUP to the gateway process:

kill -HUP $(pgrep mvgc-gateway)
# or with systemd:
systemctl kill --signal=HUP mvgc-gateway

The gateway re-reads the file, clears the previous file-sourced entries for the org, and re-seeds all connectors. A log line confirms the reload:

INFO custom connectors loaded from file path=/etc/mvgc/custom-connectors.yaml count=3 org_id=org-...

Lookup priority

When the gateway resolves custom:<connector_id>, it checks sources in this order. The first hit wins.

1. Bundle-seeded entries    (no TTL; replaced only when bundle reloads)
2. File-sourced entries     (no TTL; replaced on SIGHUP)
3. CP gRPC cache            (30 s TTL; auto-refreshed from control plane)
4. CP gRPC live fetch       (falls back when cache miss)
5. NotFound → 404

Bundle and file entries take priority over CP-managed entries. If a connector with the same ID appears in both the bundle and the CP, the bundle version wins.

See also

• Credentials — registering credentials referenced by credential_id
• Policies — writing policy rules that select custom connectors
• Configuration Overview — how all configuration pieces fit together
• NVIDIA NIM — Self-Hosted Containers — worked example: registering a self-hosted NIM container as a custom connector