Unifying Developer Tooling with an OpenAI Gateway for Bedrock

The modern developer's toolkit is heavily optimized for a single API standard: OpenAI's. Popular AI-native editors, coding assistants, and orchestration frameworks—such as Cursor, Continue.dev, aider, and the broader LangChain ecosystem—overwhelmingly expect to communicate with POST /v1/chat/completions using a simple Authorization: Bearer token.

However, in enterprise environments, security and compliance teams often mandate the use of Amazon Bedrock. Bedrock keeps inference data strictly within an organization's cloud perimeter and offers a diverse menu of foundation models. But Bedrock speaks its own proprietary API and authenticates via AWS Signature Version 4 (SigV4).

This creates a frustrating friction point. Developers are forced to choose between using their favorite cutting-edge tools or complying with corporate data-governance policies. The solution is to deploy an OpenAI-compatible gateway in front of AWS Bedrock. By acting as a lightweight, transparent translation layer, this gateway allows teams to use standard OpenAI-compatible SDKs and tools while routing all underlying model traffic securely through AWS.

The Architecture of the Bridge

The naive approach to solving this integration problem is to issue AWS IAM credentials directly to every developer's local machine. This is a security anti-pattern. Distributing long-lived IAM keys across dozens of laptops increases the attack surface, complicates offboarding, and makes it incredibly difficult to implement granular rate limiting or cost attribution.

A centralized gateway solves this by acting as the sole custodian of the AWS credentials. The gateway runs within a secure environment (such as an EC2 instance, ECS container, or Lambda function) and assumes an IAM role with Bedrock permissions. Developers are issued simple, revocable API keys (e.g., sk-...) by the gateway.

When a developer's editor sends an OpenAI-formatted payload, the gateway intercepts it, validates the custom bearer token, translates the payload into Bedrock's Converse API format, signs the request using its own IAM role, and forwards it to Bedrock. It then translates the response back into the OpenAI format before returning it to the client.

Developer Tools (Cursor, Continue, Aider)
       │ 
       │ OpenAI Payload + Bearer Key (sk-...)
       ▼
┌─────────────────────────────────────────────┐
│            API Gateway / Proxy              │
│  1. Validate & Hash Bearer Key              │
│  2. Rate Limit & Log Request                │
│  3. Translate OpenAI -> Bedrock Converse    │
│  4. Sign with AWS SigV4                     │
└─────────────────────────────────────────────┘
       │ 
       │ AWS Bedrock API Call
       ▼
AWS Bedrock (Claude, Nova, Llama, DeepSeek)

Build vs. Adopt: Evaluating Your Options

When implementing this architecture, engineering teams face a classic build-versus-buy decision.

The Custom Build

For organizations with highly specific compliance, auditing, or identity provider (IdP) integration requirements, building a custom gateway is surprisingly straightforward. A basic translation layer can be written in Node.js using Express, @aws-sdk/client-bedrock-runtime, and a lightweight database like SQLite to manage keys and usage logs.

Advertisement

AirDroid Personal Best mobile device management suite — only $3.99/mo →

By hashing API keys using SHA-256 before storing them, you ensure that a database leak won't compromise active developer keys. A custom build also allows you to calculate exact per-developer costs by logging input and output token counts directly to an internal database, enabling precise chargebacks across departments.

The Off-the-Shelf Route

For most teams, maintaining custom translation logic is an unnecessary operational burden. The ecosystem has matured with robust, open-source alternatives that support advanced features out of the box:

1. Bedrock Access Gateway (BAG): An official AWS sample project that provides a highly scalable, production-ready gateway. It supports advanced features like Server-Sent Events (SSE) for streaming responses, tool calling, cross-region inference, and prompt caching. Crucially, it supports the latest models available on Bedrock, including Anthropic's Claude 4.5 family, Amazon Nova, DeepSeek-R1, and Qwen 3.
2. Envoy AI Gateway: A powerful, cloud-native gateway built on top of the Envoy proxy. It is designed for high-throughput Kubernetes environments and natively supports AWS authentication methods like EKS Pod Identity and IAM Roles for Service Accounts (IRSA).

Developer Angle: Local Setup and Deployment

To understand how this works in practice, we can spin up the bedrock-access-gateway locally using Docker. This setup allows you to test the translation layer on your workstation before deploying it to production.

Prerequisites

Before starting, ensure you have Docker installed, an AWS account with model access enabled in your target region (e.g., us-east-1), and an IAM user or role with the following minimal permissions policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream",
        "bedrock:ListFoundationModels"
      ],
      "Resource": "*"
    }
  ]
}

Running the Gateway with Docker Compose

Create a docker-compose.yml file to run the pre-built gateway container. We will configure it to listen on port 8000 and secure it with a custom API key:

services:
  bedrock-gateway:
    image: ibehren1/bedrock-gateway:latest
    container_name: bedrock-gateway
    environment:
      - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
      - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
      - AWS_REGION=us-east-1
      - API_KEY=my-secure-local-key
    ports:
      - "8000:80/tcp"
    restart: unless-stopped

Run docker compose up -d to start the gateway. Once running, the container exposes an OpenAI-compatible REST API. You can verify the connection by querying the gateway using curl:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer my-secure-local-key" \
  -d '{
    "model": "us.meta.llama3-2-1b-instruct-v1:0",
    "messages": [{"role": "user", "content": "Hello Bedrock!"}]
  }'

Production Deployment Trade-offs

When moving this architecture to AWS, you have two primary deployment patterns, each with distinct trade-offs:

• API Gateway + AWS Lambda: This serverless approach is highly cost-effective for teams with variable usage patterns, as you only pay per request. With native streaming support now available, Lambda is an excellent default. However, you must design your application to tolerate occasional cold starts.
• Application Load Balancer (ALB) + AWS Fargate: For large engineering organizations with high-throughput, latency-sensitive workloads, running the gateway on Fargate containers behind an ALB is the superior choice. This eliminates cold starts and provides the lowest possible streaming latency, though it incurs a continuous baseline compute cost.

The Editorial Verdict

Deploying an OpenAI-compatible gateway in front of AWS Bedrock is not just a clever workaround; it is a pragmatic, production-ready pattern for enterprise AI adoption. It successfully decouples the developer experience from cloud infrastructure constraints.

While building a custom gateway in Node.js is a viable weekend project for teams with highly bespoke logging or authentication needs, the official Bedrock Access Gateway and Envoy AI Gateway are the clear winners for production deployments. They handle the complex, fast-moving nuances of streaming, tool calling, and prompt caching across a rapidly expanding matrix of models. By adopting this gateway pattern, organizations can unleash their developers to use the best tools on the market today, without compromising on security, budget tracking, or compliance.