LLM Gateways Compared 2026: OpenRouter vs LiteLLM vs Portkey vs Cloudflare

Why the Gateway Layer Exists at All

The first version of every AI feature calls one model directly. The second version calls two, because the first one had an outage. By the third version somebody in finance wants to know why the bill went from 400 dollars to 9,000 dollars, somebody in security wants prompts scanned before they leave the building, and somebody in engineering wants to try a cheaper open-weights model without touching application code.

That is the LLM gateway. It is a proxy that sits between your application and every model provider you call, and it collapses five problems into one layer:

• One API surface. Call OpenAI, Anthropic, Google, Groq and a self-hosted Llama with the same request shape.
• Routing and failover. When a provider 500s or rate-limits you, fall through to the next one instead of paging someone.
• Caching. Serve identical or semantically similar requests without paying for them twice.
• Cost control. Per-team keys, per-key budgets, hard spend caps.
• Observability. One log stream, one cost dashboard, one place to answer what did this request cost.

None of this makes your model smarter. If your problem is output quality, a gateway will not fix it - that is a hallucination and evaluation problem, not a routing one. But if your problem is operational sprawl, this is the layer that fixes it.

How We Compared

This is an editorial synthesis, not a benchmark. Everything below was checked against each vendor's own documentation, pricing page or GitHub repository in July 2026, and we have linked the source for every figure. Where a vendor publishes a performance claim, we label it vendor-reported rather than presenting it as an independent measurement. Where we could not verify a number, we left it out rather than estimating.

We looked at nine things: model breadth, self-host versus SaaS, routing and fallback, caching, cost tracking and budgets, observability, guardrails, pricing model, and the team each one actually fits. Pricing in this category changes often, so treat every dollar figure as a snapshot and confirm on the vendor's page before you commit.

The Five Gateways at a Glance

1. OpenRouter — Best for Breadth of Models

Best for: teams who want a single key and a single bill across essentially the entire model market, and who are comfortable with a third party in the request path.

OpenRouter is the path of least resistance. Its pricing page advertises 400+ models, and the platform routes across 70+ upstream providers. It implements the OpenAI API specification for chat and completions, which its FAQ describes as a drop-in replacement - any SDK that speaks OpenAI works by default, so adoption is a base-URL swap.

The routing is the interesting part. Per the model routing docs, OpenRouter offers an Auto Router that picks a model based on prompt complexity and task type, with a cost-quality tradeoff parameter on a 0-10 scale. It supports session stickiness so multi-turn conversations keep hitting the same upstream endpoint - which matters a lot for prompt caching, because a cache that lives on provider A is worthless if your next turn lands on provider B. Model shorthands let you bias routing without config: the nitro variant prioritises throughput, the floor variant prioritises price.

Caching is pass-through rather than gateway-owned. OpenRouter's prompt caching docs list support across OpenAI, Anthropic, Google Gemini, DeepSeek, Groq, xAI, Moonshot, Alibaba Qwen and Z.AI, noting that most providers enable it automatically while Anthropic and Alibaba require explicit cache_control markers.

• Pricing: the pricing page lists a 5.5% platform fee on pay-as-you-go. Critically, OpenRouter states it does not mark up provider pricing - the catalogue price is the provider's price.
• BYOK: you can bring your own provider keys. The pricing page lists a BYOK allowance of 25,000 dollars of list-price inference per month with no BYOK fee on pay-as-you-go (200,000 dollars on Enterprise), and a 5% fee above that allowance. This allowance is separate from the 5.5% platform fee that applies to credit-funded usage.
• Fallbacks: if a provider fails, OpenRouter falls through to the next one transparently.
• Free tier: free models are capped at 50 requests per day, rising to 1,000 per day once you have purchased at least 10 dollars in credits.

Limitations: there is no self-hosted OpenRouter. Every prompt transits their infrastructure, a non-starter for regulated workloads. Cost governance is thinner than Portkey's or LiteLLM's - it is a router first, a control plane second. And the 5.5% is a permanent tax on spend that grows as fast as your usage does.

2. LiteLLM — Best Open-Source Self-Hosted Gateway

Best for: platform teams who want to own the gateway, run it inside their own VPC, and pay nobody a percentage of their token spend.

LiteLLM is the default answer for self-hosting, and it has earned that position. It is a Python SDK and a proxy server that calls 100+ LLM APIs in OpenAI format, and the repository is MIT-licensed - with the caveat that content under the enterprise/ directory carries a separate commercial licence. The proxy docs cover deployment by CLI, Docker, Helm and Terraform across AWS, GCP and Azure.

What makes LiteLLM a gateway rather than an SDK is the control plane: virtual keys, spend tracking, per-key and per-team budgets, load balancing across deployments, fallback routing, and an admin UI. If you want to hand a team a key that dies at 500 dollars a month and logs every call, LiteLLM does that out of the box.

Caching is unusually flexible. The caching docs list in-memory, disk, Redis (including cluster and Sentinel), S3, GCS, and semantic caching via Qdrant, Redis or Valkey - with per-request controls for TTL, no-cache, no-store and namespacing.

• Latency: LiteLLM reports 8ms p95 at 1,000 requests per second. Vendor-reported, on their own hardware.
• Observability: it exports to essentially every backend in the LLM observability category, so you are not locked into anyone's dashboard.
• Enterprise tier: the enterprise page adds SSO, SCIM, OIDC/JWT auth, team and org admins, key and team-based guardrails, secret managers, key rotations and 24/7 support. Pricing is not published - it is contact-sales, with a 30-day trial key available.
• Open-weights friendly: it is the natural front door for self-hosted open-source models running on vLLM or Ollama alongside commercial APIs.

Limitations: you are now running infrastructure. It needs a Postgres, probably a Redis, monitoring and someone on call. Real throughput means horizontal scaling. And because enterprise pricing is not public, budgeting for the paid tier requires a sales conversation.

3. Portkey — Best for Enterprise Governance and Guardrails

Best for: organisations where the blocker is not routing but compliance - where you need SOC 2, VPC deployment, RBAC and prompts inspected before they reach a provider.

Portkey is the most complete control plane of the five, and it is unusually honest about the open-source split. The gateway itself is MIT-licensed on GitHub and routes to 1,600+ language, vision, audio and image models across 45+ providers, with 50+ built-in AI guardrails. You can run it with npx, Docker, Node or on Cloudflare Workers; enterprise private-cloud deployments cover AWS, Azure, GCP, OpenShift and Kubernetes.

The gateway feature set is the deepest here: universal API, simple and semantic caching, automatic fallbacks, conditional routing based on custom checks, automatic retries, load balancing across API keys, circuit breakers, request timeouts, canary testing, plus budget limits and rate limits at hourly, daily and per-minute granularity.

• Pricing: per the pricing page - Open Source is free with no request limit; Developer is free with 10,000 recorded logs per month and 3-day log retention; Production is 49 dollars per month for 100,000 logs, then 9 dollars per additional 100,000 requests, with 30-day log retention and 90-day metrics; Enterprise is custom, starting at 10 million-plus logs per month.
• The pricing model is worth understanding: Portkey bills on observed logs, not on your token spend. Your inference bill stays with your providers.
• Enterprise adds: SSO, granular budgeting, custom guardrail hooks, private cloud and VPC hosting, SOC 2 Type 2, GDPR, HIPAA and custom BAAs.
• Latency: Portkey advertises sub-1ms latency and a 122kb footprint. Vendor-reported.

Limitations: the free Developer tier's 3-day log retention is unusable for production debugging, so the real entry price is 49 dollars per month. Log-based billing means a chatty app can outgrow a tier faster than token spend would suggest. And the OSS gateway does not include the observability platform - the part most teams actually want.

4. Cloudflare AI Gateway — Best at the Edge and Cheapest to Run

Best for: teams already on Cloudflare, edge-deployed applications, and anyone who wants caching, rate limiting and analytics without a new invoice.

Cloudflare's angle is simple: the gateway is essentially free. Per the pricing docs, the core features - dashboard analytics, caching and rate limiting - are offered at no cost, and DLP scanning is free on all plans. You add one line of code and a URL prefix, and you get an observability and caching layer that competitors charge a subscription for.

The feature set has grown well past a logging proxy: caching, spend limits, rate limiting, dynamic routing, custom costs, guardrails, DLP, authentication, BYOK, analytics, logging and custom metadata. Dynamic routing is configured visually or as JSON and supports conditional branches on request body, headers or metadata; percentage nodes for A/B tests and gradual rollouts; and rate-limit and budget-limit nodes that fall back when exceeded.

• Limits to know: per the limits page - 100,000 logs per account on Free versus 10 million per gateway on paid; 10 MB max per log; cache TTL up to 1 month; 10 gateways per account on Free, 20 on paid.
• Unified Billing: you can buy provider credits through Cloudflare. The docs state a 5% fee on credits purchased - 100 dollars of credit costs 105 dollars - while inference itself is passed through with no markup. Coverage currently includes OpenAI, Anthropic, Google AI Studio, Google Vertex AI, xAI and Groq.
• OpenAI compatibility: there is an OpenAI-compatible chat-completions endpoint, though Cloudflare notes it is now deprecated in favour of the REST API while continuing to work for existing integrations.
• Logpush: paid plans only, with 10 million records per month included and 0.05 dollars per million after.
• Guardrails: metered as Workers AI token-based inference, so cost scales with the length of what you are evaluating.

Limitations: provider breadth is the trade-off - roughly 14 providers on the compat endpoint versus hundreds elsewhere. You cannot self-host it, by definition. And the model catalogue does not stretch to the long tail of specialist inference providers the way OpenRouter's does.

5. Helicone AI Gateway — Best Lightweight Open-Source Alternative

Best for: teams who want an open-source gateway and observability from the same vendor, without running Python in the hot path.

Helicone started as an observability tool and grew into a gateway. The AI Gateway repo is Apache 2.0, written almost entirely in Rust, and supports 100+ models across 20+ providers using OpenAI SDK syntax. It ships load balancing, rate limiting (per-user, per-team or global), Redis or S3-backed response caching, fallbacks, uptime-aware routing and OpenTelemetry support. You can run it with npx, Docker or Kubernetes.

• Performance: Helicone claims under 5ms p95 overhead and roughly 3,000 requests per second on around 64MB of memory. Vendor-reported.
• Cloud pricing: per the pricing page - Hobby is free with 10,000 requests; Pro is 79 dollars per month; Team is 799 dollars per month and adds SOC 2 and HIPAA; Enterprise is custom with SAML SSO and on-prem.

Limitations: it is the youngest gateway here and has the smallest catalogue of the open-source options. The strongest reason to pick it is if you already want Helicone's observability - as a pure gateway, LiteLLM has more surface area and Portkey more governance.

Pricing Models Decoded: Who Takes a Cut

The most useful way to sort this category is by what the vendor charges for.

The crossover point is worth doing the arithmetic on. A 5.5% fee is trivial on 2,000 dollars of monthly inference and costs 5,500 dollars a month on 100,000 dollars of inference - at which point a self-hosted LiteLLM cluster is dramatically cheaper. Conversely, if you spend 300 dollars a month on tokens, self-hosting a gateway is a way to spend more money and get less sleep.

Latency Overhead: Read These Numbers Carefully

Every vendor here publishes an overhead figure and every one is a vendor benchmark: Portkey advertises sub-1ms, Helicone claims under 5ms p95, LiteLLM reports 8ms p95 at 1,000 RPS. We have not independently reproduced any of them, and neither should you trust them without doing so.

The reason to be relaxed is arithmetic: time-to-first-token from a frontier model is typically measured in hundreds of milliseconds, so single-digit-millisecond proxy overhead is noise. The reason to be careful is geography. A gateway hosted in a different region from your app and your provider adds tens of milliseconds of round-trip regardless of how fast its internals are. Co-location beats microbenchmarks - which is the structural argument for Cloudflare's edge model, and the best reason to self-host LiteLLM next to your app rather than calling a SaaS gateway across an ocean.

Which Should You Choose?

Best for breadth of models: OpenRouter. 400+ models behind one OpenAI-compatible key, with automatic fallback and a router that picks for you. If your job this quarter is to evaluate models rather than operate them, start here and stop reading.

Best open-source / self-host: LiteLLM. MIT, 100+ providers, virtual keys and budgets built in, and the broadest caching and observability integration list. It is the answer when the requirement is that prompts must not leave your VPC.

Best for enterprise: Portkey. The combination of an MIT gateway, 50+ guardrails, conditional routing, semantic caching and SOC 2 / HIPAA / VPC deployment on the Enterprise tier is not matched elsewhere. If procurement and security are in the room, this is the one that survives the meeting.

Best at the edge and cheapest: Cloudflare AI Gateway. Caching, rate limiting and analytics for zero dollars, at the edge, with a one-line integration. If you are already on Cloudflare, there is very little reason not to put it in front of your calls today.

Best lightweight open-source alternative: Helicone AI Gateway. Rust, Apache 2.0, with a mature observability product attached.

A reasonable default: start with Cloudflare AI Gateway or OpenRouter while you are small and moving fast, then migrate to self-hosted LiteLLM or Portkey OSS when your percentage fee exceeds the cost of the infrastructure. Because all of these speak OpenAI's format, that migration is a config change, not a rewrite.

Three Mistakes Teams Make With Gateways

Not testing the gateway as a single point of failure. You have just put a proxy in front of every AI call in your product. Test what happens when it is down - every serious gateway supports a direct-to-provider fallback path, so configure it.

Assuming caching is free money. A cache only pays when requests repeat. Semantic caching in a chat product with high prompt variance can have a hit rate low enough that the embedding cost exceeds the savings. Measure the hit rate before you celebrate.

Confusing a gateway with governance. Routing traffic through one endpoint does not make it safe. Guardrails, evals and human review are separate work - the gateway just gives you the place to hang them.

Conclusion

The market has settled into a clean split. OpenRouter owns breadth and charges a percentage for it. LiteLLM owns self-hosting and charges nothing. Portkey owns enterprise governance and charges a subscription. Cloudflare owns the edge and charges almost nothing. Helicone bundles the gateway with observability.

The good news is that the decision is unusually reversible. Because all five implement the OpenAI chat-completions format, the cost of being wrong is a base URL and an afternoon. Pick the one that matches where you are today, not where you hope to be in two years, and re-evaluate when the invoice starts to sting.

This is an editorial synthesis of vendor documentation and public sources; see our [methodology](/methodology). Verify current details with each provider.