Hacker News 熱門精选63

Show HN: 我们构建了开源 OpenRouter,它会把使用情况转化为更好的模型

Show HN: We built open OpenRouter that turns usage into a better model

· 47天前
跳到正文

精选理由

帮助读者统一管理多模型并优化成本与效果

AI 摘要

嗨,HN,我们构建了一个开源模型网关。它是一个集中管理我们自托管、前沿和开源模型的单一入口。 它原生使用 Rust 构建,面向并发,并实现了跨模型和提供方的各种配置细节差异(流式格式、工具调用、模型参数、速率限制以及不同的错误行为)。 对于 BYOK 请求,这个网关增加的延迟不到 1 ms;当由 Experiential 提供供应商密钥时,延迟不到 2 ms。它覆盖所有主要推理提供方,并通过一个会发起 PR 的 codex agent 每日刷新 1000+ 个模型。 与其他类似项目相比,我们是开源的,不收取任何加价,允许你将本地模型与市场混合使用,并利用你的流量来(选择加入后)训练一个模型。简单路由并不值得收取 10% 的 token 加价。 我们的做法是:基于标准化的 OTel traces,我们挖掘具有代表性的真实任务,使用 text world models 模拟各种模型的 rollouts,应用一个 LLM judge,然后在 prompt 的 embedding 之上拟合一个最近邻分类器,以决定每个请求的最优模型。通常这比直接调用单个模型能画出更好的成本/质量帕累托曲线,但并不完美。 借助这些模拟,我们还可以做诸如建议缓存命中优化、新模型建议以及训练模型之类的事情。 它是开源的,所以你可以把它部署到自己的基础设施上,使用我们托管的版本且不收取任何加价,或者在我们的网站上阅读我们如何为最大可用性进行设计。 Comments URL: https://news.ycombinator.com/item?id=49471407 Points: 9 # Comments: 0

正文

Experiential

gateway latency

Experiential is an open source gateway and router for agent workflows:

  1. Use hosted, BYOK, and local models through one OpenAI-compatible API.
  2. Control which users and agents can use which models, for which use cases, and how much they can spend.
  3. Turn production traffic into a custom router or model optimized for quality, speed, and cost.

Experiential workspace usage dashboard showing model traffic, identities, and spend

🌐 Platform | 📚 Docs | Discord

Getting Started

Start a local OpenAI-compatible gateway. On first run, the setup wizard uses the shared provider, model, and reasoning-effort selectors, persists every selected provider connection, then shows defaults for the public alias, identity, and $50.00 command budget before printing a one-time key:

pip install experiential
exp

Choose a public alias such as opus-5, capture the issued key, and send a request:

export EXP_GATEWAY_KEY=...
curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Authorization: Bearer $EXP_GATEWAY_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"opus-5","messages":[{"role":"user","content":"Help me"}]}'

Setup / get started with the hosted gateway

Prefer a managed gateway to running one locally? The hosted platform at platform.experientiallabs.ai serves the same OpenAI-compatible (and Anthropic Messages) API at https://api.experientiallabs.ai/v1. See SETUP.md for copy-paste prompts you hand to your coding agent (Claude Code, Cursor, Codex, and similar); the agent runs the setup for you. It collects four prompts:

  • Upload your LLM traces as telemetry: create an account instantly from your email, then pull or upload your existing LLM traces onto the platform as telemetry.
  • Connect your inference provider keys (BYOK): create an account, then connect your own OpenAI, Anthropic, Gemini, Azure, Bedrock, Fireworks, or OpenRouter keys for free pass-through.
  • Start calling models on the gateway: make your first /v1 call with the OpenAI and Anthropic SDKs using an xpl_ key, and optionally repoint your existing coding agents.
  • Full onboarding: create an account instantly from your email, connect your keys, import your spend, then repoint every coding agent (Claude Code, Cursor, Codex, Aider, and similar) or Conductor at the gateway.

Using the API

Start the local gateway with exp (or exp run); the compiled native data plane serves every route on loopback. From Python, load a fitted project router as an official OpenAI client backed by its own private gateway:

import exp

with exp.load_router("my-project") as client:
    response = client.chat.completions.create(
        model="my-project",
        messages=[{"role": "user", "content": "hello"}],
    )

Optimize from Traffic

First, collect OpenTelemetry traces from your current agent. If you just want to try it out, grab the public terminal-tasks OTLP dataset:

curl -L -o traces.otel.jsonl \
  https://huggingface.co/datasets/experiential-labs/wmo-terminal-tasks-traces/resolve/540883e451dc13d34fb50fdd36b143cb0f1fb0db/traces.otel.jsonl

Then build a project. The build command walks you through providers, models, and budget, and asks for your trace file:

# Build simulation from your agent traces and optimize a router against it
exp build support-agent

After collecting traces from your router, fine-tune an open source model you own using Tinker.

exp optimize model support-agent

Telemetry

Anonymous aggregate PostHog product telemetry is enabled by default. It never includes prompts, traces, actions, observations, paths, model names, credentials, or raw customer content.

exp config telemetry status
exp config telemetry disable
exp config telemetry enable

The preference is stored locally in .exp/settings.toml.

Development

uv sync --extra dev
uv run ruff format --check .
uv run ruff check .
uv run ty check
uv run pytest -q

Repository and documentation conventions live in AGENTS.md.

查看原文(Hacker News 熱門)