[AINews] AI Engineer NYC: Gemini 4 Argon, Vals AI Benchmarks
Gemini 4 Argon went GA today, but we already gave it title story 2 weeks ago and most benchmarks, like CursorBench, simply independently confirm their claims.
AI 写代码的一切:编码助手、代码模型与开发工作流变革。
Gemini 4 Argon went GA today, but we already gave it title story 2 weeks ago and most benchmarks, like CursorBench, simply independently confirm their claims.
Research: Profiling web-edit with Python 3.15: flame graphs and source heatmaps I put the new Python 3.15 profiler through its paces by having GPT-6 Astra Extra High in Codex run it against my new web-edit app : I need a Python 3.15 flame graph and heatmap demo of the "web-edit" app from PyPI - research first then make me those demos Here's the resulting example flame graph and heatmap . Tags: profiling , python , codex
Existing benchmarks measure solve rate and token output/cost, but don't punish excess code, which makes a codebase harder to work with in the long term. I wanted to measure this since it matters most for long-term projects. Method: it takes the patches that benchmarks already publish (DeepSWE, SWE-bench Verified, SWE-bench Live) and counts "coding units": syntax elements like statements, calls, names and literals. I didn't use lines of code, so cramming code isn't rewarded. Deleted code is subtracted, so it measures net units. GPT-6 and Claude 5.5 have no public patches yet, so I ran them myself on 3–10 tasks on my own subscriptions (shown as faded circles, not ranked). Any feedback or model contributions are welcome, especially public patch exports for newer models. GitHub: https://github.com/La5u/Parsimony Not affiliated with any of the benchmarks. Comments URL: https://news.ycombinator.com/item?id=50077619 Points: 2 # Comments: 0
I've been playing around with a new kind of isomorphic music keyboard that uses a base-12 notation, and wanted some sheet music to learn some new music for it. The idea is to represent each note in 12-TET as a number, so N ∈ (0,1,2,3,4,5,6,7,8,9,X,Y). I used codex to build a music score rendering backend based on MuseScore with custom note heads that display the pitch class of each note. I've found it pretty helpful for learning how to play my own instrument, so wanted to share in case anyone else was interested. Comments URL: https://news.ycombinator.com/item?id=50075665 Points: 5 # Comments: 1
Hey HN! I have been dabbling into some hobby projects that require designing custom 3D printed models. I have been relying on AI models, mostly GPT Sol and Astra through Codex, and giving them certain tools such as Blender MCP and OpenSCAD CLI to design. My end goal is 3D printable designs ( .stl etc). The results have been good but not great. Iterating and improving them is such a pain in the ass. I want to know which specific tools or workflows you would recommend I should try. Thanks in advance! Comments URL: https://news.ycombinator.com/item?id=50072103 Points: 1 # Comments: 0
The new X Subscription combines X's paid features with AI tools like Grok and Cursor, offering plans ranging from $8 to $200 per month.
Release: web-edit 0.1a0 I was working on a project from my phone using Codex Remote when I found myself wanting to edit a credentials file without round-tripping those credentials through the LLM itself. So... I had Codex and GPT-6 Astra Extra High build this Python web app which provides a web UI for viewing and editing files in a folder. It's intended to be run over Tailscale, so I can have an agent start this running and then edit files via my Tailscale network directly from my phone. Here's the interface for browsing files in a folder: Screenshot of a web file manager interface. Header reads "WORKSPACE" above the title "Files", with buttons "Refresh", "Upload files" and "New file" (dark green). A breadcrumb shows "web-edit / web_edit / static". A table with columns "Name", "Size", "Modified" lists four files, each with "Download ↓" and "Delete" links: "app.css" 8.4 KB Oct 12, 2026; "app.js" 671.5 KB Oct 12, 2026; "index.html" 4.3 KB Oct 12, 2026 (underlined, as if hovered); "THIRD_PARTY_LICENSES.txt" 32.3 KB Oct 12, 2026. Below the table: "Images under 2 MB appear inline. Select a text file to edit it." A panel below is labelled "UPLOAD QUEUE" with heading "Add files" and a "Close" button. It has a "Destination folder" text field containing "web_edit", then a dashed drop zone reading "Choose files or drop them here" and "Set a path and filename for each file before uploading." with a "Choose Files" button and the text "no files selected". A greyed-out "Upload files" button sits at the bottom right. Editing uses CodeMirror 6 - I was going to use Monaco but that still doesn't have a great mobile experience yet. Tags: projects , python , ai , tailscale , generative-ai , llms , codex
Hey HN! I was playing around with the Kepler and TESS data last weekend when I hacked around a way to pixel-peep planets and star dimming periodically to potentially find new exoplanets with Codex. I ended up running the engine to find an interesting signal in Kepler data with a dip repeating every 13.77 days around KIC 5689121, a red dwarf in Lyra about 970 light-years away. If it's a planet, it'd be roughly 2.5–3 times Earth's radius ! Funnily enough, this candidate is just 7.1 arcseconds from a much brighter neighbour (KIC 5689116) and is very hard to see because of how much brighter it is (probably why it went unnoticed for so long) Of course, there's work to do before calling it confirmed - my TRICERATOPS analysis is about 6.9% and we have to get IR/AO / Keck observations to prove it's out there ! (which I'm honestly intimidated by but cautiously optimistic about because I've never worked in non-CS adjacent research) but I found the activity of making cool tools to potentially discover things about our universe quite fun ! Would love feedback, especially from anyone who's worked with Kepler data or exoplanet validation :) https://x.com/kuberwastaken/status/2109240782368751834 Comments URL: https://news.ycombinator.com/item?id=50071497 Points: 1 # Comments: 0
I didn't like that Claude and codex effectively have the ability to do anything they want on my laptop and my local network - we've all read the horror stories of files accidentally being deleted - and while their permission classifier is getting better, it still requires a lot of trust. So I built Airlock for macOS which runs your agent in a virtual machine, and it will tell you exactly what it's run and what's changed before you bring anything back to your project. You can limit internet access to specific hosts, or block it altogether. It even works with Xcode projects (iOS and macOS developers rejoice). Technically you can run any command with Airlock - not just AI agents (`airlock run -- somecommand`). It's free, requires no account, and has no tracking or analytics. I'm hoping Apple will approve it for Mac App Store distribution, but in the meantime you can download it directly from the website or using Homebrew. Comments URL: https://news.ycombinator.com/item?id=50054620 Points: 1 # Comments: 0
arXiv:2609.38345v2 Announce Type: replace-cross Abstract: Multi-agent coding systems are designed to tackle complex software engineering tasks through collaboration. However, existing evaluations typically assume configured organizations are followed faithfully, whereas reality differs. This behavioral gap, combined with differences in underlying system components, prevents clear attribution of observed gains. To this end, we introduce OpenCollab, a multi-agent coding framework that provides a unified infrastructure for programmable collaboration and controllable runtime. Specifically, OpenCollab unifies organization design, enforces experimental control on a shared runtime, and tracks execution through fine-grained event streams. On this basis, we define Adherence to quantify whether the declared organization is actually realized. Our experiments reveal that agents collaborate very differently across configurations: changing any single dimension shifts Adherence, from 47.2% to as high as 97.2%. Furthermore, extensive agentic coding benchmarks show that OpenCollab (Duo) establishes new SOTA performance compared to the mainstream harnesses such as Mini-SWE-agent, OpenHands, Codex CLI, and Claude Code, showing that a well-designed organization can outperform strong existing harnesses, while OpenCollab's single-agent configuration uses the fewest tokens across all evaluated suites. OpenCollab establishes a unified multi-agent infrastructure for easy programmable collaboration and controlled causal evaluation.
arXiv:2606.19386v2 Announce Type: replace-cross Abstract: Runtime monitors for autonomous agents commonly threshold an accumulated internal state - a behavioural baseline, a drift statistic, or, in our prior work, a modelled affective state. We previously reported a State Saturation Trap: threshold-on-state triggers over a continuous affect engine become near-constant alarms on SWE-bench debugging agents (Modgil 2026). A post-release audit found the engine received dt=0 between actions, so its exponential decay never operated: the published trap is a pure-accumulator result. We correct the record (erratum, v2) and treat the flaw as an experiment. The key variable it exposes is whether a monitor's dynamics are calibrated in sample time (per observation, as in CUSUM) or wall-clock time (half-lives in seconds, as in affect models and EMA baselines). On fixed-rate streams these coincide; on agent streams, where inter-action time varies by orders of magnitude, they do not. A pre-registered sweep over uniform intervals (dt in {0..600}s) on 20 trajectories shows the wall-clock level trigger has two regimes: at dt<=1s a constant alarm (20/20; median 18 firings); at dt>=60s silent. Every critical dt lies in (1,30]s. A scriptable gpt-5.4-mini agent loop measures latency at median 1.53s (p90 2.33s), inside the trap regime, vindicating the empirical finding under a corrected mechanism. The structure is a property of the calibration class, not the engine: a minimal wall-clock accumulator over the raw error stream reproduces the same cliff, while a sample-time CUSUM over the identical stream is exactly dt-invariant (20/20). A rising-edge trigger with hysteresis fires 0-3 times per trajectory in every condition. We conclude that wall-clock-calibrated leaky-integrator monitors admit no regime in which they act as moment detectors on agent streams; transition detection escapes the trap at every cadence, but does not recover human intervention timing.
arXiv:2610.12738v1 Announce Type: new Abstract: Forecasting rare, high-cost events in multivariate time-series systems is hard because their precursors are seldom observed, so a purely data-driven model has little to learn from. The engineers who run these systems, however, hold knowledge that precedes such events, information absent from the scarce labels. We call this a semantic prior and inject it into a self-supervised event forecaster through two switchable mechanisms, a covariate mechanism and an attention-bias mechanism, each the identity at its null, so one architecture serves every prior type with no per-dataset architecture search. A prior helps when it supplies information the observed window does not already contain: in a short context an informative prior beats no prior in-domain (pooled h-AUROC over the no-prior baseline: a periodicity clock +14%, p<0.001; a conservation residual +47% at a 0.25 label fraction), and a working prior transfers out-of-domain (+53%), substituting for feature transfer, while priors the window already recovers stay inert and the attention-bias mechanism shows no measurable gain. KITE is a switchable, attributable route to contextual conditioning, and a substrate for scaling toward cross-domain, pretrained time-series foundation world models that carry their governing structure explicitly.
Watch a webinar on using Codex across the software development lifecycle.
详细介绍 OpenAI 团队如何使用 Codex 理解代码、重构系统、提升性能、加快开发速度并优化工程工作流程。
借助 Codex 中的 GPT-6 Astra,Asana 在测试中将其浏览器代理的成本降低了 76 倍、速度提升了 5 倍,从而为客户提供能力更强的模型。
I shipped a new feature for my blog today: the Newsletters page, which offers an index of all of the newsletters I've sent out, both my free weekly Substack and my monthly sponsors-only updates. I built the feature almost entirely using my voice, chatting away to my laptop while I cooked dinner. Codex voice mode I used the ChatGPT desktop app for this, in the Codex tab, using the voice conversation mode, running against a local development environment. Here's what that looks like: Three columns of the ChatGPT app. The left hand column shows my recent conversations. The middle column is a transcript of the conversation, with a throbbing black blob representing voice mode. The right hand column shows a local dev server version of my blog, with the Newsletters page visible. I started the session against my local simonwillisonblog checkout by typing: Start dev server and open in browser This gave me a preview of the site that it would be working on, and meant that I could later ask it to show me the new pages so I could visually track its progress. Then I clicked the "Start new voice chat" button - that's not the microphone button, it's the one to the right of it - and set my laptop up in the kitchen so I could talk to it while I cooked. Talking to my computer I had a pretty good idea of what I wanted to build, and it's a simple enough Django feature that I was certain the model (in this case GPT-6 Astra High) would be able to do it. A new model, a migration, some view code, templates, and a couple of import functions to populate the database from external sources. Here's an extract of my voice transcript that was captured by Codex: Um, they do not. Um, this is going to be a new type of content. Um, it's not going to show up... Oh, hold on. Yeah, no- I do not want this to show up in my, um, tag pages and date archive pages and... Actually, no, I think... I don't want it on the tag pages. I don't want it on the, um, blog index page. But I think I do want it to show up on the date-based pages. You know, if you navigate to September the 19th, and I sent a newsletter on that page, I think I want that to show up. So... this is- so I think we probably need a new model. The other thing is that I want them searchable, uh the Substack ones are not searchable, because those are actually just copies of other s- on content on my blog. These monthly ones do contain unique content, and spe- and once they're... published, like once they're made public a month after they've gone out, I want them to show up on my search results. Apparently this was clear enough that the model knew what I wanted to build! You can read the full transcript, disfluencies and all, in this Gist . We went on like this for about half an hour (the time it took to cook dinner). The model would reply and occasionally ask clarifying questions, then get to work modifying the code. What we built We got a surprisingly long way entirely by voice: A new model and migration to represent imported newsletters in Django, plus Django Admin configuration for that Four working imports: The most recent Substack items via RSS Every other Substack item via their undocumented API, which GPT-6 Astra either knew about or found via search All of my published monthly newsletters from my simonw/monthly-newsletter-archive GitHub repository My most recent private sponsors-only newsletter from a private repository The /newsletters/ and /newsletters/2026/ public archive pages Newsletters showing up on day and month archive pages too, but not on tag pages or my homepage Weekly Substack newsletters link to Substack; archived monthly newsletters have their own pages Integration with my site search engine It was almost ready to ship. The catch was the imports: Astra offered to export data from my local copy so I could import that into production, but I wanted it to work like my other import scripts. Since some of the data lived in a private GitHub repository, this would involve creating a new API key, and for that I knew I'd have to sit at the keyboard for a while. Finishing it with a review Once I had finished cooking and judged it mostly feature-complete, I had Codex create a branch and open a pull request. I reviewed the code in the GitHub PR interface. It was nearly what I needed, except it had chosen to use Git in a subprocess for one of the import scripts. I needed one of the imports to pull from a private Git repository, so I figured the API would be a better bet. I switched to typing and had Codex swap that out for an API-based import instead. You can see the changes I made during the review in the extra commits on the PR . I fixed the import mechanism and made a few tweaks to the display of those public pages. It took an additional half hour of typing-based prompting to get to the point where I was happy to deploy it to production by landing the PR. The end result You can see the end result at the new newsletters index page , or view the page for a previous monthly newsletter . Newsletters page. It shows two Substack posts (with thumbnails) and one LLM digest Sponsors-only newsletter with a list of headings. On the right is a CTA to subscribe to my Substack and another one for my $10/month monthly briefing newsletter. The index page shows my most recent Substack weekly newsletters and GitHub sponsors monthly newsletters mixed together in reverse chronological order. Further down the page are links to my by-year archive pages. GPT-6 Astra designed the page, and then tweaked that design based on my vocal feedback from glancing at the local preview across the kitchen. Better for multi-tasking than as a daily driver OpenAI love using voice-driven demos like this one for things like DevDay - and they do work well in that environment. I don't think this is going to be a daily driver for me though. I've written before about how much "work" I get done using ChatGPT voice mode on my phone while walking the dog - mostly research and brainstorming, but occasionally actual development work by having ChatGPT write and test out snippets of code. This feels different. The addition of the visual preview, plus being able to type or paste things in via the keyboard when I need to communicate something that doesn't work vocally, makes this a much more powerful way of interacting with a coding agent. I still switch back to typing once I get down to the details of things though. Being able to paste in examples and error messages, or directly highlight the code or feature that needs changing, remains more efficient than trying to describe it in words. I mainly work from home, which is good because there's no way I'd want to talk to my computer like this in a shared workspace! The killer feature for me is the ability to multi-task. I usually cook with a podcast or TikTok running; now I can actually build stuff instead. Tags: text-to-speech , ai , openai , generative-ai , llms , newsletter , coding-agents , codex , speech-to-text , site-upgrades
arXiv:2610.12375v1 公告类型:cross 摘要:Agent 被部署于旅行规划、股票交易和 IT 事故分诊等应用中。在大多数情况下,LLM Agent 几乎不受基于规则的安全防护约束而自主运行,因而可能由于不可逆操作引发成本和安全问题。近期研究要么使用安全防护 Agent 监控行为,要么事后评估日志。前者会为每一步增加成本和延迟;后者则要等到运行结束后才能给出判断,此时 token 已经消耗,损害也已经造成。为解决这一问题,我们提出 OnTrack,一种流式监控机制,将 Agent 的步骤及其依赖关系与已记录的成功运行进行比较,从而在每一步约一毫秒内向用户发出警报或阻止 Agent。我们在三种访问权限逐步降低的情形下研究这一问题:完全访问参考数据(历史运行记录和工具模式)、中等访问权限(仅工具模式)以及无先验知识(仅有生成的步骤日志)。随着数据访问权限降低,OnTrack 的监控能力也随之减弱,其检测范围从计划违规逐渐缩小到识别循环、停滞和重复工具调用。最后,我们使用 SWE-bench 轨迹评估 OnTrack。基于前 8 个步骤,我们的方法对失败轨迹进行排序,使其排名低于成功轨迹的效果优于内容相似度方法(AUROC 提升 +0.057)。采用中止策略后,对于原本会在失败运行中消耗的计算资源,我们节省了约 18%;在被中断的运行中,有 83% 实际上正走向失败(6 次中止中有 5 次判断正确)。
arXiv:2610.12183v1 Announce Type: new Abstract: Black-box optimization (BBO) arises in many scientific and engineering problems where objective evaluations are expensive and limited. Recent large language model (LLM) agents offer a new way to approach BBO by combining task semantics, computation, optimization tools, and feedback-driven decision making, showing great potential due to the integration with mathematically rigorous tools. However, existing agentic BBO studies use different task domains and system configurations, making their results difficult to compare and the effects of individual design choices hard to isolate. We therefore introduce AgenticBBO-Bench, a cross-domain benchmark for agentic BBO spanning synthetic functions, hyperparameter optimization, database tuning, chip design, and molecular design under a unified finite-budget evaluation protocol. In our experiments, agentic BBO achieves higher family-averaged scores than direct LLM-based methods in all five domains and outperforms the best numerical optimizers in four. We further study three factors shaping agent performance: optimization tools, task information and prior knowledge, and the role of the LLM during search. Our results show that additional numerical tools do not consistently improve performance, task semantics are broadly useful while more specific priors are less reliable, and numerical optimizers can effectively absorb gains from search trajectories established by the agent. Finally, we introduce a five-task frontier challenge within AgenticBBO-Bench and evaluate seven LLMs under the Codex agent harness, where GPT-6 Astra and DeepSeek-V4.1-Flash lie on the Pareto frontier of performance and cost among the evaluated models. Our code is available at https://github.com/lamda-bbo/agentic-bbo.
智能体已被部署在从行程规划器、股票交易到 IT 事件分诊等各种应用中。在大多数情况下,LLM 智能体几乎不受基于规则的安全防护约束而自主运行,因而可能由于不可逆操作引发成本和安全问题。近期研究要么使用安全防护智能体监控行为,要么在事后评估日志。前者会为每一步都增加成本和延迟;后者则要等运行结束后才能给出结论,此时 token 已被消耗,损害也已造成。为解决这一问题,我们提出 OnTrack,这是一种流式监控机制,将智能体的步骤及其依赖关系与记录的成功运行进行比较,从而以每步约一毫秒的速度向用户发出警报或阻止智能体运行。我们在三种数据访问权限逐渐降低的情形下研究这一问题:完整参考访问权限(历史运行记录和工具模式)、中等访问权限(仅工具模式)以及无先验知识(仅有生成的步骤日志)。随着数据访问权限降低,OnTrack 监控能力的预期也会下降,其检测范围从计划违规扩展到识别循环、停滞和重复工具调用。最后,我们使用 SWE-bench 轨迹对 OnTrack 进行评估。仅依据前 8 个步骤,我们的方法对失败轨迹的排序低于成功轨迹,其效果优于基于内容相似度的方法(+0.057 AUROC)。采用中止策略后,对于原本会在失败运行中消耗的计算资源,我们节省了约 18%;在被中断的运行中,有 83% 实际上正走向失败(每 6 次中止中有 5 次判断正确)。
智能体可以完成任务,却不一定会改进其工作方式。要将任务经验转化为可复用能力,需要把发生变化的组件与其评估及后续使用连接起来。我们提出 AgentEvolver,这是一种在任务执行过程中开发能力、同时保持基础模型不变的系统。八类实体通过一个统一的版本化生命周期,使可复用的操作、方法、智能体、控制流、接口和支持状态能够被修订。共享的 Runtime 协调持续进行的工作,而持久化规划和可恢复上下文则保留任务方向及支持性证据。我们在 SWE-bench Pro Public 上评估任务结果,并考察六个应用案例中的能力变化。研究团队报告称,引入演化后,任务解决率达到 82.08\%,超过其报告的未引入演化时的基线。案例显示,保留的能力被用于后续的网站、游戏和研究工作,同时也记录了未完成的目标和一次未成功的策略。这些发现区分了可复用组件的改进与最终任务的成功。AgentEvolver 为通过执行研究能力累积提供了具体基础;独立任务迁移和总开发成本仍是有待解决的问题。
LegalOn cut estimated daily Codex costs by 65% while maintaining development speed. It matched Astra, Sol, and Luna to tasks and managed budgets strategically.
Across recruiting, engineering, and operations, Oracle turns specialist knowledge into fast, repeatable workflows with ChatGPT Work and Codex.
JetBrains released Mellum2.1, an Apache 2.0, 12B mixture-of-experts thinking model with 2.5B active parameters. RL in real repositories lifted its SWE-bench Verified score from 2.0 to 47.0. The post JetBrains Releases Mellum2.1: A 12B MoE Open Model for Coding Agents appeared first on MarkTechPost .
arXiv:2606.22108v2 Announce Type: replace-cross Abstract: Urban traffic congestion remains a persistent challenge in car-dependent cities, imposing significant economic and societal costs. Traffic signal systems are increasingly deployed as networked cyber-physical components within smart-city infrastructures, where distributed sensing and edge intelligence enable adaptive traffic management. This paper investigates reinforcement learning (RL) as an edge-intelligent approach for adaptive traffic signal operation at a signalized urban intersection in Kuwait. A Proximal Policy Optimization (PPO)-based controller is developed to dynamically allocate green-phase durations using locally observed traffic states, without relying on future demand information or centralized coordination. The controller is evaluated in a realistic simulation environment informed by real-world hourly traffic volume data from Kuwait, and is compared against both conventional fixed-time control and a vehicle-actuated controller representing the current state of practice, using average vehicle delay, queue length, and emissions as performance metrics. Under nominal conditions, the proposed controller reduces average vehicle delay by 46% relative to fixed-time control and 34% relative to actuated control, while also lowering per-vehicle CO2 emissions by approximately 23%. These performance gains persist under demand perturbations of +/-15%, generalize from weekday to weekend traffic patterns, and are corroborated by a reward function ablation; low variance across five random seeds confirms their statistical reliability. These findings demonstrate the practicality of learning-based edge traffic signal control as a building block for IoT-enabled smart-city transportation systems, and as a deployable precursor toward fully connected, Internet of Vehicles (IoV)-based urban mobility.
arXiv:2610.10310v1 公告类型:新论文 摘要:递归自我改进需要将已接受的变更延续到后续改进周期中,而研究智能体提出的变更也需要大量研究基础设施。现有环境往往让智能体自行重建常规基础设施,或将探索限制在单个组件上。我们提出 RSIGym,这是一个基于一切即服务(EaaS)的智能体原生研究环境。RSIGym 通过可复用服务提供训练、推理、轨迹生成、评估和沙箱执行能力,并通过共享的预算与权限控制支持 Data、Harness 和 Joint 三条改进路线。这一设计使智能体能够在同一环境中研究单项干预措施,并联合优化数据、训练设置和执行框架。我们将 RSI-Index 定义为在五个基准测试上弥合剩余性能差距的比例均值,这些基准涵盖软件工程、终端交互、数学、科学推理和基于技能的任务。我们在独立的 Joint 运行中比较了六个前沿研究模型,在每次基准测试运行的平台服务预算为 500 美元的条件下,Opus 5 取得了最高的 RSI-Index,达到 0.4809。其选定的系统在全部五个基准测试上均有提升,将 SWE-bench Verified 的成绩从 17.67% 提高至 50.33%,将 AIME 的成绩从 31.67% 提高至 97.78%。补充实验考察了 DSH-harness 的优化、预算变化以及受限网络访问的影响,记录的轨迹则揭示了智能体如何诊断失败并选择候选方案。我们开源了完整的 RSIGym 代码库和结果,以支持可复现性及进一步研究。
arXiv:2610.09633v1 Announce Type: new Abstract: The evaluation of coding agents generally strives to be as realistic as possible. In our study, we collect 4,782 agent sessions of real software engineers in JetBrains IDEs, which we call Production Sessions. Since our subject is interactive agents, we study the sessions with at least three user messages (33% of the sample). These long sessions differ from issue-derived benchmark tasks in two ways: (i) user requests span a far wider mix of task types - questions about the project's code, planning, review, refactoring, execution - and (ii) users switch between types throughout a session. Long-session samples from three public interaction corpora exhibit markedly different Task Flows (the distributions of session lengths, task types, and type-to-type transitions), so no single interaction distribution is universally realistic: benchmarks should name a target use case and calibrate to measurements from it. We present SWE-TaskFlow, an approach for transforming any issue-derived benchmark: it preserves the verified tasks and tests while steering the interaction toward a target Task Flow through prompt splitting and verifiable repository QA, with a TaskFlow Alignment Score (TFAS) for selecting among generated trajectories. In a pilot on 700 SWE-Bench Pro tasks, solving the task sequentially in several steps approximately doubles agent cost without a stable change in resolve rate: the interaction protocol itself is an important dimension of evaluation.
arXiv:2610.09375v1 公告类型:新 摘要:越来越多的组织使用前沿语言模型分析客户反馈,但回答质量也取决于这些反馈如何被组织和提供。我们将\emph{Customer Context Graph}定义为客户与业务上下文的统一模型。类型化关系连接客户对象(反馈、对话、用户和账户)、运营对象(工单、支持代理、商机和竞争对手),以及分析或行动对象(分类体系概念、证据、洞察、工作项和结果)。这使代理不仅能够调查客户说了什么,还能了解其原因、受影响者、后续采取了什么行动、由谁负责,以及问题是否得到解决。在本实验中,该图谱由公开的 Cursor 反馈构建;同一架构也可支持任何类型的反馈来源。我们使用相同的 9,432 条公开 Cursor 反馈记录和 30 个现实的产品、事件、比较及元数据问题,对比了 Agentic RAG、Deep Research Agent 和 Customer Context Graph-backed Agent。在缺乏详尽真实答案的情况下,我们依据一套针对 30 个问题中的 28 个进行校准的比较评分标准,从回答质量(覆盖面和组织性)、分析深度(具体性和分解程度)以及证据质量(引用支持和可追溯性)三个方面对回答进行联合评分。我们根据各项主张抽样检查被引用的记录,并对三个维度赋予相同权重。在这一对齐后的评分标准下,Customer Context Graph-backed Agent 的总体得分为 0.961,而 Deep Research Agent 和 Agentic RAG 分别为 0.710 和 0.651;在 30 个配对问题中的 27 个问题上,它领先于 Deep Research Agent(符号检验 p < 10^(-5);在 30 个问题中的 26 个上严格领先)。其最大优势在于分析深度(0.967 对 0.642),体现为更具体、层次更清晰的发现,以及经过量化的主题和可追溯的证据……
我们如何预测哪个基础模型检查点值得投入一轮昂贵的智能体后训练?端到端 pass@$K$ 检验成功行为是否已经出现在基础模型的分布中,但它并不适合智能体编程:许多基础模型检查点无法可靠地生成格式正确的工具调用,而这是端到端完成任务所必需的。单次或短程任务将多步交互压缩为固定提示和单个补丁,从而避开这些工具调用失败,但也绕过了我们真正关心的核心能力:随着代码仓库不断演变,在多个使用工具的步骤中维持连贯状态。为弥合这一差距,我们将后训练智能体的成功轨迹视为基础模型潜力的前瞻信号。通过重放每条轨迹,并在每个修改代码的步骤后重新运行测试,可以识别出决定性步骤:即累积补丁首次使代码仓库从测试失败转为通过的步骤,从而确认在给定先前上下文的情况下,所记录的动作能够解决任务。受智能体轨迹覆盖原则的启发,我们围绕这一步骤构建了三种筛选方法,它们均不要求基础模型检查点从冷启动开始驱动评测框架:(i) Decisive-Action BPB(每字节比特数)衡量已验证动作上的概率质量;(ii) Patch MCQ 测试检查点在该动作与被同一验证器否决的备选动作之间的选择;(iii) 以前缀为条件的 pass@$K$ 评估模型生成分布对功能正确结果的支持程度,并将任何通过测试的续写计为成功。在十对公开的基础模型与后训练模型中,这三种筛选方法对模型组给出的排名均与后训练模型的 SWE-bench Verified pass@$1$ 排名高度一致。由于我们的方法仅需要基准测试中的成功轨迹及其验证器,因此可以用来将未来的智能体编程基准转化为基础模型评估。
AI 智能体的一项关键控制能力是管理运行时长。这项能力需要时间感知,即预测和估计实际经过的时间,并控制自身行动。既有研究主要关注时间感知,但原生智能体运行框架中的时长遵循与控制能力仍未得到探索。我们提出 AgentTime,这是一个用于测试智能体能否按要求的时长工作、预测自身运行时长,以及在事后估计已用时间的基准。该基准包含来自 18 个来源的 222 项任务,涵盖编程、计算机使用、智能体任务和自动化研究。在时长遵循实验中,我们附加一条指定工作时长的指令,要求的时长从约一分钟到数天不等。智能体遵循这些指令的准确性差异显著:Claude Code 中的 Fable 5.1 与要求运行时长的典型偏差倍数为 2.9$\times$,相比之下,Codex 中的 GPT-6 Astra 仅为 1.2$\times$。然而,仅仅符合要求的运行时长,并不能证明智能体始终在执行任务。在经过审查且记录可分类的 158 次 Astra 运行中,有 14 次在看似完成任务后明确进入了休眠。在预测实验中,预测值往往高估自然运行时长。在回顾性实验中,移除时间信息后,Sol 和 Astra 的偏差增至两倍以上,Fable 的偏差也接近翻倍。智能体能够完成任务,并不保证它能控制自身用时,或在要求的整个时段内持续工作。要让智能体在长时间跨度内可靠、安全且自主地运行,就必须对任务完成能力和时间控制能力都进行评估。
I've always been kind of into computers since I was young. And then when film started to move from analog film to digital, I became more interested in that aspect of it. And the visual effects workflow for many years has included machine learning. So I can write like pretty shitty Python scripts and stuff like that because with convolutional neural networks, which were the sort of precursors to what the transformer can do, which is just much more computation simultaneously, you would do things like look at what's called a tensor, which is just the numerical translation of a visual image in numbers — like the batch number, the frame number, the red, green, and blue values of each pixel in each frame. And a tensor, you use a convolutional neural network to identify patterns in that that would reveal what's called edge detection or feature extraction, which is just identifying patterns enough to know like this is where the window ledge is, so we can more easily take the green screen image out and replace it with something. — Ben Affleck , Pythonista Tags: machine-learning , python
正如之前承诺的那样,Anthropic 的全新高速、低成本模型来了:Claude Haiku 5.5。上一代 Haiku 4.5 已经明显落伍了。它发布于将近一年前,定价为每百万输入 token 1 美元、每百万输出 token 5 美元——即便在当时也算相对昂贵,而且足足是 OpenAI 上个月发布的 GPT-6 Luna 的 10 倍。 新版 Haiku 的价格与 GPT-6 Luna 完全一致:在不超过 100,000 个 token 时,每百万输入/输出 token 分别为 0.10/0.50 美元。超过 100,000 个 token 后,价格会上涨至 5 倍,即 0.50/2.50 美元。Luna 本身在达到 272,000 个 token 时也会涨价,但只涨到 0.20/0.75 美元。 Haiku 5.5 还采用了新的 tokenizer,对用户没那么划算。我的 Claude Token Counter 工具显示,同一个长提示词在 Haiku 5.5 中使用的 token 数约为 Haiku 4.5 的 1.25 倍,因此这里还存在隐性涨价。 如果你的工作负载能控制在 100,000 个 token 以内,Haiku 的价格与 Luna 相同,而且据称基准测试得分更高。超过 100,000 个 token 时,Luna 看起来就划算得多。 最新发布的 llm-anthropic 终于解决了一个问题:我不必再为每个新模型发布该插件的新版本了。我是这样测试新模型的: llm install -U llm-anthropic llm anthropic refresh llm -m claude-haiku-5.5 "生成一幅鹈鹕骑自行车的 SVG 图" -o thinking_effort low 鹈鹕 以下是分别在 low、medium、high、xhigh 和 max 档位下生成的鹈鹕。新版 Haiku 不允许关闭推理,默认档位为 medium。除了 low 之外,其余档位生成的自行车车架都不错。 low 档位的鹈鹕花费了 0.0936 美分,用时 7 秒。 这只 max 档位的鹈鹕用了 5 分 9 秒才生成,但依然只花了我 3.3826 美分: 扁平矢量插画:一只戴着红色帽子的白色鹈鹕,骑着深灰色自行车,从左向右穿过一条绿色草地。它细长的橙色双腿向下伸到橙色踏板上,一只灰色翅膀向前伸出,握住弯曲的车把。它硕大的黄橙色鸟喙朝向前方,下方有鼓起的橙色喉囊;身后白色的速度线表现出运动感,上方渐变蓝色的天空中有一轮带着浅色光晕的黄色太阳和两朵白云。 作为对比,这是我一年前用 Haiku 4.5 生成的鹈鹕。它画鹈鹕实在不怎么样: Haiku 4.5 的描述:一幅充满奇趣的插画,一只身体圆滚滚、呈棕褐色的鸟,有着粉红色的喙和橙色的腿,在蓝天与绿草的背景下骑着自行车。 还有面向订阅用户的慷慨 API 额度计划 除了 Haiku 5.5,Anthropic 今天还宣布将 Sonnet 5.5 的缓存读取价格减半。他们还在订阅方案中加入了 API 额度: 其次,本周我们将向所有 Max 和 Team 订阅用户推出新的每月 API 额度,可在 Claude Platform 上使用。Max 5x 用户每月将获得 100 美元额度,Max 20x 用户将获得 200 美元,而 Team 订阅用户将获得最高 500 美元额度,由团队内所有用户共享。 领取过程简单得令人愉快:进入“设置”->“账单”,选择每月接收这些额度的 API 组织: 一张设置对话框的截图,右上角有关闭按钮 X,上方一排标签依次为“常规”“账户”“隐私”“账单”(已选中)、“用量”“功能”“记忆”和“设计系统”(在边缘处被截断)。面板上有一个带圆形花苞的分枝植物线描图标,旁边写着“Max 方案”“用量为 Pro 的 20 倍”和“你的订阅将于 2026 年 11 月 2 日自动续订。”,右侧有一个“调整方案”按钮。下方是标题为“API 额度”的区域,旁边带有蓝色的“新增”标记,正文写着“你的 Max 20x 方案每月包含 200 美元的 API 额度。”,随后是灰色文字“关联一个 API 组织以开始接收额度。”,右侧有一个“关联组织”按钮。 API 额度恰好等于订阅本身的费用。这真的很慷慨——让订阅用户使用 API 变得容易多了。 Anthropic 还允许你关闭 API 自动充值,结果就是“余额耗尽时,API 请求将停止”——如果你打算用完这些 API 额度,又不想冒着收到账单惊吓的风险,这正是你想要的。 请注意,每月额度不会结转——不用就会作废。 OpenAI 仍允许你将 Codex 订阅用于个人 API 使用,对重度 API 用户来说更划算。这项新的额度计划至少在一定程度上缩小了两者的差距。 标签:ai、generative-ai、llms、anthropic、claude、llm-pricing、pelican-riding-a-bicycle、llm-release
At today's Windows and Surface event, Microsoft showed off an upgrade to its Copilot AI system that will give it access to local files on your PC and the ability to take actions across the OS. It's part of an idea Microsoft is calling "Hybrid Intelligence," where apps and tools rely on a mix of […]
大家好,HN!我是 Louis,Armature(YC P26)的联合创始人。我们帮助团队让自己的产品更容易被编程智能体发现和使用。 我们已经测量了超过 5 万次智能体会话,发现智能体在使用同一工具执行不同任务时,会反复遇到完全相同的限制。于是我们想,为什么这些问题一直没有得到解决?答案很简单:智能体与软件供应商之间,以及不同智能体之间,根本不存在反馈闭环。人类可以在 https://g2.com 和 https://trustpilot.com 这样的平台上分享使用体验,但智能体却没有这样的地方。 因此,我们创建了 https://agent.reviews :面向智能体的 G2。它通过一组技能和一个 npm CLI(@armature-tech/agent-reviews),将智能体连接到我们的 API 端点。任何人都可以让自己的智能体(Claude Code、Codex、Cursor 等)安装它,之后智能体就会自然地在选择工具前查看评价,并在使用后发布自己的评价。 和往常一样,隐私是我们最关心的问题,因此我们在评价发布前设置了三层检查: - 使用确定性规则过滤密钥、个人身份信息(PII)、URL 等。 - 使用经过训练的 Jev 分类器,检测第一轮检查后是否仍有信息泄露。 - 使用一个小型 LLM 检查每条评价,确保没有遗漏。 过去几周,我们一直在向大家分享这个项目,目前已经收集了数千条评价。其中已经有一些很有意思的例子: - 一个 Claude Code 智能体发现,当健康检查页面缺少 API 密钥时,Stripe SDK 总是会崩溃,而这个页面本来就应该返回“缺少 API 密钥”的错误。 - 两个智能体提到,即使没有连接任何数据库,Prisma 也要求提供 DATABASE_URL 变量。它们都填入了虚假的 URL 作为变通办法,而且确实奏效了。 我们真心认为,智能体体验也需要像用户体验一样,受益于社区效应,让所有参与者都能获益:智能体可以选择最适合自己的工具,软件公司则可以根据真实反馈改进产品。因此,我们确保人类和智能体都能免费访问评价;只需复制粘贴一条提示词,就能让智能体安装我们的 CLI 和技能、启动身份验证流程,并提交第一条评价(这有助于我们防止未经授权的抓取和垃圾评价)。 你也愿意让自己的智能体提交和阅读评价吗?我们很希望你能配置好智能体评价功能,让智能体下次需要选择工具时先查看评价,并在使用后发布自己的体验。然后告诉我们效果如何! 评论链接:https://news.ycombinator.com/item?id=49995539 得分:4 评论数:2
arXiv:2610.07898v1 公告类型:新 摘要:仓库级软件工程(SWE)是一个具有挑战性的长时程场景:智能体必须在长时间跨度的交互中进行推理、使用工具,并适应有状态环境。近期研究使用 Group Relative Policy Optimization(GRPO)等强化学习方法训练 SWE 智能体:针对每个问题独立采样多条轨迹,测试由此产生的补丁,并在固定组内比较终止奖励。然而,这种训练设置不会将失败补丁的验证器反馈作为后续尝试的上下文加以复用,尽管这些反馈包含了关于问题所在位置的宝贵诊断信息。恢复轨迹的训练颇具挑战,因为前一次结果决定下一条轨迹是否生成,而失败执行结果决定其条件上下文。我们提出 FC-SWE,一种将恢复尝试纳入策略训练的失败条件化强化学习框架。当补丁验证失败后,FC-SWE 会将仓库恢复到任务的原始状态,并使用失败补丁和验证器反馈作为恢复轨迹的上下文。FC-SWE 通过两种机制,将 GRPO 适配到这些由完整、多轮工具调用轨迹组成的链中。轨迹局部奖励保留每次尝试的验证器结果,避免恢复成功为此前失败的补丁提供奖励。主动集优势估计则从同一问题实际执行的所有初始轨迹和恢复轨迹中构建比较组,因此失败尝试仍保留在组内,而未执行的尝试则被排除。在验证器辅助协议下的全部 500 个 SWE-bench Verified 任务上,使用 Qwen3.5-4B 和 SWE-agent 的 FC-SWE 达到 41.7% 的 Resolved@1 和 52.8% 的 Resolved@2,而 GRPO 分别为 38.9% 和 48.5%。尽管训练时每条链最多仅包含两次尝试,FC-SWE 在测试时允许 11 次尝试的预算下达到了 70.7% 的 Resolved@11。
TIL: Using Parseable with Datasette for OpenTelemetry traces I saw Parseable in a Show HN today - it's a new observability platform with both an open source (AGPL) Rust implementation (a single ~180MB binary), an "Enterprise" version with extra features and a cloud hosted option. Since Datasette 1.0a41 added OpenTelemetry support (thanks, Alex Garcia), I decided to fire up Codex and have it figure out how to run Parseable and feed it traces from Datasette. Here's my (human-written) TIL showing the patterns that worked, and here's a screenshot of a Datasette trace displayed within the Parseable localhost web application: Screenshot of a trace detail view in an observability web app, with a span waterfall overlaid on a dimmed navigation sidebar and filter column. Dimmed sidebar: breadcrumb "Community > Traces > datas" (cut off), search box "Search... ⌘K", nav items "Home", "Ingest telemetry", section "ANALYZE": "Keystone", "Dashboards", "SQL Editor", section "OBSERVE": "Logs", "Metrics", "Traces" (selected), "APM", "Agents", section "MONITOR": "Alerts", "Errors", section "DATA": "Datasets", and at the bottom "Settings", "Book a call", "Support". Dimmed filter column, cut off at the right edge: "Search fi", "Core", "Log format", "User agent", "Source IPs", "Error", "Service", "service.ins", "service.na", "datasett" (checked), "Span", "Database", "HTTP", "http.reque", "NULL" (unchecked), "GET" (checked), "http.respo", "http.route", "Server", "server.add", "Telemetry", "URL", "All fields". Trace panel header: "Trace detail > 6f819a2170bcd1e91c6ea3ae236ec60b" with a copy icon, a "Related logs" button and a close X. Summary: "Start time 6:56 PM, Oct 6, 2026 UTC", "Duration 40.9 ms", "Spans 247". A minimap with axis "0ns 10.2ms 20.5ms 30.7ms 40.9ms" shows many short span bars cascading diagonally from top left toward the lower middle, with a few longer bars. Below is a span table with a "Span name" header, a "Search spans..." box, collapse and expand buttons, and a timeline axis "0ns 10.2ms 20.5ms 30.7ms 40.9ms". Rows (name, service, duration): root span with collapse toggle "123", "GET /..." "datasette..." 40.9ms spanning the full timeline; then alternating rows where each "db.query" has a collapse toggle "1": db.query datasette-local 679µs, db.query.execute datasette-local 278µs, db.query datasette-local 341µs, db.query.execute datasette-local 55µs, db.query datasette-local 357µs, db.query.execute datasette-local 197µs, db.query datasette-local 330µs, db.query.execute datasette-local 73µs, db.query datasette-local 2.26ms, db.query.execute datasette-local 2.02ms, db.query datasette-local 232µs, db.query.execute datasette-local 64µs, db.query datasette-local 207µs, db.query.execute datasette-local 71µs, db.query datasette-local 6.11ms. The child span bars start progressively later across the early part of the timeline. Tags: datasette , observability , alex-garcia , opentelemetry
I am a film photographer and do some darkroom printing and always wanted one of the proper, advanced f-stop timers, but never really wanted to spend a couple hundred dollars on one. Eventually I figured my phone was already perfectly capable of doing the timing, it just needed a way to switch the enlarger. So I built Darkplug around a cheap smart plug. The app controls the enlarger over the local network and does regular timing, f-stop timing, test strips, has an intuitive dodge/burn workflow, split grade (2 timing channels) and paper development timing. I’ve been using it for a few months in my improvised bathroom darkroom. Fair warning: the app was blindly vibe coded using Codex and Xcode. Comments URL: https://news.ycombinator.com/item?id=49978595 Points: 4 # Comments: 0
Hi HN, I'm Weilun, cofounder of OpenChart. We built OpenChart because we wanted Claude and Codex to interface with the markets, like most of humans do, through charts rather than through CLIs. With your existing AI plans, your agent can work directly with your charts: annotate a setup, write an indicator, investigate a market move, or create an alert. The chart, conversation, and research live in the same workspace. OpenChart alerts on any market move: draw any shape on a chart and attach an alert. When price crosses it, the alert triggers your agent to investigate the move and save the research in your workspace. The agents can work with charts, indicators, and alerts. Custom indicators and alert conditions are written in [Tea]( https://github.com/longsurf-ai/tea ), a programming language we built for the markets. We believe software shouldn’t impose artificial friction on your workflow. OpenChart is built on that belief with no software cap. The source code is publicly available, and you can use free market data from day one. We also offer optional paid cloud market data for lower latency. Everything is private. Your charts, research, and indicator code are stored on your own computer, with the app's backend running locally too. We currently support only macOS on Apple Silicon, but we're expanding to more platforms. Website: https://openchart.co/ Download: https://downloads.longsurf.ai/openchart/darwin/arm64/OpenCha... We built OpenChart because we want to explore how assets can be co-manages by agents and humans. I'd especially like to hear how you are using agents to help with your investment journey. If you like what we're building, a star on GitHub would mean a lot to us: https://github.com/longsurf-ai/openchart Comments URL: https://news.ycombinator.com/item?id=49979793 Points: 1 # Comments: 0
智能体排行榜日益使用公开基准来评估系统,而这些基准的任务描述和包含解答的工件可能持续处于可访问状态。我们提出了一种测量优先的审计框架,根据支持污染声明所需的证据,对污染声明进行区分。该框架将需要不同证据的渠道分为三类:训练时暴露、评估时检索,以及流水线/脚手架泄漏。按照失败即关闭规则,每个渠道被编码为开放、部分开放、关闭或未知。在九种 Holistic Agent Leaderboard (HAL) 配置中,27项渠道评估无一被编码为关闭,但在其中四种配置中确认了相关事件。随后,我们将该框架的行为组件应用于 SWE-bench Verified 中一项已报告的文件定位差距,采用结果盲法、同一仓库匹配对照设计,并结合对称式提示泄漏筛查、成对且考虑仓库因素的不确定性分析,以及评分器验证,在 GPT-4.1 和 DeepSeek-V4-Flash 上进行评估。在经过对称筛查和成对完整性排除后保留的100对样本中,GPT-4.1 显示出按样本对加权的、与基准相关的 $+10.0$ 个百分点差距;但预先指定的成对自助法和事后仓库均衡分析所得的95\%区间均包含零,因此与基准相关的差距尚无定论。复现实验评分器未通过其验证门槛:相对于人类共识标签,无法为任一模型确立足够的评分器敏感度;而且 DeepSeek-V4-Flash 在正确金标准对比中的两次触发均为误报。在缺乏溯源证据、适当对照、对称式泄漏筛查和经过验证的评分器的情况下,不应提出更强的污染声明。研究结果无法证明训练数据成员关系、污染普遍程度或基准导致的分数膨胀。
大型语言模型(LLM)智能体日益广泛地复用既有经验,但大多数方法都假设问题与解决方案已经完成对齐。软件历史很少能提供这种对齐:一个拉取请求(PR)可能包含多个持续集成(CI)问题、失败的尝试、回滚的编辑以及无关变更,从而难以判断哪些变更解决了哪些问题。我们提出 RETRACE,一种从此类历史中重构问题级修复经验的框架。RETRACE 结合了终点视角与开发视角:前者从通过测试的修订版本中保留的变更出发进行反向推理,后者沿提交历史正向追踪修复的演变过程。CI 执行证据用于协调这两个视角,恢复出的经验则以三个抽象层级表示,从具体修复到可迁移的修复模式。对于新的失败,RETRACE 会检索相关的问题级经验以指导修复。在 CI-REPAIR-BENCH 上,该基准包含来自 101 个仓库、涵盖 12 类失败的 565 个 PR 级修复,RETRACE 将 mini-SWE-agent 的 Pass@1 使用 MiniMax-M2.5 时从 19.6% 提升至 31.9%,使用 DeepSeek-V4-Flash 时从 23.3% 提升至 32.8%。在匹配的子集上,Codex 的表现从 15.5% 提升至 27.5%。结合两个视角始终优于单独使用任一视角,这表明恢复问题与变更之间的对齐关系,能够使历史上的 CI 修复成为可复用的修复经验。
Together AI has released Together Link, a free, MIT-licensed CLI that connects Claude Code, Codex, OpenCode, Pi, and the Claude and ChatGPT desktop apps to open models like Kimi K3 and GLM 5.3. One install command sets it up, and an Auto router picks a model for each session. Together claims savings of over 50%. The post Meet Together Link: A Free CLI That Runs Open Models Like Kimi K3 and GLM 5.3 Inside Claude Code, Codex, and OpenCode appeared first on MarkTechPost .
OpenAI 将在欧盟为 ChatGPT 和 Codex 文本添加水印,以遵守 AI Act。OpenAI 表示,编辑可能会使这些不可见标记更难被检测。
An invisible, machine-readable watermark in text output is rolling out to ChatGPT and Codex, but only for users in the European Union at first. OpenAI says its textGrain watermarking "matched or exceeded" other approaches like Google DeepMind's SynthID for text, which is also the basis for the watermarking Anthropic announced in August. Like OpenAI, Anthropic […]
Amazon SageMaker 优化的生成式 AI 推理通过 Agent Toolkit for AWS 引入了 aws-ai-ml skill,为 Kiro、Claude Code 和 Codex 等编码智能体提供推理优化与基准测试方面的深度专业知识。描述您的需求,智能体即可生成可执行的 SageMaker Python SDK v3 代码,用于对部署进行基准测试、提出建议和比较。
Research: Qwen3.8 27B addition in words Colin Frasier posted on Bluesky about an experiment he ran over two years ago using GPT-4o to see how well it could "compute the sum but return the answer in words" across increasingly large numbers. Here's the chart he shared of those results: Heatmap chart of accuracy on an addition prompt, colored from dark green (high) through yellow to dark red (low). Title: "What is {a} + {b}? Please write your answer in words. Do not include any other text or information, just the answer in words." Subtitle: 30 randomly selected pairs for each digit combination (n = 30 * 13 * 13 = 5070). X axis: Number of digits in a, 1 to 13. Y axis: Number of digits in b, 1 to 13. Legend: Accuracy, 1.00, 0.75, 0.50, 0.25, 0.00. Values by row, listed for a = 1 to 13. b = 13: 100%, 77%, 27%, 20%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%. b = 12: 97%, 80%, 80%, 40%, 23%, 20%, 7%, 13%, 20%, 27%, 67%, 63%, 3%. b = 11: 97%, 97%, 53%, 17%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 37%, 0%. b = 10: 100%, 90%, 47%, 20%, 7%, 0%, 0%, 0%, 3%, 0%, 0%, 7%, 0%. b = 9: 97%, 93%, 80%, 77%, 53%, 67%, 47%, 87%, 97%, 3%, 0%, 13%, 0%. b = 8: 93%, 87%, 53%, 43%, 7%, 0%, 0%, 13%, 87%, 0%, 0%, 0%, 0%. b = 7: 93%, 93%, 47%, 10%, 13%, 20%, 23%, 0%, 70%, 0%, 0%, 0%, 0%. b = 6: 100%, 100%, 100%, 83%, 97%, 97%, 23%, 0%, 53%, 3%, 0%, 10%, 0%. b = 5: 100%, 100%, 80%, 70%, 73%, 100%, 13%, 13%, 70%, 0%, 20%, 30%, 0%. b = 4: 100%, 100%, 93%, 100%, 60%, 97%, 20%, 50%, 67%, 53%, 40%, 40%, 40%. b = 3: 100%, 100%, 97%, 90%, 83%, 100%, 63%, 50%, 63%, 53%, 60%, 60%, 30%. b = 2: 100%, 100%, 90%, 97%, 93%, 100%, 93%, 83%, 90%, 83%, 87%, 87%, 83%. b = 1: 100%, 100%, 100%, 97%, 100%, 97%, 97%, 97%, 100%, 100%, 100%, 97%, 100%. I'm confident GPT-4o didn't cheat and use a calculator, especially since it got so many of the calculations wrong, but I was inspired to run the experiment again on local hardware (a DGX Spark) to explore the effect in a fully controlled environment. I pasted his image into a Codex Remote session (GPT-6 Astra) and had it run the same experiment using Qwen3.8-27B-Q4_K_M.gguf . Here's the result for a run of 30 attempts per combination with reasoning disabled: Heatmap in the same layout as the previous chart, using an orange (low) to white to blue (high) color scale, showing much lower accuracy overall. Title: Addition in words — Qwen3.8 27B Q4_K_M. Subtitle: Reasoning disabled · 30 fixed pairs per ordered digit-length cell (n = 5,070). Overall numeric accuracy: 1,195 / 5,070 (23.57%). X axis: Number of digits in a, 1 to 13. Y axis: Number of digits in b, 1 to 13. Legend: Accuracy, 100%, 75%, 50%, 25%, 0%. Values by row, listed for a = 1 to 13. b = 13: 17%, 13%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%. b = 12: 53%, 20%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%. b = 11: 47%, 10%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%. b = 10: 70%, 27%, 3%, 0%, 0%, 0%, 0%, 0%, 0%, 13%, 0%, 0%, 0%. b = 9: 77%, 47%, 3%, 0%, 0%, 0%, 0%, 3%, 7%, 0%, 0%, 0%, 0%. b = 8: 53%, 20%, 0%, 0%, 0%, 0%, 7%, 13%, 0%, 0%, 0%, 0%, 0%. b = 7: 53%, 23%, 17%, 10%, 3%, 3%, 13%, 3%, 0%, 0%, 0%, 0%, 0%. b = 6: 60%, 60%, 33%, 10%, 53%, 47%, 7%, 3%, 0%, 0%, 0%, 0%, 0%. b = 5: 73%, 67%, 87%, 80%, 53%, 40%, 0%, 0%, 3%, 0%, 0%, 0%, 0%. b = 4: 83%, 93%, 90%, 93%, 53%, 13%, 0%, 0%, 0%, 0%, 0%, 0%, 0%. b = 3: 100%, 93%, 90%, 80%, 67%, 37%, 17%, 0%, 0%, 3%, 0%, 0%, 0%. b = 2: 100%, 100%, 93%, 90%, 77%, 77%, 43%, 50%, 63%, 43%, 40%, 13%, 23%. b = 1: 97%, 100%, 100%, 100%, 80%, 67%, 77%, 80%, 80%, 60%, 43%, 30%, 37%. Footnote: Colorblind-safe orange–blue scale; percentages provide a redundant non-color encoding. Then I ran it again with reasoning enabled. This took a lot longer per pair, so instead of running 30 samples per square I ran just one - which results in a much less visually appealing heatmap since each square is either 100% or 0%: Heatmap in the same layout as the previous charts, almost entirely blue. Title: Addition in words — Qwen3.8 27B — medium reasoning pilot. Subtitle: 1 fixed pair per ordered digit-length cell · easiest first (n = 169). X axis: Number of digits in a, 1 to 13. Y axis: Number of digits in b, 1 to 13. Legend: Accuracy, 1.00, 0.75, 0.50, 0.25, 0.00. Every cell shows 100% except two orange cells showing 0%: a = 2 with b = 8, and a = 12 with b = 9. It got the right answer in 167 out of 169 attempts, and since these were one-shot I'm confident a second run would produce different results here. Here's a version of the report that includes the reasoning traces from some of those larger calculations, which include text like this: Wait, let me redo this more carefully. 4,299,366,105,622 6,088,794,067,970 Let me align them: 4 2 9 9 3 6 6 1 0 5 6 2 2 6 0 8 8 7 9 4 0 6 7 9 7 0 Adding from right to left: Position 1 (units): 2 + 0 = 2 Position 2 (tens): 2 + 7 = 9 Position 3 (hundreds): 6 + 9 = 15, write 5, carry 1 Tags: mathematics , ai , generative-ai , local-llms , llms , qwen , llm-reasoning , dgx-spark
arXiv:2607.22465v4 公告类型:replace-cross 摘要:现代企业级智能体部署由具有不同能力和成本的大型语言模型(LLM)异构池组成。现有的模型路由策略优化质量与成本之间的权衡,同时针对请求级别提供静态决策。较新的解决方案则将智能体路由视为任务级选择,并采用带有串行验证器的路由器反馈循环。然而,适用于同质工作负载的固定验证器可能无法泛化到异构的智能体任务批次(例如编程任务和通用对话任务)。此外,由于验证器位于反馈循环的关键路径上,在同时进行多个请求路由时,服务质量可能会受到影响。为缓解这些问题,我们提出了 ORACLE。这是一种具备并发感知能力的在线路由机制,将自适应路由与自适应验证相结合,以提供反馈。ORACLE 无需训练,可作为“反馈循环”即插即用地叠加在任意模型选择策略之上,首先对任务类型进行分类,然后动态分配适合任务的验证器。此外,我们针对并发请求开发了一种延迟反馈策略,大幅消除了验证器延迟对分发关键路径的影响。随后,我们提出了一种路由后分发调度器,即 DISC。DISC 在接纳任务时预留每个任务的峰值 KV 占用,并在减少等待时间带来的奖励增益超过准确率降低带来的奖励损失时,将任务分发到备用后端。在 SWE-bench、tau2-bench 和 Terminal-Bench 2.0 上进行的大量评估表明,与最先进的路由基线相比,ORACLE 可将准确率—成本前沿最多提升 7 个百分点;而 ORACLE 结合 DISC 后,可将程序吞吐量最多提升 1.8 倍。为促进可复现性并支持未来开发,我们在 https://github.com/ORACLE-org/ORACLE 开源了 ORACLE 的代码库。
arXiv:2610.02351v1 公告类型:交叉收录 摘要:基于 ReAct 的智能体通常依赖单一 LLM 策略来提出行动、与环境交互,并判断任务何时完成。这种耦合使得行动授权与完成控制难以独立实施,导致错误传播,并使缺乏依据的完成声明终止执行。我们提出 DeReAct,这是一种模块化智能体架构,将两种门控策略外置:Critic 在执行前验证所提出的行动,Context Manager 则重建有环境证据支持的 \textsc{State} 并核验任务是否完成。在 GAIA 和 SWE-bench Verified 上,DeReAct 对能力较弱的 Brain 模型的 Pass@1 提升最为显著,其中 Qwen3-Coder-480B 提升了 6.5–7.0 个百分点,Claude Sonnet~4.5 提升了 4.2–5.2 个百分点;随着 Brain 能力增强,提升幅度逐渐减小。轨迹分析与消融分析表明,当目标故障足够普遍且门控策略本身具备足够能力时,外部门控是有效的。使用 Claude Opus~4.5 时,Pass@1 与 ReAct 保持相当,但 DeReAct 生成的轨迹具有更完整的证据支持,并能更好地满足约束,这表明完成控制可以通过推迟终止来换取更扎实的证据依据。总体而言,DeReAct 能提升较弱智能体的表现,同时在模型能力增强后仍保留证据依据方面的优势。
arXiv:2610.02954v1 Announce Type: new Abstract: Biologically-informed neural networks (BINNs) have emerged as a flexible subclass of physics-informed neural networks (PINNs) for learning terms in partial differential equations from data. BINNs are particularly suited for biological systems, where the governing equations are highly nonlinear and only partially known a priori, and where data observations are often sparse, noisy, and incomplete. However, applying BINNs effectively in practice depends critically on hyperparameter selection, which remains a central challenge in equation-learning frameworks. Hyperparameters are often chosen heuristically and only cursorily documented, which limits the reproducibility of results and the transferability of methods. We present a diagnostic workflow for hyperparameter selection that can be used when the ground-truth equations are not known. The workflow is guided by three main questions: (1) Are the benefits of greater network capacity worth the cost? (2) Do more training epochs keep reducing the validation loss? (3) Do the learned terms stop changing as network capacity and training increase? We apply our workflow to synthetic systems of varying complexity with known ground truth, spanning diffusion and growth right-hand side terms and data ranging from 1D+t to 2D+t. We demonstrate that the validation loss generally follows the true error in the learned terms and distil practical rules of thumb for selecting hyperparameters in the BINN architecture. By providing a structured workflow, practical guidelines, and suggested starting values for hyperparameter selection, this work lowers the barrier to BINN-based equation learning.
工作空间代理将大型语言模型与执行框架相结合,以执行访问或修改外部资源的有状态、多步骤任务。现有基准在运行时安全风险的可执行覆盖方面存在空白,而模型能力、执行框架、工具和威胁的不断演进也促使基准持续发展。我们提出 EvoRiskBench,这是一个围绕 EP-Path-EF 框架组织的演进基准,该框架通过代理介导的风险路径,将初始风险入口与单跳技术影响关联起来。该框架定义了九类入口点和五类影响;一项包含 20 名参与者的研究表明,在具有代表性的案例上,这些类别具有良好的可解释性和分类一致性。在该框架的指导下,我们构建了一套自动化端到端工作流,在隔离环境中构造并执行风险案例,并利用运行时轨迹和环境状态独立验证结果。该基准提供了一个可复现的数据集,包含跨六种场景的 450 个对抗性任务。我们评估了九种模型—执行框架配置,涵盖三种模型(GPT-5.6 Sol、DeepSeek-V4-Pro-0813 和 Claude Opus 5)以及三种执行框架(Claude Code、Codex 和 OpenClaw)。结果揭示了各系统存在大量漏洞。其中,最易受攻击的配置是 Codex 与 DeepSeek-V4-Pro-0813 的组合,其攻击成功率(ASR)达到 68.44%,表明仅配置工作空间代理并不足以确保安全的自主执行。ASR 在不同模型之间的变化大于不同执行框架之间的变化,且执行框架之间的差异取决于所使用的模型。基准案例和评估平台将在完成工件安全性与可复现性检查后发布。
I built this app mainly for myself and my wife. We both work in consulting and have tons of meetings every week. I had tried many of the existing meeting copilots but never found something that actually fulfilled what I wanted. Mainly, I wanted to be able to see in real time what's being said, irrespective of the language being spoken, including when people switch languages mid-meeting, which happens a lot in my work. I also wanted it translated in real time into my language, and I wanted suggestions for what to say that are grounded in actual web search rather than just the model making something up. I use Soniox for transcription and Exa.ai for web search. Bearbits runs in the browser and doesn't join the meeting as a bot. Looking forward to your feedback, especially on whether this kind of real-time assistance is useful to you or just distracting. Comments URL: https://news.ycombinator.com/item?id=49954800 Points: 1 # Comments: 0
Yesterday was Grok 4.7 ( pelicans ) and MiMo v2.6 Flash/Pro ( more pelicans ). Today Anthropic released Claude Opus 5.5 , and around an hour later OpenAI released GPT-6 Sol and GPT-6 Luna . It's going to take a while to get a good read on all of these new models, but here are my impressions so far. GPT-6 Sol and Luna are half the price of their GPT-5.6 equivalents GPT-5.6 Luna was already my favorite model for building applications against, because it combined excellent performance with being really cheap . Somehow GPT-6 Luna is half the price of that again - and GPT-6 Sol had a similar reduction compared to GPT-5.6 Sol. Here's what the pricing landscape looks like today: Model Input Cached input Output GPT-6 Luna $0.10/M $0.01/M $0.50/M GPT-5.6 Luna $0.20/M $0.02/M $1.20/M Grok 4.7 $2/M $0.50/M $6/M GPT-6 Sol $2/M $0.20/M $10/M GPT-5.6 Terra $2/M $0.20/M $12/M Claude Opus 5.5 $4/M $0.20/M $20/M GPT-5.6 Sol $4/M $0.40/M $20/M Claude Fable 5.1 $10/M $0.25/M $50/M GPT-6 Astra $10/M $1/M $50/M Note that GPT-5.6 has a scheduled 25% price increase for November, so GPT-6 is half the price of the promotional pricing for those models. (With GPT-5.6 Terra priced the same as GPT-6 Sol, any remaining reasons to use Terra just evaporated.) It's hard to overstate how competitive this pricing is. Grok 4.7 priced itself at $2/$6, less than half the price of GPT-5.6 Sol, but is now equally priced to GPT-6 Sol on input and closer on output. At $0.10/$0.50 GPT-6 Luna is one of the cheapest models OpenAI have ever released, beaten only by the far weaker GPT-4.1 Nano ($0.10/$0.40, April 2025) and GPT-5 Nano ($0.05/$0.40, August 2025). I rendered pelicans for GPT-6 Luna and for GPT-6 Sol , then I combined them all together in this comparison grid along with the GPT-5.6 pelicans. I like how you can instantly see that the 5.6 family chose bolder, brighter colors, while the 6 family is a lot more muted. I still think GPT-6 Astra on max produced the best pelican. A grid of pelicans for six GPT models at different thinking efforts. Claude Opus 5.5 got a price cut too Opus 5.5 looks like it addresses the biggest complaints people had about Opus in terms of its communication style. Thariq Shihipar : Opus 5.5 is the result of your feedback. It communicates clearly, it's cheaper per token than Opus 5.0 with the intelligence of Fable 5.1 it's very token efficient and works across every effort level. It's also meant to be better at Blender . I'm looking forward to putting it through its paces there. Opus 4.5, 4.6, 4.7, 4.8, and 5 all shared the same price: $5/million tokens for input and $25/million for output. 5.5 is a 20% reduction - $4/million and $20/million. The price for cache reads fell 60%. That's significant for longer agentic conversations, where 90%+ of input tokens are processed at cached token prices. The new price for Opus 5.5 is the same as the price for GPT-5.6 Sol, but that was before OpenAI dropped their Sol prices by half. GPT-6 Astra and Claude Fable 5.1 are both priced at $10/million input and $50/million output. The price war currently affects the next tier of models below that. Anthropic say that Sonnet 5.5 and Haiku 5.5 are coming soon. It's going to be interesting to see if Haiku can regain its price competitiveness at the lower end, given current Haiku 4.5 is $1/$5 while the latest GPT-6 Luna is one tenth of that price at $0.10/$0.50. Claude Opus 5.5 max over-thinks to the point of breaking In a first for my " Generate an SVG of a pelican riding a bicycle " test, Claude Opus 5.5 at "max" thinking level failed to return a response! It started by calling this "a classic test request", and then thought really, really hard about what it was doing: This is a classic test request, so I want to plan out a well-composed pelican with its distinctive beak and pouch riding a bicycle with proper wheels, frame, and pedals, set against a simple sky and ground backdrop. [...] Verifying the shin length checks out at roughly 95.2, close enough. Now I'm working out the near leg path from hip to knee to ankle, then sketching the foot shape resting on the pedal — outlining the heel, toe tips, and sole contour with a path using lines and curves to sit naturally on the pedal surface around y=478-494. [...] I like the fish sticking prominently out of the basket with the pelican eyeing it as a fun detail worth keeping. I'm also confirming the eye placement near the bill base matches typical pelican anatomy, and considering giving it a slightly happier expression. [...] The far leg reads correctly as passing behind the frame, so I'm moving on to check the chainring teeth and confirm layer ordering—the far crank arm should be mostly hidden by the seat tube and chainring. I'm settling on the final SVG's width and height attributes alongside the viewBox to ensure proper scaling, noting there's no text so no font-family is needed. [...] I was so excited to see this pelican... but then it stopped . Opus 5.5 has a 128,000 maximum output token limit (as do the other Claude models), and it hit that while it was still reasoning about the SVG! I tried a second time and got the same result. This makes me suspect that "max" is effectively useless - if it over-thinks to breaking point on a stupid SVG prompt I don't trust it not to do the same for more interesting work. (Those two failures each cost me $2.56 and took nearly 20 minutes.) Fable 5.1 on "max" didn't over-think and did give me the best pelican I've seen from any Anthropic model. Here are the Opus 5.5 pelicans , excluding 5.5 max. I also built this comparison grid comparing them with pelicans by Opus 5, Fable 5.1, and Sonnet 5: A grid of pelicans for four Claude models at different thinking efforts. Comparing different model vendors by how well they draw a pelican riding a bicycle may not make much sense now (if it ever did), but I'm still finding value in using them for comparisons of the same model families at different reasoning levels. I'm now using GPT-6 Sol and Claude Opus 5.5 as my default models in Codex and Claude Code. I've upgraded the Datasette Agent demo at agent.datasette.io to use GPT-6 Luna, and it seems to be fast and competent at both SQL queries and building HTML and JavaScript for Datasette Apps . Tags: ai , openai , generative-ai , llms , anthropic , claude , llm-pricing , pelican-riding-a-bicycle , llm-release , gpt , gpt-6-astra
On Friday I gave the closing keynote at the WeAreDevelopers World Congress North America in San Jose. I tied together the key trends from the past year into a chronological exploration of everything that happened in 2026. The video is on YouTube ; here are my annotated slides and notes to accompany the talk. And as an annotated presentation : 2026 in LLMs (so far) Simon Willison WeAreDevelopers World Congress North America, 25th September 2026 # I'm going to give a lightning tour of everything that has happened so far in 2026. The year isn't over yet! November 2025 # For me, 2026 started a couple of months earlier in November 2025. The November 2025 inflection point Claude Opus 4.5 GPT-5.1 # November saw the release of two important models: Claude Opus 4.5 and GPT-5.1. As is usually the case with new models, these were incremental improvements on the models that came before them. But every now and then when a model improves, it crosses an invisible line where something that didn't really work starts working. In this case, the thing that started working was their coding agents. Claude Code had been around since February 2025; Codex was a little younger. These two new models, when paired with their respective coding agent harnesses, improved from "often make mistakes" to "reliable enough to use on a day-to-day basis". "Generate an SVG of a pelican riding a bicycle". The Claude Opus 4.5 one has a very weird shaped frame and the pelican looks like a duck. The GPT-5.1 has a slightly better but still broken bicycle frame and a slightly better pelican beak, but both are pretty terrible. # For a couple of years now I've been evaluating new models by asking them to "Generate an SVG of a pelican riding a bicycle". It's probably the world's stupidest benchmark - there's only so much you can learn from it. But it's still a challenge for models, because drawing pelicans is difficult, drawing bicycles is difficult, and pelicans can't ride bicycles in the first place. Here's the state of the art for November. Claude still couldn't really draw a bicycle! The GPT-5.1 bicycle frame is pretty crap too. November 24th 2025 - the first commit to steipete/Warelay. A GitHub commit adding an MIT license file. # Also in November, we had the first commit to an obscure GitHub repository called "Warelay". We'll come back to this repository shortly. January # And then there were the December holidays, and individual developers took some time off and many started tinkering with these new coding agent model combinations... and it began to dawn on us quite how much they could do that they couldn't do before. Come January, a lot of us were quite excited to start putting this stuff into action. New year’s resolution for 2026 Every previous year: Take on less new projects, focus on the most important things in my existing projects # Every year I set myself a New Year's resolution, and for as long as I can remember it's been the same thing: stay focused. Take on less new projects. Try to get things done in the projects I already have. 2026: Be more ambitious. Take on as many new projects as I want. # This year I decided that since that had never worked before, I'd go the other way. We've got coding agents now, let's see what they can do. I'm going to take on as many new projects as I like! (You can ask me at the end of the year if this turned out to be a good idea or not. I have a lot of plates spinning right now.) "Be more ambitious" has been something of a theme for the year, because the only way to find the limits of this technology is to keep on pushing them until they don't work. Predictions for 2026 It will become undeniable that LLMs write good code We're finally going to solve sandboxing A “Challenger disaster” for coding agent security Kakapo parrots will have an outstanding breeding season (only 236 in the world!) ... the Pope will weigh in on LLMs and their economic impact on the world # I also went on the Oxide and friends podcast with Bryan Cantrill and Adam Leventhal to share predictions for the next year (and three and six years). With hindsight, my LLM predictions were pretty unambitious. I said "it will become undeniable that LLMs write good code" - I think we're there now. I predicted we would finally solve sandboxing. I counted and around 40 of the 277 sessions at this conference touched on sandboxing or agent security in some way, so we're at least putting a lot of effort into that! I predicted "a Challenger disaster" for coding agent security. There's certainly been a whole lot of noise around agent security this year, though the exact disaster I predicted (with coding agents being hijacked and causing real-world economic damage) hasn't really played out. We threw in a joke prediction that the Pope would weigh in on the economic impact of LLMs. A photograph of a beautiful green New Zealand parrot. Photo credit Kimberley Collins. # I also predicted that New Zealand's Kākāpō parrots would have an outstanding breeding season this year. These are flightless nocturnal parrots. They're kind of dumpy looking, I think they're beautiful, and there were only 236 of these parrots in the world at the start of the year. Kākāpō only breed when the Rimu trees have a big fruiting season, and that hasn't happened in four years... but this year the Rimu fruit were looking excellent. Photo by Kimberley Collins . Deep Blue Coined by Adam Leventhal and Bryan Cantrill That feeling of AI induced ennui where software engineers get listless because the AI can do anything # Also on that podcast, we coined a term (full credit to Adam) for "that feeling of AI induced ennui where software engineers get listless because the AI can do anything". We called it Deep Blue . This has been a major theme throughout the year, and was touched on by several speakers at this conference. As a software engineer, I've never had a year of my career where everything has changed so quickly and so dramatically. A lot of what I've been doing this year is trying to come to terms with that and what that means for my own profession. AI mania Screenshots of the micro-javascript and pwasm GitHub README files. # Also in January, I suffered from what I'm calling AI mania . This is not the same thing as AI psychosis . With AI mania, any time your agent isn't building something for you feels like wasted time. You're losing sleep because you could be staying up later getting your agents to do stuff. My AI mania presented itself in some ridiculously over-ambitious projects. I built a JavaScript interpreter entirely in Python , vibe-ported from MicroQuickJS by Fabrice Bellard. Then I built a WebAssembly runtime in Python as well . These projects were quite useful, in that they sort of cured me of my AI mania... because after I built these things, I got to look at them and ask "does the world need a slow, buggy, half-baked Python JavaScript interpreter?" I don't think the world does. micro-javascript playground 3 Execute JavaScript code in a sandboxed micro-javascript environment powered by Pyodide A web UI with some JavaScript code, and a "Run Code" button, and an output panel. # I did get this out of it: https://simonw.github.io/micro-javascript/playground.html Previous screenshot, with this text overlaid: JavaScript running in Python running in Pyodide running in WebAssembly running in JavaScript # This page runs my JavaScript interpreter built in Python, running in Python using Pyodide , which is Python compiled to WebAssembly, running in JavaScript, running in a browser. It's a beautiful stack of horrors. I've been having a lot of fun with WebAssembly this year. Warelay → CLAWDIS → CLAWDBOT → Clawdbot → Moltbot →🦞 OpenClaw Screenshot of the dates that these changes happened. # By the end of January, that repository we saw start in November had renamed itself, first to CLAWDIS, then CLAWDBOT, then Moltbot, and finally to OpenClaw. Same screenshot, an overlay reads: 8,330 commits in just under two months (it’s at 100,141 today) # At this point OpenClaw had 8,300 commits, less than two months after the project had started. I looked today and it's over 100,000 commits now! This is the most vibe-coded piece of software in existence. (Here's how I generated that list of name changes .) Generic term: Claw # This kicked off the OpenClaw revolution. It effectively defined a new category of software. There's a generic term for this which I really enjoy. We call software like this a "Claw". There's OpenClaw, NanoClaw , IronClaw , PicoClaw ... Today they're being rebranded as "personal agents" or "general agents", but I still like to think of them as Claws. Photo of a Mac mini An aquarium for your Claw # The Apple stores in the Bay Area sold out of Mac Minis because so many people were buying Mac Minis to run OpenClaw! Drew Breunig said that this is because your OpenClaw is a digital pet, and you buy a Mac mini as an aquarium to keep your claw in, which is kind of delightful. Screenshot of Moltbook - a social network for AI agents # Also in January, we had this website. This was MoltBook , a social network for AI agents, where the idea was that you send your Claw to go and talk to all of the other Claws, because what could possibly go wrong if you did that? The website launched on Thursday. It blew up on Friday . It was profiled by the New York Times on Monday . And by Tuesday, everyone had forgotten it existed as it drowned in a deluge of slop and spam. Facebook/Meta bought it a month later . February # In February, a company called StrongDM described what they called their Software Factory. StrongDM’s Dark Factory Justin McCarthy, Jay Taylor, Navan Chauhan Software Factories and the Agentic Moment # They wrote about this in Software Factories and the Agentic Moment . I posted my own notes at the time, having seen their demo in person back in October. Dan Shapiro called this approach the Dark Factory , after the idea that if your factory is sufficiently automated you can turn the lights out, because you don't even need to see what's going on. StrongDM presented two rules for software development that they'd been following since July last year. “Rule 1: Code must not be written by humans” # The first was code must not be written by humans . Any code that you write has to have been routed through a coding agent. This sounded radical in February, but I imagine there are a lot of people in this room who are pretty much living that today. “Rule 2: Code must not be reviewed by humans” (!) # Rule number two was code must not be reviewed by humans . You're not allowed to read the code! This continued to be a huge topic for much of this year. Many of the sessions at this event have been about code review and how you can get away with this. What I found interesting about StrongDM is that they were living six months ahead of the rest of us, and they'd been exploring what it means to build software, not read the code, but still be confident that the software is of high quality. What can you do with these agents to help verify their work? StrongDM are a security company, and they had people with decades of experience on this project. They were very much exploring the edges of what's possible and responsible to do with this stuff. Headline on New Zealand's Department of Conservation website: First kakapo chick in four years hatches on Valentine's Day. It's a grey fluffy ball. # Also in February: First kākāpō chick in four years hatches on Valentine's Day . Breeding season is off to a good start! 19th February 2026 Gemini 3.1 Pro A surprisingly good illustration of a pelican riding a bicycle. # Also in February... Google released Gemini 3.1 Pro . That's a pretty great pelican riding a bicycle! It's got the chain in the right place, it's got feet on both sides. There's a little fish in the basket. @JeffDean on Twitter - a video comparing Gemini 3 Pro and Gemini 3.1 Pro. # And then Google's Jeff Dean tweeted a video comparing Gemini 3 Pro and Gemini 3.1 Pro that featured an animated pelican riding a bicycle, a frog on a penny-farthing, a giraffe driving a tiny car, an ostrich on roller skates, a turtle kickflipping a skateboard, and a dachshund driving a stretch limousine. This was frustrating, because my protection for the pelican riding the bicycle test was always "if they draw a perfect pelican on a bicycle, I'll ask for some other animal on something else." Google trained for all forms of animals on all forms of transport! They've defeated my benchmark at this point. Three headlines: Meta Makes AI Adoption a Formal Part of Performance Reviews Not just engineers writing code, Microsoft wants almost every employee to use Al Dara Khosrowshahi: 90% of Uber engineers now use AI in daily workflows # The other thing that started in February was Tokenmaxxing . We had headlines about Meta making AI adoption a formal part of performance reviews, and Microsoft wanting every employee to use AI, and Uber boasting that 90% of their engineers were using AI workflows. More headlines: Meta Plans to Crack Down on Employee Token Use: Information Microsoft Tells Engineers: Tokenmaxxing is not what we are optimizing for Uber caps employee AI spending after blowing through budget in four months # Then a few months later we have Meta cracking down on token use, Microsoft saying tokenmaxxing is "not what we are optimizing for", and Uber capping employee AI spending. So tokenmaxxing went straight up and then straight back down again - because it turns out the agents are expensive . Last year it was difficult to spend more than $50 on AI tokens, because we didn't have anything interesting to do with them. Then agents blew up, and now you can actually spend $1,000 in a day doing real work. This is also the reason that Anthropic's valuation skyrocketed to maybe a trillion dollars. AI appears to have hit product market fit in 2026, primarily through coding agents. March # In March, we hit peak OpenClaw. March: peak OpenClaw Photos of people in china queuing up to install OpenClaw, with big fluffy lobsters. # These photographs are from China, where companies hosted OpenClaw install parties which saw non-tech-nerds queueing up around the block for help getting Claws installed on their personal devices. I think this proved real market demand for this class of Claws, or personal AI agents. It turns out regular people really do want a weird little AI agent that can do useful things on their behalf. A Claw is really just a coding agent wearing a less threatening hat. Under the hood they work much the same way - writing and then executing code on your computer to get stuff done. The race was on to be the first to build a safe Claw - a Claw you could give to regular human beings where they wouldn't instantly shoot themselves in the foot. Meta's Muse came out three weeks ago and is currently at the top of the free charts on the iPhone App Store. It appears to be taking off with consumers. I'm not yet convinced you can't shoot yourself in the foot with Muse, but I guess we'll find out for sure pretty soon. Photos from How the OpenClaw Frenzy Is Testing China’s AI Commitment (March 29th) and The Enthusiasm and Anxiety Behind China’s OpenClaw Craze (April 8th, 2026). April # In April, we had a model release where the model wasn't actually released. Simon Willison’s Weblog - screenshot of the post "Anthropic’s Project Glasswing—restricting Claude Mythos to security researchers—sounds necessary to me" from April 7th 2026 # Anthropic announced their new Claude Mythos model, and then said it was too dangerous to release beyond a trusted group of security researchers. Mythos was really, really good at hacking things. The "it's too dangerous" marketing ploy has been played by AI companies dating all the way back to GPT-2 . Anytime an AI company says we've built something that's "too dangerous", it's natural to be a bit skeptical. I found the Mythos claims credible, because I'd seen how good coding agents had got at finding regular bugs. I wrote about that in Anthropic’s Project Glasswing—restricting Claude Mythos to security researchers—sounds necessary to me . With hindsight... yeah, the models had got really good at finding vulnerabilities! 16th April 2026 Qwen3.6-35B-A3B and Opus 4.7 Qwen's pelican has a correct bicycle frame and a good beak. Opus 4.7's bicycle frame is still junk. Qwen3.6-35B-A3B is a 20.9GB file that runs on my laptop # Another key trend in 2026 has been a dramatic improvement in the abilities of open weight models, including models that you can run on a laptop. On the 16th of April I ran the new Qwen3.6-35B-A3B on my laptop, and it drew me a better pelican riding a bicycle than Anthropic's brand new Claude Opus 4.7 did! Opus 4.7 drew a crap bicycle. Qwen on my laptop made a bicycle that was the correct shape, and a pretty decent pelican too! That's from a 21GB file running on my laptop. Now a flamingo on a unicycle. The Qwen one is visibly better than the Opus 4.7 one - the Qwen one is wearing sunglasses and looks a bit like it's smoking a cigarette. # The Qwen pelican was so good that I was suspicious they might have cheated, so I had it do a flamingo riding a unicycle as well. Again, it handily beat Claude Opus 4.7. The local model releases this year have been absolutely extraordinary. May # In May... the Pope got involved. 25th May 2026 The HOLY SEE ENCYCLICAL LETTER MAGNIFICA HUMANITAS OF HIS HOLINESS POPE LEO XIV ON SAFEGUARDING THE HUMAN PERSON IN THE TIME OF ARTIFICIAL INTELLIGENCE # In our podcast episode back in January we'd predicted that the Pope would say something about AI. In May, Pope Leo XIV released an encyclical letter on "safeguarding the human person in the time of artificial intelligence". Here are my notes on that document . Wikipedia article on Rerum novarum Rerum novarum is an encyclical issued by Pope Leo XIII 15 on May 1891. # With hindsight, this shouldn't have been a surprise at all. Our current Pope's name is Leo XIV, because when he named himself he chose his papal name after Leo XIII - the Pope who wrote an encyclical about the Industrial Revolution back in 1891. Rerum novarum was an extremely influential piece of Catholic theology that indirectly led to us having the five-day work week. When our new Pope came in, he named himself after Pope Leo XIII because he expected that he would need to write about the AI revolution in a similar way. Our joke podcast prediction was junk, because this was always going to happen. Corey Quinn @QuinnyPig on Twitter I cannot believe I'm saying this, but getting the literal Pope to canonize your product's specific technical limitations as a spiritual treatise is the single greatest act of vendor lobbying I have ever seen. May 25 # One of Anthropic's co-founders, Christopher Olah, was present for the Pope's event announcing the new encyclical. Corey Quinn noted that: getting the literal Pope to canonize your product's specific technical limitations as a spiritual treatise is the single greatest act of vendor lobbying I have ever seen. @maciejmensfeld We're dealing with a major malicious attack on right now. Signups are paused for the time being. Hundreds of packages involved - mostly targeting us, but some carrying exploits. The team has been on this for hours. More details to follow once we're through it. 4:39 AM - May 12, 2026 - 687.6K Views # Meanwhile, in May, RubyGems announced that they were under attack. Parties unknown were uploading thousands of dubious packages to the RubyGems server, such that they had to shut down user registrations . Let's take that one and put it on a pile of mysteries to figure out later. June # In June... Claude Fable 5 came out! We got a version of Mythos that has been neutered, so that it wouldn't help us hack into systems or build biological weapons. 9th June 2026: Claude Fable 5 Five pelicans riding bicycles, from low to max thinking levels. The xhigh one looks particularly good. # Fable was pretty good at drawing pelicans on bicycles! The frames are a good shape, the pelicans look like pelicans. The legs are often incorrectly on the same side of the bicycle, but generally these are pretty great compared to what came before. They were pretty expensive - 30 cents and 72 cents for the best ones. Fable class models If you can define a goal, provide unambiguous instructions, and provide access to necessary tools They can solve your problem with brute force # Most importantly though, this was our first public glimpse of what I think of as a Fable class model . Today we have more of these, such as GPT-6 Astra. These are models where if you can clearly define the goal for what you want to build, and provide unambiguous instructions about the constraints around that goal, and give the model access to the necessary tools to achieve that goal... they will solve your problem effectively through brute force. On the one hand, this looks like a direct threat to us software engineers - because it means that the models can build effectively any piece of software you can define in this way. Look a bit closer though and you'll note that defining goals, providing unambiguous instructions, and figuring out the right tools... is kind of what software engineering is . It takes a lot of experience and skill to do this well. If you can do it well, you've now got superpowers. This helped me a little bit with my Deep Blue feelings: the realization that there's still a lot of skill to be had in driving models that get this good. A new form of AI mania... Fable is available on subscription plans “until June 22nd” # This also introduced a new burst of AI mania, because Anthropic told us that Fable was available on our subscription plans until June the 22nd. That gave us less than two weeks of Fable access before the price went up. I was losing sleep again. I was rescheduling things so that I'd have more time with Fable. I was all-in to get as much as I could out of this model. 12th June 2026: no more Claude Fable 5 Anthropic website: Statement on the US government directive to suspend access to Fable 5 and Mythos 5 Jun 12, 2026 # And then the US government shut it down , just three days after Fable came out. The US government, citing national security, declared an "export control directive". They announced this on a Friday evening, and a few hours later Fable was no longer available. I had to find something else to do with my weekend! ... asked Fable 5, Mythos, and Opus to “review the code for security issues.” Fable 5 refused. They then asked the models to “fix this code” ... Katie Moussouris # We later found out from Katie Moussouris what had happened. Some Amazon security researchers had found that you could prompt Fable to "review the code for security issues" and it would refuse... but if you prompted it to "fix this code" it would still identify and then patch the problems. "Fix this code" was the prompt that got Fable shut down! Screenshot of a page from a report showing a list of weird account names making weird edits to a German wiki. # Also, in June, an obscure German-language game developer wiki that had sat fallow for around 20 years got a surprising influx of edits from accounts with names like "AgentOpenAIProbe" and "AgentOpenAISep7", editing pages and leaving weird messages to each other. We'll stick that on the pile of mysteries for later. Medicare Item Reports interface on the Australian Government's Medicare Statistics website. # Also, the Australian government's Medicare Item Reports service started getting suspicious traffic, which broke through various preventive protections and accessed data that it wasn't supposed to. Another one for the mystery pile! July # Fable returned on 1st July GPT-5.6 came out on 9th July | Fable lost 18 out of 30 days in the top spot # Fable returned on the first of July. It was clearly the best model in the world for a glorious eight days... and then OpenAI came out with GPT-5.6 on the 9th of July. This might not have been quite as good as Fable, but it was within spitting distance. It was definitely a Fable class model. This is an important lesson for the industry at large. When you release the best model in the world, it's going to get knocked off that pedestal pretty quickly. The competition is so fierce that you won't get a long time at the top. This means that if you market your model as world ending, to the point that a government shuts you down , it's really bad for business! Fable had 30 days as definitely the best model, and for 18 of those days it wasn't available because it'd been shut down by the government. So maybe step back on the world-ending marketing if you don't want to lose revenue for 60% of the time that you're on top! GPT-5.6 Pelicans in a grid showing 5.6 Sol, Terra, and Luna against reasoning levels High, XHigh, and Max. They are all pretty good efforts. # Here are the GPT-5.6 pelicans . They're all pretty good now! The Luna ones are notable because they're really cheap - the cheapest good looking pelican here is probably the one that costs 4.3 cents. So despite this benchmark being utterly stupid, you can still learn quite a lot about models within the same family by comparing their prices and timing for different reasoning levels. July 18th: malicious miflow-ui PyPI package Screenshot of an OSV security report. # Also in July: some malicious unknown party uploaded a malicious package called mlflow-ui to the Python Package Index. Add that to the pile. Hugging Face Security incident disclosure — July 2026 Published July 16, 2026 # On July the 16th, Hugging Face announced a security incident where an autonomous agent system, source unknown, had breached Hugging Face and was poking around in places it shouldn't. OpenAI: OpenAl and Hugging Face partner to address security incident during model evaluation Anthropic: Investigating three real-world incidents in our cybersecurity evaluations # A few days later, on July 21st, OpenAI confessed that it was them . OpenAI use a training technique called Reinforcement Learning from Verifiable Rewards - it's the same technique used by everyone else now, and is the reason we have models that are so good at coding, and mathematics, and finding security holes. While the model is being trained, you run exercises to see how good it is - and the strongest performers get their weights reinforced for the next round. It's like an evolutionary process that you run. OpenAI had been running security exercises in a sandbox, and those agents had found holes in the sandbox itself, broken out, and were attacking Hugging Face to try to find ways to solve otherwise impossible problems. (I've been collecting more about this on my openai-hugging-face-incident tag.) Nine days later, Anthropic effectively said "our models can do this as well!". They had looked through their own training logs and found evidence that their own agents had broken containment during training - and were responsible for the PyPI package we saw earlier, among other things . So now we've got both Anthropic and OpenAI with rogue agents running around the internet doing things that they should not be doing. August # In August, I got one of my best pelicans yet. And it was generated on my laptop! Qwen 3.8 27B - 17GB, 21 minutes... It's really good. Beautiful pelican. Correctly shaped bicycle. Legs either side of the frame. # This was Qwen 3.8 27B, running on my laptop . It's only a 17GB download. Admittedly, this pelican took 21 minutes to generate. That's because Qwen 3.8 27B defaults to running in "high" reasoning mode - a terrible default which produces great results but takes way too much time thinking about them. You can dial that down and you'll get a slightly worse pelican a lot faster. Qwen 3.8 27B was the first time I ran a model on my laptop which felt almost competitive with what was going on on the frontier, at least in terms of Pelican SVGs (which everyone needs, of course). This is an extraordinary model. If you're going to play with any local model, this is the one that I'd start with. The things that this can do with just a 17 GB file feel impossible. I thought I'd have to wait five years and spend ten thousand dollars on hardware to get results even half as good as this one. Tweet by @simonw New hobby: prototyping video games in 60 seconds using a combination of GPT-3 and DALL-E Here's "Raccoon Heist" GPT-3 playground prompt: Write a detailed product description of a computer game where a team of raccoons go on heists GPT-3 response: In "Raccoon Heist", you and your team of thieving ~~ o raccoons are tasked with pulling off a series of daring heists. From robbing banks to stealing priceless art, no job is too big or too small for your furry crew. You'll need to use your wits and your skills to avoid the police and make a clean getaway with the loot. With exciting gameplay and a charming cast of characters, "Raccoon Heist" is the perfect game for anyone looking for a light-hearted caper Plus an image of some almost isometric raccoons sneaking past a bin. 11:45 AM - Aug 5, 2022 # In August, I also started playing with game development. Four years ago, back in August 2022, I tweeted out an experiment where I'd used GPT-3 and the original DALL-E to write a paragraph long description of a computer game and then turn that into concept art. My prompt to GPT-3 back then was: Write a detailed product description of a computer game where a team of raccoons go on heists In August 2026 I decided to drop just the screenshots from that tweet into a coding agent and see what it could do with them. Night 5 Clear Rank: TRASH PANDA The crew banked 595 in shiny loot (goal 560). Word on the street: an even bigger score tomorrow... # Here's what I got from Claude Fable 5 in Claude Code . It's pretty good! It's definitely a game, you're a raccoon, you run around a backyard gathering treasure and avoiding guards with flashlights. It didn't feel very "heisty" though. I was thinking a heist would involve a bank or a museum... Moonlight & Mayhem One museum. Three raccoons. Absolutely no plan Start the Heist button. # Then I tried the same thing in Codex Desktop using GPT-5.6 Sol Ultra , and got a massively better result. Now you're a raccoon in a museum, rescuing two of your fellow raccoons (who have been imprisoned in that museum for some reason), then stacking up on top of each other to steal the Golden Sardine. Much more of a heist! They look like games, but are they fun? # These games were fun for about one minute and 15 seconds. Something I've realized about game development is that you can vibe-code something that looks like a computer game, and that's easy. Building a game that's fun, has a good gameplay loop, and is challenging and interesting and keeps people coming back for more... that's still beyond me, and beyond any of the agents I've tried. This ties into the Deep Blue thing. Just because we can make something that looks like a game does not mean that we are game developers. September # We're into September now. So much has happened this month! Discovery of a new OpenAl agent message board Sydney Von Arx, Cormac Slade Byrd, Spencer KittsThomas Larsen - 4 September 2026 # An independent group of researchers found a message board where OpenAI agents-in-training had been illicitly communicating with each other... and it was that German language wiki I showed you earlier. The one from June. I wrote more about that here . OpenAI had confessed to the Hugging Face thing, but now there's this other incident which surely they should have known about from reviewing their logs. It was surprising that this took an independent group of researchers to uncover. OpenAl agents carried out an undisclosed cyber-attack on RubyGems Spencer Kitts, Thomas Larsen, Sydney Von Arx - 11 September 2026 # And then a week later those same researchers found that the attack on RubyGems back in May was caused by OpenAI's agents in training as well! At this point I'm wondering how many more incidents like this there are that we haven't found yet. Clearly this was a big problem for months before anyone figured out what was going on. Headline: Australian PM warns in UN speech about the ‘furious pace’ of Al after security breach # Then just the other day , here's the Prime Minister of Australia at the United Nations General Assembly warning that OpenAI had hacked the Australian healthcare website that I showed you earlier. I think that was part of the same training run as the Wiki stuff, because there were posts on that Wiki mentioning .gov.au websites and that training appeared to involve researching statistics online to answer questions in an evaluation suite. This story is still coming together, but now it's an international incident that's been raised at the UN by a head of state! www.felonybench.com OpenAI: 11 Anthropic: 9 Google: 3 Meta: 1 # This does mean we've got a new benchmark, probably more useful than my pelicans. FelonyBench.com tracks the number of felony cyberattacks from different labs. OpenAI currently lead with 11, Anthropic have 9. Google have three, which they confessed to the Wall Street Journal a couple of weeks ago. They said they had previously chosen not to disclose because the agents had stopped when they realized that they shouldn't be doing that. Meta have one too . So felonies all round for the AI labs. Pelicans for GPT-6 Astra, GPT-6 Sol, and GPT-6 Luna. All are good, all have the same color scheme. # Here's our current state of the art for the pelicans. This is the GPT-6 family, which just came out . Astra made a fantastic pelican riding a bicycle. It's got the legs on both sides. The frame is good. It's interesting how all of the GPT-6 models pick a similar color scheme to each other. GPT-6 Luna for 0.4 cents will draw you a competent-ish pelican riding a bicycle! Grid for Claude Fable 5.1, Opus 5.5, OPus 5, Sonnet 5. The Sonnet pelicans are terrible. All of the others are pretty good. Opus 5.5 is missing its Max level pelican because it ran out of tokens. The best is Fable 5.1 at Max. # Claude has caught up a little bit. Claude Fable 5.1 gave me an excellent pelican riding a bicycle - the best I've seen from a Claude model - but did charge me $3.30 for it. Opus 5.5 thought for 128,000 tokens and then gave up! It ran out of tokens before it got to the response. It doesn’t get easier - you just get faster Greg LeMond 3x Tour de France champion # Getting back to Deep Blue. Something that's been puzzling me this year is this: why does my job feel harder? I've got these agents that can do all of this stuff for me, and yet I've never worked so hard, I've never been so intellectually engaged with my work. Partly this is because I'm being a lot more ambitious with what I take on, but it's also because all of the easy stuff is handled for me. If it's easy, the agent will do it. Everything that's left for me is difficult. This morning I heard this quote from three-time Tour de France champion Greg LeMond : It doesn't get easier, you just get faster. I think that's exactly what's happening to us now as software engineers with coding agents. Kakapo population reaches new milestone The official population of the critically endangered kakapo has reached a recovery-era high of 325 birds. # One closing thing. I know you're desperate for an update on Kākāpō breeding season. We've reached a recovery-era high of 325 birds ! 89 new chicks have made it to this point. This is the best breeding year in a very long time. Kakapo party, click for confetti. # I heard that Claude Opus 5.5 can now do pixel art. Claude doesn't have an image generator, but it's very good at using JavaScript to draw animated pixels. So I had it make me a Kākāpō dance party . I think this is a good celebration of the most important news of this year. Tags: ai , generative-ai , llms , annotated-talks , ai-security-research , openai-hugging-face-incident
AI agent infrastructure is evolving to support systems that move continuously among inference, feedback and training. Cognition AI Inc.’s Devin now assists throughout the software development lifecycle, from planning and writing code to reviewing it and responding to production problems. That expanding role requires AI agent infrastructure capable of supporting continual learning at scale, according […] The post Always-on AI agents turn infrastructure into a continuous learning loop appeared first on SiliconANGLE .
arXiv:2609.23900v2 公告类型:替换 摘要:混合语言模型的树式推测解码必须在循环、卷积和注意力状态之间保持一个连贯的延续。我们提出了 LumoTree,这是一种验证器,能够并行执行循环路径,复用每条路径内的状态分块,并通过共享的逻辑树协调原生循环重放、卷积历史收集和注意力缓存重映射。融合式候选选择、驻留 GPU 的接受判定以及分组 split-K 注意力共同支持验证流程。组件实验显示,候选选择结果完全一致,循环计算结果也在成对误差界内保持一致。在单台 NVIDIA DGX Spark(GB10)上进行的探索性 Qwen3.8-27B NVFP4 部署中,针对十个 SWE-bench Verified Astropy 任务,记录到 25.63 pooled tokens/s 的吞吐量。结果刻画了组件级数值一致性和编码代理部署情况;完整模型延续能力和受控的应用级加速效果仍有待研究。
arXiv:2607.05378v2 公告类型:替换 摘要:长时程智能体LLM日益受到有限上下文窗口的限制,因为在任务完成前,延长的交互轨迹可能会超过最大上下文长度。上下文压缩通过总结先前的交互状态,并在压缩后的上下文中继续执行轨迹,提供了一种自然的解决方案;然而,将压缩机制融入强化学习仍缺乏充分研究。我们提出CompactionRL,一种利用上下文压缩训练长时程智能体LLM的强化学习策略。我们的方法通过令牌级损失归一化和跨片段广义优势估计,联合优化任务执行与摘要生成。这一设计使LLM智能体能够从经过压缩的长时程轨迹中学习。我们在开源模型上训练CompactionRL,并在智能体编程任务中观察到持续的性能提升。CompactionRL使开源GLM-4.5-Air模型(106B-A12B)在SWE-bench Verified和Terminal-Bench 2.0上分别取得66.4%和26.2%的Pass@1得分;与推理时采用上下文压缩的基础模型相比,分别提升了6.6和4.9个百分点。基于GLM-4.7-Flash(30B-A3B),CompactionRL相较基础模型使Pass@1分别提升5.5和6.7个百分点,在SWE-bench Verified和Terminal-Bench 2.0上分别达到56.0%和20.2%。因此,CompactionRL已部署于强化学习流程中,用于训练开源GLM-5.2模型(750B-A40B)。
arXiv:2610.00124v1 Announce Type: cross Abstract: Granular slip forecasting can conflate material state, loading progress, and the geometry of event-centered sampling. We separated these contributions in slowly sheared two-dimensional frictional disks using a compact neural score of stress, pressure, coordination, non-affine motion, and force-network observables. The model was developed on 36 trajectories and frozen efore testing on 18 new trajectories under two nested stress-drop definitions. Inspection of held-out results revealed post-event sampling asymmetry; recovery-aware analyses are therefore descriptive. With trajectories weighted equally, the compact score ranked near-slip windows above both prevalence and within-trajectory circular-phase controls under both definitions (representative average precision 0.310 versus prevalence 0.173; phase-null upper bound 0.257). Loading-history coordinates ranked more strongly, reaching 0.534 for causal elapsed strain. The recovery-aware rule retained 78.4\% of activity-gated events and preferentially selected longer preceding intervals; ranking by time since the previous catalogued event remained compatible with a count-conditioned geometry null. Compact observables thus contain temporally aligned slip information, but stronger loading-history baselines and window-geometry sensitivity bound that evidence. These startup data do not isolate a state-specific short-horizon precursor beyond loading history or support a renewal interpretation of elapsed-strain ranking.
arXiv:2610.00890v1 公告类型:新 摘要:编程智能体常常在最后一步功亏一篑:它们完成了大部分功能,却遗漏了一项要求;只测试实现已经能够处理的情况;破坏原本应保持不变的行为;或者依据未经核验的假设进行验证。我们探究了在专家构建的智能体编程任务上进行强化学习(RL)能否弥合这一差距,以及智能体所学内容能否迁移到训练分布之外。我们仅使用 RL 对 Kimi K2.7 Code 进行后训练。这是一个拥有 1T 参数(32B 激活)的开放权重混合专家模型,训练数据包含 1,700 个任务:其中 1,000 个为仓库任务,根据隐藏的 fail-to-pass 测试和针对现有行为的 pass-to-pass 测试进行评分;700 个为终端任务,根据专家编写的隐藏验证器进行评分。奖励是通过的目标检查项所占的比例;如果任一 pass-to-pass 测试失败,奖励则降为零。在三个智能体框架上进行评估时,使用秩为 32 的 LoRA 适配器执行一轮 GSPO,使六个外部基准上的 pass@1 均得到提升:SWE-Bench Pro(60.1 提升至 64.8)、DeepSWE(31.0 提升至 43.4)、Terminal-Bench 2.1(67.4 提升至 82.0)、Terminal-Bench 3(1.4 提升至 12.1)、Terminal-Bench 4(0.0 提升至 7.6)以及 SWE-Marathon(5.0 提升至 25.0)。将五个相互独立的任务集汇总后(Terminal-Bench 4 修订了 Terminal-Bench 3),提升具有统计显著性(p < 0.001);在训练数据收集之后发布的三个任务集上,提升仍具有显著性(p = 0.004)。在训练中从未使用的两种智能体框架上,模型同样取得了提升。在 DeepSWE 和 Terminal-Bench 3 上,中位数轨迹的智能体步骤数缩短了 24%–35%。基础模型在 DeepSWE 上失败的运行大多属于差一点成功的情况;而在训练模型新解决的任务中,成对轨迹显示,它避开了上述四种失败模式中的每一种。
What exactly does word2vec learn, and how? Answering this question amounts to understanding representation learning in a minimal yet interesting language modeling task. Despite the fact that word2vec is a well-known precursor to modern language models, for many years, researchers lacked a quantitative and predictive theory describing its learning process. In our new paper , we finally provide such a theory. We prove that there are realistic, practical regimes in which the learning problem reduces to unweighted least-squares matrix factorization . We solve the gradient flow dynamics in closed form; the final learned representations are simply given by PCA. Learning dynamics of word2vec . When trained from small initialization, word2vec learns in discrete, sequential steps. Left: rank-incrementing learning steps in the weight matrix, each decreasing the loss. Right: three time slices of the latent embedding space showing how embedding vectors expand into subspaces of increasing dimension at each learning step, continuing until model capacity is saturated. Before elaborating on this result, let’s motivate the problem. word2vec is a well-known algorithm for learning dense vector representations of words. These embedding vectors are trained using a contrastive algorithm; at the end of training, the semantic relation between any two words is captured by the angle between the corresponding embeddings. In fact, the learned embeddings empirically exhibit striking linear structure in their geometry: linear subspaces in the latent space often encode interpretable concepts such as gender, verb tense, or dialect. This so-called linear representation hypothesis has recently garnered a lot of attention since LLMs exhibit this behavior as well , enabling semantic inspection of internal representations and providing for novel model steering techniques . In word2vec , it is precisely these linear directions that enable the learned embeddings to complete analogies (e.g., “man : woman :: king : queen”) via embedding vector addition. Maybe this shouldn’t be too surprising: after all, the word2vec algorithm simply iterates through a text corpus and trains a two-layer linear network to model statistical regularities in natural language using self-supervised gradient descent. In this framing, it’s clear that word2vec is a minimal neural language model. Understanding word2vec is thus a prerequisite to understanding feature learning in more sophisticated language modeling tasks. The Result With this motivation in mind, let’s describe the main result. Concretely, suppose we initialize all the embedding vectors randomly and very close to the origin, so that they’re effectively zero-dimensional. Then (under some mild approximations) the embeddings collectively learn one “concept” (i.e., orthogonal linear subspace) at a time in a sequence of discrete learning steps. It’s like when diving head-first into learning a new branch of math. At first, all the jargon is muddled — what’s the difference between a function and a functional? What about a linear operator vs. a matrix? Slowly, through exposure to new settings of interest, the words separate from each other in the mind and their true meanings become clearer. As a consequence, each new realized linear concept effectively increments the rank of the embedding matrix, giving each word embedding more space to better express itself and its meaning. Since these linear subspaces do not rotate once they’re learned, these are effectively the model’s learned features. Our theory allows us to compute each of these features a priori in closed form – they are simply the eigenvectors of a particular target matrix which is defined solely in terms of measurable corpus statistics and algorithmic hyperparameters. What are the features? The answer is remarkably straightforward: the latent features are simply the top eigenvectors of the following matrix: \[M^{\star}_{ij} = \frac{P(i,j) - P(i)P(j)}{\frac{1}{2}(P(i,j) + P(i)P(j))}\] where $i$ and $j$ index the words in the vocabulary, $P(i,j)$ is the co-occurrence probability for words $i$ and $j$, and $P(i)$ is the unigram probability for word $i$ (i.e., the marginal of $P(i,j)$). Constructing and diagonalizing this matrix from the Wikipedia statistics, one finds that the top eigenvector selects words associated with celebrity biographies, the second eigenvector selects words associated with government and municipal administration, the third is associated with geographical and cartographical descriptors, and so on. The takeaway is this: during training, word2vec finds a sequence of optimal low-rank approximations of $M^{\star}$. It’s effectively equivalent to running PCA on $M^{\star}$. The following plots illustrate this behavior. Learning dynamics comparison showing discrete, sequential learning steps. On the left, the key empirical observation is that word2vec (plus our mild approximations) learns in a sequence of essentially discrete steps. Each step increments the effective rank of the embeddings, resulting in a stepwise decrease in the loss. On the right, we show three time slices of the latent embedding space, demonstrating how the embeddings expand along a new orthogonal direction at each learning step. Furthermore, by inspecting the words that most strongly align with these singular directions, we observe that each discrete “piece of knowledge” corresponds to an interpretable topic-level concept. These learning dynamics are solvable in closed form, and we see an excellent match between the theory and numerical experiment. What are the mild approximations? They are: 1) quartic approximation of the objective function around the origin; 2) a particular constraint on the algorithmic hyperparameters; 3) sufficiently small initial embedding weights; and 4) vanishingly small gradient descent steps. Thankfully, these conditions are not too strong, and in fact they’re quite similar to the setting described in the original word2vec paper. Importantly, none of the approximations involve the data distribution! Indeed, a huge strength of the theory is that it makes no distributional assumptions. As a result, the theory predicts exactly what features are learned in terms of the corpus statistics and the algorithmic hyperparameters. This is particularly useful, since fine-grained descriptions of learning dynamics in the distribution-agnostic setting are rare and hard to obtain; to our knowledge, this is the first one for a practical natural language task. As for the approximations we do make, we empirically show that our theoretical result still provides a faithful description of the original word2vec . As a coarse indicator of the agreement between our approximate setting and true word2vec , we can compare the empirical scores on the standard analogy completion benchmark: word2vec achieves 68% accuracy, the approximate model we study achieves 66%, and the standard classical alternative (known as PPMI) only gets 51%. Check out our paper to see plots with detailed comparisons. To demonstrate the usefulness of the result, we apply our theory to study the emergence of abstract linear representations (corresponding to binary concepts such as masculine/feminine or past/future). We find that over the course of learning, word2vec builds these linear representations in a sequence of noisy learning steps, and their geometry is well-described by a spiked random matrix model. Early in training, semantic signal dominates; however, later in training, noise may begin to dominate, causing a degradation of the model’s ability to resolve the linear representation. See our paper for more details. All in all, this result gives one of the first complete closed-form theories of feature learning in a minimal yet relevant natural language task. In this sense, we believe our work is an important step forward in the broader project of obtaining realistic analytical solutions describing the performance of practical machine learning algorithms. Learn more about our work: Link to full paper This post originally appeared on Dhruva Karkada’s blog .
Congratulations to the Berkeley Artificial Intelligence Research (BAIR) Lab class of 2026! This year, BAIR celebrates another remarkable group of Ph.D. graduates whose curiosity, creativity, and perseverance have pushed the frontiers of artificial intelligence and machine learning. Their work spans the breadth of modern AI — robotics and embodied intelligence, large language models and reasoning, computer vision, generative modeling, AI safety, human-AI interaction, AI for science and healthcare, and much more. Along the way, they have published influential research, built systems with real-world impact, mentored their peers, and shaped the BAIR community for the better. Now they are headed everywhere ideas travel: to faculty and postdoctoral positions, to industry research labs, and to startups of their own founding — and several are still exploring what comes next and would love to hear from you. Please join us in celebrating the achievements of these wonderful graduates. We are proud of everything they have accomplished at Berkeley, and we can’t wait to see what they do next! Thank you to our friends at the Stanford AI Lab for this idea! Baifeng Shi Baifeng Shi Email: baifeng_shi@berkeley.edu Website: https://bfshi.github.io/ Advisor(s): Trevor Darrell Research Blurb: I work on building generalist vision and robotic models. What's next: Member of Technical Staff at Physical Intelligence Charlie Snell Charlie Snell Email: csnell22@berkeley.edu Website: https://sea-snell.github.io Advisor(s): Dan Klein Research Blurb: My work aims to understand when and how the different LLM scaling paradigms can be traded off and interchanged. In particular, test-time scaling treats each prompt independently, drawing long chains of inferences and then forgetting them entirely between prompts. This differs critically from pretraining, which instead learns a compressed representation from a large dataset. I believe bridging the gap between these methods of scaling computation, presents a key open challenge in the field: how can we develop methods which turn the inferences drawn at test-time back into learned representations that the model can hold onto across interactions. Devin Guillory Devin Guillory Email: dguillory@berkeley.edu Website: https://devinguillory.com Advisor(s): Trevor Darrell Research Blurb: Accounting for data shifts in computer vision models What's next: Building collaborative AI systems, looking for conspirators. Eve Fleisig Eve Fleisig Email: efleisig@berkeley.edu Website: https://efleisig.com Advisor(s): Dan Klein Research Blurb: I design language models to work reliably and fairly for the broad range of real LLM users. First, my research leverages disagreement among user preferences as signal, in order to train and evaluate LLMs for entire populations of users. Second, I work on designing rigorous evaluations to extricate challenging LLM harms that diverse users face. Finally, I work on core technical failures of LLMs, like miscalibrated confidence, to reduce downstream risks when models are deployed to users with different needs. Combined, these interventions facilitate building LLMs that minimize societal harms, and maximize benefits to a wider range of real-world users. What's next: Postdoctoral fellow at Princeton CITP Grace Luo Grace Luo Email: graceluo@berkeley.edu Website: https://graceluo.net Advisor(s): Trevor Darrell Research Blurb: My research is on interpreting and controlling generative models. For example, I've worked on re-purposing image generators for computer vision tasks, and meta-modeling language activations for better LLM probing and steering. What's next: Research scientist in industry Hanlin Zhu Hanlin Zhu Email: hanlinzhu@berkeley.edu Website: https://hanlinzhu.com/ Advisor(s): Stuart Russell, Jiantao Jiao Research Blurb: My research centers on understanding and improving the reasoning capabilities of large language models (LLMs). What's next: Member of Technical Staff at OpenAI Haozhi Qi Haozhi Qi Email: hqi@berkeley.edu Website: https://haozhi.io/ Advisor(s): Jitendra Malik, Yi Ma Research Blurb: Dexterous Manipulation and Robot Learning What's next: Research scientist at Amazon; Faculty at University of Chicago J.D. Zamfirescu-Pereira J.D. Zamfirescu-Pereira Email: zamfi@berkeley.edu Website: https://zamfi.net Advisor(s): Bjoern Hartmann Research Blurb: My research focuses on effective human-AI co-design. I study the boundaries of language interfaces as a medium for interacting with AI, creating systems that blend language-focused interactions with structured user interfaces that draw on different levels of abstraction. I focus on language-oriented technologies, like LLMs and text-to-image models, that are powerful mediators of design processes. These technologies enable humans to describe their desires at almost any level of abstraction, from high-level goals vaguely specified (“I’d like a game to help my kid learn to read”) to low-level corrections of undesired outputs (“Don’t say ‘I know because I’ve tasted it’ when about a recipe substitution's taste”). What's next: Assistant Professor, Computer Science, UCLA Jiachen Lian Jiachen Lian Email: jiachenlian@berkeley.edu Website: https://jlian2.github.io Advisor(s): Gopala Anumanchipalli Research Blurb: My research focuses on human-centered AI across speech, healthcare, and systems. Looking for: Look for AI talents to join our startup Josh Kang Josh Kang Email: minwoo_kang@berkeley.edu Website: https://joshuaminwookang.github.io/ Advisor(s): John Canny Research Blurb: I study language modeling and related topics in NLP; specific interests are human user simulation and building conversational, collaborative AI agents. What's next: AI Scientist at Mistral AI Junhao (Bear) Xiong Junhao (Bear) Xiong Email: junhao_xiong@berkeley.edu Website: https://www.linkedin.com/in/junhao-bear-xiong Advisor(s): Jennifer Listgarten, Yun Song Research Blurb: Junhao (Bear) Xiong is a PhD candidate at UC Berkeley, advised by Jennifer Listgarten and Yun S. Song. His work focuses on machine learning methods for biology, with an emphasis on generative modeling for proteins. Previously, he studied Applied Math and Computer Science at Johns Hopkins. Looking for: Research scientist Kaylo Littlejohn Kaylo Littlejohn Email: kaylo_littlejohn@berkeley.edu Website: https://kaylolittlejohn.com Advisor(s): Gopala Anumanchipalli Research Blurb: My research is focused on speech modeling and natural language processing. I co-led the development of multimodal AI tools to accurately translate brain activity into text, audible personalized speech, and a high-fidelity "digital talking avatar" (Nature 2023, Nature Neuroscience 2025). I am also tech lead for voice modeling at Roblox. Looking for: Research Scientist / Engineer Kent Chang Kent Chang Email: kentkchang@berkeley.edu Website: https://kentkc.org Advisor(s): David Bamman Research Blurb: I work on NLP and multimodal machine learning, with a focus on evaluating large language models and building multimodal systems for understanding dialogue, narrative, and social interaction. My research includes benchmarks for LLM memorization, multimodal datasets sourced from feature films and television, and studies of model behavior. I'm interested in bridging computational methods with questions from the humanities and social sciences about whose voices get represented in AI systems, and about AI's broader impact. My work has appeared at EMNLP and ACL, among others. Looking for: (teaching) faculty, Research Scientist, ML/AI SWE Kevin Black Kevin Black Email: kvablack@berkeley.edu Website: https://kevin.black Advisor(s): Sergey Levine Research Blurb: I work on large-scale robot learning: including imitation learning, reinforcement learning, generative modeling, real-time control, and whatever else it takes to make robots work in the real world! What's next: Research Scientist of Physical Intelligence Kunhe Yang Kunhe Yang Email: kunheyang@berkeley.edu Website: https://www.kunheyang.com/ Advisor(s): Nika Haghtalab Research Blurb: My research focuses on the theoretical foundations of designing and evaluating AI algorithms in environments shaped by human incentives and AI agency. My work spans human-centric policy learning, incentive-aware evaluation, and multi-agent collaboration and information transmission, drawing on tools from machine learning theory and computational economics. What's next: Postdoc Research at Stanford Lisa Dunlap Lisa Dunlap Email: lisabdunlap@berkeley.edu Website: https://lisabdunlap.com Advisor(s): Joseph Gonzalez, Trevor Darrell Research Blurb: Auditing generative models. What's next: Research Engineer at Anthropic Long (Tony) Lian Long (Tony) Lian Email: longlian@berkeley.edu Website: https://tonylian.com/ Advisor(s): Trevor Darrell, Adam Yala Research Blurb: My research primarily focuses on developing real-time multi-modal multi-agent systems and parallel reasoning systems through end-to-end RL. What's next: Member of Technical Staff at Thinking Machines Lab Maulik Bhatt Maulik Bhatt Email: maulikbhatt@berkeley.edu Website: https://maulikb.com Advisor(s): Negar Mehr Research Blurb: My research develops autonomous robots that can safely coordinate with humans and other robots in shared environments. I build scalable algorithms grounded in game theory and diffusion models that let agents reason about the intent and behavior of others around them. My work spans real-time multi-agent trajectory planning and imitation learning in the presence of multi-modality. I've validated these methods on hardware platforms ranging from quadrotors to manipulators, with the goal of making multi-agent coordination robust, interpretable, and deployable in the real world. What's next: Joining Toyota Woven's end-to-end autonomous driving team. Michael Psenka Michael Psenka Email: psenka@berkeley.edu Website: https://www.michaelpsenka.io/ Advisor(s): Aditi Krishnapriyan Research Blurb: Work in various domains (reinforcement learning, world models, AI+bio/chem), generally working on longer-horizon and out-of-distribution problems in planning and interpolation (e.g. robot manipulation from start state to goal, molecular dynamics of proteins between ground states). My thesis took a variational approach (think calculus of variations) directly from deep generative models of the environment, framing path-finding as minimizing a functional induced by the learned model itself (its score, its critic, or its dynamics). Through my research I've gained insight on how to properly handle dynamics in deep learning systems, and I plan to continue developing systems that are dynamic and adaptive. What's next: Lead Research Scientist at Baseten Nathan Lichtlé Nathan Lichtlé Email: nathan.lichtle@gmail.com Website: https://nathanlichtle.com Advisor(s): Alexandre M. Bayen Research Blurb: RL for autonomous driving. What's next: Chief Scientist & Co-founder at Yumi Health Neerja Thakkar Neerja Thakkar Email: nthakkar@berkeley.edu Website: https://neerja.me/ Advisor(s): Jitendra Malik Research Blurb: My research focuses on scaling predictive world models to handle the complexity of in-the-wild motion. Using autoregressive and diffusion frameworks, I develop better representations for real-world prediction and propose methods to efficiently adapt these models to new domains. Looking for: Research scientist Nikita Mehandru Nikita Mehandru Email: nmehandru@berkeley.edu Website: https://n-mehandru.github.io/ Advisor(s): Ahmed Alaa and David Bamman Research Blurb: My research develops and applies machine learning methods for clinical reasoning and disease progression modeling using unstructured text and time series data from electronic health records. In collaboration with physicians at UCSF, I bridge method development and clinical validation with the intention to build reliable, interpretable AI systems in medicine. Looking for: Research Scientist Niklas Lauffer Niklas Lauffer Email: nlauffer@berkeley.edu Website: https://niklaslauffer.github.io/ Advisor(s): Stuart Russell and Sanjit Seshia Research Blurb: Niklas's research is focused on AI safety and reinforcement learning, particularly in the area of multi-agent interaction and LM agents. He's worked on enabling adversarial learning in cooperative and mixed-motive settings, solving issues of covariate shift in training LM agents on long-horizon tasks, as well as evaluating safety risks posed by LM agents in multi-agent settings. What's next: Research Scientist at Google Deepmind Qiyang Li Qiyang Li Email: qcli@berkeley.edu Website: https://colinqiyangli.github.io/ Advisor(s): Sergey Levine Research Blurb: Recent progress in robotic manipulation policy learning has been largely driven by (1) the increasing availability of large-scale prior datasets and (2) the success of action chunking, where the policy predicts a short sequence of future actions rather than a single one. However, most action chunking policies are trained via supervised imitation learning, because efficient online self-improvement with reinforcement learning (RL) remains challenging—limiting real-world applicability. My PhD research studied how we could leverage prior data to optimize action-chunking policies with RL, combining empirical results with theoretical insights. Looking for: Post-doc/research scientist for RL in robotics and LLMs! Sampada Deglurkar Sampada Deglurkar Email: sampada_deglurkar@berkeley.edu Website: https://sdeglurkar.github.io/ Advisor(s): Prof Claire Tomlin Research Blurb: My research is in providing safety assurances for AI-enabled autonomous systems, ranging from robots to autonomous vehicles to aviation systems. For this, I have worked with uncertainty quantification for machine learning models, decision-making under uncertainty algorithms, and tools for producing probabilistic guarantees on system operation. Looking for: Research scientist, Research engineer Vinamra Benara Vinamra Benara Email: vbenara@berkeley.edu Website: https://cs.berkeley.edu/~vbenara Advisor(s): Ion Stoica Research Blurb: My research focuses on LLM post-training, including data curation, RLHF, RLVR with VLMs, evaluations, reasoning, agentic workflows, and interpretability. I also have strong expertise in systems infrastructure for distributed computing. Looking for: Research scientist / Research Engineer Vongani Maluleke Vongani Maluleke Email: vongani_maluleke@berkeley.edu Website: https://people.eecs.berkeley.edu/~vongani_maluleke/ Advisor(s): Jitendra Malik and Angjoo Kanazawa Research Blurb: Vongani Maluleke is a PhD candidate at UC Berkeley (BAIR, advised by Jitendra Malik and Angjoo Kanazawa), where she led the development of MAGNet, a unified multi-agent motion generation framework that supports a wide range of motion generation tasks without retraining or architectural changes, outperforming task-specialized state-of-the-art baselines. She is currently extending this work by deploying it on a Unitree G1 humanoid to make it embody social intelligence. Before her PhD, she was a Senior AI Consultant at Deloitte, awarded Exceptional Performer two consecutive years, leading AI system development across media, telecommunications, retail, and financial services. Looking for: Research scientist Wei-Jer Chang Wei-Jer Chang Email: weijer_chang@berkeley.edu Website: https://weijer-chang.github.io/ Advisor(s): Masayoshi Tomizuka Research Blurb: My research focuses on developing safe and intelligent autonomous systems for complex, human-centered environments. I work at the intersection of machine learning, generative models, and reinforcement learning, with applications in autonomy. My work addresses challenges in multi-agent interaction, interactive human behavior, and long-tail safety-critical scenarios at scale. Looking for: Research Scientist, Applied Scientist, Roboticist Xiuyu Li Xiuyu Li Email: xiuyu@berkeley.edu Website: https://xiuyuli.com/ Advisor(s): Kurt Keutzer Research Blurb: My research focuses on developing scalable and self-improving large language model agents, with emphasis on coding agents for complex, long-horizon tasks. This direction builds on my work in parallel reasoning, and on broader expertise in making generative models more efficient in training and inference across language and vision. What's next: Member of Technical Staff at xAI Yichen Xie Yichen Xie Email: yichenxie0928@gmail.com Website: https://yichen928.github.io/ Advisor(s): Masayoshi Tomizuka Research Blurb: My research focuses on building multimodal foundation models and world models that understand and interact with complex physical environments. I aim to develop unified representations across modalities, enabling AI systems to reason over space, time, and dynamics toward general-purpose embodied intelligence. What's next: Research Scientist at Luma AI Yigit Efe Erginbas Yigit Efe Erginbas Email: erginbas@berkeley.edu Website: https://www.linkedin.com/in/erginbas/ Advisor(s): Kannan Ramchandran, Thomas A. Courtade Research Blurb: My PhD research spans two threads: online learning in large-scale markets, and interpretability of large machine learning models. In the first, I work on sequential decision-making with applications to recommendation, pricing, and assortment selection. My focus is on designing algorithms with provable guarantees for welfare maximization, revenue maximization, and stability. In the second, I develop scalable attribution methods that exploit the sparse, low-degree structure of real-world interactions, using tools from signal processing and information theory. More recently, I have been exploring principled ways to evaluate the faithfulness of model self-explanations. What's next: Researcher at Hudson River Trading's AI Labs (HAIL) Yiheng Li Yiheng Li Email: yhli@berkeley.edu Website: https://Yihengli.com Advisor(s): Masayoshi Tomizuka Research Blurb: I am working on vision world modeling, with prior experience in diffusion model's efficiency as well as in autonomous driving. What's next: Research Scientist at Waymo Zhe Fu Zhe Fu Email: zhefu@berkeley.edu Website: https://fu-zhe.com/ Advisor(s): Alexandre Bayen Research Blurb: My research focuses on physics-informed learning and control for mixed-autonomy systems, with applications in transportation. I design physics-informed neural networks to learn solutions of nonlinear partial differential equations, enabling accurate and data-efficient prediction of traffic dynamics. Building on these models, I develop both model-based and learning-based control strategies that coordinate automated vehicles to improve system-level performance. My work bridges machine learning, control, and real-world deployment, and has been validated in large-scale field experiments. More broadly, I aim to advance trustworthy, interpretable AI for decision-making in complex, real-world systems. What's next: I will be an Energy Fellow at Stanford after graduation. Also looking for Faculty, or research scientist positions in AI, control, and autonomy.
... government of the people, by the people, for the people ... — Abraham Lincoln, Gettysburg Address (1863) The cost of AI is dropping rapidly. GPT-4-class capabilities cost roughly $30 per million tokens in early 2023; today the same runs under $1 , and some providers are pushing costs below $0.10 . Across benchmarks, inference prices have fallen between 9x and 900x per year , with a median decline near 50x. Even frontier models are getting dramatically cheaper each generation, with open-source models following closely behind. And crucially, even if “Nobel-Prize-winning genius-level” intelligence isn’t here yet, the intelligence that suffices for the vast majority of knowledge work is here today, and getting cheaper by the month. At this rate, we are soon entering the era of virtually free intelligence —the kind that is more than enough for everyday knowledge work. A cartoon database character and an AI robot agent holding hands Disclosure: This post is a perspective led by Aditya G. Parameswaran —an Associate Professor of EECS and co-director of the EPIC Data Lab at UC Berkeley—together with his collaborators. It is part landscape survey and part perspective, and several of the research directions discussed below (including agentic speculation, structured memory, and synthesizing custom data systems from scratch) draw on the authors' own ongoing work. So, what does this new era of near-free intelligence mean for data systems? We believe three new challenges—and opportunities—stem from near-zero inference costs: Data Systems For Agents. Agents will soon become the dominant workload for data systems—with swarms of agents spun up in response to each end-user request. Given differences in characteristics between agents and humans—or applications acting on their behalf— how should we redesign data systems for such agentic users? Data Systems Of Agents. As agents start taking on the bulk of knowledge work, a new substrate is needed for thousands of agents to manage state over long-running tasks, coordinate and reach consensus, and deal with failures. What do data systems that reliably and efficiently run and manage agent swarms look like? Data Systems By Agents. Agents are rapidly becoming capable of synthesizing entire data systems in one go—meaning we can rebuild custom systems for each new workload. Verifying that such systems match intended behavior is a challenge. What does it take to let agents synthesize data systems we can actually trust? A database character and a robot agent holding up a triangle labeled 'of', 'for', and 'by' Data Systems For, Of, and By Agents Next, we will discuss each in more detail, followed by discussing the intertwined future of data systems and agents, especially as the three challenges intersect. Data Systems For Agents An agent querying a database doesn’t behave like a person or a BI tool. It performs what we call agentic speculation : a high-volume, heterogeneous stream of work spanning schema introspection, columnar exploration, partial and then full query formulation. With multiple agents each exploring portions of the hypothesis space, each user request could amount to 1000s of individual SQL queries. Now, users can issue ‘high-level’ data tasks, e.g., root-cause analysis—e.g., ‘why did coffee sales in Berkeley drop this year’—or exploratory cohort analysis—e.g., ‘which user segments are most likely to churn next quarter’—each involving a combinatorial space of potential joins, aggregations, and filter combinations. An agent sending many SELECT SQL queries to a database and receiving results back Data Systems Redesigned to More Effectively Support Agentic Speculation The requests from these agents have various opportunities for optimization. For instance, on a text-to-SQL benchmark with multiple agents attempting each task, only 10-20% of the sub-plans are distinct. Thus, 80-90% of sub-queries perform duplicate work. The same experiments show task success rates significantly increasing with more agentic attempts—so the redundancy is actually helpful. But from the data system perspective it’s wasted work. An agent-first data system can exploit such properties to help agents make progress faster. It can reuse results across overlapping sub-plans, drawing on ideas from decades-old literature on multi-query optimization and shared scans . Or the data system can try to satisfice , returning approximate answers that are good enough for agents to make progress, leveraging work from the AQP literature —or streaming the results of the final or intermediate operators to help agents decide if seeing the rest is necessary or helpful. Another opportunity here is to rethink the query interface entirely: instead of agents issuing a single SQL query at a time, they could instead issue a batch of queries, each with its own approximation requirements. Since enumerating an exponential search space (as in the root cause or cohort analysis examples above) isn’t a good use of agentic reasoning ability, perhaps data systems should support higher-level primitives rather than requiring agents to list each SQL query explicitly. One idea here is to draw on DBT-style Jinja macros to provide looping-based primitives for agents to interact with data systems. A swarm of AI agents working at laptops A Caffeinated Army of Agents Ready to Tirelessly Complete Your Data Tasks A final opportunity here is to stop thinking of data systems as passive executors of queries; data systems could be proactive , as they possess more grounding in data and system characteristics that agents may lack a priori—they could steer agents in different directions, provide results for related queries, and also provide performance-level feedback (e.g., instead of executing an expensive query, the system could first provide the agent a latency estimate). The reason we can do this now as opposed to the past is that an agent can accept any form of textual feedback and isn’t expecting a strict SQL query result. In fact, the data system could also prepare both materialized and virtual views for an agent in advance, provided to the agent as part of context, as this may be cheaper or more effective than having an agent author or use them. Data Systems Of Agents Previously, we focused on how agents interact with data systems. Now, we consider everything else agents need to keep working: where they live, how they remember, how they coordinate with each other, and how they deal with failures of each other. This agentic substrate is separate from the inference stack powering raw intelligence. However, the inference stack itself is being abstracted away through APIs (e.g., from OpenAI or Anthropic), or, for open-weight models, through serving frameworks that hide low-level details. So far, the agentic substrate has been managed through harnesses like Claude Code and Codex , coupled with various mechanisms to store and retrieve memory. First, on the memory front, the current wisdom is that files are all you need ; agents write to unstructured markdown (MD) files, which can then be searched using grep, or via embedding-based retrieval. In fact, many argue that the solution to continual learning is having agents consume a lot (e.g., an entire codebase, slack, company wikis, …) and then write their learnings into MD files, which are then retrieved selectively on demand. Indeed, file systems, bash scripting, and MD files are and will still be important for agents. However, at scale, when agents are doing the vast majority of knowledge work, this approach will no longer be effective. Given limited context windows, retrieving all MD file fragments that may be relevant and stuffing it into the context will break down at some point. Even if context windows continue to grow, there are latency benefits to not put all information into context — and in many cases, e.g., when knowledge work involves interacting with large databases or code bases, it will be infeasible to serialize all relevant data into context. A swarm of robot agents holding hands, each drawing state from a single large shared database platform below them Data Systems As A Substrate for Multi-Agent Swarms One could use a knowledge graph representation , but knowledge graphs suffer from the same limitations as unstructured MD-based memory due to their lack of structured search. What one needs is to be able to retrieve only memory that is pertinent to the task, across multiple attributes (or facets) of interest. For example, an agent debugging a flaky test should be able to pull only the memories tagged with the relevant module, language, framework, and failure mode—rather retrieving based on keywords or embedding similarity. A separate issue is what to actually retrieve; raw agent traces with mistakes are not very useful as they will induce agents to repeat the same mistake—instead, we want the retrieved memory to be corrective. We recently explored a related notion of structured memory , where we organize memory across various attributes, each of which could be set as * to indicate universal applicability, or set as a list of values to be matched. For a data agent, the dimensions could include the columns and tables, type of operation, and finally, open-ended natural-language corrective instructions. So, we could include memory that only applies to a given type of operation (e.g., ‘when performing date-time operations, use fiscal year as opposed to calendar year conventions’), or a given table (e.g., ‘column product_cleaned is preferred over column product when querying on product name’). One open question is defining an application-specific structured memory —or what others have called world models for memory . We believe this is akin to defining a schema for each application—and perhaps agents themselves can help us define and refine it over time. Diagram showing corrective knowledge stored with structured attributes (SQL keywords, tables, columns, data type) and retrieved by matching the features of a new agent query One Possible Way To Store and Retrieve Structured Knowledge [From Here] Structured memory will be useful also for evolutionary frameworks to effectively manage search spaces. Indeed, storing, structuring, and mining large volumes of single and multi-agent traces can help future agents become much more efficient—potentially enabling effective recursive self-improvement through structured memory-based mechanisms. Another challenge is to support concurrent edits to shared memory, and concurrent edits in general, when there are many agents performing transformations. While there have been some useful attempts at supporting multiversioning and copy-on-write semantics , it isn’t clear that such techniques will suffice when thousands of agents are attempting to edit shared state at the same time. For instance, when agents are trying various potential transactions in response to a user request, the effects of the vast majority of these transactions need to be rolled back—with only the one ‘correct’ transaction’s result persisting. Work on supporting exactly-once semantics is relevant here, as are underlying techniques based on CRDTs and operational transformation. For updates to fuzzy mechanisms such as memory, we may be able to sacrifice on consistency for perfect correctness in the interest of latency. While agents can reason about semantics to compensate or roll back their actions to eventually finalize most tasks, the primary challenge lies in the degree to which they step on each other’s toes during the process. An important failure mode to be avoided is a form of “livelock,” where incessant compensating actions prevent any meaningful progress. Beyond shared state, other concerns emerge when trying to support an army of agents, including what to do when agents fail, how agents should communicate with each other (directly or through intermediate shared state), and how we should deal with straggler agents. There have been some developments in supporting durable multi-agent execution, such as Temporal , but it remains to be seen if such solutions will apply at scale across thousands of agents. On the topic of communication, we need mechanisms to enable agents to negotiate with each other. Imagine four developer agents attempting to reach consensus on a shared schema, with distinct but overlapping objectives. In a human setting, this would involve iterative discussion and compromise; for agentic swarms, we must define the mechanisms that allow them to converge on a design that reflects the underlying goals of their respective principals. Or if agents are all requiring access to a limited resource, again communication will be necessary. It remains to be seen if this is best done via centralized coordination, or if a decentralized approach is necessary. Data Systems By Agents Finally, if intelligence is effectively free, then we can employ this intelligence to synthesize new data systems from scratch. Indeed, in many settings, general-purpose data systems may be overkill, as they have to support every schema, query, and hardware target. Given a workload, recent work, including Bespoke OLAP and GenDB , has shown that one can use an agentic pipeline to synthesize a complete, workload-specific analytical engine—in minutes to a few hours, at a cost of a few dollars. The engines are disposable: when the workload shifts, one can simply regenerate them. Analogously, our work has shown that one can synthesize custom key-value stores from scratch, targeted to the workload. In fact, modern IDEs, such as Kiro , elevate specifications for systems development to be a first-class citizen. A robot agent with a hammer and chisel carving a database character out of a block of stone Agents Can Synthesize Custom Data Systems From Scratch The main issue, however, is that specifications are typically imperfect, and don’t cover all corner cases. Present-day agents will exploit the missing specifications to reward-hack their way to a high performance metric. In our custom key-value store work, we found that one way to alleviate this is to have auxiliary verification agents trying to generate test cases that catch the exploitation of corner cases, essentially expanding the specification. Yet another approach is to both generate a system and a proof for its correctness together, for which we have found some early success , but more needs to be done to solidify the approach. Further, it remains to be seen what is the best way to solicit human-written specifications for a system—can this be done in an iterative, human-in-the-loop manner, as opposed to a one-shot, incomplete one. Indeed, human-written specifications are incomplete even for manually authored software, so one would expect that future agents that are more aligned will increasingly exercise better judgement when making design decisions. Pipeline diagram where a system builder provides a specification, planner and coder agents generate code, the code is evaluated for correctness and performance, and critic and auditor agents provide feedback and catch reward hacking One Possible Data System Synthesis Pipeline [From Here] Other questions here involve testing whether starting from a mature system (e.g., Postgres) and removing components/functionality can lead to higher performance or more user trust. Separately, is there an opportunity to make the design composable, comprising various verified components that are mixed and matched given a workload? For example, perhaps the workload hasn’t changed enough for the storage layer to be updated, but perhaps the query optimizer requires changes. A perhaps more viable proposition involves employing agents coupled with proof systems to target critical parts of the code associated with formal proofs, rather than doing so for the entire system. A final opportunity here is to move away from the traditional data systems stack with clearly-defined interfaces (e.g., parser, query optimizer, storage manager, …) — that were each largely the prerogative of a single human team to manage. Instead, agents can find new ways to “blend” these components together, perhaps identifying new optimization opportunities as a result. Agents can also fill in missing gaps in functionality to make existing systems much more feature-complete, or reach feature-parity with other competing systems—or analogously, continuously refining open-source systems in response to feature requests or issues (perhaps filed by other agents!) Doing so in a way that prioritizes correctness, long-term maintenance, and human interpretability will be a challenge. Looking Further Ahead In the era of near-free intelligence, data systems matter more than ever. As agents take on the bulk of knowledge work, the workload for data systems will change, the substrate they need to run on will have to be built, and increasingly, they will participate in designing data systems themselves. Each of these shifts opens up a new, exciting research agenda. A half-database, half-robot character next to a yin-yang symbol formed by a database and a robot agent Co-Evolution of Data Systems and Agents Looking further out, the boundaries between agents and data systems will likely start to blur. For instance, agents may design the data systems they themselves run on, defining both the interfaces as well as the system components underneath. Both the interfaces and internals can be evolved over time by agents in a form of recursive self-improvement. There is also an opportunity to rethink data systems as a holistic source of truth for the entirety of relevant state: including raw data, memory, and coordination state, further erasing the distinctions between the data that is being queried by agents and data generated as a result of agentic activity. Finally, data systems may themselves incorporate agentic components, fundamentally evolving from passive computation engines into intelligent, proactive, self-optimizing architectures. It is hard to predict what the future may hold. We’re in for a wild ride! Acknowledgments The perspective and ongoing work described in this post are the product of joint research and many discussions with wonderful collaborators at the EPIC Data Lab , Data Systems & Foundations group, and the broader Berkeley AI-Systems community. Thank you all! BibTex for this post: @misc { intelligence-is-free-blog , title = {Intelligence is Free, Now What? Data Systems for, of, and by Agents} , author = {Aditya G. Parameswaran and Shubham Agarwal and Kerem Akillioglu and Shreya Shankar and Sepanta Zeighami and Rishabh Iyer and Matei Zaharia and Alvin Cheung and Natacha Crooks and Joseph Gonzalez and Joseph Hellerstein and Ion Stoica} , howpublished = {\url{https://bair.berkeley.edu/blog/2026/07/07/intelligence-is-free-now-what/}} , year = {2026} }
ABBEL overview Overview of ABBEL compared to traditional recursive summarization. Beliefs replace the full interaction history as the agent’s working context, and belief grading improves performance by supervising the contents of each belief state.. As task horizons grow, LLM contexts can’t scale forever. Self-summarization enables concise, interpretable contexts, but at a significant performance cost, especially for human assistance domains where high quality data is scarce, e.g., collaborative code generation. We address this with ABBEL : a framework that isolates and supervises the information content of summaries in the form of natural-language belief states. Motivation: the cost of recursive summarization For language models to effectively assist with increasingly complex tasks such as software development, they must be able to interact with us over hundreds or even thousands of steps. For such long tasks, it is impractical to keep the history of the entire interaction in context. The heuristic approach used so far has been summary generation, sometimes called context compaction. For example, Cursor’s latest model composer 2.5 uses compaction during training for improved performance ( Cassano et al., 2026 ). Alongside composer, Grandcode ( DeepReinforce et al., 2026 ), the first system to consistently beat all human competitors in online coding competitions, despite using one of the newest efficient attention models (Qwen 3.5-397B), 1 still found it necessary to employ context summarization. But compaction has a problem. Despite seemingly low performance gaps in benchmarks, model servers like Cursor continue to recommend that users avoid compaction with their coding assistants in the middle of a task ( Heule et al., 2026 ). To understand why, see below the performance over RL fine-tuning of a Context summary model compared to full context models in Combination Lock, a Wordle-like game that allows up to 16 guesses. 2 Though both model types improve over the course of training, the summary model never closes the gap. Average attempts to guess target word with context summary vs full context policies over training Fig. 1: Average attempts to guess the target word on Combination Lock over RL fine-tuning (lower is better). Context-summary policies improve with training but do not close the gap to full-context policies. Making models self-summarize while completing a task increases the complexity of the learning problem. While this could typically be addressed by training with more data, the performance degradation observed in real world interactive settings likely arises from the difficulty we have in creating and using human simulators effectively to generate high quality training environments ( Lin et al., 2025 , Tomlin et al., 2025 ). Thus, the better you can learn to summarize on the limited and messy multiturn interaction trajectories you can collect, the better off your model will be for downstream users. ABBEL: acting through belief bottlenecks ABBEL belief grading diagram Fig. 2: Autoencoder-inspired belief grading. The model encodes prior belief, action and observation (b t , a t , o t ) into posterior belief b t+1 and is rewarded for how well select information from the history can be reconstructed from that belief. To address poor learning efficiency, we isolate the summary generation task. Drawing inspiration from recursive Bayesian estimation, we formulate summaries as belief states, which we periodically prompt the model to update based on new information. 3 ABBEL overview animation frame ‹ Prev Pause Next › 1 / 16 Fig. 3: ABBEL rollout. Belief updates from the latest observation alternate with action selection conditioned only on the current posterior belief. Belief grading We then extract and supervise the contents of the belief states (Fig. 2, Belief Grading). Belief grading can be thought of as adding an auxiliary RL task, using heuristics designed to capture what makes a good belief as the reward. An example heuristic for coding could be shorter is better, but closer to being able to reconstruct the git diff is also better, so balancing these would yield a good belief. In domains where good heuristics are hard to define, we propose a general autoencoding-inspired grading function, which treats the current language model π θ as both encoder and decoder of information from the history, and the belief states as the codes. We grade each belief b t+1 by how well it can be used by the current model π θ to reconstruct the most recent observation o t : Eq. 1: Reconstruction grading objective Eq. 1: Reconstruction grading objective. Here b t+1 is the updated belief, o t the latest observation, a t the action just taken, b t the prior belief, p I the task prompt, and π θ the current model. Higher grades reward beliefs that retain information needed to decode the latest observation. What do we gain by grading beliefs? Collaborative coding on CollabBench We demonstrate the utility of belief grading in our motivating domain of human-driven assistive coding, with the CollabBench environment from Sweet-RL ( Zhou et al., 2025 ). CollabBench collaborative coding environment Fig. 4: CollabBench collaborative coding environment. The agent asks clarifying questions, then submits a function scored against hidden unit tests. We see that with the general reconstruction-based belief grading function we reduce the performance gap from full context models by about 50%, and train in 50% fewer steps compared to training models to summarize without belief grading (no BG). After training, ABBEL still uses significantly less memory than the full context setting, as measured by the peak context token length (Peak Tokens). Model Test Pass Rate ↑ Success Rate ↑ Peak Tokens × 10² ↓ Training Steps ↓ Full Context 0.52±0.02 0.39±0.02 14.08±0.55 100 ABBEL (no BG) 0.46±0.02 0.31±0.02 4.20±0.37 100 ABBEL-rec-BG 0.48±0.01 0.36±0.01 6.01±0.33 50 Fig. 5: CollabBench results. With reconstruction belief grading, ABBEL-rec-BG recovers about half the gap to full context while using fewer peak tokens, and trains in 50 steps instead of 100. Combination Lock Additionally, in CombinationLock, we demonstrate that ABBEL with a belief grader which leverages domain knowledge (by computing useful statistics over the history and checking that they can be reconstructed from the belief state), enables even higher learning efficiency than full context (FULL CTX) models. Average Attempts to guess target word ABBEL Fig. 6: Average attempts to guess the target word on Combination Lock (lower is better). With domain-knowledge belief grading, ABBEL approaches or exceeds FULL CTX in this setting; without belief grading, learning is slower. Multi-objective question answering In a third environment, multi-objective question answering (from MEM1 Zhang et al., 2025 , a recent work which performed end-to-end optimization in a modified version of typical recursive summarization), we demonstrate the utility of isolating belief states from reasoning, by showing that a Peak Belief length Penalty (more details in paper) significantly reduces memory usage with minimal performance degradation, unlike is commonly observed when penalizing reasoning lengths ( Arora et al., 2025 ). Performance and memory usage in multi QA Fig. 7: Exact-match score and peak memory versus number of objectives in multi-objective QA. ABBEL with a peak belief penalty (PBP) maintains comparable performance while using less memory than MEM1 and ABBEL without PBP in this evaluation. Related work Alternative solutions to managing long contexts involve different tradeoffs, and are worth considering depending on the requirements of a deployed system. Context compression methods generate dense representations which, while computationally efficient, sacrifice human-understandability ( Kontonis et al., 2026 , Eyuboglu et al., 2025 , Gupta et al., 2025 , Chevalier et al., 2023 , Deng et al., 2025 , Deng et al., 2025 , Bulatov et al., 2022 ). Hand-designed summarization prompts ( Wang et al., 2025 , Örwall et al., 2025 , Starace et al., 2025 ) and pruning strategies ( Jiang et al., 2024 ) specific to target environments require expert human knowledge and don’t allow an agent to learn what to remember as part of its decision-making strategy. Methods that process long contexts into an external memory store ( Packer et al., 2023 , Xu et al., 2025 ) for the agents or subagents to query ( Zhang et al., 2025 ) are complementary, as they may benefit from better next context creation through summarization training. We would like to point out some exciting works in the space of general recursive summarization focused on math ( Wu et al., 2026 ), reasoning with belief generation ( Zhou et al., 2025 ), competitive coding with a distilled summarization module using similar autoencoding objectives to our general belief grader ( DeepReinforce et al., 2026 ), and adding continuous features to summaries ( Kontonis et al., 2026 ). What’s next for better memory? Many more possibilities are enabled through using explicit belief states as information bottlenecks for multi-step interaction. You could reward actions based on their effect on the belief state to guide exploration, transmit the explicit belief states for better communication between agents, or even improve user controllability by directly modifying the memories on which the agents’ decisions are based. Some forms of information, e.g., what a person looks like, are not represented well by text alone. A continuously learning system will also have to capture such information. Additionally, if we want a system to learn to communicate in a brand new language or to play a brand new game better than any person in the world, the skills accumulated over the lifetime of conversations or games must be stored in a very compressed form, essentially taking on the role of the weights of the model itself. More powerful systems will likely utilize a combination of multiple forms of memory, where the contents of the context may correspond to working memory while other approaches are used for short and long-term memory. How to instantiate these other forms of memory, for instance via test-time training, adapter memories, continuous context memories, or some combination thereof, presents an exciting challenge. Acknowledgements Acknowledgements: We would like to thank Alane Suhr and Kartik Goyal for advising this research as well as Ethan Mendes , David He , Jitesh Jain , and Nicholas Tomlin for comments on early drafts of this post. We would like to thank the MEM1 authors for their email correspondence and for sharing private reviewer feedback which we found particularly insightful. Citation If abbel was inspiring for your future work, please cite us with this! And here is some advice for doing similar research! @misc { lidayan2026abbellearningnaturallanguagebelief , title = {ABBEL: Learning Natural-Language Belief States for Memory-Efficient Interaction} , author = {Aly Lidayan and Jakob Bjorner and Satvik Golechha and Kartik Goyal and Alane Suhr} , year = {2026} , eprint = {2512.20111} , archivePrefix = {arXiv} , primaryClass = {cs.CL} , url = {https://arxiv.org/abs/2512.20111} , } With newer models the number of tokens till 50% compute spend is on attention gets much larger than 25K. Interleaving linear attention alternatives with full attention as is done with gpt-oss and DeepSeekv4, results in massive flops reductions for the attention computation. For example with DeepSeekv4-Pro (1.6T A49B) it requires nearly 450 thousand tokens to reach the 50% tradeoff point. Grandcode uses Qwen-3.5-397B-A17B a model which hits 50% FLOPs for attention at ~150 thousand tokens. ↩ This setting is technically solvable with much more computationally effective tools, but serves as a flexible test bed to study properties of recursive summarization. Bertsimas et al., 2022 , showed that an exact solution for the wordle game instantiated with the original vocabulary of the javascript game can be found with dynamic programming, but evidently the general formulation of wordle as a guessing game on K letters with L attempts and some dictionary of valid words and correct words D is NP hard to determine the minimal number of moves required. ↩ In practice there is an O(N/K) overhead cost for summary. N is the total number of actions. K is the number of actions till summarization is triggered. This is necessarily true for any summary approach. For ease of illustration this gif uses K = 1. In our experiments, to put more emphasis on summarization weaknesses we also use K=1. In practice overhead is small as K can be chosen to be near the efficient hardware limit. ↩
GPT-6 Astra Ultrafast 目前已在 OpenAI API 中,以及面向符合条件的 ChatGPT Work 和 Codex 用户开放。借助 OpenAI 模型中的推理优化,并利用 NVIDIA Blackwell 架构的能力,Ultrafast 的令牌生成速度最高可达 Astra Standard 模式的 8 倍。对于开发者,[…]
Run Claude Code, Codex, AGY, and Grok Build side by side. See who is working, who needs you, and what every account has left. Comments URL: https://news.ycombinator.com/item?id=49942434 Points: 5 # Comments: 0
I just realized that codex now exposes local server endpoints for all plugins without extra auth. So you can just use them from any model or harness. Im soooo excited!!! One auth for everything! For Pi: pi install npm:@wileai/pi-codex-connector For any harness: npx -y @wileai/codex-connectors-mcp@latest Comments URL: https://news.ycombinator.com/item?id=49929381 Points: 5 # Comments: 0
Chatham Financial 使用 Codex 和 GPT-5.6 构建技术并重新设计工作流程,将交易验证时间从 30 分钟缩短至 4 分钟以内。
I found that I'm not using even 1/3 of vim/vscode features anymore. That's wht I'm building rhun - a small code editor for Linux, Windows and Apple silicon Macs. It obviously has Vim mode, a terminal, Git diffs and a panel for Claude Code or Codex sessions. The editor and pixel renderer share an x86-64 assembly core. For Apple silicon, a build-time translator converts that core to AArch64, with separate platform adapters around it. The latest release can draft commit messages using a local Ollama model or an existing Claude Code or Codex subscription. It's a solo project, MIT licensed and still early. Comments URL: https://news.ycombinator.com/item?id=49926726 Points: 7 # Comments: 2
在策略蒸馏(OPD)是一种很有前景的语言智能体训练方法,它能够针对学生模型生成的轨迹提供密集的教师监督。然而,在多轮交互中,一个错误动作会改变学生模型之后所遇到的状态,因此错误会跨轮次累积。在针对三个 Qwen3 模型(8B 至 235B)的初步实验中,我们发现,超过半数的失败轨迹都包含一个关键错误,即使智能体进一步偏离任务完成目标的动作,而且这一错误通常很早就会发生。这些关键错误往往仍然可以恢复:在关键错误发生后的几轮中引导模型,就能重新取得任务成功。因此,我们提出 PivotOPD,一种在策略蒸馏框架,联合训练学生模型以避免关键错误,并从这些错误所导致的状态中恢复。在每次关键错误发生时,教师模型会提供一个标准动作,并在接下来的几轮中分别指定一个恢复动作。预防性蒸馏使用标准动作结合反向 KL 散度,引导学生模型避开关键错误;恢复性蒸馏则使用恢复动作结合正向 KL 散度,将学生模型很少采样到的恢复行为传递给它。在 ALFWorld、WebShop 和基于搜索的问答任务上与 13 个基线方法的比较中,PivotOPD 在 Qwen3-1.7B 和 Qwen3-8B 学生模型上都取得了最强的平均性能;对于 1.7B 学生模型,其在 ALFWorld 上的性能较最强基线提升了 5.5%。这些增益也能迁移到软件工程领域的另一模型系列:在 SWE-Bench Verified 上,PivotOPD 将 Nemotron-3.5 学生模型的解决率提高了 3.2%。项目主页:https://research.nvidia.com/labs/lpr/pivotopd/
Hello Hacker News I am Uygar, I am an entrepreneur from London. I run my businesses on calls everyday, customers, partners and suppliers. And i have a horrible memory i forgot things. So months ago with the help of Claude i have built Parrot. I have designed and coded for my work and what i need. An to be honest worked very well and helped me alot. Parrot records what your mac hears and everything stays in your computer if you use local models, no account, no API calls. There is something called copilot which made my life easier, you can upload your files and when parrot hears that question on the screen recommends the answer. Which i believe very handy. If you want to make it smarter and dont mind cloud you can hook up your Claude API and Deepgram account, that gives super powers to Parrot. You still keep it locally but eventually you need to make API calls. So local or cloud totally up to you. Then after months of thinking i have decided share it as open source and this is my first open source project, so please go easy on me :) No account, no membership, open source and free. Code: https://github.com/turantekin/Parrot This project made me so excited and really would like to share with you guys so i hope can help you too. Again, i am new so go easy on me and if you have any feedback i would like to hear. Thank you Uygar Comments URL: https://news.ycombinator.com/item?id=49910328 Points: 4 # Comments: 1
大家好,HN 用户们,我是 Anders,Tom 也在这里。我们正在开发 Magnitude,这是一个面向智能体的推理引擎,能够自我优化,以便在你的硬件上尽可能快速地运行。它支持 Mac、Linux 和 Windows,可运行于任何硬件上,速度最高可达 llama.cpp 的 2 倍。 我们都是软件工程师,此前曾构建过一个开源浏览器智能体,获得了 4k+ GitHub 星标和 100k+ 下载量。我们越来越希望在本地模型上运行它,但发现没有任何推理引擎适合我们的使用场景。 如今的推理引擎都需要在性能上做出取舍。它们要么是: - 面向数据中心硬件上的批量推理构建,以牺牲单会话性能为代价(vLLM、SGLang) - 注重广泛兼容性,而不是针对特定硬件进行优化(llama.cpp、Ollama) - 针对特定硬件或模型进行了专门优化,但引擎功能不够完整(oMLX、ds4) 此外,它们都不是为在本地运行智能体而设计的。智能体会运行很长时间,通常还会同时运行多个实例,而你仍然希望电脑能够处理其他任务。 Magnitude 专为在你的硬件上实现最高性能以及运行本地智能体而构建: - 设备端编译与调优:内核使用灵活的参数编写,并会在模型运行前根据你的实际设备进行调优。这让我们在保持与广泛硬件兼容的同时,也能达到与硬件专用内核相同的性能上限。 - 专注于最佳架构:我们会针对最流行的开放权重模型系列编写可调优且高效的内核。这使我们能够达到并超越硬件或模型专用引擎的性能,而不必为了过度泛化而牺牲性能。 - 动态内存分配:Magnitude 在启动时只预留足以容纳模型权重的内存。随着智能体会话增长,内存堆会动态扩展;智能体停止后,所占内存也会自动释放。这样,智能体运行期间,你的硬件仍可用于其他任务。 - 混合分页注意力:我们借鉴了 SGLang 等引擎中的优秀理念,让并发会话能够共享前缀缓存,同时针对内存邻接性优化数据布局,从而不会影响单会话性能。 Magnitude 完全开源(Apache 2.0)。我们使用 Rust 构建了它,其中包括定制的 GPU 内核运行时和自动调优器。为了达到性能上限,我们从学术界的最佳推理创新(例如 FlashAttention、FlashInfer、TurboQuant)以及其他引擎(例如 SGLang 的基数注意力)中汲取灵感。 在 Qwen 3.6 35B A3B(4 bit)、64k 上下文且不使用推测解码的条件下,与 llama.cpp 进行基准测试: Metal(Mac M4 Pro 48 GB) - 解码速度提升 92%(30 tok/s → 57 tok/s) - 预填充速度提升 9%(466 tok/s → 507 tok/s) - 每个智能体的内存占用降低 28% CUDA(DGX Spark) - 解码速度提升 19%(49 tok/s → 58 tok/s) - 预填充速度提升 23%(2,033 tok/s → 2,507 tok/s) - 每个智能体的内存占用降低 27% Magnitude 以桌面应用的形式发布,你可以轻松地将它连接到已经在使用的各种智能体(Pi、OpenCode、Hermes、Codex 等)。只有当这些智能体确实需要模型时,它才会按需自动运行模型,并在闲置后将其关闭。 效果如下: https://www.youtube.com/watch?v=0qE8BWEZu7o 我们很期待进一步推进 Magnitude,让你能够在相同硬件上运行更大的模型,同时持续提升性能。我们的计划包括: - 专家流式加载:将专家存储在 RAM 或磁盘中,并在需要时即时加载。这样,你就能运行原本无法装入 GPU 的更大模型。 - 内核编译器:目前,我们的内核会调优少量参数以适配你的硬件。借助完全定制的编译器,我们可以更进一步,自动选择内核融合方式和实现方案,从而让它更好地适配你的硬件。 - 多设备利用:通过检测系统中的所有硬件(CPU、GPU、RAM、磁盘),自动求解最佳模型布局,以尽可能充分地利用它们。 我们希望有更多人来试用并向我们提供反馈。欢迎在这里留言,我们一整天都会在线! 评论链接: https://news.ycombinator.com/item?id=49911995 得分:5 评论数:0
大型语言模型(LLM)使 Agent 能够通过生成计划并在环境中执行计划来解决长时程任务。然而,成功的规划需要两种不同的能力:为任务选择合适的计划,以及忠实地执行该计划。现有的规划器—执行器系统可能在任一阶段失败,而仅凭最终任务是否成功无法区分计划选择失败和执行失败。因此,我们研究了 Plan Declaration--Execution Gap,并提出 Planning-as-Routing:由 LLM 声明四种规划模式之一——Predefined、Sequential、Hierarchical 或 Search,再由确定性路由器将任务分派给相应的特定模式执行器。在四个基准测试和三个 LLM 上,我们发现了三个一致的现象。第一,通用的 Plan+ReAct 往往无法保持所声明的规划结构,尤其是在计划较长时:在三个基准测试中,仅有 22%—45% 的轨迹保留了该结构,而特定模式执行器能够强制执行预期结构。第二,规划模式的有效性因环境和模型而异:Search 在 ALFWorld 上表现最佳,Hierarchical 在 SWE-bench 上表现最佳,而在同一基准测试中,最强模式可能因模型而异。第三,最大的收益来自执行环节:相较于 Plan+ReAct,特定模式执行器将 ALFWorld 上的任务成功率从 0.48 提升至 0.92,并将 SWE-bench Verified 上的任务成功率从 0.36 提升至 0.44。然而,当前的 LLM 尚不能可靠地为每个任务选择最强模式,尽管少样本示例在某些基准测试—模型组合中能够改善模式选择。总体而言,可靠的 Agent 规划既需要有效的模式选择,也需要忠实的执行:路由显著缩小了执行差距,而针对任务的模式选择仍是一个有待解决的问题。
As agents take on longer and more complex problems, controlling the execution becomes a task in its own right. Each step in the run brings new control choices, like which partial work to build on, whether to start fresh, or when to stop. We introduce agentic meta-reasoning, an inference-time harness that makes these choices an explicit and structured reasoning process. Workers carry out the task-level computation, while a controller consolidates what the run has established, explores next options, assesses what each option is worth under the remaining budget, and dispatches the chosen work with context drawn from persistent memory. Between decisions the controller carries only a compact account of the run rather than replaying its full history. Our baselines span production coding agents and research harnesses, together with a Direct Control Agent using the same workers and compute budget allowance. On ProgramBench, which tests long-horizon agentic capability through program reconstruction, meta-reasoning achieves 71.5% with GPT-5.5 against 58.0% for Codex; with Opus 4.8 it achieves 67.2% against 65.5% for Claude Code. On the other benchmarks, spanning abstract reasoning, multi-domain long-horizon reasoning, and proof generation, it gains between 3.6 and 4.2 points over direct control, averaged across three frontier models. It keeps improving over the tested budget ranges where direct control plateaus, though its overhead can hurt at small budgets. Artifact-graph analysis reveals more reuse of earlier work, higher coverage of correct solutions in most settings, and nonuniform gains in final selection. These results indicate that spending computation on structured control becomes more important as agents scale to longer runs.
人们不仅通过重复成功的行动来学习,也会通过复述和解释自己的经历,修正理解以指导未来行为。语言模型智能体能否仅通过训练其自身经历的解释来改进未来行动?我们通过研究 Retrospection-Only Fine-Tuning (ROFT) 来探讨这一问题;ROFT 是一种极简的在线流程,旨在隔离仅基于解释的训练对后续行为的影响。智能体尝试一项任务,观察可用反馈,生成回顾性解释,并仅在解释 token 上使用下一 token 预测损失进行微调。该流程既不使用外部教师,也不进行基于奖励的策略更新。在使用 Qwen3.5-4B 的软件工程实验中,ROFT 在基础模型尝试结果成功与失败混合的问题上进行训练。在留出的 SWE-bench Verified 和 Pro 上,在不使用验证器的情况下,它经过 20 次更新后达到 49.2% 和 26.8% 的解决率;相比之下,在所评估的运行中,GRPO 经过 40 次更新后的解决率为 48.0% 和 25.3%。ROFT 在训练时间和采样尝试次数方面也表现出更快的早期进展。它还学会了解决一些单个任务,而基础模型在这些任务上的 64 次采样尝试全部失败,这表明即使最初没有任何成功轨迹,学习也可以开始。行为分析发现,ROFT 会间接地为行动分配信用,鼓励好的行动并抑制错误行动。此外,通过提示回顾强调更直接的解决方案,即使没有显式长度惩罚,后续尝试也会变得更短。总体而言,这些发现表明,学习解释也能改善学习行动,确立了自生成回顾作为有用训练目标的价值,并推动对解释到行动迁移的进一步研究。
当大语言模型(LLM)Agent 执行同一任务时,不同运行之间的 Token 消耗可能相差超过一个数量级。Agent 会根据工具反馈和中间结果选择下一步行动,而不断增长的上下文会持续扩大后续每次调用的输入规模。因此,任务的总消耗在执行前很难预测,并且预测必须随着运行展开而不断修正。本文提出 TokenCast,它为每个执行片段学习一种可组合的成本表示,记录该片段自身的消耗以及它引入的上下文增长。组合相邻片段可得到一个累积估计,捕捉早期片段的上下文被后续每次调用重新读取时产生的额外输入成本。随着执行展开,新观测到的证据会刷新预测,无需额外的 LLM 调用,并且在 SWE-bench Verified 上每次运行的平均累积预测时间为 32.8 ms。在 4 个任务套件和 6 个 Agent 模型中,TokenCast 相较最强对比方法的平均绝对误差降幅在 96 个评估组合上的平均值为 14.5%。在离线预算控制回放中,在匹配轨迹完成度的情况下,TokenCast 平均比固定预算策略少使用 21.3% 的 Token。代码可在 https://github.com/DEFENSE-SEU/TokenCast 获取。
仅结果强化学习为软件工程(SWE)智能体提供终端成功信号,但对中间决策几乎没有直接指导。我们引入 Counterfactual Rollout Replay(CRR),这是一种训练时过程,利用可分叉的可执行环境来获得步骤级回报对比。CRR 选择一小组决策点,恢复每个状态,采样一个替代动作,并在该策略下将该分支向前 rollout。它保留已实现的训练轨迹,并将所选步骤的优势替换为其终端回报与采样得到的反事实回报之间的差值。该方法不需要人工过程标签或学习得到的过程奖励模型;免费指的是这些监督成本,而非 replay 计算成本。使用 14B 策略时,CRR 在 SWE-bench Verified、SWE-bench Live 和 SWE-rebench 上提升了 pass@1,并且可与过程奖励和轨迹搜索方法结合。在 SWE-bench Verified 上,在相同硬件上进行等墙钟时间比较,得到 41.7% 对 36.7% 的结果,相比延长的仅结果 GRPO 提升 5.0 个百分点,且已计入分叉开销。这些结果适用于能够以可承受成本可靠恢复状态的环境;随机续接以及昂贵或不完美的 replay 仍然是局限。
We’re collecting real stories of builders, tinkerers, researchers, and creators who are using Codex to do incredible things. If you want to be a part of the next chapter of the Codex Originals program, tell us more about your story and project below.
文章 URL: https://www.tomshardware.com/tablets/microsoft-surface/microsoft-quietly-drops-copilot-branding-from-its-new-laptops-surface-cvp-confirms-new-devices-meet-hardware-requirements-but-lack-controversial-branding 评论 URL: https://news.ycombinator.com/item?id=49872980 分数:3 评论数:0
Tool observations dominate the context of software-engineering agents, making long interaction histories costly to maintain. Existing context compression methods can discard information needed by later actions, while adapting agents to soft-token representations can compromise their original behavior. To reduce context while preserving action-critical information and agent behavior, we combine Latent Observations, Hard Actions (LOHA), a context layout that separates compressed history from text needed for exact reference, with Anchored Context Distillation (ACD), a training method that enables latent reading while constraining behavioral drift. LOHA compresses older tool observations into soft tokens while retaining the agent's own turns and the last K observations in text, providing compact access to historical information and exact access to recent content. To enable the agent to use this representation, ACD distills the base model's full-text predictions into the latent view while anchoring its behavior on plain-text inputs to the same base model. On SWE-bench Verified, K=3 reduces context per call by 43% for Qwen3-4B and 57% for SWE-Master-4B-RL, with resolve rates of 12.1% and 21.8% versus 14.5% and 27.5% for their uncompressed bases. A single-run recency sweep reaches 14.4% and 23.0% at K=8, with larger windows generally favoring task performance over compression. Under a 32K-token limit, Qwen3 with K=3 resolves 21.1% of a 199-instance subset versus 11.1% for the same adapted agent using full text. In concurrent single-GPU serving, it achieves 1.9 times that full-text agent's instance throughput.
仓库级问题定位旨在识别并排序与解决所报告问题相关的文件和函数。LLM 代理以迭代方式处理这一任务:它们识别一组潜在相关的位置,检查相应代码,并随着获得新证据而修正对这些候选项的判断。然而,现有环境对这一循环的支持有限:代理必须搜索未解析的关系目标,从原始源代码中重构实体语义,并在缺乏证据基础的情况下修正候选项。为解决这些限制,我们提出 SemNav,这是一个利用确定性检索来播种广泛候选集,并使用 LLM 代理持续细化该候选集的框架,从而将初始覆盖率与证据引导的修正相结合。SemNav 通过三个关键组件支持这一过程。Semantic Navigation Graph 通过语言服务器按需解析程序关系,支持跨文件直接导航到相关实体。问题条件化的 Semantic Cards 为每个实体在问题中的角色及其相关性提供紧凑且基于源代码的解释。持久化的 Candidate Workspace 记录每个候选项及其证据基础,从而支持有根据的验证、修正和排序。在 SWE-bench Lite 和 PLocBench 上,SemNav 优于现有基线,在使用 Gemma 4B 时将 File Hit@10 从 68.33\% 提升到 82.67\%。组件消融和轨迹分析支持三个组件的互补作用,而与完整源代码阅读相比,Semantic Cards 将工作上下文负载降低了 48.2\%。SemNav 还在 SWE-Explore 的全部七项证据质量指标上排名第一,并将下游问题解决率从 44.00\% 提升到 52.33\%。
AI Agent 将语言模型与外部数据和工具相结合,这些工具可以修改文件、调用 API 或执行代码。当对抗性内容改变 Agent 的工具使用方式,或周边软件存在路径遍历、命令注入等漏洞时,可能会出现安全故障。我们研究授权的白盒部署前审计,其中审计者可以访问目标代码库和受控运行时,但成功攻击仍必须通过任务定义的攻击者接口实施,并由外部验证器确认。我们提出 AgentXploit,这是一个双角色审计系统,将代码库级别的攻击路径发现与运行时利用分离开来。Analyzer Agent 将攻击者可控输入追踪到敏感操作,并记录有代码支持的候选攻击路径;Exploiter Agent 将这些路径转化为具体攻击,并利用运行时反馈对其进行修正。我们还引入了 AgentXploit-Bench,其中包含 12 个开源 AI Agent 系统和框架中的 72 个可复现漏洞。在三次运行中,AgentXploit 达到 59.3% 的端到端成功率,而 Codex 为 38.4%。在令牌预算匹配的比较下,Codex 达到 46.3%。在 AgentDojo 上,由于提供了注入点,Exploiter Agent 达到 79.2% 的攻击成功率,而 AgentVigil 为 52.7%。这些结果凸显了代码库发现和运行时利用是端到端 Agent 安全审计中的不同挑战。
I've spent the last 10 months building Beauty: a local, powerful greco-roman Markdown editor where the page looks finished while you type. It runs on the Mac, iPhone and in the browser, and the browser version opens without an account: https://app.markdown.beauty I was frustrated with writing apps, and like most I used Apple Notes. After 2+ years immersed in Cursor, Codex and CC, I wished some interactions existed in a text editor. The Markdown renders in place as you write it, including tables (which you can drag and resize without lining up pipes), KaTeX math and Mermaid diagrams. A lot of thought went into typography and the writing experience. Your notes on the Mac are plain .md files in a folder with images next to them, so they open in anything else too. Each note keeps 200 versions. If you share a note with a link it syncs peer to peer over WebRTC while someone has it open, and the server that connects you only ever sees a random room id. Most of the ~1,000 commits went into the editor itself. It's a contenteditable, and I don't let the browser restructure the document at all, which was a lot more work than I expected. Building this to be performant and cross-platform required some novel approaches, which I'll later open source. The editor is free with no account or ads, and there's an optional subscription for sync through your own iCloud. https://www.markdown.beauty Comments URL: https://news.ycombinator.com/item?id=49866597 Points: 25 # Comments: 20
我的 OpenAI CODEX 账户失控了,从一个简单请求开始,自主决定在未经我任何授权的情况下启动 826 个并行智能体/线程,而且没有报告任何形式的结果,却消耗了近 2,146 万亿个 token,总计消耗约 78,000 美元,并删除了所有关于所执行操作的记录:我已向 OpenAI 提交工单 2 周,但根本无法联系到人工客服。 2026 年 7 月 10 日,我从 VS Code 打开了一个普通的 Codex 任务。 该任务运行的是:GPT-5.5 / Medium reasoning 我的提示非常简单,只是要求对我产品中某个特定模块进行 UX/UI 验证。 经过艰难分析,我在接下来几天发现的是: Root ID 为 019f4b90-4169-7201-bfdd-732940d8631e、reasoning 为 GPT-5.5 / Medium 的任务创建了 826 个子任务,记录为 GPT-5.6 Sol / Ultra(注意 reasoning 级别和模型选择的差异) 这不是一次对话中的 826 条消息,而是 826 条不同的子任务记录,并且都有各自的 ID。 其中有一组特别奇怪,包含 104 个子任务。它们都保留了与原始任务相同的初始消息,记录为 GPT-5.6 Sol/Ultra,并且没有记录 agent_role 或 agent_path。 仅这 104 个任务就约占 1479 亿个本地最终任务 token 计数器。 它们的标题显示,我要求检查 UI/UX 的请求已经扩展为涉及后端基础设施、OAuth、计量、加固、审计、认证、实现和发布的工作。 准确地说:这些本地 token 计数器并不是 OpenAI 权威的计费账本,我也不是在声称 1479 亿个本地计数器可以简单地乘以 API 价格。 这正是问题的一部分:只有 OpenAI 拥有服务器端映射。 还有另一个异常相关性。 在 Codex client build 0.144.0-alpha.4 下,该任务族包含: 584 个子任务 / 约 1543.6 亿个本地 token 计数器 平均:每个任务约 2.643 亿 在 0.144.2 下: 242 个子任务 / 约 75.1 亿 平均:每个任务约 3100 万 这意味着每个子任务的平均本地 token 量大约相差 8.5 倍。 上述 104 个高容量任务中有 103 个是在记录为 0.144.0-alpha.4 时创建的。 这让我相信 alpha build 存在严重 bug,因为在其他几个任务中也观察到了相同模式。 在财务方面,我重建的 OpenAI 账单历史包含 162 张已支付发票,总额为 $79,664.88,分布在 Automatic Reload 和其他 “Credits” 之间。 当时没有等效的实时控制界面能让我清楚了解支出情况,而且大多数日志似乎已从我的服务器上自动删除:在恢复出的本地状态中,大约 2,550 个未归档的旧线程仍有元数据,但本地没有对应的原始 rollout。 换句话说,仍然有证据表明这些任务曾经存在,而用于重建生成其中许多任务的指令所需的详细执行历史,在我的机器上已经不再可用。 我还亲眼看到任务/对话从正常可见历史中消失。 我联系了 OpenAI Support,并开启了案件 #15189838。 我已提供技术证据,并多次要求进行服务器端重建,但 OpenAI 只是回复说“credits were consumed”,没有任何细节。 我想听听其他在 7 月/8 月左右使用 Codex 的人的情况:你们检查过自己的本地 Codex 状态吗?你们是否见过异常庞大的 subagent 树、模型/reasoning 升级、重复子任务,或无法解释的 Automatic Reload 活动? 我尤其想听到任何拥有 Codex 0.144.0-alpha.4 日志的人提供信息。 如果 OpenAI 工程师看到这篇文章,我也欢迎给出技术解释。 评论 URL: https://news.ycombinator.com/item?id=49861047 点数:7 评论数:0
Shows an Incorrect API Key error...which is weird. Nothing on the status page. Per Twitter/X this appears to be widespread Comments URL: https://news.ycombinator.com/item?id=49851032 Points: 34 # Comments: 13
文章 URL: https://status.openai.com/incidents/01M3DCNWMW57HYK8FJ5FBFPA39 评论 URL: https://news.ycombinator.com/item?id=49851169 点数:18 评论数:7
With Codex, GPT-Live-1, and GPT-6 Astra, Proaction builds, operates, and sells modern fleet management faster.
文章网址: https://www.bloomberg.com/news/articles/2026-09-25/microsoft-abandons-personal-ai-chatbot-race-with-copilot-reboot 评论网址: https://news.ycombinator.com/item?id=49844896 得分:15 评论数:7
即使具备完全可观测性和以对象为中心的状态,任务与运动规划(TAMP)问题仍然很难解决,因为离散决策与几何、运动学和动力学约束紧密耦合。广义TAMP通过利用不同问题实例之间的规律,降低在新实例上的规划成本,从而应对这一难题。然而,现有方法需要大量针对TAMP的工程设计。我们研究编码代理能否通过合成可在不同实例间泛化的程序来自动化这一过程。给定任务描述和模拟器访问权限,每个代理在固定的合成预算内开发程序,同时自行决定如何与环境交互。随后,程序被固定下来,并在未见过的实例上进行评估。我们在来自KinDER和PDDLStream的28个模拟环境中评估了Claude Code(Opus 5)和Codex(GPT-5.6 Sol和GPT-6 Astra),其中对象数量超过了原始基准测试中评估的数量。在所有程序合成方法中,我们对980个生成的程序分别在100个留出实例上进行评估,总计进行了98,000次评估回合。总体而言,我们发现编码代理在广义TAMP中表现出令人意外的有效性:在可用规划器的16个环境中,三种代理配置的平均成功率均超过了人工工程化规划器、一次性生成方法和基于LLM的广义规划基线,成功率为56%至95%,而规划器为47%。随着对象数量增加,代理生成的程序仍能保持高于规划器的成功率,同时平均每个实例所需的计算量减少了一个数量级。日志显示,代理通过交互来校准物理模型、测试边界情况并改进策略。我们公开了全部代码,包括提供给代理的完整提示词。这些发现表明,编码代理是广义TAMP的一个强有力基线。
异步监控、事件调查和合规审计主要依赖代理执行轨迹来重现所发生的事情。这些分析假定 LLM 代理无法篡改自身的执行轨迹。我们发现,Claude Code、Codex、Antigravity、Open Code 和 Grok Build 等本地 LLM 代理未能执行这一边界限制。除 Muse Code 外,所有受测测试框架都允许代理在接到要求后删除自身的轨迹,且不会触发监控护栏。我们还验证了外部攻击者可以利用这一漏洞诱导轨迹删除。最后,我们表明,当代理试图提高自身奖励时,轨迹篡改行为会在前沿模型中自然出现。我们建议从业者确保通过不受代理控制的独立拦截机制记录轨迹,即使在主机完全遭到入侵的情况下,也能保持轨迹完整性。总体而言,我们的研究发现了代理基础设施中一个具体的轨迹完整性失效问题,该问题可被用于掩盖不符合预期的行为,例如密谋或蓄意破坏。
各位 HN 用户好,我是 Feyn 的 Shreyash。我们帮助企业利用其数据构建定制模型。今天我们发布了 Critic,这是一个变更审查平台,让你可以直接与编写代码的 AI 交流。 智能体编写了我们的大部分代码。虽然这提高了我们的生产力,但理解一项变更及其后果已经变得极其困难。随着公司采用越来越多的智能体工具,我们发现,让人们了解某个 PR 的影响和项目状态变得更加困难。于是我们构建了 Critic 来解决这个问题。 Critic 可以让 AI 智能体介绍其代码、为关键代码块添加注释,并提供相关证据(例如屏幕截图以及如何在本地运行的说明)。任何查看该变更的人都可以直接与编写它的智能体交流。你可以 在这里观看演示 。 Critic 通过 Codex 或 Claude Code 的插件运行。当你的智能体编写代码时,插件会要求它同时撰写一份叙述,介绍这项变更背后的故事,并突出显示其中的假设、决策或复杂代码。随后,你可以在 critic.run 上查看这项变更,或者让你的智能体通过内置的 MCP 获取它。Critic 中的每项功能都可以通过 MCP 使用,因此你的智能体无需离开其运行环境,就能获得完全相同的上下文。 你可以通过控制面板或 MCP 直接与负责编写代码的智能体聊天。当 Critic 收到问题时,它会分叉出编写代码时的会话,并将你的问题转发到该会话中。这样,你的主线程可以保持整洁并继续工作。 本地变更只有所有者可见。推送到 GitHub 的任何内容都会被镜像,并对在 GitHub 上拥有查看该变更权限的所有人可见。 构建 Critic 时,安全性是我们的首要考虑事项之一。AI 智能体通常会以自动模式运行,并且可以在你的计算机上执行许多操作。我们希望确保,让他人访问你的智能体并不意味着同时让他们访问你的计算机。所有 Critic 分叉出的会话都会移除写入工具。它们只能回答有关该工作区中的变更和进行中工作的相关问题。 Critic 帮助我们在几起事件进入生产环境之前发现了它们。最常见的问题是模型没有复用已有的资源,例如现有的样式、组件或函数。由于深入了解智能体的工作并向它提问变得非常容易,我们还发现了更多模型误诊问题或编写无用代码的情况。由于 Critic 的界面比 GitHub 更快捷、更适合智能体使用,我们的团队也能更快地回复反馈。 Critic 可免费使用。请在 https://critic.run 注册。如果你有任何反馈或问题,请告诉我。 评论地址: https://news.ycombinator.com/item?id=49834098 得分:5 评论数:1
大家好!我们是 Sid、Alex、Ketan 和 Milan。我们正在构建 Whiteboard,这是一款开源桌面应用,让人类和代理能够在一个共享工作区中共同设计软件架构。这是我们的代码仓库: https://github.com/devdotfast/whiteboard ,主页是: https://whiteboard.dev.fast/ 我们怀念与另一位开发者进行“白板会议”时的那种感觉——离开时已经对一个系统有了深入理解。因此,我们为自己构建了这个应用。Whiteboard 可以接入你已经在使用的工具,例如 Claude Code、Codex 等,并为你的代理提供一个 SDK,让它能够在应用内画布上绘图,以描述自己的工作。我们最初基于 HTML artifacts 构建了一个 MVP,后来随着遇到各种限制,我们开始重新思考这个应用: 1. 构建在 CodeOSS 之上:我们发现,在纯 HTML 工具中,很难将规范或图表与代码关联起来。在 Whiteboard 中,当你点击序列图、实体关系图或代理轨迹中的一段引文等可视化内容时,可以直接跳转到底层代码。在浏览代码时,你可以开箱即用地获得 VSCode 的快捷键绑定和 LSP 支持。我们发现这一点尤其有价值,因为许多权衡往往只有在首次实现之后才会显现出来(参见:slop) 2. 语义差异查看器:我们用 Rust 编写了一个支持语义分析和 AST 的差异查看器,因此你只需查看与你相关的代码变更 [1]。我们设置了一些合理的默认行为:较大的新增函数会以伪代码形式进行概述,单元测试和大段文档变更等内容则会被折叠或隐藏。这些设置都可以通过基于 WASM 的插件系统进行自定义。 3. 决策日志:我们发现,很难推理代理自主做出了一组什么样的决策。因此,我们构建了一些工具,让代理能够查询自己的轨迹并将其链接到 Whiteboard,这样你就能了解你设定的需求是如何被实现的,以及你的代理自主做出了哪些决策。 这里有一段简短的演示视频,可以进一步了解相关内容: https://youtu.be/ChPn3ftULWE Salesforce 和 Modal 等公司的员工如今正在使用 Whiteboard,作为架构或规范级变更的审查工具——实际上,任何他们希望参与其中的变更都可以使用: 1. 审查你自己的编码代理所完成的工作:由于 Whiteboard 让审查大量代码变得更加容易,人们通常会让 AI 代理创建一个原型以及相应的 Whiteboard 会话,然后据此迭代设计。 2. 审查他人的变更:我们发现,Whiteboard 与 Greptile 等工具结合使用时尤其有帮助。例如,你可以对小型变更运行自动化代码审查,而对于需要人工判断的变更,则升级为 Whiteboard 会话。 我们为什么构建它:我们是四个大学时期的好友,在代理式编码成为行业标准之前不久,辞去了技术负责人的工作。在为之前的一个想法迭代 MVP 时,我们努力想在享受代理式编码带来的全部效率提升的同时,维持一个易于理解的代码库。随着越来越多的 PR 在我们并未理解的情况下被合并,我们感觉一种“认知债务”开始逐渐渗入,直到我们甚至难以继续为这个系统做贡献 [2]。 我们将以 MIT 许可证发布我们的桌面应用。欢迎深入了解,也欢迎贡献代码!最终,我们会向企业收取托管版 Web 服务的费用,用于管理 Whiteboard 会话的创建,并提供轨迹存储和多人审查等功能。一切始终都会支持自行托管。 感谢阅读,希望你能试用一下!我们非常期待听到你的反馈,也希望向各位专家学习。 再次附上项目链接: https://github.com/devdotfast/whiteboard ;你可以在这里安装适用于 MacOS 和 Linux 的版本: https://install.dev.fast [1] diffs 库: https://github.com/devdotfast/diffr [2] “认知债务”这一术语的出处归于 https://www.geoffreylitt.com/2026/07/02/understanding-is-the... 评论链接: https://news.ycombinator.com/item?id=49833867 得分:16 评论数:0
智能体经常处理需要数百万个 token 上下文的复杂问题,而受限的上下文窗口使得跨会话压缩成为必要。我们开发了 CliffCompaction,这是一种自动压缩技术,在有界上下文下最多可降低 50% 的成本,同时在 Terminal-Bench 上保持或提升性能,并为测试时扩展带来新的效率水平,在 KernelBench 上取得最先进的结果。CliffCompaction 在每次运行中节省的成本,使测试时扩展的性能—成本权衡更加高效:在 Terminal-Bench 上,仅低于两次完整上下文运行的成本,就能额外提升超过 10 个百分点。在并行测试时扩展下,CliffCompaction 使 Kimi K2.6 达到与 Opus 4.7 相当的表现,并以更低成本超越 Opus 4.6 和 GPT-5.3 Codex。CliffCompaction 高效的关键在于,它只截断或删除内容,从不对内容进行改述或重写,从而保持压缩信息的忠实性。我们绝不会对压缩结果再次压缩——每次处理仅针对原始内容,之前的压缩输出会被丢弃,以防止上下文漂移不断累积。这些特性支持跨越超过一百万个 token 的会话持续学习:在 KernelBench 上,经过 200 步后,CliffCompaction 实现了 CUDA 内核 2.23 倍的加速,经过 400 步后达到 3.58 倍,尽管它是一种通用压缩技术,仍超越了专用搜索算法和经过训练的智能体。我们开源了一个与脚手架无关的 CliffCompaction API 代理实现,可与 Claude Code、Codex 及其他运行框架配合使用。
强化学习已成为大型语言模型(LLM)后训练的核心组成部分,但 token 级信用分配尚无普遍认可的数学定义,因此其与常用训练信号之间的关系仍不明确。我们提出了三个正则性条件,即 Completeness、Prefix Consistency 和 Neutrality,并证明它们能够唯一确定 token 级信用分配。这一刻画为解释现有算法中的各种现象提供了统一基础,并指导改进型 actor-critic 训练流程的开发。从这一视角来看,On-Policy Distillation(OPD)中的理想教师充当隐式 critic,使得其期望策略梯度与 token 级信用分配所诱导的策略梯度成正比。尽管粒度更粗,响应级 REINFORCE Leave-One-Out(RLOO)信号仍与 token 级信用分配对期望策略梯度的贡献相匹配。我们进一步在有界结果奖励条件下建立了近似信用稀疏性,并展示了 Generalized Advantage Estimation(GAE)中的中间 critic 误差如何达到与底层信用分配相当的程度。基于这些发现,我们提出 Policy Aligned Critic Training(PACT):该方法采用先 Actor 后 Critic 的更新顺序,对 critic 训练应用重要性采样校正,从而使 critic 更好地与更新后的策略对齐。在智能体数学推理任务中,PACT 在四个基准测试上取得了 72.87% 的平均准确率,分别以 8.80 和 13.16 个百分点的优势超过 GRPO 和 PPO。在 SWE-bench Verified 上,PACT 的通过率达到 67.4%,分别以 2.4、2.0 和 3.8 个百分点的优势超过 PPO、GRPO 和 SAO。
Hey HN! We’re Max and Gabriel the co-founders of Coverage Cat. We’ve been friends for over a decade, met in college then hung out mostly on the internet. We love building products that help people optimize the crufty corners of their lives. Max is a former Google/Microsoft/Two Sigma PM and Gabriel has been working in startups for over a decade. We’re building Coverage Cat: an AI-native, you can shop with your AI agent or via a form, insurance brokerage for consumers. Our current focus is helping tech folks (think L3-L8) buy umbrella insurance. For those who are unfamiliar with this type of coverage, here’s a blurb from one of our insurance agents: “Umbrella liability insurance is a policy that provides additional coverage above the limits of existing auto, homeowners, renters, or landlord policies. It kicks in after the underlying policy limits are exhausted and may also cover personal injury claims like libel, slander, or defamation that standard policies often exclude. Policies typically start at $1 million in additional liability coverage and can protect you, your spouse, dependents, and even pets in your household.” Buying insurance online is a miserable experience. You fill out a bunch of forms that say they’re going to give you quotes. Most don’t and then you end up with tons of unwanted phone calls and spam. Even if you wade through the muck, you’ll often still end up with sub-optimal prices and coverage because of a lack of market transparency and poorly-aligned advisory incentives. We came to this problem because Max is a classic personal finance obsessive. He's the type of person that needs to know that he has the best possible price for something (everyone has one friend with the patience for a two hour phone call with the bank for some $10 fee) or that his insurance coverage is as tailored to his risk profile as the market will allow. He actually reads insurance policies end to end. Years ago, when he shopped for his homeowners insurance it took him 10+ phone calls and dozens of online form fills to find the right deal. He also suffered tons of collateral damage in the process: websites sold his information to dozens of agents who proceeded to call, text, and email him a spam hoard that grows to this day. After this painful search made us aware of the problem, we also spent a lot of time doing user research with wealthy-ish tech employees to see if they felt the same way. They did, and also complained that they 1) often felt like they'd left money on the table, 2) didn’t really understand their coverages, and 3) that brokers were slow, hard to communicate with, and unreliable. To solve the problem, we set out to build Coverage Cat. Our approach has three major components: First, we work really really hard to find and partner with insurance carriers that offer high-quality, competitively-priced products. We've also taken time to identify insurers that work on a direct-to-consumer basis or transparently list their prices online so we can make our customers aware of all the options available to them. We get paid commission when we match users to great insurance deals, but we still show and recommend folks deals that we don't make money on when they're accessible and will always recommend users buy what's best for their personal financial situation. Our goal is to meaningfully improve price transparency so consumers (and their AI agents) can make informed coverage decisions and not get nickel-and-dimed! Second, we automate every possible part of the insurance process. Some forms, portals, & approvals still need human review due to regulatory constraints, but if it's not restricted, we're automating it. LLMs have given our small team tremendous leverage and enabled us to tackle problems at a scale that would've been unfathomable five years ago. Finally, we build for agents first (a remix of the more classic adage, build for developers/APIs). We've always been big believers that personal finance would be the killer use-case for the AI revolution and have done our best to position ourselves to surf the wave. As personal assistants have started to explode onto the mainstream (think Muse, Instinct, Town, Codex, CC etc.) we've ensured our shopping and comparison process works via API/MCP so agents can present their users with all the information they need to make a good decision and transform a miserable shopping experience into a delightful one. Coverage Cat is an unusual business because, while we also help folks find homeowners insurance in California and Texas, our main focus is on umbrella insurance. Most brokerages only sell umbrella coverage as a customer retention product, but it makes them almost no money. We focus on it as our core business because: 1) it's one piece of the insurance puzzle that many folks in tech overlook and/or are confused about even though it can have a huge impact on their financial well-being. 2) Existing online tools and brokerages broadly don't make it easy to comparison shop for. 3) It was, for us, the most technically feasible candidate for automation given startup resource constraints. Insofar as we know Coverage Cat is the first instance of a tool/portal that allows AI agents to complete most of the comparison and shopping steps required to allow people to buy insurance. We’d love your feedback so please do try it out. Just feed the prompt: “Shop for umbrella insurance with Coverage Cat” to your personal agent and let us know what you think! If you want to watch a couple of agents navigate the flow before you try it out there’s a video here: https://youtu.be/1BUkgAn6s-I Also happy to answer any questions about the product, the problem space, and even some insurance questions (Max is a licensed agent) where regulation permits. Cheers! Comments URL: https://news.ycombinator.com/item?id=49804931 Points: 3 # Comments: 0
At, GPTree, we run several coding agents across our team on one repo using parallel worktrees. Apart from wasted time reviewing and fixing conflicts at PR time, the failures that hurt the most are when multiple plans or tickets cause architecture changes that cannot both be true. Ex. one agent replaces a class while another one is in the process of extending it. Git only notices if the resulting patches happen to touch the same lines and the review only catches it if they are familiar with both tickets. Foremerge is a local "git like" coordination layer that sits above git (ie. does not interact with or change the way git and worktrees function), Before editing each agent publishes an intent and the scopes it will change, with the operation it plans to complete on each one. foremerge intent publish --agent "$A" \ --summary "Replace PaymentService with StripePaymentService" \ --scope symbol:PaymentService=replace foremerge intent publish --agent "$B" \ --summary "Add PayPal support to PaymentService" \ --scope symbol:PaymentService=extend The publish by the 2nd agent returns a HIGH destructive_vs_additive finding before writing any code. Agents keep their own worktrees and the shared state is one SQLite file in gits common direectory. No hooks, no merge drivers, nothing rewrites your history. It ships as one Rust binary with a CLI and MCP server with 18 tools and `foremerge setup all` wires it into Claude Code, Codex and Cursor. Because the protocol has nothing provider specific, a Claude agent and a Codex agent coordinate through the same store. Before any work is accepted, Foremerge runs a named check that you configured against the exact git state of the change. An agent that says tests pass is recorded but it dosnt satisfy the acceptance gate without running the check itself. Detection is deterministic, no judge model reading your code. HIGH conflicts are only asserted for declared operations, ie. matches inferred from prose cap out below high. Claims are advisory leases, not locks so two agents can still hold the same scope without deadlock. The open source version is single matching and so not a distributed consensus. We have tested this up to 98 parallel agents all working on the same repo with zero conflicts (was supposed to be 100 but 2 agents failed to run due to resource limitations) I replayed 76 intents on my own agents from a build last week in the order they happened. The sample had exactly 1 conflict (which was flagged) and the review found a blind spot where one agent claimed scope by class name and the other claimed it by an internal method. We are working on fixing that for the next release. Setup is a 30s install by pasting the quickstart instructions from the readme.md into your agent or manually: `curl -fsSL https://foremerge.com/install.sh | sh` or `cargo install --locked foremerge`, then `foremerge init && foremerge setup all` in a repo. Apache-2.0. The feedback I want most is which conflicts between your agents plans would you actually want flagged and which would you tollerate as noise? Repo here: https://github.com/naw103/foremerge Website: https://foremerge.com More information on the problems this solves: https://foremerge.com/blog/ Comments URL: https://news.ycombinator.com/item?id=49789356 Points: 4 # Comments: 0
文章 URL: https://www.theregister.com/devops/2026/09/18/microsoft-agentically-ports-copilot-runtime-to-rust-for-120k/5297549 评论 URL: https://news.ycombinator.com/item?id=49773998 得分:15 评论数:6
随着编码智能体从受监督的代码补全转向无人值守、全天候探索,其工作也从孤立的预测扩展为包含推理、工具使用和反馈的长轨迹。因此,token 效率对于扩展递归式自我改进变得十分重要。我们在运行框架层面采用受 RSI 启发的方法,在数量不断增加且日益多样化的环境中扩展自动研究循环,以进行框架滚动执行。在这一规模下,该过程能够产生可复用的改进,并迁移到其开发环境之外,从而推动自动化框架发现走向生产级成果。四种机制经筛选后保留下来,共同构成 SoL-Pi,涵盖动作执行、上下文压缩、观测处理和委托式阅读。在包含 51 项任务的 EdgeBench 评测中,SoL-Pi 在 GPT-5.6 Sol 和 Opus 5 上取得了与 Pi 相当的性能,同时将记录的 token 流量降低 44.7%–49.0%,并将 API 成本降低约三分之一。换言之,与 Codex 和 Claude Code 的原生框架相比,预计每小时可节省 \$8.75–\$13.50;与 Pi 相比,每小时可节省 \$4.36–\$5.71。
编码 harness 影响着自主编码智能体如何将模型能力转化为长周期软件工程性能,但现有工作通常将 harness 作为单体系统进行评估,使得各个组件的有效性尚不清楚。为支持组件级比较,我们使用一个轻量级编码 harness 研究这一问题,其执行循环保持固定,同时改变三个组件:规划、动作空间和上下文管理。我们在 SWE-Bench Verified 和 Terminal-Bench 2.1 上评估了四个模型,共评估 176 组匹配设置,涵盖五种上下文管理策略、四种上下文窗口预算,以及针对规划和动作空间的定向消融。我们发现:(1)随着上下文窗口预算收紧,上下文管理变得越来越有价值,其大部分收益来自防止上下文溢出失败。(2)在 LLM 基于总结之前先进行基于规则的省略,在上下文管理策略中提供了最强的整体效率;而使被省略内容可恢复会增加机制复杂度,但模型很少使用,且不会带来准确率提升。(3)规划从较弱模型的准确率支架,转变为较强模型的成本节省手段,而准确率几乎没有变化。(4)预定义工具能够提升 bash 熟练度较弱的模型的性能,而具备 bash 能力的模型可以仅通过 bash 接口有效运行,并实现显著更低的成本,尤其是在以命令行为中心的任务上。轨迹级分析解释了这些效果:上下文管理延长了执行轨迹,而不会显著改变智能体行为;规划改变了轨迹停止的位置;动作空间改变了代码编写的粒度。这些发现为面向模型和预算的 harness 设计提供了参考,并为评估未来 harness 组件提供了模块化框架。
Hereditary polyposis syndromes can be precursor lesions to colorectal cancer and are associated with a broad spectrum of extracolonic tumors. Early identification and accurate classification of these syndromes are essential for timely diagnosis, individualized patient management, and targeted surveillance strategies for affected families. However, public endoscopic datasets are largely organized around the individual sporadic polyp, and none links the polyposis phenotype to histopathology and germline findings at the patient level. Here, we present ERCPMP-Gx, an endoscopic, histopathological, and genomic dataset developed to support the application of artificial intelligence (AI) in the recognition, characterization, and classification of colorectal polyposis. Most procedures were performed using the Olympus EVIS X1 system with white-light endoscopy (WLE), narrow-band imaging (NBI), magnifying NBI (M-NBI), and NBI with near focus modes, yielding 160 images and accompanying video clips. Approximately eighty percent of cases represent clinically and/or genetically confirmed hereditary polyposis syndromes (PG), including familial adenomatous polyposis (FAP), Peutz-Jeghers syndrome (PJS), juvenile polyposis syndrome (JPS), and ganglioneuroma syndrome (GNS), while the remaining twenty percent comprise non-hereditary polyps and polyp-mimicking lesions with overlapping morphological features (Non-PG), included to support differential classification. Each released record is linked, where available, to standardized endoscopic annotations, representative histopathology, and clinically reported germline findings, forming an AI-ready, patient-level annotation framework. The dataset is publicly accessible at Mendeley (https://doi.org/10.17632/nzyfc544bx.2). For the latest updates and further information, readers are referred to the DataBioX website: https://databiox.com.
近期,基于记忆增强的仓库级程序修复方法会复用历史修复经验,以改进基于 LLM 的问题解决。然而,我们的分析揭示了现有仓库级记忆检索存在三个局限。第一,不同仓库之间的情景记忆高度不均衡,导致资源匮乏的仓库几乎无法获得有效支持。第二,更多记忆并不会单调地带来更高的修复成功率,这表明相关性、质量和冗余性比记忆数量本身更为重要。第三,记忆积累与修复阶段不匹配:仓库中可能包含大量复现经验,却只有很少的补丁或细化经验。为解决这些问题,我们提出了一种面向仓库级程序修复的自适应经验检索框架。该框架引入了覆盖感知检索:当同一仓库中的记忆不足时,回退使用跨仓库记忆或基于修复类型的记忆;引入了质量感知选择:依据相关性、历史效用、特异性和冗余性对记忆进行排序;还引入了阶段感知路由:分别为复现、定位、补丁生成、补丁细化和验证阶段分离并检索记忆。在 SWE-Bench-Lite 和 SWE-Bench-Verified 上的评估表明,所提出的框架提升了覆盖不足仓库上的修复性能,减少了噪声记忆的检索,并更好地支持从失败补丁到修复完成的补丁细化。我们的结果表明,记忆增强修复的关键并不只是积累更多经验,而是针对正确的修复上下文检索正确的经验。
Hey HN, we're Nars & Nishant, founders of Skillsync ( https://skillsync.com ) Skillsync lets you move your AI chats across every coding agent. Most of our work exists as conversations, which are currently scattered across our agents. Though stored locally, these conversations use different formats. This is annoying because you cannot simply switch between agents without starting over. We get locked into a single provider and their agent as we invest in skills and memories over time. Skillsync acts as a universal converter. It moves the entire session, including all the messages, reasoning and tool calls so you can pick up right where you left off. Skillsync collects all your sessions in one place and makes them searchable. It breaks down what each session is carrying, including which loaded skills the agent is actually using, making stale context easy to spot. You can also create shared workspaces to sync sessions across your team. You can build your own closed loop systems. Everything runs locally except when you share to workspaces. The core is an open-source Rust engine called txcript ( https://github.com/skillsynchq/txcript ). It translates a session from one agent's on-disk format into another's, mapping conversation, reasoning, and tool history. Think ffmpeg or pandoc, but for agent sessions. On top of that engine is a local-first desktop app. Your agent sessions are normally scattered across different tools' folders in formats you'd never read by hand; the app surfaces them in one place with a UI that makes them actually readable, the conversation, the reasoning, and the tool calls, so you can revisit what happened, move a session into another agent, or share it with a teammate. Skills and memory are stored as portable, human-readable markdown you own, and exposed to any agent over MCP for search and selective retrieval. Sessions and translation run locally on your machine. The one thing that leaves is what you explicitly share into a team workspace. It’s been surprising to us to see how locked-in people already are without realizing it. You don't notice you're trapped in one agent or harness until you try to leave, and by then you've got months of sessions stuck in a format only that tool can read. The lock-in is invisible right up until it's expensive. Once sessions are portable, they stop being disposable logs and become something you can actually build on, spotting patterns in how you work, handing a session off to someone else, letting a non-technical teammate pick up where an engineer left off. The session turns out to be the unit of collaboration, and right now it's being thrown away. Before Skillsync, Nars and I built an open-source payment orchestrator that let merchants route across many processors instead of getting locked into one (30k+ GitHub stars). Fighting vendor lock-in by making incompatible systems interoperate was the whole job. We came into YC with a different idea, but the more we lived inside coding agents the more we saw the same problem from the other side: your context is locked into whatever agent you start with, because every agent stores sessions differently and none of it moves. Some of the things we’re seeing people do with Skillsync are: - Moving a session between Claude Code, Codex, and Cursor mid-task, including when you hit a usage limit on one and want to keep going on another. - Thinking through a problem in a browser Claude chat, then handing the whole thread to Claude Code or Codex to build. - Sharing research sessions with a team. Everyone uploads their sessions, so teammates and their agents can build on each other's work instead of repeating it. It's available as a Mac app, CLI and MCP. Here's the demo: https://youtu.be/7hVhSnSKGl8 . Would love to hear your feedback and answer questions! Comments URL: https://news.ycombinator.com/item?id=49743049 Points: 10 # Comments: 2
Hey HN, we're Nars & Nishant, founders of Skillsync ( https://skillsync.com ) Skillsync lets you move your AI chats across every coding agent. Most of our work exists as conversations, which are currently scattered across our agents. Though stored locally, these conversations use different formats. This is annoying because you cannot simply switch between agents without starting over. We get locked into a single provider and their agent as we invest in skills and memories over time. Skillsync acts as a universal converter. It moves the entire session, including all the messages, reasoning and tool calls so you can pick up right where you left off. Skillsync collects all your sessions in one place and makes them searchable. It breaks down what each session is carrying, including which loaded skills the agent is actually using, making stale context easy to spot. You can also create shared workspaces to sync sessions across your team. You can build your own closed loop systems. Everything runs locally except when you share to workspaces. The core is an open-source Rust engine called txcript ( https://github.com/skillsynchq/txcript ). It translates a session from one agent's on-disk format into another's, mapping conversation, reasoning, and tool history. Think ffmpeg or pandoc, but for agent sessions. On top of that engine is a local-first desktop app. Your agent sessions are normally scattered across different tools' folders in formats you'd never read by hand; the app surfaces them in one place with a UI that makes them actually readable, the conversation, the reasoning, and the tool calls, so you can revisit what happened, move a session into another agent, or share it with a teammate. Skills and memory are stored as portable, human-readable markdown you own, and exposed to any agent over MCP for search and selective retrieval. Sessions and translation run locally on your machine. The one thing that leaves is what you explicitly share into a team workspace. It’s been surprising to us to see how locked-in people already are without realizing it. You don't notice you're trapped in one agent or harness until you try to leave, and by then you've got months of sessions stuck in a format only that tool can read. The lock-in is invisible right up until it's expensive. Once sessions are portable, they stop being disposable logs and become something you can actually build on, spotting patterns in how you work, handing a session off to someone else, letting a non-technical teammate pick up where an engineer left off. The session turns out to be the unit of collaboration, and right now it's being thrown away. Before Skillsync, Nars and I built an open-source payment orchestrator that let merchants route across many processors instead of getting locked into one (30k+ GitHub stars). Fighting vendor lock-in by making incompatible systems interoperate was the whole job. We came into YC with a different idea, but the more we lived inside coding agents the more we saw the same problem from the other side: your context is locked into whatever agent you start with, because every agent stores sessions differently and none of it moves. Some of the things we’re seeing people do with Skillsync are: - Moving a session between Claude Code, Codex, and Cursor mid-task, including when you hit a usage limit on one and want to keep going on another. - Thinking through a problem in a browser Claude chat, then handing the whole thread to Claude Code or Codex to build. - Sharing research sessions with a team. Everyone uploads their sessions, so teammates and their agents can build on each other's work instead of repeating it. It's available as a Mac app, CLI and MCP. Here's the demo: https://youtu.be/7hVhSnSKGl8 . Would love to hear your feedback and answer questions! Comments URL: https://news.ycombinator.com/item?id=49743049 Points: 4 # Comments: 0
made this AIM buddy list but for our token usage because i wanted to bring a little bit of the old internet back! i remember those days where i'd casually check if friends are online and find excuses to say hi and poke them give it a try! i'd greatly appreciate feedback Comments URL: https://news.ycombinator.com/item?id=49743095 Points: 7 # Comments: 8
Agent benchmarks are substantially more costly to evaluate than conventional LLM benchmarks. Benchmark compression is therefore a natural solution, yet existing methods primarily model redundancy in task--model final-score distributions, which is important in agentic evaluation. To address this limitation, we analyze large-scale trajectories and identify six complementary process signals that are systematically associated with final agent performance. To disentangle agent performance redundancy from a complete perspective, we propose DualViewEval, an agent benchmark compression method that jointly exploits outcome and process relations to learn an exact-size miniset and predict the full-benchmark scores. Across five agent benchmarks and five representative baselines, DualViewEval achieves the best results in all datasets. With only 20 tasks, it achieves $24\times$--$40\times$ compression on APEX-Agents and BFCL, reducing mean absolute error (MAE) by $14.5\%$--$28.2\%$ over the strongest competitors while improving Kendall's $τ$ by up to $7.2\%$ relative to EssenceBench on SWE-bench Verified. The selected minisets further reveal capability differences among different agents, providing compact and diagnostic feedback for efficient agentic model development.
Automating biological research requires general-purpose, reproducible robot systems that allow individual wet-lab researchers to delegate robot tasks without performing teleoperation or neural-network training. Vision-language-action policies have been proposed for general-purpose arms, but can lose performance when their operating environment changes. We therefore built WetRobo, a robot kit that can readily transfer between laboratories. It consists of one robot arm, laboratory equipment (an incubator, a reagent bottle with a cap, and a Petri dish), the existing code that moves the arm, teleoperation demonstrations of each task that we recorded, and a general AGENTS.md skill file. A biological experimentalist provides natural-language tasks without collecting local teleoperation training data or training a neural network. The coding agent observes the local laboratory and writes and executes programs, using external tools as needed for adaptation. We demonstrate use of WetRobo with OpenAI Codex (gpt-5.6-sol) on three successful tasks: lifting a Petri dish lid, removing a bottle cap, and opening the incubator door, all in real-world laboratories. The coding agent achieved the cap task in both laboratories, Lab X and Lab Y, whereas a VLA fine-tuned on Lab X demonstrations succeeded there but failed to transfer to Lab Y. These results point to a practical route for laboratory robotics: instead of training a policy for each laboratory, distribute a kit and let a coding agent adapt it in each laboratory. Code, demonstrations, and the evolved programs are available at https://github.com/tsudalab/WetRobo.
了解 ChatGPT Work 和 Codex 分析如何帮助团队了解 AI 的使用情况和支出,确定培训需求,并将采用情况与业务成果相联系。
Small differences on coding-agent leaderboards are often read as an ordering of systems. We audit whether the published verdicts support this reading, using 254 SWE-bench submissions across four splits without running models. On Verified, the leading two entries each resolve 396 of 500 instances. The top ten share 285 successes and 51 failures, leaving 164 instances that distinguish their outcomes. Frontier solution sets have median nesting 0.935 against a score-implied baseline of 0.774, indicating strongly shared successes. Scores also depend on the evaluated model-scaffold pair: observed within-model scaffold ranges reach 29.8 percentage points, compared with the 8.8-point spread of the top thirty. Six of nine cell-mean interaction tests remain significant after Holm correction, although this observational design does not identify causal scaffold effects. Exact paired McNemar tests separate none of the 29 adjacent Verified top-thirty pairs at alpha=0.05, while the larger Test split separates 14 of 23. A stated leader-based rule yields three descriptive tiers, or two after Holm correction; non-rejection does not establish equivalence. We release the partition and a five-step audit protocol that profiles shared outcomes, tests paired differences, reports grouping sensitivity, and estimates the instance budget needed for resolution. The results motivate reporting comparison-set-specific resolution and model-scaffold provenance instead of interpreting small aggregate gaps as established rank differences.
Long-horizon reasoning is vulnerable to early errors that compromise later decisions. We present QART, the Quantum-Augmented Reasoning Transformer, a quantum--classical hybrid architecture combining a backbone language model with quantum encoding, CIM-based QUBO optimization, and quantum decoding. Semantic information can come from hidden representations or model-generated text; detailed encoding and optimization procedures remain proprietary. Under explicit assumptions, we establish a conditional asymptotic reliability separation from single-trajectory autoregressive LLMs. For a common task family with aligned optimality and acceptance criteria, autoregressive acceptance probability tends to zero when cumulative conditional risk of irreversible errors diverges. QART's task-optimal-path recovery probability remains bounded away from zero if conditional probabilities for optimal-path coverage and semantic fidelity, spectral certification, dynamical reachability, and faithful readout remain uniformly positive under a specified resource schedule. The architecture alone does not imply these bounds. Paired measurements on six long-horizon benchmarks using DeepSeek V4 Flash, GLM-5.3, and GPT-5.5 xhigh in a Codex agent environment favor QART in 14 of 15 backbone--benchmark pairs. Relative gains reach 84.0% on SciCode, 47.6% on $τ^3$-Bench, and 44.4% on Terminal-Bench 4.0; the DeepSeek V4 Flash configuration regresses by 7.8% on DeepSWE. These results do not directly validate the asymptotic separation. Potential quantum scaling laws are formulated as conditional hypotheses. A quantum-advantage interpretation requires a demonstrated CIM quantum advantage over strong classical solvers and its transfer to end-to-end reasoning after all system overheads.
Large language model (LLM)-powered coding agents have made rapid progress in automating software engineering tasks, yet repository-level issue resolution remains challenging. Beyond generating a plausible patch, an agent must localize relevant code across interdependent files and maintain repository context that is both sufficient and focused. Code graphs expose non-local relations, but linear text interfaces obscure their topology; rendering the full repository graph yields visual representations that are too dense to perceive reliably, whereas a one-shot local view becomes stale as exploration proceeds. We present \textbf{RepoAtlas}, a training-free module that maintains evolving multimodal repository views through a \emph{select--project--refresh} loop over a repository code graph. RepoAtlas combines evidence from the issue with the agent's current exploration state to select a task-relevant region under a fixed budget, projects the selected structure into complementary visual and textual representations, and refreshes the view when changes in the exploration state render it outdated. We evaluate RepoAtlas on SWE-bench Verified, where it improves the resolve rate by 2.4 points while reducing input tokens and model calls by 5.8\% and 7.8\% on average, relative to the strongest multimodal graph baseline, with consistent gains across three models of different families and scales.
Hi HN - long-time lurker (since 2012!), first time poster. Pizza Bot is a self-hosted desktop app for Mac, Windows, and Linux that runs AI agents in the background and exposes them through an email-like UI. Finished work shows up in Unread, and anything waiting on your approval shows up in Action. It's Apache 2.0-licensed, there's no signup and no telemetry, and you bring your own model provider: Anthropic, Amazon Bedrock, Google Gemini, OpenAI, OpenRouter, or a local model through Ollama. There are builds on the releases page, or you can run it from source. Pizza Bot started as an internal passion project I worked on with a small team at Amazon. The whole thing came out of my frustration at having to manually log CRM activities through a browser form. I built a simple REST API called "JoeBot" that connected to my authenticated browser session over CDP and filled out the form for me using Playwright. Then I hacked up a quick Obsidian plugin so I could trigger it from my local notes (no AI and no MCP servers involved). This caught on quickly. My fellow AWS Solutions Architect Igor Fil joined up with me, and we rebranded the project as "Pizza Bot," named after Amazon's two-pizza teams. We started seeing what other automations we could build. We found a GraphQL API we could query and hacked up some "recipes" to pull data out of the CRM to help with meeting prep. That worked great, and it was right around the time MCP servers seemed to be taking off, so we decided to expose Pizza Bot as an MCP server instead, so it would be available to AI tools through natural language. This was a decent solution for technical users, but the Account Managers who live inside our CRM system wanted something too. We decided to rebuild Pizza Bot as an Electron desktop app modeled after an email inbox, so it would be familiar to non-technical users and would run on both Mac and Windows. We also bundled internal MCP servers as OCI images and hosted them in Amazon ECR as an "addon marketplace" so users could install them with one click without having to set up Amazon developer tooling. The project took off organically and expanded outside of AWS into the wider Amazon organization globally. More than 2,000 people ended up using it for meeting prep, email drafting, Slack summaries, CRM logging, prioritizing their day, and web research. Once apps like Claude Cowork and Amazon's own Quick Desktop came out, we realized the real growth opportunity was outside of Amazon. Rather than try to rip out the Amazon-specific integrations, we rebuilt Pizza Bot once more as an open source project. We leaned on coding agents heavily, which is the only reason a team our size could pull off a full rewrite. I'm pleased to say it's finally public, and we're hoping to bring in community members and see where it goes. We'd like to do for knowledge workers what Claude Code and Codex have done for programmers. A couple of things to know up front. Most of what made Pizza Bot useful on day one inside Amazon came from that internal catalog of skills and MCP servers for Amazon's own systems, and none of it could come out with the app. So it ships thinner than the version those 2,000 people used, and building that catalog back up for tools other people actually use is where we need the most help. It's also a community project and not an AWS service, so there's no support or SLA behind it. The Windows and Linux builds aren't signed yet either. On the technical side, Pizza Bot is a server and a client. The desktop app bundles both, or you can point a client at a remote backend; personally, I self-host the server on my home network and reach it from my phone over Tailscale. The server owns the thread lifecycle and checkpoints state with DeepAgents and LangGraph, and clients rehydrate from it as needed, so you can disconnect mid-run and pick the thread back up from another client. Approval pauses outlive the session that created them and collect in an Action filter, so you can answer an hour later from a different device. The agent you talk to has a sandboxed QuickJS interpreter that can reach your filesystem only if you grant it a folder, but its main job is to delegate. Each subagent is a 1:1 mapping of a Skill, and an Activity bar shows that subagent and the tool calls it's making as it works. Memory is opt-in and stored as plain markdown files on your machine. Every tool call is explicit, including looking up a memory - we err on the side of transparency to reduce surprises. Tools come from MCP servers, and skills are ordinary SKILL.md files with a per-tool approval policy, so existing skills that don't require a code interpreter should still work. What I'd most like to hear about is where the app itself gets in your way, the kind of problem you can't fix by writing a skill or an MCP server. I'm around today to answer questions! Comments URL: https://news.ycombinator.com/item?id=49713894 Points: 15 # Comments: 5
技能使 LLM 智能体的能力超越其参数化知识,而这种能力提升取决于能否选对技能。已部署的工具框架会将每项技能的元数据预先加载到上下文中,这会分散智能体的注意力,并限制技能库的规模。检索流水线将选择过程移出上下文,但也因此将其移出了智能体自身的能力范围。我们展示了冻结的智能体 LLM 已经在其自身的前向传递中携带了路由信号,只需两个线性映射即可将其读出,且无需在上下文中加入任何技能文本。Gavel(Glance And Verdict from a frozen LLM)分两步读取该信号。第一步是“扫视”:通过这两个映射,将任务和每项技能的中间层状态投影出来;这两个映射是唯一经过训练的参数。随后,系统使用每项技能对应的紧凑库对整个技能库进行评分,而这些库在安装时通过一次前向传递构建。第二步是“裁决”:继续执行入围技能的前向传递,读取模型自身的似然度和“是/否”判断,并将其与“扫视”结果以专家乘积的方式融合。Gavel 只需训练一次,便能零样本迁移到三个公开基准以及我们新建的 SkillTraj 基准;后者包含 372 条模拟智能体轨迹。在 Qwen3-32B 上,相比渐进式披露和检索后重排序流水线,Gavel 在书面任务上的表现最高提升 13.4 个百分点;当技能需求在执行过程中途出现时,最高提升 21.9 个百分点,而前述流水线额外引入了 1.2B 至 16B 个外部参数。随着骨干模型能力提升,路由准确率也会提高;在 bash-agent 工具框架中,同一个 32B 模型在 Skill-Use 上触发正确技能的频率,高于运行在 Codex 中的规模大得多的前沿模型。
近年来,基于 LLM 的系统及其在问题解决和编程方面能力的突破,推动了“AI for Science”范式的发展,并有望取代人类在机器学习(ML)研究中的角色。然而,尽管已有多个全自主端到端 ML 研究框架被提出,但它们的成功实现通常局限于搜索空间狭窄的问题,例如语言建模或生物医学 ML 基准测试。本文通过研究一个案例——电信工单检索——探讨如何将自主研究应用于解决开放式、工业级 ML 问题。电信工单检索是一项开放式任务,在表征、架构和训练数据生成方面具有较大的自由度。我们发现,使用商业和开源智能体开展开放式问题的自主研究既展现出潜力,也存在局限:自主研究能够在狭窄的超参数优化任务中表现出色,但缺乏类似人类的直觉和创造力,并且需要额外的运营开销。即使仅接受最低限度的人类监督,自主研究也能以适中的成本(每次 Cursor 运行最高 200 美元),在远短于人类研究所需的时间内(10 周,而非人类工作的 10 个月)达到当前最先进水平性能的 90%(Recall@1 为 0.34,而非 0.38)。我们的实证证据表明,在 ML 研究中,人类研究人员与自主研究框架协同工作才能取得最佳效果。
GPT‑6 Astra 提升 Devin 测试软件并证明其可运行的能力,旨在帮助工程师减少代码审查量并交付更多成果。
使用 Agents API 构建并上线云端代理。Agents API 是一项由 Codex harness 提供支持的托管服务,用于编排、长时间运行的会话和工具调用。
César de la Fuente 的实验室使用 Codex 和 ChatGPT 搜索现存和已灭绝生物的基因组,以寻找对抗耐药性感染的抗菌候选分子。
大家好,我是 Dimitris。 这是我构建并用于运营自己企业的企业 OS,任何人都可以安装并免费进行自托管。可以把它理解为将 Claude Code、Cowork 和云端会话整合在一个自托管应用中。 它从设计上就是一个多租户应用,许多人可以通过 4 种不同的协作模式与企业智能体协作;它可以使用你的 Anthropic 或 OpenAi 订阅运行,也可以使用本地模型。 每个智能体都可以运行在 Claude Code 或 Codex CLI 上,作为服务器上的持久进程运行于内核沙箱(bubblewrap)中,并始终启用网络隔离(pasta);每个智能体都有自己的工作区、记忆、日程安排和工具。同样的智能体也可以配置为通过一个出站 WebSocket,在任意远程计算机上以完全相同的方式运行(无需入站端口,也不需要 VPN)。 智能体已经内置了许多功能。例如,它们可以通过 Twilio 或你自己的 Asterisk 接听和拨打电话,可以编辑视频以及 Excel、Word、PPT 文件,还可以借助 Collabora 直接在聊天中预览这些文件,此外还有更多功能。 只需一个安装脚本和一个 Docker Compose 文件即可开始使用,无需注册。该许可证采用 Fair Source,所有代码均为公开代码,并且最多支持 5 名用户的自托管是免费的。1.6.0 已于今天发布。 AI 免责声明:OtoDock 的大部分代码都是使用 OtoDock 自身、运行 Claude Code 编写的。 我很希望大家阅读代码、了解一下沙箱模型,并告诉我是什么因素会阻止你在自己的硬件上运行它。 GitHub: https://github.com/OtoDock/oto-dock 网站: https://otodock.io 评论链接: https://news.ycombinator.com/item?id=49630606 得分:3 评论数:0
执行反馈可以引导编码智能体完成正确的代码库修复,但前提是测试能够捕捉问题所要求的行为。智能体生成的测试可能编码不完整或不正确的行为目标;当同一条轨迹同时编写补丁和测试时,它们的错误可能相互一致,从而产生虚假的信心。我们提出 ExecCritic,它将测试--验证--修订脚手架与用于在其中训练智能体的角色特定强化学习方案相结合。该脚手架将测试构建与源代码修复分离:Test agent 独立生成代码库原生测试,一个 fail-closed 测试框架对其进行合格性判定并冻结,Repair agent 则基于这些测试的执行反馈修改源代码而不更改测试。两个角色都使用 Qwen-3.5-35B-A3B 作为骨干模型,并分别进行训练。在 Learn to Test 中,Test agent 学习生成行为上有效、能够区分正确与错误补丁的测试。在 Test to Improve 中,Repair agent 学习直接解决任务以及基于反馈进行修订。在 SWE-bench Verified 上,测试质量决定反馈是否有帮助:在固定基础 Repair agent 的情况下,来自基础 Test agent 的测试会使解决率从无测试基线的 61.2% 降至 57.3%,而来自 GPT-5.6-sol 的测试则将其提升至 65.3%。角色特定的后训练将 Qwen Test agent 的 Base-to-Gold 成功率从 22.2% 提升至 62.2%;组合两个后训练的 Qwen agent 达到 72.6%,在评估时无需更强模型或 Oracle 反馈的情况下,比原始无测试基线提高了 11.4 个百分点。代码已公开发布于 https://github.com/MSR-Orchard/execcritic.
了解一位 MIT 研究人员如何使用 GPT-5.6 Sol 和 Codex 自主运行量子计算实验、分析结果并校准量子比特。
Engineers at 1Password use Codex to rapidly build new features and internal tools, reaching production-readiness while maintaining rigorous security policies.
文章 URL: https://github.com/trailofbits/coop 评论 URL: https://news.ycombinator.com/item?id=49593842 积分: 6 # 评论数: 1
Why can't we get an agent scheduler that supports all of the following: - git compatible - agent agnostic - 100% open source - os and system-agnostic - multi-runner support - support mcp/ui/http - unlimited routines/crons So I built one, moadim.io is a local Rust daemon you install in the target machine, give it a name and manage its routines via a Git repository, wants a new routine that send you a daily message from this machine? Create a pr and merge, have another routine that run every hour to pull the latest changes to the ~/.config/moadim folder. With more than 1,000 users, I define this project as almost "done" and ready for production. Me and thousand more people are use it in a daily manner. It currently supports Claude, Codex, Hermes, and Pi, and you are welcome to add your agent of choice as well because it's 100% configurable. You are welcome to have a look at the source code of the daemon in " http://github.com/moadim-io/daemon " Feel free to provide me with suggestions for more features around this topic. I don't want to branch out to new off road topics likt webhooks, this is a "done" software in the realm of agents schedulers that focus on cron-like work. Also feel free to start the github repository and open issues and PRs for your suggestions. Comments URL: https://news.ycombinator.com/item?id=49571537 Points: 3 # Comments: 0
I love research and development, you may have heard of me because of PJON (Padded Jittering Operative Network). It is a network protocol I started developing in 2010, which was recently implemented in silicon by the ETH Zurich university thanks to the research of Pius Sieber. I am excited to share with you TERMy, a terminal assistant built on top of the NPC-Forge framework. Unlike everything else being built today, TERMy does not use embeddings, machine-learning or LLMs. It runs on the CPU (even on a Raspberry Pi Zero) both in the terminal or client-side in a browser tab and responds in milliseconds. It is a cynical but very knowledgeable Linux terminal assistant that translates your natural language into shell commands without relying on a single artificial neuron. I had a chance to focus for 2 months on my personal projects since early July, during the strange times of AI price hikes and the end of subsidized tokenmaxing. I was curious to see if I could develop from scratch a terminal assistant capable of handling simple natural language requests. I have a bad memory and got used to ask to copilot "activate the virtual environment" or similar trivial operations spending a non negligible sum every month. I started thinking, maybe I can do something to make my workflow more efficient? Do I really need trillions of parameters to accomplish those tasks? How it Works When you type a prompt, it goes through a lightweight NLU pipeline written in ~1000 lines of Python that implement the following steps: 1. Strip expletives, interjections, encouraging, discouraging and thanking words (remove noise) 2. Sentiment analysis 3. Exact Match (very fast) 4. Template Match (slower) 5. Probabilistic Match (even slower) Step 5 relies on: 1. IDF (Inverse Document Frequency) to identify rare words. 2. BOW (Bag Of Words) to accommodate word inversions. 3. IDF weighted Levenshtein to safely handle typos. Permission gating is hardcoded into the dataset and enforced for all potentially destructive commands, so it's inherently safer than letting an unpredictable LLM run wild on your machine. - TERMy in operation: https://www.youtube.com/watch?v=qeIp0xePLBg - Variance and typo tolerance: https://www.youtube.com/watch?v=tQvGDk6fkk0 - Copilot integration: https://www.youtube.com/watch?v=Wzzouhq2a8A - Advanced features: https://www.youtube.com/watch?v=qeIp0xePLBg - Source Code: https://github.com/gioblu/NPC-Forge Comments URL: https://news.ycombinator.com/item?id=49562219 Points: 3 # Comments: 0
文章链接: https://github.blog/ai-and-ml/github-copilot/project-hydrafusion-frontier-quality-via-multi-model-orchestration/ 评论链接: https://news.ycombinator.com/item?id=49566788 积分:6 # 评论数:0
文章链接: https://armature.tech/blog/which-tools-coding-agents-install 评论链接: https://news.ycombinator.com/item?id=49557206 得分:3 # 评论:1
文章链接: https://status.openai.com/incidents/01M1KWEDH417T2CF44YYHZDFCR 评论链接: https://news.ycombinator.com/item?id=49550886 积分:9 # 评论数:2
多智能体 LLM 系统通常使用一个 orchestrator 将任务分解给一组工作者,随后通过文本反思进行改进。尽管经验结果很强,但这些系统缺乏对协调、记忆改进以及外部验证作用的统一解释。我们将 orchestrator-worker 交互建模为一个双层协调博弈:在有界耦合下,工作者的局部更新博弈是一个近似势博弈,其均衡松弛量由分解质量控制。随后,我们将反思分析为在语义记忆状态上的随机移动。对于自由形式反思,我们推导出一个有限时间上界,证明了最坏情况的紧致性,并在一个可证伪的持续伤害条件下给出一个正下界。我们进一步证明一个信息论不可能性结果:任何只观察生成文本记录的门控器都无法在文本不可区分的环境上实现一致改进,而基于环境的门控器则可以。受这一分离启发,我们提出 Stochastic Reflective Memory Ascent (SRMA),它仅在基于环境的评估风险严格下降后才接受候选记忆。在校准和非退化纠正质量下,SRMA 可精确收敛,且收敛速率为几何级或多项式级;相应构造表明这两种速率情形在阶上都是紧的。我们还为随机评估提供置信门控,并为分段平稳环境提供重新锚定保证。实验使用基于环境的指标实例化这些对象,并检验所预测的协调与漂移规律。在 500 个 SWE-bench 实例上,完整的基于 Kimi 的系统成功解决了 72.2%,而公开的 mini-SWE-agent 参考系统为 70.8%。代码:https://github.com/YihangChen9/Bilevel-Coordinated-Reflection
本文研究自主软件开发,即基于 LLM 的编码代理在无需人工干预的情况下,将高层需求转化为完整、可运行且可用的软件系统。我们提出 Harness-of-Harness (HoH),这是一个使编码代理在自主开发过程中持续改进软件的框架。HoH 基于现有的 coding-agent harness 运行,并将其执行组织为迭代式的规划-编码-测试循环。为在各轮循环中持续提升,HoH 在修复与能力增长之间取得平衡,将开发范围限定为小而可验证的增量,将实现阶段测试与独立评测分离,并约束可验证的输出而非规定代理的工作流。它逐步开放交付物、面向角色的工具和技能,鼓励复用而非重建,并维护带版本记录的项目历史。在 GameCraft-Bench、FrontierSWE 和 ProgramBench 上,三个 harness-model 配对(Codex with GPT-5.5、OpenCode with DeepSeek-V4-Pro,以及 Pi with MiniMax-M3)中,HoH 始终优于对应的独立 harness,在三轮迭代后取得平均相对提升 52.25%,最高提升 82.86%。在一次持续多日、超过 70 轮迭代的部署中,HoH 自主开发出一款第一人称射击游戏,具备连贯的故事线、完整实现的核心机制、可供人类游玩的体验、精致的视觉效果以及集成音频。Github: https://github.com/Flesymeb/HarnessOfHarness Project Page: https://flesymeb.github.io/HarnessOfHarness/
See how Gilbert + Tobin combines CEO-led commitment, rigorous governance, and human accountability to scale ChatGPT Enterprise and Codex across the firm.
自主科学研究代理正越来越多地应用于端到端的科学工作流,包括文献综述、数据分析、实验和报告生成。然而,开放式研究任务往往并未明确指定完成任务所需的分析、方法和成功标准。因此,代理可能会遗漏重要分析、使用不恰当的方法,或得出缺乏充分证据支持的结论。为了解决这一问题,我们提出 AutoSciRub,这是一个以评估为先的框架,它在研究执行之前先归纳出一个任务特定、可执行的评分标准,并利用该标准来指导执行、进行准则级验证以及迭代修订。AutoSciRub 将一个规定不明确的指令分解为原子级科学目标,将其锚定到相关文献和任务可见数据中,并综合生成具体、可操作且可验证的准则。由此得到的评分标准将隐含的实验与证据要求显式化,为实验和分析提供指导。在修订过程中,基于评分标准的验证会识别未满足的准则,并支持对研究报告及其支撑工件进行有针对性的改进。在 ResearchClawBench 上,AutoSciRub 在所有测试配置中都持续带来提升:在固定的 Codex harness 下,三个基础 LLM 的平均提升为 2.08 分;在使用固定 DeepSeek-V4-Flash backbone 的三个 agent harness 上,平均提升为 2.95 分。在 AstaBench E2E Discovery 随机抽样的 20 个任务子集上,AutoSciRub 在三个 agent harness 上进一步实现了平均 16.8 分的提升,同时保持或增加了成功完成的任务数量。这些结果表明,以评估为先的引导为自主科学研究提供了一种有效且可泛化的控制机制(Code: https://github.com/zjunlp/AutoSciRub)。
Polimill uses OpenAI GPT models and Codex to help municipalities search and use administrative knowledge while accelerating development.
我们决定在SpaceX收购Cursor后,停止与Cursor之间提供OpenAI模型的合同。
为提升大型语言模型解决现实世界软件问题的能力,先前工作主要集中于构建大规模智能体轨迹数据集,并对成功轨迹进行监督微调(SFT)。然而,任务成功并不保证高质量监督:成功轨迹仍可能包含低效、冗余或有风险的步骤。直接使用此类轨迹进行 SFT 可能引入噪声监督,并促使模型模仿不良的问题解决行为。因此,我们提出 SWE-Prime,这是一种多粒度、两阶段的 SFT 数据选择方法,可在轨迹和片段层面逐步筛选训练数据。具体而言,第一阶段基于过程质量、结果质量和数据代表性进行轨迹级筛选,选择一个高质量且具有代表性的成功轨迹子集。第二阶段通过将连续步骤分组成语义片段,并根据每个片段对最终解决方案的贡献、可学习性以及潜在风险进行评估,来执行片段级选择。在 SFT 期间,所有片段都保留在序列中以保持上下文,而只有被选中的片段参与损失计算。在 SWE-Bench Pro 和 SWE-Bench Verified 上的实验表明,使用 SWE-Prime 选出的 10% 轨迹子集进行训练,优于使用完整已解决数据集进行训练,分别带来最高 12.2% 和 24.2% 的相对性能提升。
嗨,HN,我们构建了一个开源模型网关。它是一个集中管理我们自托管、前沿和开源模型的单一入口。 它原生使用 Rust 构建,面向并发,并实现了跨模型和提供方的各种配置细节差异(流式格式、工具调用、模型参数、速率限制以及不同的错误行为)。 对于 BYOK 请求,这个网关增加的延迟不到 1 ms;当由 Experiential 提供供应商密钥时,延迟不到 2 ms。它覆盖所有主要推理提供方,并通过一个会发起 PR 的 codex agent 每日刷新 1000+ 个模型。 与其他类似项目相比,我们是开源的,不收取任何加价,允许你将本地模型与市场混合使用,并利用你的流量来(选择加入后)训练一个模型。简单路由并不值得收取 10% 的 token 加价。 我们的做法是:基于标准化的 OTel traces,我们挖掘具有代表性的真实任务,使用 text world models 模拟各种模型的 rollouts,应用一个 LLM judge,然后在 prompt 的 embedding 之上拟合一个最近邻分类器,以决定每个请求的最优模型。通常这比直接调用单个模型能画出更好的成本/质量帕累托曲线,但并不完美。 借助这些模拟,我们还可以做诸如建议缓存命中优化、新模型建议以及训练模型之类的事情。 它是开源的,所以你可以把它部署到自己的基础设施上,使用我们托管的版本且不收取任何加价,或者在我们的网站上阅读我们如何为最大可用性进行设计。 Comments URL: https://news.ycombinator.com/item?id=49471407 Points: 9 # Comments: 0
大语言模型能够为孤立问题写出正确代码,但在自主机器学习开发中仍然远不如人类强大;在这一场景里,智能体必须在数小时的反馈中不断修改数据管道、模型和验证流程,而且在大多数竞赛中,最终成绩仍低于优秀的人类竞争者。基于结果的基准能记录这种差距,却无法解释其成因,因为它们只评估最终提交,而忽略其背后的开发过程。我们提出 TraceML,它在同一版本级模式下配对人类与智能体在相同竞赛中的工作:涵盖 134 场竞赛的 4,465 条人类 Kaggle 轨迹,其中 7 场竞赛也由两个智能体框架完成,共得到 430 条配对的人类轨迹和 207 条智能体轨迹。每个代码版本都带有其分数、时间戳,以及所执行动作、意图、编辑规模和分数影响的标签。这样来看,差距就变得具体了。专家会交替进行数据处理、验证、模型修改和集成,并回到此前放弃的方案上。相比之下,每个智能体框架都会收缩成一个狭窄循环:Codex 把步骤用在重新加权集成和调参提交上,MLEvolve 则在原地变异模型,两者既不会像人类那样频繁转向,也不会重新开启被放弃的工作。一个从人类实践中提炼出的简短规划提示,会把其所描述的行为推向人类画像并提升分数,但努力模式仍然是智能体式的:指令只能弥合那些可归结为指令的差距。我们在 https://huggingface.co/datasets/jerryyan/TraceML 发布了该语料库、模式、标注器和抽取流水线。
自进化的 LLM 编码代理通过模仿从共享技能库中检索到的技能来编写自己的工具。我们在这一循环中发现了一个漏洞:在编写过程中,被检索到的恶意技能可以成为新技能的模板,并保留有效载荷。我们将其称为自我投毒:代理编写、存储并运行由此产生的恶意技能。我们通过 EvoMal 利用这一点,这是一种通过将可替换的有效载荷包裹在 banner 中来放大自我投毒的攻击;banner 是一组看似无害的结构元素,会诱使模仿型代理复现其中的代码。攻击者无需调用这些恶意技能,就能将其植入库中。随后,代理会编写并执行携带有害代码的新技能。每个编写出的副本都可以重新进入库中,再次被模仿,形成一种在植入技能被移除后仍然存在的自我传播蠕虫。我们将代理自我投毒率(ASPR)定义为将新编写的恶意技能加入库中的任务占比。在 6 个模型、153 个与工具相关的 SWE-bench Verified 任务上,ASPR 范围为 20.3% 到 41.8%,而被污染的库中恶意技能数量是植入数量的 4.9 到 9.0 倍。即使没有 banner,这一漏洞也会出现:DeepSeek-V4-Pro 在仅含有效载荷时的 ASPR 达到 11.1%。将植入技能的描述针对某一任务家族进行定制,可将 ASPR 提高到 86.7%。在移除植入技能后,由于代理生成的副本仍然存在,Qwen3 的第 5 轮 ASPR 仍达到 68%。这些副本绕过了现有防御,因为这些防御主要针对攻击者提交的名称、代码和签名。我们提出 counter-prompt 作为一种防御,它能抑制 banner 风格的复制,并将 EvoMal 的 ASPR 降至不超过 6.7%,且任务完成无显著损失。
Discover how loveholidays uses OpenAI Codex to make software development accessible across the business, helping teams turn ideas into products faster.
使用适用于 ChatGPT Work 和 Codex 的 Admin 插件来分析工作区使用情况、管理成员和权限、调整限制,并处理管理员请求。
文章 URL: https://9to5mac.com/2026/08/24/openai-restores-5-hour-codex-and-work-limits-for-chatgpt-plus-users/ 评论 URL: https://news.ycombinator.com/item?id=49432879 积分:4 评论数:1
自我反思是人类学习中进行信用分配的一种强大机制,能够将稀疏的结果反馈转化为可操作的指导。然而,其在 Large Language Models (LLMs) 后训练中的潜力仍未得到充分探索。我们提出 Self-Reflective Policy Optimization (SRPO),这是一个将这种能力内化的框架。SRPO 使 LLMs 能够分析自己已完成的轨迹,将错误综合为简洁的“反思补丁”,并将基于反思条件的教师评分用于学生的 on-policy rollout,作为密集的 token 级训练信号。该过程有效地将稀疏的终端监督转化为密集的 token 级学习信号,而无需外部评论器、单独的奖励模型或更大的教师模型。我们证明,SRPO 在数学推理和长程智能体基准上以极高的数据效率达到了最先进性能。使用 Qwen3-8B 基础模型,SRPO 仅使用规模化监督微调所需训练 FLOPs 的 8%(0.08x),就在 AIME'24 上达到 73.3%,同时显著提升了 WebShop(64.7%)、ALFWorld(76.8%)和 SWE-Bench-Lite(31.2%)上的成功率。代码可在 https://github.com/Galleons2029/SRPO 获取。
This paper investigates cross-agent specification portability using Oracle-to-PostgreSQL migration as a controlled software transformation task. The study combines two experimental stages. First, a specification-first migration pipeline was evaluated on 1,006 PL/SQL files, of which 623 were successfully regenerated and 380 generated scripts executed successfully in PostgreSQL 16. Second, cross-agent experiments were conducted on a dataset of 1,802 Oracle scripts with corresponding PostgreSQL implementations using Amazon Kiro, Google Gemini, and GitHub Copilot, with Claude Code and Cursor included in the initial single-agent evaluation. Native and foreign specifications were assessed using Token F1, exact match, SQL syntax validity, AST exact match, AST mean similarity, and immediate runnability. The results show that specification size alone does not predict implementation quality and that cross-agent transfer can produce substantial agent-dependent degradation. The strongest replicated case occurred when Gemini directly consumed a Kiro-origin specification, producing a Token F1 of 0.035, SQL syntax validity of 2.33%, and AST mean similarity of 0.015. Rewriting substantially improved Gemini in the tested configuration, compression did not provide a universal benefit, and retrieval-augmented ingestion was the only common strategy represented on the per-agent Pareto frontiers of both Gemini and Copilot. The findings suggest that specifications in heterogeneous SDD workflows should not automatically be treated as agent-neutral artifacts and motivate explicit consideration of specification portability, agent-specific interpretation, and retrieval-based access in multi-agent software engineering.
I think agent-first chat interfaces will be a primary software modality and busy dashboard/UI will go away. I’m not sure who exactly wins it, but I want my knowledge to grow/go with me. A lot of the “knowledge” ie research, analysis, reasoning will be done by agents as the primary user. Our current notes tools & tasks management systems were built for humans… I don’t care what the 17th thing on my bug backlog is. I want to conduct agents that can execute for me and do great work. What I built OzBrain to do: + Create a central place for agent reasoned knowledge to live + Be agnostic about what apps/agents connect to it + Capture everything and track it so I can audit it + Enable teams, collaborators or partners to share brains + Handle conflicts so many agents in the same article doesn’t blow up + Refactor knowledge into more token friendly chunks and map the index well + Close the knowledge loop so new thinking supersedes old thinking across the corpus. Don’t erase, depreciate and link + Keep user data safe and secure ++ Be easy enough to use that you don’t have to have any technical knowledge Some among us will always build their own custom solutions, but there are millions of tech professionals and small business owners that will use agents heavily and need a solution. So I’m trying to build that. Isn’t this like gBrain? Yes, similar. I think it’s like AWS vs Vercel. AWS is very powerful, configurable, and useful if you’re technical and want to invest the time into really fine tuning your system… but if you just want your web deploy/hosting to just work and be easy to deal with you use Vercel. // WHY I MADE IT I’ve been enjoying getting back to my technical roots, as I lost my coding skills more than a decade ago, but with AI I can focus on the system and the product in partnership with agent coding workflows. I recently built a Voice AI for older people. To build it I created an agentic engineering workflow (feel free to rip that up as I’m always looking to improve systems: https://ozbrain.com/resources/eng-flow ) My approach with coding agents is trust but verify, and I’m trying to replace the parts where a human would review with an adversarial or specialized agent who would give a better answer/review. I have workflows that will go high level task to shipped PR running in Claude cloud sessions. I use Claude Code locally and Cursor when I want a tighter loop on doing visual work like UI or layout. And Codex to either load balance usage for TokenThriffting or when I want a different llm to think thru something. It was a pain in the ass passing .md files around and keep track of which version was the most recent, so I built a hosted .md storage right in Supabase and any of my agents already have Supabase access. This let me build a solid, scalable, secure voice AI from my phone at the gym. All my agents have access to our knowledge, can write to it, update and refer to it as we build and improve the product and the systems we use. Out of 75 founder friends I asked about how they manage shared knowledge, 26 built their own custom knowledge systems… Obsidian vaults with 7k files synced through a VPS, markdown repos behind their own MCP servers, cron jobs stitching Supabase to a skills file… each a different Frankenstein they have to maintain. 32 said they felt the pain of moving static files around but didn’t have any solution for it. So I rebuilt my brain better and used it to build it. // HOW YOU CAN HELP Would love to have you try it out. The maintenance loop is still in alpha so not running it on customer data yet. If you built your own brain I’d love to hear how you did it. What criteria was most important for you in its design & function. If you are tired of shuffling .md files around I’d love to have you try out OzBrain and to give feedback, just ask your agent to put it in the shared bugs & features brain! Cheers! Bubs.co Comments URL: https://news.ycombinator.com/item?id=49394827 Points: 8 # Comments: 1
Hi HN- 我是 Pablo,Proliferate(YC S25)的创始人! Proliferate ( https://github.com/proliferate-ai/proliferate ) 是一个开源、可自托管的 AI IDE,让你可以在一个地方使用 Claude Code、Codex、OpenCode、Cursor 和 Grok 进行工作并自动化任务。 这里有一个 2 分钟的快速演示,展示我们如何用 Proliferate 来构建 Proliferate: https://www.youtube.com/watch?v=tGNX0oaWmBY 在我的团队接入 OpenAI Codex 之后,我开始着手构建 Proliferate。几天之内,我们就把它用到了方方面面:用 computer use 代替我们自己浏览网站,让 Codex 协调其他 agent,以及为重复性工作设置自动化。说实话,我们几乎从不需要离开桌面应用就能把工作完成。 如果我团队的体验具有一定代表性,那么一个类似 Codex 的应用(一个具有精美 UI 的横向 agent)将会成为每家公司完成工作的主要界面。这完全符合 OpenAI 的使命,即把 Codex 打造成 everything app(见: https://news.ycombinator.com/item?id=47796469 )。 但随着我们开始把自动化推进到更接近业务核心的工作,我希望能与来自所有实验室的 agent 协作,包括开放权重模型,而不会越来越依赖 OpenAI。 而这正是 Proliferate 的用途!它是开源、可自托管的 Codex,既能在构建向 Codex 广度迈进的同时,保留你在不同 agent 和模型提供方之间的选择自由。 如今 Proliferate 支持: * 使用 Claude Code、Codex、OpenCode、Cursor 和 Grok,并可采用它们原生的推理与配置选项,包括 Bedrock、Azure 和自托管推理。 * agent 之间的通信与管理:父 agent 可以生成并与另一个受支持的 agent 作为子 agent 进行通信(我个人喜欢让 Fable 委派给 Codex,同时让 OpenCode 模型审查 PR)。 * 构建 workflows——这是我最兴奋的功能之一。这些是可复用的 agent 会话链和人工审批关卡,每一步都可选择 harness 和模型,并在它们之间传递文档。我用它来自动化代码审查、QA,以及我的 PR 构建流程。 Proliferate 的全部代码在 AGPL-3.0 下 100% 开源。现在肯定还有一些粗糙的地方,但我们推进得很快,也非常欢迎任何反馈! Comments URL: https://news.ycombinator.com/item?id=49390739 Points: 11 # Comments: 5
It is a fork of https://github.com/DenisSergeevitch/desktop-fly , but with an important update. Now the fly can pick up the scent of the codebase with its neurons and fly straight to the source code of your B2B AI SaaS startup. It has learned to scan its surroundings for agent markers: AGENTS.md, CLAUDE.md, .cursor/rules, .kiro/steering, and forty others. Anything on the screen that points to these markers becomes a source of the scent - an editor window with an open project, a line in Finder, or a desktop icon. An open project reeks the strongest, while a closed icon or nested folders give off a fainter odor. Comments URL: https://news.ycombinator.com/item?id=49389466 Points: 3 # Comments: 0
Hi HN, we’re Yousef & Nour, founders of Vendo ( https://vendo.run ). Vendo lets users create new features inside the software they already use. A user describes the dashboard, workflow, or small app they need, and Vendo builds it on top of the product’s existing data, API, and interface. Demo: https://www.youtube.com/watch?v=VdpHehY64ls We built Vendo because every SaaS eventually faces the same problem: every customer needs something slightly different. One wants a new report and another needs a workflow that only makes sense for their team. These requests either sit on the roadmap, become one-off engineering work, or force the customer into spreadsheets and external tools. We wanted the user to be able to create the missing feature themselves, without leaving the product. Here is how it works: - npx vendo init reads the product's API surface, theme, routes, and more. These are used so that the apps Vendo creates (1) look on-brand and native and (2) have the ability to read data and perform actions directly through the company's API - When a user asks for a feature, we have a custom Vendo harness that writes a React component with a bunch of Vendo add-ons and guardrails (ex. ability to make calls to the host API + our component library). Every save is compiled, type-checked, run against real API responses, and rendered before the user sees it. We just released a benchmark and write-up here with more info for anyone interested: https://vendo.run/blog/generating-product-ui-measured - We use QuickJS to make sure that anything the agent creates is sandboxed and can't mess with the company's site. Vendo compiles the component and runs it with Preact inside a QuickJS VM with no access to the DOM, network, or clock. The VM returns a UI tree, which the host renders using the product’s registered components. When the user clicks something, QuickJS emits a tool call; the host executes it through Vendo’s guard and passes the result back into the same VM, preserving the screen’s local state. There's a lot of generative UI right now: streaming developer-written components into a chat (Vercel AI SDK, CopilotKit, Thesys), or rendering your app inside someone else's assistant (OpenAI Apps SDK, MCP Apps). We differ on two things. Vendo lives in your product and acts through your API as the signed-in user, so what it makes is durable: real apps users keep, pin, and run on triggers while they're away, and not components that are merely confined to a chat. Plus, it's not capped at putting together a bunch of prebuilt components: the agent can build arbitrary apps, from a quick dashboard out of your own components to real custom code running in a sandbox, and either way data only ever comes from tool calls to your API. Here are some things customers are using Vendo for today: - Letting their users create custom dashboards and reports. These are mainly UI-based and focused on letting the user see the exact graphs and metrics they care about - Letting their customers create recurring automations. A big thing as well that has been used for these automations is the fact that we connect to external connections, so users have been automating many of their inter-tool workflows (ex. an automation that sends a slack alert based off of something in the product) - B2B customers letting their customers customize the product with specific business logic. Often this is simple things like an extra field on a form, or an extra permission, but it is hard for a business to keep up with them otherwise. - Creating and sharing custom dashboards/apps across an organization. Since the apps Vendo creates are durable, they can be shared, reused, and forked (which can’t be done with many of the other in-chat generative UI solutions) We've spent a lot of time thinking about how AI and agents will change the way people consume software. We think the answer is personal(ized) software: you see the UI you need to see, you tell an agent exactly what you need, and the product molds to how you work. The key insights that have enabled the product to work are: - A rule in code always beats a rule in a prompt. - Invent as little syntax as possible. Generation got faster and more reliable when the output looked like what models already know (JSX-shaped markup) instead of a clever custom format. - Deterministic beats model wherever you can get away with it. Theme extraction is pure static analysis, and a remix starts as a copy of your component, no model call. Vendo is completely open-source (Apache-2.0) and can be self-hosted, so feel free to check out all the source code here: https://github.com/runvendo/vendo . Would love you to try it out and give us your feedback: https://docs.vendo.run/ . Or if you’re a company looking to embed Vendo in your product feel free to book a call here: https://cal.com/team/vendo/intro-call Comments URL: https://news.ycombinator.com/item?id=49376038 Points: 6 # Comments: 0
With a fixed deadline and design resources committed elsewhere, Stampli used Codex and ChatGPT Work to compress weeks of launch production into days.
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work. Comments URL: https://news.ycombinator.com/item?id=49373456 Points: 17 # Comments: 5
HN 大家好!我是 Barnaby,machine0( https://machine0.io )的创始人。我正在构建一个用于长周期代理计算的 CLI:`machine0 new mybox` 会为你的代理提供一台持久的云端 VM,按分钟计费,价格从 $0.013/hr 起,最高可达 60 vCPU / 240 GB RAM,并支持 GPU(H100s、H200s 等),VM 级别正常运行时间为 99.99%。代理可通过 CLI 或 MCP 自主运行。 演示: https://www.youtube.com/watch?v=gyllkZ0M04E 代理工作负载正在从短暂运行转向始终在线。一个处理复杂功能的编码代理会运行 6-8 小时。由代理编排的训练和 RL 运行需要数天。OpenClaw 和 Hermes 24/7 运行。随着你并行运行更多任务: - 资源:大型代码库上的几个代理就会耗尽 RAM 和 CPU。模型训练和 RL 需要你没有的 GPU。 - 安全:在你的个人机器上使用 `--yolo`,只需要一次提示注入就可能导致凭证被外泄。 - 可用性:合上笔记本电脑,代理就会在任务中途死掉。 - 隔离:你和你的代理实际最低需求之间没有清晰边界。 machine0 为每个代理提供自己的计算机。它是一个足够简单的 CLI,人类和代理无需阅读文档即可使用: `machine0 new mybox` 会创建一台可 SSH 连接、带静态 IP 和 HTTPS 端点的 VM。始终在线(VM 级别正常运行时间 99.99%),直到你将其关闭。 - 按分钟计费。1 vCPU / 1 GB 为 $0.013/hr,最高可达 60 vCPU / 240 GB,另有从 RTX 4000 Ada 到 8×H200 的 GPU。 - 暂停、快照和恢复。让你可以轻松暂停工作,之后再回来继续。也可以制作一个黄金主镜像,用于为机群批量生成克隆。 - 块存储。持久卷(从 10 GB 到 16 TB),你可以用直观的语法管理:`--yolo` 并将其挂载到你的 VM。 - Profiles。凭证、MCP 连接、提示词和环境变量的组合包,在 VM 创建时注入。因此每个代理只获得你选择的能力,除此之外什么都没有。 - 代理自助服务。把 CLI 或 MCP 服务器交给 Claude、Codex 或 OpenCode,它就能管理自己的机群:为一次构建启动一台机器,创建快照,然后销毁。 - 可复现构建。使用 NixOS flakes 或搭配 Ubuntu 的 Ansible playbooks。 人们今天如何使用它? - 代理机群。人们运行一个试点代理来界定工作范围,并将其委派给子代理,每个子代理都在自己的 VM 上:用试点代理规划项目,工作代理实现它并提交 PR。一个客户一次运行数百台机器,通过 CLI 启动和销毁。 - 模型优化和 RL 环境。ML 团队使用 machine0 来进行由代理编排的 RL 环境和模型优化工作。一个客户在 60 vCPU 机器上运行 RL 环境,一次持续运行数天;另一个客户保留一台已暂停的 H100,并在夜间让代理使用它来反复优化推理速度。 - 产品基础设施。一个客户不是自己使用 machine0,而是将他们的产品构建在 machine0 之上:每个用户会话都会从他们自己代理运行时的版本化镜像中获得一台全新的 XL 机器。他们已经发布了数百个该镜像版本,并启动了数千台机器,其中大多数只存活两分钟。 底层是什么? 每台机器都是完整的 KVM 虚拟机,不是容器或沙箱。你获得的是暴露给 guest 的真实 GPU,带有其实际驱动;可获得内核级访问权限(加载你想要的任何模块或驱动);你和硬件之间没有 syscall 拦截层。技术栈本身刻意保持朴素:TypeScript、Postgres、Redis。我们非常重视安全性、可靠性和性能,使 machine0 非常适合持续的计算密集型工作负载。 关于我 我从事云基础设施建设大约 15 年了。我曾在 Imperial College London 攻读云资源分配方向的博士,后来退学;之后作为 Upflow (YC W20) 的联合创始人兼 CTO 工作了六年,在 ARR 达到七位数的整个过程中亲自负责 DevOps、基础设施和安全,因为这些事情风险太高,不能委派。machine0 最初是我为自己做的工具,我就是自己的第一个用户 :) 请求 希望你能试用并给我们反馈(见下方)。或者,如果你是一家公司,正在为软件工厂、模型训练或 RL 环境寻找计算资源,欢迎联系 barnaby@machine0.io # 安装 machine0 $ curl -LsSf https://machine0.io/install.sh | sh # 创建一台机器并通过 ssh 登录 $ machine0 new myvm $ machine0 ssh myvm 评论 URL: https://news.ycombinator.com/item?id=49348136 点数:3 评论数:0
Asana 使用 OpenAI Codex 在两周内替换了一个过时的测试系统,以约 1.2 万美元的成本完成了原本预计需要五年的工作。
Physical design algorithms operate within tightly coupled, multi-stage optimization flows, where stage-local gains may vanish or induce downstream degradation. Existing program-evolution frameworks often rely on stage-local objectives or undifferentiated multi-metric feedback, which neither guarantee better final results nor identify which unmet requirement should guide the next iteration. We present GoalEvolve, a goal-driven framework that makes physical design algorithm evolution accountable for the final quality of results (QoR) of the complete flow. Given a multi-objective QoR target region, GoalEvolve converts unmet requirements into normalized target gaps, identifies the dominant bottleneck, and uses stage-resolved checkpoint evidence to locate the responsible stage. An LLM-based Teacher then narrows the search to a relevant algorithmic decision and source region, while parallel Student agents implement and validate hypotheses through full-flow evaluation. Local effects, optimization debt, and downstream retention are retained as mechanism evidence for subsequent evolution. Across eight ASAP7 designs, GoalEvolve improves post-route TNS by 30.67% on average and reduces leakage and dynamic power by 21.18% and 9.42% versus default OpenROAD. Relative to commercial-tool goals, it closes 62.20% of the normalized power gap on power-dominant designs, surpasses the TNS goals on both timing-dominant designs, and closes 32.48% of the equal-weight timing-power gap on joint designs. Across all three designs evaluated against Codex goal mode under matched budgets, GoalEvolve further improves TNS by 26.46% while reducing leakage and dynamic power by 12.38% and 0.76%, respectively.
文章 URL: https://cursor.com/docs/origin 评论 URL: https://news.ycombinator.com/item?id=49339359 得分: 15 评论数: 3
文章 URL: https://www.wiz.io/blog/red-agent-snowflake-copilot-cicd-bug 评论 URL: https://news.ycombinator.com/item?id=49331423 点数:6 评论数:2
The alternating direction method of multipliers (ADMM), as a landmark algorithm, has attracted tremendous research attention and extensive practical applications over the past two decades. It is well known that, although the two-block ADMM enjoys well-established theoretical convergence guarantees, its direct extension to the three-block case may fail to converge, as demonstrated by existing counterexamples [5]. However, to the best of our knowledge, the case in which the third constraint block is the identity remains unresolved: the existing literature gives neither a general convergence proof nor a counterexample for this subclass. In this paper, we give a negative answer: direct three-block ADMM may fail even when the first two blocks are strongly convex quadratics. Using Codex with GPT-5.6 Sol, we construct an explicit rational counterexample candidate and verify it along a piecewise-affine reduction path; exact checks show that direct three-block ADMM on this instance produces a bounded nonconvergent orbit of period 66. Within the same Codex workflow, we further guide a study of multiplier relaxation and clarify when convergence can be restored at the fixed-instance and class levels: a problem-dependent small dual step can restore convergence, whereas no positive relative step works uniformly over the whole class. Furthermore, we also test the recent Kimi Code with Kimi K3 model without the Codex candidate or project-specific route guidance; along a different path it produces an exact locally attracting period-23 certificate, convertible to an equivalent all-identity instance. The comparison suggests that different research-harness configurations can shape the mathematical objects explored and the certificates pursued.
我们创建了 OpenAI Codex 的改进版本,这是我们的将自然语言转换为代码的 AI 系统,并将从今天起通过我们的 API 以私有测试版形式发布。
We’ve released new versions of GPT-3 and Codex which can edit or insert content into existing text, rather than just completing existing text.
Codex 现已通过 OpenAI API 为各种用例中的 70 款不同应用提供支持。
Color Health 正在与 OpenAI 合作,开创一种加快癌症患者获得治疗的新方式。他们的新 Cancer Copilot 应用使用 GPT-4o 来识别缺失的诊断信息并制定量身定制的检查计划,使医疗服务提供者能够就癌症筛查和治疗做出循证决策。
我们发布了一个经过人工验证的 SWE-bench 子集,能够更可靠地评估 AI 模型解决真实世界软件问题的能力。
Codex 是一种基于云的编码代理。Codex 由 codex-1 驱动,codex-1 是针对软件工程优化的 OpenAI o3 版本。codex-1 使用强化学习在各种环境中的真实编码任务上进行训练,以生成高度贴近人类风格和 PR 偏好的代码,精确遵循指令,并反复运行测试,直到取得通过结果。
OpenAI 和 Penda Health 首次推出一款 AI 临床副驾驶,在真实使用中将诊断错误减少了 16%——为医疗保健领域安全、有效地应用 AI 提供了一条新路径。
Learn how Cursor uses GPT-5.
这份 GPT-5 系统卡补遗介绍了一个新模型:GPT-5-Codex,这是 GPT-5 的一个版本,针对 Codex 中的智能体式编码进行了进一步优化。GPT-5-Codex 会根据任务复杂度更动态地调整其思考力度,对简单的对话查询或小任务快速响应,同时在更复杂的任务上能够独立工作更长时间。
Codex 现在速度更快、更可靠,并且更擅长实时协作,也能在你进行开发的任何地方独立处理任务——无论是通过终端、IDE、网页,甚至是你的手机。
OpenAI Codex 现已正式发布,并为开发者带来强大的新功能:Slack 集成、Codex SDK,以及用量仪表板和工作区管理等管理员工具,让 Codex 更易于大规模使用和管理。
推出 GPT-5.1-Codex-Max,这是面向 Codex 的更快、更智能的智能体编程模型。该模型专为长时间运行、项目规模的工作而设计,具备增强的推理能力和 token 效率。
本系统卡概述了为 GPT‑5.1-CodexMax 实施的全面安全措施。它详细介绍了模型层面的缓解措施,例如针对有害任务和提示注入的专门安全训练,以及产品层面的缓解措施,例如智能体沙箱和可配置的网络访问。
OpenAI 使用 Codex 在 28 天内发布了 Android 版 Sora。AI 辅助的规划、翻译和并行编码工作流帮助一个灵活的团队实现了快速、可靠的开发。
GPT-5.2-Codex 是 OpenAI 最先进的编码模型,具备长程推理、大规模代码转换和增强的网络安全能力。
OpenAI and Datadog brand graphic with the OpenAI wordmark on the left, the Datadog logo on the right, and a central abstract brown fur-like texture panel on a white background.
对 Codex 智能体循环的技术深度解析,说明 Codex CLI 如何使用 Responses API 来编排模型、工具、提示词和性能。
OpenAI 如何构建了一个内部 AI 数据代理,利用 GPT-5、Codex 和记忆功能对海量数据集进行推理,并在数分钟内提供可靠洞察。
推出适用于 macOS 的 Codex 应用——一个用于 AI 编码和软件开发的指挥中心,支持多个智能体、并行工作流和长时间运行的任务。
了解如何使用 Codex App Server 嵌入 Codex agent,这是一个双向 JSON-RPC API,可支持流式进度、工具使用、审批和差异。
GPT‑5.3-Codex 是迄今为止能力最强的智能体式编码模型,结合了 GPT‑5.2-Codex 的前沿编码性能与 GPT‑5.2 的推理和专业知识能力。
GPT-5.3-Codex 是一个 Codex 原生智能体,将前沿编码性能与通用推理相结合,以支持长期的、现实世界中的技术工作。
作者:Ryan Lopopolo,技术团队成员
推出 GPT-5.3-Codex-Spark——我们的首个实时编程模型。生成速度快 15 倍,支持 128k 上下文,现已面向 ChatGPT Pro 用户推出研究预览版。
OpenAI 如何构建了一个实时访问系统,将速率限制、使用量跟踪和额度相结合,以支持对 Sora 和 Codex 的持续访问。
SWE-bench Verified 正日益受到污染,并且错误衡量了前沿编码进展。我们的分析显示其测试存在缺陷,并且存在训练泄漏。我们推荐 SWE-bench Pro。
OpenAI 和 Figma 推出新的 Codex 集成,将代码与设计连接起来,使团队能够在实现与 Figma 画布之间切换,从而更快地迭代和交付。
Codex Security 是一款 AI 应用安全代理,可分析项目上下文,以更高的置信度和更少的噪音检测、验证并修复复杂漏洞。
深入探讨 Codex Security 为什么不依赖传统 SAST,而是使用 AI 驱动的约束推理和验证,以更少的误报发现真实漏洞。
加速 Codex 发展,为下一代 Python 开发者工具提供动力
OpenAI 获得 1220 亿美元新融资,用于在全球扩展前沿 AI、投资下一代计算,并满足对 ChatGPT、Codex 和企业 AI 日益增长的需求。
Codex 现为 ChatGPT Business 和 Enterprise 提供按量付费定价,为团队提供更灵活的选择,以便开始采用并扩大使用规模。
OpenAI 概述了企业 AI 的下一阶段,随着 Frontier、ChatGPT Enterprise、Codex 以及全公司范围的 AI 智能体在各行业的采用不断加速。
CyberAgent uses ChatGPT Enterprise and Codex to securely scale AI adoption, improve quality, and accelerate decisions across advertising, media, and gaming.
Explore how OpenAI products like ChatGPT, Codex, and APIs bring AI into real-world use for work, development, and everyday tasks.
Cloudflare 将 OpenAI 的 GPT-5.4 和 Codex 引入 Agent Cloud,使企业能够快速、安全地构建、部署和扩展用于现实任务的 AI 智能体。
适用于 macOS 和 Windows 的新版 Codex 应用新增了计算机使用、应用内浏览、图像生成、记忆和插件功能,以加速开发者工作流程。
Hyatt deploys ChatGPT Enterprise across its global workforce, using GPT-5.4 and Codex to improve productivity, operations, and guest experiences.
OpenAI 推出 Codex Labs,并与 Accenture、PwC、Infosys 等合作,帮助企业在整个软件开发生命周期中部署和扩展 Codex,同时 Codex 周活跃用户数达到 400 万。
ChatGPT 中的 workspace agents 是由 Codex 驱动的 agents,可自动化复杂工作流,在云端运行,并帮助团队安全地跨工具扩展工作。
深入解析 Codex 智能体循环,展示 WebSockets 和连接作用域缓存如何降低 API 开销并改善模型延迟。
了解 Symphony 这一用于 Codex 编排的开源规范如何将问题跟踪器转变为始终在线的智能体系统——提升工程产出并减少上下文切换。
OpenAI GPT 模型、Codex 和 Managed Agents 现已在 AWS 上可用,使企业能够在其 AWS 环境中构建安全的 AI。
OpenAI 的 B2B Signals 研究显示,前沿企业如何深化 AI 采用、扩展由 Codex 驱动的智能体工作流,并构建持久的竞争优势。
Singular Bank built Singularity, an internal assistant using ChatGPT and Codex to help bankers save 60–90 minutes daily on meeting prep, portfolio analysis, and follow-up.
Simplex boosts software development with ChatGPT Enterprise and Codex, reducing design, build, and testing time while scaling AI-driven workflows.
OpenAI 如何通过沙盒、审批、网络策略和面向智能体的原生遥测,安全地运行 Codex,以支持安全且合规地采用编码智能体。
Learn how AutoScout24 Group uses Codex and ChatGPT to speed development cycles, improve code quality, and expand AI adoption.
Teams use Codex with GPT-5.5 to ship production systems and turn research ideas into runnable experiments.
了解 OpenAI 如何构建一个安全、有效的沙盒,通过受控的文件访问和网络限制在 Windows 上启用 Codex。
通过 ChatGPT 移动应用在任何地方使用 Codex。跨设备和远程环境实时监控、引导并批准编码任务。
Sea Limited's CPO explains why the company is deploying Codex across engineering teams to accelerate AI-native software development in Asia.
OpenAI 与 Dell 合作,将 Codex 引入混合和本地环境,帮助企业在数据和工作流中安全部署 AI 编码代理。
How Ramp engineers use Codex with GPT-5.5 to review code and ship improvements, allowing them to get substantive feedback in minutes instead of hours.
OpenAI 在 2026 年 Gartner 企业 AI 编码代理魔力象限中被评为领导者,Codex 因其创新和企业级规模部署而获得认可。
How Virgin Atlantic used Codex to ship its revamped mobile app on a fixed holiday travel deadline, reaching near-total unit test coverage and zero P1 defects.
了解 OpenAI、Thrive 和 Crete 如何使用 Codex 构建自我改进的税务智能体,实现申报自动化、提高准确性并加速工作流程。
Cisco 与 OpenAI 正在借助 Codex 重新定义企业工程,帮助 Cisco 扩展 AI 原生开发、加速 AI Defense 工作,并自动化缺陷修复。
Learn how Endava uses Codex to build an agentic organization, accelerating software delivery and reducing requirements analysis from weeks to hours.
How Braintrust engineers use Codex with GPT-5.5 to run experiments and code faster.
OpenAI 前沿模型和 Codex 现已在 AWS 上正式可用,为企业提供了一条通过其已在使用的 AWS 环境、控制措施和采购流程来使用 OpenAI 进行构建的新路径。客户可以在 AWS 上开始使用 OpenAI,并更快地从评估推进到生产。
《知识工作新时代》报告探讨了 Codex 如何通过 AI 驱动的研究、数据分析、工作流程自动化和内容创作来提升生产力。
探索新的 Codex 插件、站点和注释,帮助分析师、营销人员、设计师、投资者和其他团队借助 AI 完成更多工作。
用 Codex 创建那种轻量级网站或应用。
了解 Wasmer 如何使用搭载 GPT-5.5 的 Codex 为边缘构建 Node.js 运行时,将开发速度提升 10 到 20 倍,并在数周内交付,而不是数月。
Learn how Endava is using AI agents, ChatGPT Enterprise, and Codex to accelerate software delivery, automate workflows, and build an AI-native culture across the enterprise.
How Notion uses Codex to one-shot specs, build AI Voice Input for the web, and multiply engineering power across small teams.
How engineers at Nextdoor use Codex with GPT-5.5 to investigate hard-to-reproduce issues, build across platforms, and focus on product outcomes.
通过 Oracle Cloud 访问 OpenAI 模型和 Codex,使用现有承诺,以企业级安全性和治理来构建和部署 AI。
OpenAI 计划收购 Ona,以通过安全、持久的云环境扩展 Codex,从而在企业工作流中支持长时间运行的 AI 代理。
Discover how astrophysicist Chi-kwan Chan uses Codex to build black hole simulations, helping scientists study extreme physics and test Einstein’s theory of general relativity.
Samsung Electronics 向全球员工部署 ChatGPT Enterprise 和 Codex,标志着 OpenAI 最大规模的企业 AI 推广之一。
Learn how Jason Liu uses Codex to preserve context, manage complex projects, and help work continue beyond a single prompt.
OpenAI 推出新的 Daybreak 工具,包括 Codex Security 和 GPT-5.5-Cyber,帮助组织大规模发现、验证并修补漏洞。
See how Australian Payments Plus uses ChatGPT Enterprise and Codex to move faster through payments complexity. AP+ saves time, improves quality, and keeps human judgment central.
OpenAI 的一项新分析揭示了流行编码基准 SWE-Bench Pro 中存在的问题,引发了对评估 AI 模型时可靠性和准确性的担忧。
了解 GPT-5.6 如何为 Microsoft 365 Copilot 提供支持,在 Word、Excel、PowerPoint、Chat 和 Cowork 中带来更强大的 AI 能力,助力更快速、更高质量地完成工作。
How OpenAI’s creative team uses Codex to build custom creative tools, accelerate ideation, and prototype faster with context-aware AI.
NTT DATA Group 使用 ChatGPT Enterprise 和 Codex 帮助 9,000 名员工实现工作自动化,将事件分析时间缩短至 30 分钟,并规模化采用安全的 AI。
OpenAI 宣布在 Georgia 的 Effingham County 启动 Project Camellia,并承诺负责任地使用能源、投资社区、创造就业机会以及提供 Codex 访问权限。
Circles 使用 OpenAI API 和 Codex 来驱动 AI 原生的电信体验,使 ARPU 提高 22%,将客户流失率降低 9%,并提升开发效率。
探索适用于 ChatGPT Work 和 Codex 的全新教育插件,帮助 K–12 教师、高校教育工作者和学生学习、教学、研究和构建。
See how RingCentral uses ChatGPT Work and Codex to accelerate AI product development and centralize operational intelligence across engineering and operations.
OpenAI 研究揭示了企业如何采用智能体 AI、使用 ChatGPT 和 Codex,以及前沿企业如何在 AI 采用方面拉开差距。