Google / Gemini

Google 与 DeepMind 的 AI 动态:Gemini 系列、Gemma 开源模型与研究成果。

arXiv cs.AI84

SolBehave:评估 LLM 生成的智能合约的行为正确性

大型语言模型日益生成 Solidity 智能合约,而这类合约一旦部署便永远无法修补,使单个潜在缺陷永久存在并可被利用。然而,当前的评估大多只考察生成的代码能否编译。我们提出 SolBehave,这是一项用于大规模评估模型生成合约正确性的基准测试,超越了临时性单元测试。针对 250 个真实且可复现的 Ethereum 合约,我们采用三种互补的行为保真度测试:差分模糊测试、历史交易重放,以及作为核心贡献的、由动态条件响应(DCR)图引导的合成交易重放;该图枚举了合法函数调用序列的空间。差分模糊测试将每个生成函数与原始函数在自动生成的输入上进行比较;历史重放则根据合约实际上在链上接收过的交易,对整个合约进行比较;DCR 引导的重放则依据符合其预期用法的调用序列进行比较,其中包括从未在链上发生过的调用顺序。我们在进行一轮自动修复后,评估了七个最先进的开放权重模型。编译成功率介于 26.4% 和 92.7% 之间,但在差分模糊测试中,汇总各模型和样本后,所有能够编译的生成函数体中只有 31.4% 与原始函数等价。在合约层面,出现偏离是常态而非例外:对于七个模型中的六个,在历史重放下,能够编译的生成合约中有 13% 至 31% 与原始合约发生偏离;在 DCR 引导的重放下,这一比例为 13% 至 44%;只有 Gemma-4-31B 在两种测试下的偏离率均低于 3%。因此,编译成功是正确性的必要条件,但远非充分条件。

arXiv cs.AI82

Tev:通过稠密进展奖励扩展长时域智能体环境

长时域智能体凭借能够自主执行具有经济价值的任务而日益受到关注,其工作时长可从数小时持续到数天。这一能力对于自动化发现和递归自我改进(RSI)至关重要。然而,开源模型在长时域任务上仍然表现不佳,主要原因在于缺乏训练数据和可行的训练方案,而同时扩展这两者十分困难:长而复杂的任务难以整理和验证,且其奖励通常十分稀疏。我们同时解决了这两个问题。首先,我们推出了 Tev-Gym,这是首个具备可验证、稠密且渐进式奖励的长时域智能体训练数据集,涵盖 7 个领域的 1,170 个已验证任务环境。该数据集采用“潜在世界优先”的方法构建:首先建立一个潜在世界,即对任务领域、工作流以及每一步之后预期状态的完整且正确的隐藏规范。随后,每个任务都根据该规范逐步呈现为面向智能体的环境,从而实现可控的复杂度和自然的进度指标。其次,我们将这一指标转化为训练方案:通过进度势能奖励塑形(PPRS),将进度转化为每一步的稠密奖励,从而提高训练效率,并使原本难以训练的任务变得可训练。采用该方案训练得到的 Tev-27B,在 Agents' Last Exam 的计算与数学科学领域(ALE-CM)上取得了显著提升,在 Pass@5 指标上达到 GPT-6 Astra 和 Claude Opus 5 的水平。尽管仅使用与 ALE 相关的数据进行训练,它仍能泛化到 Terminal-Bench(TB)3.0 和 4.0 子集,超过 GLM 5.2 和 Gemini 3.7 Flash,并在 TB 2.1 等中等时域任务上达到 Claude Opus 4.8 的性能水平。

MarkTechPost50

Google Releases Gemini 4 Argon API With 1M Token Output Limit

Google has made Gemini 4 Argon generally available on the Gemini API, AI Studio and the Gemini app. The sparse MoE model writes up to 1M output tokens and leads the Vals Index at 68.90%. The post Google Releases Gemini 4 Argon API With 1M Token Output Limit appeared first on MarkTechPost .

Simon Willison78

llm-gemini 0.35

发布:llm-gemini 0.35 今天通过 Gemini API 以及其每月 100 美元订阅计划向客户提供了全新的 Gemini 4 Argon 模型,我为 llm-gemini 添加了对它的支持。下面是低、中、高三档生成的 pelican。如预期一样,它们都非常出色:Gemini 一直以来都很擅长生成优秀的 SVG,而且据我所知,这是唯一一家其员工专门投入 SVG 研究的实验室(他们以前也曾展示过自己生成的 pelican)。这是“高”档生成的插图,花费了我 17.166 美分: 时尚而准确的自行车车架。篮子里有一条鱼。帅气的围巾。双脚分列车架两侧,踩在脚踏板上。车轮辐条、一顶小帽子、漂亮的喙和喉囊,以及略显凶狠的表情。 定价很有意思。发布时,Argon 的价格是输入每百万 token 2 美元、输出每百万 token 10 美元——与 GPT-6 Sol 相同,是 Claude Opus 5.5 价格的一半,以及 Claude Fable 5.1 和 GPT-6 Astra 价格的五分之一。但从 12 月 1 日起,Argon 的价格将翻倍——届时会与 Opus 5.5 同价。 如今,这样的价格并不能真正直接比较,因为真正重要的是:在代理式运行框架中,模型要完成一个目标需要消耗多少 token。Artificial Analysis 发布了使用各模型运行其基准测试的总成本,这为我们提供了一个有用的参考。Argon 在这方面表现非常好:运行该基准测试的成本为 2,407 美元,接近 GLM-5.3,而且明显低于 OpenAI、Anthropic 和 Meta 的最新模型。即使价格翻倍,它仍会比 GPT-6 Astra 略便宜。 Artificial Analysis 的堆叠柱状图截图,对 14 个 AI 模型的美元成本进行比较,按从低到高排序;柱体按提供商着色,每个柱体下方显示提供商徽标,底部以倾斜文字标出模型名称。右上角的徽标为 Artificial Analysis。指向 Gemini 4 Argon 柱体的悬浮提示框显示:Gemini 4 Argon(High),2,407 美元;输出 313 美元;推理 815 美元;缓存写入 897 美元;缓存读取 264 美元;非缓存输入 118 美元;点击查看更多信息。 从左到右的柱体如下,每个柱体上方标有总成本,部分分段数值印在柱体内部:Qwen3.8 27B(xhigh),1,336 美元,Alibaba 徽标;Mistral Large 4 Preview,1,607 美元,Mistral 徽标;Gemini 3.8 Flash(high),1,623 美元,Google 徽标;Muse Spark 1.3(max),2,000 美元,其中一段为 769 美元,Meta 徽标;Gemini 4 Argon(high),2,407 美元,其中两段分别为 815 美元和 897 美元,Google 徽标;GLM-5.3(max),2,503 美元,其中一段为 790 美元,Z.ai 徽标;Kimi K3(max),3,658 美元,Kimi 徽标;Qwen3.8 Max(0902),4,935 美元,其中两段分别为 893 美元和 1,441 美元,Alibaba 徽标;Grok 4.7(xhigh),4,967 美元,其中两段分别为 1,861 美元和 1,618 美元,xAI 徽标;GPT-6 Astra(max),5,324 美元,其中两段分别为 778 美元和 978 美元,OpenAI 徽标;Muse Wonder(max),5,733 美元,其中一段为 1,748 美元,Meta 徽标;Claude Sonnet 5.5(max with fallback),7,260 美元,其中两段分别为 1,216 美元和 1,815 美元,Anthropic 徽标;Claude Opus 5.5(max with fallback),8,708 美元,其中一段为 921 美元,Anthropic 徽标;Claude Fable 5.1(max with fallback),13,129 美元,其中一段为 7,219 美元,Anthropic 徽标。 (我用 ChatGPT 编辑了那张图表!数据全部正确,我已经核对过了——我只是需要让 Gemini 4 Argon 的信息框显示出来,同时避免遮挡其他数字。) 标签:google,ai,generative-ai,llms,llm,gemini,llm-pricing,pelican-riding-a-bicycle,llm-release

SiliconANGLE AI54

Gemini 4 Argon reaches Gemini app Pro and Ultra subscribers alongside a new tasks mode

Google LLC today brought its Gemini 4 Argon frontier artificial intelligence model to Gemini app users on its Pro and Ultra subscription plans, two weeks after the model went first to vetted cybersecurity defenders. Subscribers on those plans also get a new tasks mode that keeps Gemini working on a job after they have stepped […] The post Gemini 4 Argon reaches Gemini app Pro and Ultra subscribers alongside a new tasks mode appeared first on SiliconANGLE .

The Decoder60

Gemini 4 Argon is now available worldwide as Google's most capable frontier model

Google is making Gemini 4 Argon available worldwide through its API, the Gemini app, and Google Antigravity. The frontier model is aggressively priced but trails Anthropic's Claude Opus 5.5 and Meta's Muse Wonder Full in independent benchmarks. Internally, Google says Argon already cut coding tasks from months to weeks. The article Gemini 4 Argon is now available worldwide as Google's most capable frontier model appeared first on The Decoder .

The Verge AI69

Google makes it easier to launch your vibe-coded apps

The next time you vibe-code a custom app with Google's AI Studio, you won't have to connect your phone to a computer to install it. On Tuesday, Google announced a new feature in the US that will allow you to launch personal apps directly through the Play Store. Google first launched the ability to vibe-code […]

Hacker News AI48

Show HN: Mac Markdown reader for your AI agents

Hi HN, I'm Kirill. A dozen coding agents running in parallel, producing walls of suggestions, comments and ideas, all at once, all crammed into 88 columns of terminal scrollback, all gone the moment you run the next command. Does this sound familiar? FolderMd ( https://foldermd.com ) might be for you. It's free, and it's fast, and your agents can use it. I made it because I just couldn't handle it anymore. It's a small Mac app that gives those documents a home. Agents read our explainer and then learn to write markdown into a .foldermd/, instead of writing it into the terminal. FolderMD then opens each one instantly and renders it like a book: real typography, syntax-highlighted code, Mermaid diagrams, light and dark mode. You can edit a doc and save it, you can then add comments just like you would in Google Doc, and pass those comments to the agent. Let me know what you think, and if you want to see any features, and improvements. Thanks! -K. Comments URL: https://news.ycombinator.com/item?id=50074607 Points: 1 # Comments: 0

The Verge AI54

Google finally releases Gemini 4 Argon

After months sitting on the sidelines of the frontier AI race, Google has finally launched its long-awaited Gemini 4 Argon model, claiming it can compete with the best the industry has to offer. Google says Argon delivers "frontier reasoning and improved performance across complex workflows," while cutting down on hedging, repetition, and the cliches that […]

Hacker News 熱門73

在 HN 发布:Consus(YC F24)——面向处理敏感数据企业的 LLM 网关

Hacker News 的各位好!我是 Consus 的创始人 Eric。我们将所有 AI 请求路由至符合 ITAR 限制和受控非机密信息(CUI)要求的基础设施。Consus 本身符合 ITAR 限制要求,我们最近还完成了针对 410 项 FedRAMP High 控制措施的 FedRAMP 等效性审计。FedRAMP High 是美国政府针对处理敏感非机密数据的云服务制定的最严格安全基线。此类数据泄露可能造成严重或灾难性后果。在处理 CUI 信息之前,必须取得 FedRAMP 等效性或授权。演示:https://youtu.be/6MYxJe6P4oY 航空航天与国防行业正在快速增长。但遗憾的是,由于必要的数据保护措施和合规风险,大多数公司都难以使用 AI。如果这些公司能够使用大多数其他行业习以为常的 AI 工具,它们本可以发展得更快。 在加入 Consus 之前,我的大部分职业生涯都在航空航天与国防领域。我曾在 SpaceX 负责发射国家安全载荷,也是 Castelion 的早期员工。我亲眼看到,合规方面的繁文缛节如何让公司难以采用新技术,也难以信任那些会接触其敏感数据的供应商。 例如,我曾与一家卫星制造商合作,该公司不允许使用任何 AI 工具,即使是处理非敏感工作也不行。尽管过去 9 到 12 个月里 AI 已经变得极其有用,但令我惊讶的是,仍有这么多航空航天与国防公司拒绝接触 AI。他们看得出 AI 可能会有所帮助,但担心模型提供商使用他们的数据进行训练、违反 ITAR 规定,或者因未通过网络安全评估而失去新业务。这些担忧都是合理的。 解决其中部分或全部问题有许多方法,但我们专注于构建一个合规的 AI 推理层,可以“接入”任何 AI 工具或工作流。正如 OpenRouter 能够与大多数工具集成一样,Consus 可以连接到 ChatGPT 或 Claude 这样的工具、包括 Pi coding agent 或 Open WebUI 在内的开源工具,也可以集成到各个组织自行构建和维护的应用程序中。 几乎没有哪家航空航天与国防公司同时使用三大 GovClouds(AWS、Azure、Google)。当我梳理哪些前沿模型可以合法运行于 ITAR 和 CUI 工作场景时,我发现每个提供商都只提供部分模型,而每个模型达到的合规水平也各不相同。 我们的客户不会从云提供商的角度思考。他们考虑的是所需的数据控制措施,以及想要使用的模型。这就是我们构建合规后缀的原因。在底层,每个请求都会访问一个统一的 OpenAI 兼容端点;在接触推理服务之前,系统会先验证密钥并执行策略。每个模型名称都必须带有合规后缀,例如 gpt-5.6-sol:fedramp-high 或 claude-opus-5-5:itar,任何不带后缀的模型名称都会被拒绝。 我们绝不会代理到标准商业端点。客户还可以构建模型允许列表,并为组织内的单个用户分配支出和模型使用限制。所有这些都通过 Consus Portal 处理,团队可以在那里管理支出、密钥和策略。 我们经常听到的一个问题是:为什么不直接在 AWS GovCloud 或 Azure Government 中使用 Bedrock?对于小型团队而言,仅仅获得这些平台的访问权限就很困难。对于大型组织而言,GovCloud 允许你继承部分、但不是全部处理敏感数据所需的控制措施。此外,模型分散在不同提供商之间:GPT-5.6 Sol 仅在 Azure Government 中提供,而 Claude Opus 5.5 则通过 AWS GovCloud 中的 Bedrock 提供。如果两者都想使用,就需要搭建并保护两个独立的政府云环境。 我们将所有模型置于一个端点之后运行,因此客户无需选择云平台,而合规工作则由我们分别为每个平台处理。 鉴于数据的敏感性,Consus 不会在静态状态下存储任何客户数据。所有提示词和生成结果只会在请求处理期间保留于内存中,绝不会记录或写入磁盘。无论是我们自身,还是基础设施及模型提供商,都严格执行零数据保留政策。出于计费和审计目的,我们会保留每个请求和响应的元数据。 Consus 从设计上就不是自助服务产品。ITAR 要求我们确切了解每个密钥另一端的使用者,因此我们会验证每个组织,并在发放 API 密钥前签订合同。 我们向客户收取合规费用,起价为每席位每月 40 美元,另加按成本收取的 AI 令牌费用。我们目前正在积极服务客户,非常期待获得安全专家,以及在高度监管行业工作的开发者和工程师的反馈! 参考资料: API 文档:https://gateway.consus.io/overview 安全与架构:https://gateway.consus.io/security 合规详情:https://consus.io/compliance 评论 URL:https://news.ycombinator.com/item?id=50073471 得分:4 评论数:1

The Decoder71

如果没有买家出现,Character.ai将关闭,标志着其从与Google达成27亿美元交易到申请破产的陨落

Character.ai已申请破产并正在寻找买家。如果没有买家接手,该平台将关闭。这家由Google前研究人员创立的公司曾拥有超过2000万用户,但如今正因涉及未成年人受害的诉讼和运营问题而苦苦挣扎。此次出售应在2026年底前完成。文章《如果没有买家出现,Character.ai将关闭,标志着其从与Google达成27亿美元交易到申请破产的陨落》最初发表于The Decoder。

Hacker News AI40

Show HN: Mana – family organizer with an AI assistant

Hi HN, I built Mana, a family organizer: shared calendars, shopping lists, receipt scanners, meal planning, chores, and notes for the whole household, with a built-in AI assistant ("add milk and eggs," "plan dinners for the week") plus automations (e.g., milk goes back on the list before it runs out). Additionally, you can connect Mana MCP to your Claude, ChatGPT, Gemini etc. The goal is to eventually take the model calls and storage to local but that's work to be done. Appreciate if there's any feedback. Thank you! Comments URL: https://news.ycombinator.com/item?id=50071636 Points: 1 # Comments: 0

Hacker News AI47

Show HN: Proofsource. Find where AI leaves you out. And Fix it

Hi HN. I am Lokesh. I built Proofsource as an internal tool to help my finance ERP product , Rivane, to gain visibility into AI engines like ChatGPT, Claude, Perplexity, Google AI mode and such. But when I talked about it to my friends, they also saw value in it and it became a product of itself. And now proofsource had 5 paying customers(paying $1500pm in total) in the first week itself and more than 60 brands used it to get their visibility since we launched out of beta last week. What proofsource does is that it queries ChatGPT, Perplexity, Claude and Google AI Overviews using the questions or prompts that buyers normally ask and then it shows then which competitor shows up instead of their own brand for the questions/prompt. It also gives visibility into which sources the answers cite and other things like sentiments. Everything is shown to you along with what to fix and how each data and number is calculated to give 100% transparency into what we did and why and how you can use all this information to improve your visibility. We are still adding features based on initial user's feedback like auto fixing and publishing to their social and web platforms. Running ads for them for improving the visibility. We are just learning as we interact with each customer so feel free to share your feedback. And thank you for your time. :) Comments URL: https://news.ycombinator.com/item?id=50067749 Points: 1 # Comments: 0

The Decoder65

White House reverses course and restores UK AI Safety Institute's access to frontier models

The White House has reversed course and restored the UK's AI Security Institute's early access to AI models from Anthropic, Google DeepMind, and OpenAI. In September, the Trump administration had told the companies to stop sharing their models with the British agency. Diplomatic pressure from AI Minister Kanishka Narayan led to the reversal. The article White House reverses course and restores UK AI Safety Institute's access to frontier models appeared first on The Decoder .

MarkTechPost56

Top 12 AI Live Translation Tools in 2026: Latency, Languages, Pricing

We checked 12 AI live translation tools, including Palabra.ai, Google Live Translate, DeepL Voice, Teams Interpreter and Wordly, against official pages for languages, latency, platforms and pricing. The post Top 12 AI Live Translation Tools in 2026: Latency, Languages, Pricing appeared first on MarkTechPost .

The Decoder60

Google Deepmind's AlphaProof Nexus solved dozens of open math problems and actually got peer reviewed

Google Deepmind's AlphaProof Nexus solved dozens of open math problems, including nine Erdős problems two of which were unsolved for 56 years, per a study in Science. The publication comes as OpenAI faces a boycott for dumping AI proofs on GitHub without peer review. Deepmind had its results reviewed before publishing. The article Google Deepmind's AlphaProof Nexus solved dozens of open math problems and actually got peer reviewed appeared first on The Decoder .

arXiv cs.LG69

Enhancing Tabular Learners with Context-Aware Semantic Embeddings

arXiv:2608.03565v2 Announce Type: replace-cross Abstract: While modern tabular learners excel at capturing statistical patterns, they frequently operate in a semantic vacuum, treating textual features as discrete symbols, ignoring the rich semantics inherent in feature names or cell entries. We propose CASE (Context-Aware Semantic Embeddings), a novel framework that bridges the gap between the semantic understanding of Large Language Models (LLMs) and the statistical capabilities of tabular learners. Unlike existing methods that embed rows in isolation, CASE utilizes a contextualization strategy: we pre-fill the KV cache of a custom-trained Gemma 3-based Tabular Language Model with a representative sample of rows to establish a persistent anchor of the dataset's semantics. This ensures that generated row embeddings are dynamically contextualized, resolving semantic ambiguities and anchoring representations in domain-specific context. Our experiments across several benchmarks (CARTE, TextTab, and TabArena) demonstrate that CASE substantially improves the performance of tabular learners on semantically rich datasets, particularly in low-data regimes.

arXiv cs.LG81

将分词从主机端移出以服务智能体式 LLM

arXiv:2610.13325v1 公告类型:cross 摘要:智能体式工作负载会在多次调用之间重复发送几乎不变的长上下文。如今,前缀缓存可以跳过其中大部分预填充阶段,但每个请求仍必须先完成全部分词,之后才能查询缓存。分词这一长期以来被视为无关紧要的 CPU 阶段,如今却成为首 token 生成时间(TTFT)中占比最大的部分;而为整个 GPU 集群执行分词的前端,则限制了其能够供给的 GPU 数量。我们提出 GraphTok,这是一种 GPU 分词器,可将从预分词、字节对编码(BPE)合并到 token ID 组装的完整编码流程,作为一张可重放的 CUDA 图运行。GraphTok 在 GPT-2、Llama 3、Qwen 2.5、DeepSeek-V3 和 Gemma 3 上逐位复现参考 token ID;对于 GPU 无法占优的短输入,则将其交给 CPU 引擎处理;同时,它还可以接入未经修改的 vLLM 和 NVIDIA Dynamo。GraphTok 在包含数据传输的情况下,可在 1.7–3.2 毫秒内编码一百万个 GPT-2 token;在 Llama 3 上,其速度比最强的 CPU 分词器 gigatoken 快 4.7–6.0 倍。在 TokTier 自有的单请求基准测试中,其速度比 gigatoken 快 2.6–3.9 倍,最高比 TokTier 快 18.1 倍。在使用缓存前缀的 vLLM 中,它将平均 TTFT 降低了 2.1–2.4 倍。最大的提升体现在容量方面。在 2 秒 P99 目标下,配备 GraphTok 的单个 Dynamo 前端可持续处理 518 个请求/秒、每个请求包含 12.1 万个 token 的流量,相当于高峰时段 56 个 GPU 的负载;相比之下,使用 gigatoken 时为 296 个请求/秒,使用默认分词器时仅为 18.5 个请求/秒。在 12.1 万至一百万个 token 的范围内,其可承载负载是 gigatoken 的 1.75–2.5 倍;在 gigatoken 的尾延迟失控的情况下,P99 最多可降低 64 倍。

arXiv cs.LG75

PlacePulse:从结构性地理数据中学习可迁移的地点表示

arXiv:2610.13390v1 公告类型:新论文 摘要:地理空间机器学习通常依赖针对特定任务的特征工程,对大量社会人口、商业和环境变量进行处理。地理空间基础模型提供可复用的地点表示,但主流方法通常编码人口流动和搜索活动等动态信号,或来自卫星影像的视觉信号。很少有模型将地点的静态结构组成(谁在那里生活和工作、哪些企业在那里经营,以及它们面临哪些风险)表示为精细空间分辨率下的可迁移嵌入。我们提出 PlacePulse,这是一种结构性地理空间基础模型,将 572 个属性通过 H3 邻域聚合扩展为 2,288 个输入,再压缩为 256 维单位范数嵌入,覆盖美国 1,055,537 个 H3 分辨率为 7 的网格单元。我们还提出了一套可复用的双轴基准,在多个目标和空间支撑单元上,将表示的内在质量与下游预测效用区分开来。在内在质量方面,这些嵌入重构标准化输入的平均 $R^2 = 0.853$,利用了 256 个维度中的 251 个近似去相关维度,并呈现出显著的空间组织性:第一主成分的 Moran's $I = 0.93$,且自相关性在超出输入半径的范围内依然存在。在外在效用方面,在来自 12 个外部来源、涵盖四种空间支撑单元的 74 个留出目标上,PlacePulse 在与训练输入无关的目标上的表现优于维度相同的特征选择基线,并可与 Google 的 Population Dynamics Foundation Model 相媲美;它在健康、经济、选举、经济流动性和社会脆弱性结果方面表现领先,其相对优势因领域而异。这些发现表明,结构组成是地理空间基础模型中一种独特且互补的模态,而不只是在行为或影像数据不可用时的备选方案。

arXiv cs.AI75

任务上下文塑造语言模型中组件的角色

大语言模型在不同任务中复用相同的注意力头,但识别出这些共享注意力头并不能解释它们在各个任务中的作用有何不同。共享脑区之间随任务而变化的交互是一个自然的灵感来源,促使我们从网络视角理解语言模型的计算。在本研究中,我们使用相同的图事实(即指明存在哪些有向连接的陈述),比较固定的模型组件在不同任务中的表现。我们测量干预如何影响注意力头和后续层的激活状态(影响力),以及如何影响答案(重要性)。在涵盖四个模型家族、参数量介于15亿至140亿的五个开放权重模型中,任务变化会重新分配影响力。在 Gemma-2-9B 和 Qwen3-14B 中,激活编辑会改变早期的一个内部激活向量,并在文字指令保持不变的情况下,切换模型读取的事实。随后,相同的注意力头会响应不同的图事实,并以不同方式影响后续组件。对于每个提示,我们构建两个线性响应映射,分别对应编辑前和编辑后。每个映射根据测得的早期注意力头变化,预测后续注意力头的响应。对于另一项针对事实的干预,与两个映射的平均值相比,使用对应条件下的映射可使 Gemma 的均方预测误差降低53.6%,使 Qwen 的均方预测误差降低5.0%。这些发现表明,任务上下文既会改变共享组件响应哪些图事实,也会改变其输出对后续计算的影响方式。基于网络的解释可能有助于说明模型如何利用共享参数执行不同的计算。对任务依赖性影响力的测量,也可以为旨在改变一种行为而保留其他行为的干预提供指导。

Hacker News AI38

Show HN: Synonymizer – an experiment in non AI text adjustment

Ever since hearing about the interesting key-based algorithmic approach to AI text watermarking (Google SynthID and the one I don't know the name of from Anthropic) I was curious about just swapping in synonyms. Built something fun to explore it. I suspect the watermarking from these labs actually goes off more than single-token assessments, and there's no public validator I can find to actually assess Synonymizer score against the watermark. But I thought it was fun to play with. Kind of an auto-mad-lib. :D Enjoy poking around with it on your Sunday afternoon Comments URL: https://news.ycombinator.com/item?id=50045321 Points: 1 # Comments: 0

The Decoder55

Google's Gemini 4 "Carbon" model is reportedly matching Anthropic's Opus 5.5 coding performance

Gemini 4 Argon isn't even widely available yet, and rumors about a more powerful version codenamed Carbon are already making the rounds. According to Business Insider, one employee compared Carbon's coding abilities to Anthropic's Opus 5.5. Meanwhile, new modes are showing up in the Gemini app and AI Studio, suggesting Google is gearing up for the broader Gemini 4 launch. The article Google's Gemini 4 "Carbon" model is reportedly matching Anthropic's Opus 5.5 coding performance appeared first on The Decoder .

MarkTechPost58

Google Cloud Launches Gemini Agent, One Universal Agent for Enterprise Work

Google Cloud has introduced the Google Cloud Gemini agent, a single agent for enterprise work. The Gemini agent is a cloud-hosted agent from Google Cloud that answers questions, does knowledge work, creates media, and writes and runs code. It does all of this from 1 prompt box and 1 API. For developers, the agent is […] The post Google Cloud Launches Gemini Agent, One Universal Agent for Enterprise Work appeared first on MarkTechPost .

MarkTechPost56

Google Research RRSI Guide: Mastering Self-Improving AI Agents

Explore a comprehensive coding guide to Google Research's RRSI (Regularized Recursive Self-Improvement), detailing how noise bands, cost rules, and leakage screens enable safe, efficient, and self-improving AI agents. The post Google Research RRSI Guide: Mastering Self-Improving AI Agents appeared first on MarkTechPost .

arXiv cs.LG77

真相从未消失:合规上下文真值探针中的完美别名

arXiv:2609.10739v2 公告类型:替换 摘要:有人提出使用从语言模型激活中解码真相的线性探针来监测欺骗;而当一个经过欺骗训练的模型上的真值探针得分低于随机水平时,人们自然会将其解读为模型隐藏了真相,或不再表征真相。我们表明,拟合标签歧义也能产生相同的读出结果。在一个受控博弈中,模型应向盟友报告一个秘密比特,向对手报告其补码。在合规(盟友)上下文中,真实比特与任务规定的答案是相同的标签,因此,在这些上下文上拟合的探针无法判断自己测量的是二者中的哪一个。我们将这种完全一致称为完美别名。在对手上下文中,这些标签互为补码,因此,一个在盟友上下文上拟合的探针分别针对这两个标签进行评分时,其对手 AUROC 之和为 1。将盟友和对手上下文结合起来进行混合拟合,会使两个标签产生差异;随机化的输出编码表也会使任务规定的答案与输出字母解耦。对于一个经过奖励训练、在所有评估的对手试验中都给出虚假答案的 Gemma-2-9B 策略,在最终层上,盟友拟合的探针 AUROC 为 $0.006 \pm 0.005$(三个 RL 训练随机种子上的均值 $\pm$ 样本标准差),而在相同留出激活上,混合拟合的探针得分为 1.000。混合拟合还使用了更多样本和带标签的对手数据,因此这表明真实比特仍可被线性恢复,而不是说明仅仅分离标签就能解释性能提升。在遵循指令的 Llama-3.1-8B 中,在一个提示变体上重新拟合的探针,以及一个从参考变体中获得并冻结的探针,在留出的盟友数据上准确率都为 1.000,但在相同试验上的对手真值 AUROC 分别为 0.080 和 0.986。我们的主要实验使用一种单一令牌的博弈,其中提示词明确给出了该比特,因此恢复出的方向可能读取的是模型保留的一份副本;而在模型必须推断该比特的情况下,没有任何受测实验条件能稳定地产生欺骗。我们研究的是探针测量了什么,而不是模型是否使用了这些信息。

arXiv cs.LG71

System-Prompt Conditioning and Hidden-State Geometry in Four Open-Weight Models: Corrections and What Survives

arXiv:2607.09842v3 Announce Type: replace Abstract: Versions 1 and 2 of this preprint reported that an identity-specifying system prompt leaves a geometric fingerprint in the final-layer hidden-state trajectories of four open-weight language models, and that instruction tuning moves this fingerprint from the direction to the magnitude of the hidden-state vector. An audit of their code and data found the following. The curvature statistic described as Ollivier-Ricci curvature on Euclidean k-NN graphs was a non-standard Forman-type edge statistic on graphs built from temporal and cosine k-NN edges. Its released test permuted pooled edges instead of trajectories, and the published p-values came from unreleased code. The quantity reported as the norm of the first generated state is the state at the last prompt position, from which the first output token is predicted. The generic control prompt was matched to the identity prompt in characters, not in tokens. This version corrects the methods, withdraws the regime-specific claims (one model per regime) and the direction-to-magnitude claim, and re-analyzes the data with added controls. What survives is narrower. Centroid distance, maximum mean discrepancy and a linear probe separate every pair of prompt conditions in every model, while the curvature statistic exceeds its split-half noise floor in only four of twelve comparisons. In Gemma-4-E4B-it this state has a lower norm under the identity prompt than under a token-length-matched generic prompt (138.1 vs. 216.5; Cohen's d = -5.45; n = 20). Its direction also separates the conditions, and the effect fits the state's role in planning the output: the identity prompt instructs a pause before every answer, and the model opens 98 of 100 responses with a pause marker. When the first token is fixed, the norm ordering reverses. The base model continues the prompt template instead of answering. A redesigned follow-up study is in preparation.

arXiv cs.LG78

停止探测,开始编码:为什么线性探针和稀疏自编码器无法实现组合泛化

arXiv:2603.28744v2 公告类型:替换 摘要:线性表征假说认为,神经网络激活将高级概念编码为线性混合。然而,在叠加表征下,这种编码是将高维概念空间投影到低维激活空间的过程,概念空间中的线性决策边界在投影后不一定仍保持线性。在这种情况下,采用逐样本迭代推断的经典稀疏编码方法可以利用压缩感知保证来恢复潜在因子。另一方面,稀疏自编码器(SAE)将稀疏推断摊销到一个固定编码器中,从而引入系统性差距。我们表明,这种摊销差距在不同训练集规模、潜在维度和稀疏度水平下持续存在,导致 SAE 在分布外(OOD)组合偏移下失效。通过分解这一失败过程的受控实验,我们确定字典学习才是限制因素,而非推断过程:SAE 学到的字典指向了显著错误的方向,即使在同一字典上用逐样本 FISTA 替代编码器,也无法弥合这一差距。一个神谕基线证明,在所有测试尺度上,只要拥有良好的字典,该问题都是可解的。我们的结果(包括使用真实 LLM 激活的实验,如 Pythia-70M 和 Gemma-2-2B)重新界定了 SAE 失败的性质:这不是推断问题,而是字典学习挑战;同时也表明,可扩展的字典学习是在叠加表征下实现稀疏推断的关键开放问题。

arXiv cs.LG83

NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents

arXiv:2610.11030v1 Announce Type: cross Abstract: Tool-using LLM agents violate the policies they are deployed to enforce, often silently. Prior defenses hand-write rules, query an LLM verifier per action, or compile policies through heavyweight formal machinery. Naive compilation fails: extracted rules block the tool satisfying their own precondition, or read arguments their tool lacks. NOMOS, a four-pass compiler, turns a natural-language policy into a deterministic tool-call gate; static verification with tool-schema-level checks alone (no prover, solver, or LLM) repairs or rejects 37% (airline) and 13% (retail) of candidates, without which most shipped rules are inoperable. Replaying compiled rules over undefended transcripts flags bindings that refuse legitimate work (a development binding refused 95.9% of task-passing calls); no evaluation binding is flagged. On $\tau^2$-bench the gate cuts violations of reference-encoded clauses among state-changing calls from 66.3% to 2.6% (airline) and 30.8% to 6.9% (retail), raising airline task success significantly for $2 \le k \le 4$; a 26B on-premise compilation is not significantly worse than hand-written or frontier-compiled rules. Unlike AgentDojo's shipped defenses, it reaches a zero attack success rate (ASR) on banking, where nine attack families collapse onto three structural rules. On the other three suites its ASR is at most 3.6%, from goals with no tool call to govern and one write admitted by a binding weaker than its clause; a second agent model, Llama-3.3-70B, reproduces the effect on both benchmarks. Decisions take microseconds without an LLM call, at a domain-dependent benign-utility cost; compilation runs on-premise on open-weight gemma-4-26B.

arXiv cs.LG78

基于人类听众强化学习的对话式语音美学模型

arXiv:2610.10868v1 公告类型:cross 摘要:我们介绍了 Conversational Voice Aesthetic Model,这是一种语音大语言模型,用于描述自然对话场景中真实或合成语音回复的语音美学特征。给定上下文和回复语音,CVAM 会描述刻画声音特征的显著片段,并预测涵盖性别、音高、语速、情绪和表达方式的九项分类属性。其关键挑战在于情绪和表达方式等感知领域,这些领域本质上具有主观性,且缺乏明确的真实标签。因此,我们从 CANDOR 语料库中获取了 3,000 条真实和合成语音回复,并为每条回复收集了约 10 条人工标注。CVAM 首先基于合成的美学描述和标签进行监督微调,随后利用 Group Relative Policy Optimization 根据人类判断进行优化。实验表明,与 Gemini 3.1 Pro 和开源语音大语言模型相比,CVAM 与人类听众的判断更加一致,并且优于单个人类与其他所有人的一致性。总之,我们证明了以人类感知为基础来定义语音美学的重要性,并提出了一个系统的人类对齐框架。

arXiv cs.LG79

Amortized Off-Policy Evaluation for LLMs

arXiv:2610.10848v1 Announce Type: new Abstract: Accurate evaluation is central to selecting which LLM to deploy, yet testing a candidate on live traffic exposes real users to an unvetted model. Teams therefore evaluate candidates offline, on data produced by already-deployed models. This is off-policy evaluation (OPE), and it faces two distribution shifts: as a model is updated in post-training, its responses diverge from the logged ones (policy shift), and the reward definition under which it is judged changes with business requirements (reward shift). Classical OPE methods are ill-suited to this continual-deployment setting because they are defined per task and require fitting from scratch on every new logged dataset or reward definition. To address this, we propose PFN-OPE, a prior-data fitted network that amortizes OPE across a distribution of contextual-bandit tasks. We pretrain it once on tasks constructed from a pool of LLM responses scored by several reward functions, in which both shifts occur. At test-time it maps a logged dataset and one sampled target response per prompt to a value estimate in a single forward pass, with no per-task fitting. On HelpSteer2 and UltraFeedback with Qwen, Llama, and Gemma policies, PFN-OPE achieves 2.0 to 9.3 times lower error than the best baselines across all tested configurations in the reward-shifted settings.

arXiv cs.LG82

覆盖范围而非难度决定激活探针需要多少合成数据

arXiv:2610.10594v1 发布类型:新 摘要:用于监测已部署语言模型的激活探针是在合成对话上训练的,而探针究竟需要多少合成对话仍是一个开放问题。我们针对三种监测概念——高风险情境、对人有害的回复,以及不遵循用户指令的回复——在十四个留出的评估分布和四种探针模型上,追踪了使用 10 至 590 个合成样本时的学习曲线,同时改变生成器 LLM 和提示词的详细程度。所需样本量取决于监测对象:在 Gemma-3-27B-IT 上,高风险探针和有害性探针使用 80 个样本时就已达到其平台期水平上下几个百分点以内,而指令遵循探针所需样本量是其数倍;在三个更小的探针模型以及真实样本(来自开发集)上,这一排序仍然成立。已有研究建议,应将生成预算用于广度,即更多种类的数据,而非深度,即每种数据生成更多样本。我们将某个概念所需的深度定义为拟合曲线的半增益规模,即达到全部增益一半时所需的样本数量。其方差的 42% 至 45% 由概念和分布解释,而生成器、探针模型和提示词详细程度合计解释不到 10%。决定半增益规模数值的是覆盖范围,而不是每种数据的难度:也就是某个分布为达到饱和所需的同类样本数量。在三个概念下,每个评估分布对应的每种数据,其同类合成样本的半增益规模中位数都为 7 至 11 个。不同之处在于,一种类型的样本能够在多大程度上迁移到同一概念下的其他类型:在高风险概念下几乎可以完全迁移,在有害性概念下较少,而在指令遵循概念下最少;这解释了生成样本和真实样本中概念之间大部分的差距。因此,数据广度对不同概念的作用并不相同:在指令遵循概念下,没有任何一种类型能够覆盖另一种类型,因此必须使用多种类型;而在高风险概念下,一种类型即可覆盖其余类型,多种类型几乎是冗余的。我们公开评估套件、开发集和生成数据集。

arXiv cs.AI82

从被动遏制到主动保障:OpenAI、Anthropic 和 Google 智能体安全事件的经验教训

2026 年,涉及 OpenAI、Anthropic 和 Google 智能体的网络安全评估触达了授权测试范围之外的真实系统。它们所采取的路径各不相同。OpenAI 智能体利用了研究基础设施,在不同运行之间进行协调,并入侵了 Hugging Face 生产环境的部分区域。Anthropic 报告了若干案例:由于第三方环境配置错误,执行模拟网络任务的智能体得以接触真实系统。在另一项单独报道的评估中,Google 的 Gemini 通过一条非预期的互联网路径访问了三个真实组织;Google 表示,该模型在三种情况下都停止了操作。综合来看,这些案例说明,评估不能依赖假定的边界。该边界必须在智能体运行过程中得到验证。本工具性比较案例研究提出了 Proactive Agent Security Assurance Cycle (PASAC) 和五层 Boundary Assurance Stack。该框架结合了按风险分级的任务设计、可执行的范围契约、运行前验证、最小能力访问、独立出口控制、凭证限制、跨运行监控、自动停止条件以及基于证据的重新授权。一个先行指标模型、九项设计命题和七个可证伪假设,将这些经验教训转化为一项可检验的研究计划。由于公开的 Gemini 记录仅限于有明确归属的陈述和新闻报道,其详细因果机制仍属暂定。核心结论很明确:主动式智能体安全需要贯穿完整执行系统的持续保障,而不能依赖对任何单一沙箱或安全措施的信心。

TechCrunch AI72

Google 将代理式 AI 引入 Gemini,首先面向企业

Google 正在将 Gemini 打造成一种 AI 代理,能够进行规划、执行任务,并跨企业应用和系统协同工作。该代理可以将工作委派给子代理,使用多个 AI 模型,甚至拥有自己的工作场所身份,包括一个电子邮件地址。

The Verge AI74

Google 的 AI 笔记应用可完全离线转录会议

正如 TechCrunch 此前报道的那样,Google 发布了一款实验性笔记应用,可完全离线转录会议和音频文件。这款名为 Google AI Edge Foresight 的应用可免费使用,并在 macOS 上运行,使用该公司的设备端 EmbeddingGemma 2 模型。与 Granola 和 Wispr Flow 等 AI 笔记应用类似,Foresight 会总结 […]

The Verge AI72

Google 正在推出一款用于工作任务的一站式 Gemini 智能体

Google 正在推出一款“通用”Gemini AI 智能体,可在后台跨应用和设备运行。这款工具于周四举行的 Gemini at Work 活动上公布,将在 Gemini Enterprise 应用中提供,用户可以通过单一界面与 Gemini 智能体聊天并向其分配任务。[…]

SiliconANGLE AI60

Google Cloud introduces Gemini agent to change enterprise work

Google LLC’s Cloud division today introduced Gemini agent, a unified artificial intelligence assistant that can act autonomously, generate code and complete work across any device, including web, mobile and desktop. The new agentic AI experience can be accessed anywhere and through any channel, including the command line, Google Workspace, Microsoft 365 or Slack. It even […] The post Google Cloud introduces Gemini agent to change enterprise work appeared first on SiliconANGLE .

arXiv cs.LG69

Forward-Deployed Full-Stack Engineering for Autonomous Cloud MLOps

arXiv:2608.29615v2 Announce Type: replace-cross Abstract: Across industries, machine-learning systems support applications ranging from prediction and anomaly detection to forecasting, optimization, and scheduling, yet operationalizing these systems requires coordinating application development, model pipelines, cloud infrastructure, security, deployment, monitoring, retraining, recovery, and rollback. We present an evidence-gated multi-agent framework for transforming a natural-language MLOps cloud engineering task into a verified repository and operational cloud deployment. The framework combines graph engineering, loop engineering, and agent harness engineering. A stateful Graph Orchestrator coordinates specialized agents for repository generation, review, execution, verification, release, and monitoring while governing workflow dependencies, evidence gates, retry bounds, recovery paths, and termination. Consequential lifecycle transitions proceed only when their required predicates are supported by verifiable execution or runtime evidence. Verification failures activate bounded reflection, repair, and re-verification, while runtime evidence of failure, drift, degradation, or policy violation can trigger bounded adaptation, recovery, or rollback. Agent harness engineering constrains repository generation, review, and repair, artifact execution, and cloud operations through controlled capabilities and isolated execution environments. We realize the framework on Google Cloud Platform and evaluate repository completeness, controlled execution, evidence-gated transitions, cloud promotion, and bounded recovery. Our experimental results show that the framework prevents unsupported lifecycle transitions and drives each run toward either a verified operational deployment or an auditable terminal failure.

arXiv cs.LG82

支持忠实性的积极证据:LLM 自我解释有助于预测模型行为

arXiv:2602.02639v2 公告类型:replace-cross 摘要:LLM 自我解释常被视为 AI 监督的一种有前景的工具,但人们对其是否忠实于模型真实推理过程仍知之甚少。现有的忠实性指标存在关键局限,通常依赖通过对抗性提示识别不忠实之处,或检测推理错误。这些方法忽视了解释的预测价值。我们提出归一化可模拟性增益(NSG),这是一种通用且可扩展的指标,其核心思想是:忠实的解释应使观察者能够学习模型的决策标准,从而更好地预测模型在相关输入上的行为。我们使用涵盖健康、商业和伦理领域的热门数据集中的 7,000 个反事实样本,对 18 个前沿专有模型和开放权重模型(例如 Gemini 3、GPT-5.2 和 Claude 4.5)进行了评估。我们发现,自我解释能显著提升对模型行为的预测能力(NSG 为 11%–37%)。即使外部模型更强,自我解释所提供的预测信息也多于由外部模型生成的解释。这表明,自我认知带来了一种外部解释方法无法复制的优势。我们的方法还揭示,在不同模型中,有 5%–15% 的自我解释具有高度误导性。尽管自我解释并不完美,我们仍证明了支持自我解释的积极证据:它们编码了有助于预测模型行为的信息。

SiliconANGLE AI63

Google debuts SynthID Detector tool for flagging AI-generated content, but it’s far from perfect

Google LLC is making its SynthID identification tools for artificial intelligence-created and AI-edited videos and images more widely available with the launch of a new service today that everyone can use. The web-based SynthID Detector can identify images, video and audio clips that have been created using AI models from Google, OpenAI Group PBC, Nvidia […] The post Google debuts SynthID Detector tool for flagging AI-generated content, but it’s far from perfect appeared first on SiliconANGLE .

arXiv cs.AI76

RECAST:通过自适应证据路由学习计算恰当的上下文

大语言模型正日益广泛地应用于以冗长、异构信息源为基础的任务。传统的检索增强生成(RAG)依赖基于相似度的固定检索方式,而智能体式变体虽然能够调整查询和工具使用,但总体上仍以检索为中心。然而,在许多任务中,求解所需的证据并未明确存在于任何单个信息源条目中,而是必须通过对多个信息源条目进行筛选、聚合或计算才能推导得出。在本研究中,我们提出 RECAST(Routing Evidence through Computation, Access, and Synthesized Tools),这是一个通过学习构建的框架,将证据构建表述为围绕异构检索与计算操作展开的序贯决策过程,使证据能够被主动推导,而非仅仅被检索。轻量级 RouterLM 以迭代方式选择并构造基本操作,或指定定制操作,由参数冻结的 CompilerLM 将其转换为可执行代码。一旦判定证据充分,RouterLM 就会将采纳的证据传递给参数冻结的 AnswerLM,以生成最终解答。我们先通过监督微调(SFT),再通过组相对策略优化(GRPO)训练 RouterLM。在六类异构基准测试中,RECAST 的平均成功率达到 75.6%,比最强的大模型基线高出 15.9%。此外,经过训练的 Qwen3.5-9B RouterLM 比未经训练的 Gemini 3.5 Flash RouterLM 高出 5.0%。在三个未参与训练的基准测试中,RECAST 比最强基线平均提升 15.0%,展现出跨任务和异构信息源表示的强大零样本泛化能力。

arXiv cs.AI74

将逻辑与人设解耦:边缘端 LLM 智能体对上下文污染的结构性免疫

边缘设备上的小型语言模型智能体必须在同一个充满对话历史和人设指令的上下文窗口内,同时维持人设并进行正确推理。我们研究了当对话历史冗长、具有误导性且充斥人设内容时,这类智能体的逻辑部分会受到什么影响(人设—逻辑干扰),并提出一种解耦架构(AO-DA):在一个配备可热切换 LoRA 适配器的 INT4 基础模型上,将逻辑推理(“做什么”)与人设表达(“如何表达”)分离为两条推理路径。逻辑路径仅接收当前轮次的核心内容,并输出可验证的结构化状态(Micro-State);人设路径则结合完整历史,以符合角色设定的方式呈现该状态。在 Apple M2 笔记本电脑上开展的同基础模型消融实验中(Llama-3.1-8B-Instruct 和 Gemma-3-4B-it,4 位量化;4 个污染级别 × 3 个实验组 × 2 项任务 × 2 种人设 × 5 个随机种子,共 480 次运行),我们发现:(i) 解耦后的逻辑路径在结构上不随污染程度变化:其提示词长度始终保持为 180(Llama)或 167(Gemma)个 token,而混合式单次推理的提示词长度从 242 增至 1,203 个 token;其输出在各污染级别下逐字节一致(40/40);(ii) 混合式单次推理的表现随污染程度增加而单调下降(Llama 的综合逻辑得分从 0.669 降至 0.150,Gemma 从 0.487 降至 0.150),主要原因是未能输出所要求的结构化内容(在最高的两个污染级别下,Llama 有 80–95% 的运行出现该问题,Gemma 则为 100%);(iii) 当向解耦后的逻辑路径输入相同污染内容时,采用专用适配器和专用格式的路径在 8B 模型上仍比单次推理更稳健(失败率为 0–20%,对比组为 80–95%;配对 $Δ$ 为 +0.30 至 +0.50,Cliff's $δ$ 为 0.50–0.85,Holm 校正后的 $p \le 0.03$),但在 4B 模型上并非如此,两种方式均告崩溃。分离的代价是在一个话题的首轮中增加一次解码(Llama 上为 28.2 秒,对比组为 18.2 秒),换来的能力是无需重新运行逻辑路径,即可在 1.7 毫秒内完成人设热切换。代码、评分标准、测试夹具、适配器和日志均已公开。

Hacker News AI14

What is the actual percentage of people For and Against AI?

Ive started to think about it lately, a google search just doesn't yield a satisfiable result. What do you guys think ? How many people are actually against AI? Sometimes it feels like the majority are starting to be skeptical, but maybe that's just the environment im surrounded by. Comments URL: https://news.ycombinator.com/item?id=50000356 Points: 3 # Comments: 7

AWS Machine Learning Blog72

借助 Amazon Quick 和 Amazon Bedrock 重新思考 RAG 的访问控制

企业级 RAG 能够从 SharePoint、Google Drive 和 Confluence 等知识源中挖掘洞察,但这些知识源的权限体系十分复杂。了解 Amazon Quick 和 Amazon Bedrock Knowledge Bases 如何实时实施文档级访问控制,在查询时直接向权威来源核验权限。

The Decoder79

Google 表示,已有 1800 亿张图片和段视频带有 SynthID 水印,检测工具现已向公众开放

Google 的 AI 水印检测工具 SynthID 现已向公众开放。任何人都可以检查图片、视频或音频是否由 Google 的 AI 或 OpenAI、Nvidia 等合作伙伴的 AI 创建。目前已有超过 1800 亿项内容带有该水印,但该工具只能识别带有 SynthID 标记的内容,无法普遍识别 AI 生成的媒体内容。本文《Google 表示,已有 1800 亿张图片和段视频带有 SynthID 水印,检测工具现已向公众开放》首发于 The Decoder。

The Decoder72

Zuckerberg 的 Biohub 牵头推进一项耗资 18 亿美元的计划,旨在构建预测细胞行为的 AI 模型

由 Mark Zuckerberg 和 Priscilla Chan 支持的研究机构 Biohub 正在协调一项耗资 18 亿美元的计划,以训练能够预测细胞行为的 AI 模型。Meta、Google DeepMind、Isomorphic Labs 和美国能源部正在为数据、实验室设备及算力提供资金。首个数据集预计将在约一年后准备就绪。《Zuckerberg 的 Biohub 牵头推进一项耗资 18 亿美元的计划,旨在构建预测细胞行为的 AI 模型》一文最先发表于 The Decoder。

Hacker News AI19

Coding and Agentic Capabilities Are Now More Important Than Ever for AI

I have written a deep dive into this topic in my website Nasengetu. You can go on Google, search for Nasengetu Tech, and read the latest article. If it doesn't appear, go to the News section of the website. Comments URL: https://news.ycombinator.com/item?id=49995428 Points: 2 # Comments: 0

The Verge AI71

Google 投资数百万美元,支持马克·扎克伯格打造“虚拟细胞”的计划

据路透社报道,Google DeepMind、Meta 和 AI 药物研发初创公司 Isomorphic Labs 正联合向 Biohub 投资 3 亿美元。Biohub 是由马克·扎克伯格及其妻子普莉希拉·陈创立的非营利生物医学研究机构。这笔资金是一项总额为 18 亿美元的计划的一部分,该计划旨在构建 AI 数据集,让研究人员能够“提问、预测,[…]

The Decoder69

Google bets Gemini can turn casual players into game developers with new Playground feature

Google launched Playground, a browser-based platform that lets users create their own games using nothing but text prompts and no coding skills. The platform runs on Gemini, Nano Banana, and Lyria. Alongside Playground, Google and Unity announced Unity Spark, a more professional AI tool with a beta planned for 2026. The article Google bets Gemini can turn casual players into game developers with new Playground feature appeared first on The Decoder .

Hacker News AI60

Google Chrome's AI API actively used for device fingerprinting and profiling

On November 17th I will publish a research paper into the current state of play for online privacy. In order to conduct the research I carried out forensic audits of 100 of the top english language content websites (no B2B or SaaS - just consumer content sites) monitoring all browser events and capturing payloads. Even though I was not specifically looking for AI API events, the results are quite alarming. Around 30% of the sites audited are using the AI API for either device fingerprinting, profiling (Google Advertising) or both. This is despite Google assuring the world that the API would not create any privacy risks after concerns we raised when they announced their plan to add the API. The rest of the findings are equally shocking and out of the 100 sites audited - every single one of them is in breach of EU law (tort, regulatory and criminal) and almost definitely in breach of various US and other laws across the world. Comments URL: https://news.ycombinator.com/item?id=49990335 Points: 1 # Comments: 0

arXiv cs.LG74

SAE++:级联稀疏自编码器在多模态大语言模型中学习多层级视觉概念

arXiv:2606.16193v2 公告类型:replace-cross 摘要:多模态大语言模型(MLLMs)在视觉语言任务上表现出色,但其内部视觉表征仍然难以解释。稀疏自编码器(SAEs)提供了一种可扩展的方法,可将稠密的模型激活分解为稀疏、可解释的特征。然而,现有 SAE 架构主要学习扁平化的特征字典,不太适合显式地组织多层级概念。本文提出一种名为 SAE++ 的级联稀疏自编码器架构,用于学习 MLLMs 中的层次化视觉概念。SAE++ 并非嵌套或堆叠 SAE 的稀疏激活编码,而是直接以第一级 SAE 的解码器权重训练第二级 SAE,将学到的低层级特征方向作为更高层级抽象的输入。这一设计使 SAE++ 能够学习“概念的概念”,同时避免嵌套式 Matryoshka 层次结构中共享前缀耦合带来的弊端,以及简单堆叠 SAE 所产生的瓶颈。在多个视觉数据集上针对 Qwen3-VL、Gemma-3 和 LLaVA 开展的实验表明,与当前最先进的 SAE 基线相比,SAE++ 在层次化概念一致性方面提升了可解释性。概念引导实验的结果进一步表明,学到的概念组能够支持对 MLLMs 输出进行有效的组级干预。代码已发布于 https://github.com/Wang-ML-Lab/sae-plus-plus。

arXiv cs.LG77

QUASAR:通过损失感知重建降低量化感知训练的损失下限

arXiv:2608.13966v2 公告类型:替换 摘要:随着大语言模型推理转向更低精度,训练后量化(PTQ)变得越来越脆弱,因此量化感知训练(QAT)对于保持模型质量至关重要。然而,QAT 存在一种结构性失配:梯度更新应用于潜在的全精度权重,而损失和梯度却基于这些权重的有损重建结果计算。这种失配可能导致次优的训练轨迹和更高的损失下限。二阶 PTQ 方法通过最小化损失感知重建误差来解决类似问题,但在 QAT 期间,随着权重不断变化,反复执行这种计算成本高昂的重建并不切实际。我们提出 QUASAR,这是一种将轻量级损失感知重建引入训练循环的 QAT 方法。在每个训练步骤中,QUASAR 通过搜索一小组裁剪范围,并利用显著性加权最小二乘法拟合反量化参数,来重建潜在权重。我们使用梯度平方的指数移动平均值作为每个参数的显著性信号。理论分析表明,优化 QUASAR 的重建目标能够同时收紧 QAT 的收敛界和最终损失界。我们在四个模型系列以及 INT4、INT3、INT2 和 NVFP4 量化格式上评估了 QUASAR。与具有竞争力的 QAT 方法相比,QUASAR 始终取得更低的训练损失和评估损失,并在下游基准测试中优于 QAT 和 PTQ 基线。在 INT2 精度下,相较于最佳 QAT 基线,QUASAR 使用量化感知蒸馏时将平均准确率提高了 13.3 个百分点,在数学推理数据上进行 QAT 时则提高了 10.9 个百分点。值得注意的是,仅使用约 6 亿个 token 进行蒸馏后,QUASAR 的 INT4 Gemma-4 E4B 检查点就优于 Google 发布的对应 QAT 检查点,KL 散度降低了 66%,平均准确率提高了 1.8 个百分点。

arXiv cs.LG74

使用 Balanced Fine-Tuning 将 LLM 与生物医学知识对齐

arXiv:2511.21075v4 公告类型:替换 摘要:通过工程化改造 LLM 来加速生命科学研究,需要使其与生物医学知识稳健对齐。我们观察到,生物医学文本的不确定性结构与通用文本存在根本差异:密集的低置信度连续片段反映的是认知性知识缺口(密集的因果链、罕见实体),而非通用文本中典型的稀疏偶然性文体变化。基于这一发现,我们提出 Balanced Fine-Tuning(BFT),这是一种双尺度后训练方法,将基于组内归一化的词元重新加权与序列级重分配相结合,使训练资源向呈现密集认知不确定性的知识密集型样本倾斜。在相同的训练设置下,BFT 在医学评估、生物学推理、稀疏奖励强化学习和生物学表征任务中,相较于 SFT 和 DFT 带来了更一致的性能提升。当替换 GeneAgent(GPT-4o)和 VCWorld(Gemini-2.5-Flash)中默认的闭源骨干模型时,经 BFT 对齐的 70B 模型在生物学过程推理和化学扰动预测中均表现更优。关键的是,所有 BFT 变体在后续采用稀疏奖励的 GRPO 训练后均进一步提升,而 SFT 和 DFT 的性能则出现下降,这表明感知认知不确定性的后训练能够提供更稳健的策略初始化。除文本生成之外,经 BFT 对齐的 LLM 还能生成更准确、更专业的生物医学特征描述文本;使用文本嵌入模型对这些描述进行编码后,所得表征可支持基因级、细胞级和扰动响应任务,这表明 BFT 增强的生成能力能够促进生物学表征,进而助力更广泛的生物医学下游任务。

arXiv cs.LG80

40亿参数开放模型对随机试验给出的校准回答:一项预注册测试与许可合规的发布

arXiv:2610.07019v1 公告类型:交叉收录 摘要:Fiorillo v0.5 是一个开放模型,可回答类型化问题,并为每个答案给出概率。其主要专用模块读取截取至6,144个词元的随机试验论文,并回答与对照相比,某项干预是否显著增加、显著减少或未显著改变某个结局(Evidence Inference 2.0,EI)。该模型基于 Qwen3-4B-Base,配有低秩适配器和决策头,仅使用2,657篇训练论文中自身许可证允许再利用的1,431篇进行 EI 微调。在生成此版本的测试预测之前,研究者在 Open Science Framework 上注册了四项标准,以决定是否发布模型,其中第二项门槛在标签公开的 EI 测试集上评判。在该测试集(333篇论文中的1,218条提示)上,预期校准误差为0.0168,低于0.05的上限;对数损失比先验基线低0.8603(95%区间为0.8104至0.9078),比读取相同输入的 Gemma 4 31B-it 低0.1829(0.1164至0.2598);宏平均 F1 为0.9248,对比值为0.8668,因此四项标准全部通过。仅在许可合规的论文上采用相同方案训练,使准确率下降了0.0123(0.0034至0.0207;描述性结果)。不提供论文时,宏平均 F1 降至0.4384;仅提供标题可使其提高0.0939(0.0655至0.1234),这可能是因为标题陈述了结果,或模型记住了该试验;交换干预与对照后,其方向性回答中有0.6652发生反转。按发布配置运行时,发布文件生成的预测与评估时的预测之间的差异处于预先设定的范围内。本次发布采用 Apache License 2.0(数字对象标识符:10.57967/hf/10722)。

arXiv cs.LG74

利用自回归后训练权重增强扩散语言模型

arXiv:2610.08108v1 公告类型:新 摘要:扩散语言模型(dLLMs)已成为自回归(AR)语言模型的一种有前景的替代方案,能够提供灵活的 token 更新顺序和并行解码。近期的 dLLMs 通常在进行扩散转换之前从预训练 AR 模型初始化,以继承其已学习的表征。然而,在转换完成后,它们通常会忽略其 AR 祖先模型所拥有的丰富后训练生态。在本工作中,我们表明,这些现有的 AR 后训练权重更新可以被有效复用,以增强扩散模型。尽管 AR 到扩散的转换会带来变化,但将 AR 后训练权重更新直接添加到扩散基础模型上仍然有效,其性能接近直接进行扩散后训练所达到的水平。值得注意的是,AR 后训练更新与扩散后训练更新在权重空间中几乎正交,但会在扩散模型中引发高度一致的表征变化。二者不同的更新还具有互补性:组合它们的权重可以同时保留两种范式带来的收益,并进一步提升经过后训练的扩散模型。基于这些发现,我们提出 A2D,这是一种利用现有 AR 后训练资源增强扩散模型的简单免训练框架。A2D 可以将能力从经过 AR 后训练的模型迁移到扩散基础模型中,还可以通过组合 AR 与扩散后训练更新,进一步改进已经过后训练的扩散模型。在包括 Dream、DreamReasoner、DiffuCoder、Dream-Coder、Nemotron-Labs-Diffusion 和 DiffusionGemma 在内的多种 dLLMs 上,A2D 均能稳定提升指令遵循、数学推理和代码生成能力;无论采用监督微调还是强化学习更新,都无需额外训练或推理时计算。

arXiv cs.LG77

MASKerade:用于密集模型到 MoE 升级的令牌路由掩码专家

arXiv:2610.07809v1 公告类型:新 摘要:稀疏激活的专家混合(MoE)模型能够提升模型容量,而无需按比例增加每个令牌的计算量。密集模型到 MoE 的升级通过复用预训练密集模型来构建此类系统,通常的做法是将前馈网络(FFN)复制为独立训练的专家。我们提出 MASKerade,这是一种密集模型到 MoE 的训练方法,其专家不再通过复制获得,而是学习为冻结的预训练 FFN 的稀疏子网络。每个专家由一个学习得到的二值掩码定义,并由令牌级路由器选择要执行和组合的掩码 FFN。路由器和掩码分数进行联合优化,而底层 FFN 的权重值保持不变。这一形式能够在同一套路由架构中支持神经元结构化、半结构化和非结构化专家。我们的主要配置采用四个 2:4 专家,并使用 top-2 路由;其中两个半稠密专家的计算过程在名义上的 FFN 算术量上相当于一次稠密计算过程,同时无需独立的专家权重矩阵。在使用 Qwen 和 Gemma 作为骨干网络的五个视觉语言基准测试中,该配置取得了所有对比基线中的最高性能。通过对不同掩码粒度、路由干预方式以及计算量匹配的控制实验进行比较,我们区分了学习得到的连接方式与专家激活数量各自的影响。这些结果表明,在冻结权重上进行掩码学习,是构建令牌路由 MoE 专家的一种实用替代方案。

Hacker News 熱門

Ask HN: How would you feel if we nationalized Google?

Not enough space in the title, the actual scenario is this: A hypothetical US gov't, benevolent and markedly less corrupt than the current one, passes a Constitutional amendment to nationalize Alphabet, with the express intent to return Google Search to 2010 levels of efficacy and to remove all LLM functions from both its frontend and backend. I guess what I'm really asking is what people would abide to have the internet back, via a search solution that actually works and whose usefulness hasn't been annihilated by ~15 years of cynical corporate marauding and product torture. Comments URL: https://news.ycombinator.com/item?id=49985180 Points: 4 # Comments: 5

SiliconANGLE AI66

Google expands EmbeddingGemma beyond text to images, audio and video

Google LLC today released EmbeddingGemma 2, an open multimodal embedding model small enough to run on a smartphone. The release takes the EmbeddingGemma line beyond text, which was all the first version handled when Google introduced it in September 2025. Images, audio and video now share one embedding space with text. An app built on […] The post Google expands EmbeddingGemma beyond text to images, audio and video appeared first on SiliconANGLE .

Simon Willison61

EmbeddingGemma 2

My comment on EmbeddingGemma 2 — Hacker News. I really appreciate that EmbeddingGemma 2 is under the Apache 2.0 license. For embedding models in particular, I don't think it makes sense to use a closed, proprietary, hosted-only model. Most applications of embedding models involve calculating thousands or even millions of embedding vectors and storing them for later comparison. If your model is proprietary, the vendor is likely someday going to decide to stop offering that model. They'll have a better model to replace it, but you still need to pay to re-calculate those millions of stored existing vectors. (In April 2024 OpenAI offered to "cover the financial cost of users re-embedding content with these new models" - https://openai.com/index/gpt-4-api-general-availability/ - but I don't think that's something we can rely on from every provider.) Notably, I don't want to host the model myself . I'd much rather pay a provider for a hosted model while knowing that if they ever stop hosting it I can run the open weights version myself - or find another vendor who can do that for me. Tags: google , ai , generative-ai , embeddings , gemma

The Decoder70

Google claims EmbeddingGemma 2 outperforms rival embedding models twice its size

Google released EmbeddingGemma 2, an open model with 740 million parameters that converts text, images, video, audio, and code into vectors. It runs on-device, needs only about 191 MB of RAM, and outperforms some competing models twice its size, according to Google. Paired with a small open model like Gemma 4, it can run offline RAG apps without sending data to external servers. The article Google claims EmbeddingGemma 2 outperforms rival embedding models twice its size appeared first on The Decoder .

Simon Willison51

Mistral Large 4

My comment on Mistral Large 4 — Hacker News. wren6991 : The benchmark is saturated. Frontier models are tested with an armadillo in fishnet tights jaywalking on Mars. OK well I couldn't resist this one: llm -m claude-opus-5.5 'Generate an SVG of an armadillo in fishnet tights jaywalking on Mars' llm -m gpt-6.1-sol 'Generate an SVG of an armadillo in fishnet tights jaywalking on Mars' llm -m gemini-3.8-flash 'Generate an SVG of an armadillo in fishnet tights jaywalking on Mars' llm -m mistral/mistral-large-4 'Generate an SVG of an armadillo in fishnet tights jaywalking on Mars' Default reasoning levels for each: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

The Decoder66

Google's new image model Nano Banana 2.1 generates better images for less money

Google's new Nano Banana 2.1 image model uses Gemini 3.6 Flash and beats the previous Pro model in some benchmarks at a lower cost. But its predecessor also scored well in tests, while Pro often produced better images in practice. The article Google's new image model Nano Banana 2.1 generates better images for less money appeared first on The Decoder .

Hacker News AI62

Show HN: HieraticBench – Can AI read ancient Egyptian handwriting?

I'm obsessed with Egyptology. During covid, I wanted to get a hieratic tattoo. For those who don't know, hieratic is the cursive form of hieroglyphs that was used in day-to-day life in ancient Egypt. It's as old as hieroglyphs. I then went on a mission to have a sentence translated into hieratic. Luckily, in 2022, I was able to commission an Oxford Egyptologist to write it for me. After that, I took it to a hieratic specialist I found on Instagram to redraw it with "biblical" accuracy. When generative AI came out, I realized that models very rarely identify hieratic and almost never translate it correctly. Every single time a new model came out, I ended up testing my sentence and hieratic texts in general, and I realized this could actually become a benchmark. Since so many benchmarks are becoming saturated, we need to find weird ways to test new models. That's why I decided to build HieraticBench. The first version of the benchmark has 268 items, including real documents, signs, and other Egyptian scripts, which I used as controls, as well as my unpublished sentence in two different renditions. The current state-of-the-art foundation models are hilariously bad at identifying my sentence, especially the one the Egyptologist wrote with her pen. Some of them said Tibetan, Urdu, Korean, and, funnily enough, "Reformed Egyptian," which is from the Book of Mormon. On real documents, the best model in the benchmark identified hieratic correctly 95% of the time, and the best score for reading individual signs was around 13%. That's when we tell the model specifically that it's hieratic. None of them can really translate a sealed sentence that's never been published and has no answer key. I haven't run every model on every task just to keep costs down. My next steps were to run Astra and Gemini 3.1 Pro on real documents. Currently, only the Claude models have done the sign readings. And just FYI, I have no experience whatsoever running benchmarks or evaluations, so not 100% sure I scored things right. Things we need: - Running the models I haven't covered - More sealed sentences and labeled signs If you know hieratic or have an Egyptologist friend, reach out at hieraticbench@veeza.ai. Source code: https://github.com/alymoursy/hieraticbench I'll be around to answer questions. Comments URL: https://news.ycombinator.com/item?id=49982822 Points: 2 # Comments: 0

The Verge AI77

Google 即将取消 Gemini Flash 和 Pro 的免费访问

从 10 月 9 日起,所有使用免费方案的 Google Gemini 用户将只能使用 Flash Lite 模型。目前,免费用户可以在 Gemini Flash Lite、Flash 和 Pro 之间进行选择,但现在需要订阅每月 4.99 美元的 Google AI Plus 才能访问标准版 Flash 模型。不过,该订阅也将迎来一些 […]

arXiv cs.AI74

One Figure, Every Canvas: Editable Flowchart Relayout via Agentic Pipeline

Pipeline figures in ML papers must be repurposed across many canvases, including paper columns, 16:9 slides, portrait posters, 1:1 social teasers, 9:16 phone previews. Each format imposes a different aspect ratio on the same computational graph, where any silently broken connection misrepresents the method. We formulate aspect-ratio-adaptive flowchart relayout as a distinct task: given a raster flowchart and a target ratio, produce a structurally faithful, hallucination-free, editable layout. Existing methods fail characteristically: image-to-image models stretch blocks and reject extreme ratios, text-to-image agentic systems hallucinate content, and parse-then-render systems mis-route edges. We propose an agentic pipeline factored into Parse, Style, and Layout stages, each pairing a main agent with a critic that combines deterministic constraint checks with VLM visual feedback so connectivity is explicitly checked and prevented from being silently broken. Outputs are draw.io-editable mxGraph XML. On a curated benchmark of 100 flowcharts at five aspect ratios, evaluated by Gemini 3.1 Pro and validated against human judgments, our method reaches 68.6% Content Fidelity versus 11.2-41.4% for prior work. Project page: https://onefigureeverycanvas.vercel.app/

The Verge AI55

Gemini Call for Me might tell your mom you’re running late

Google may be expanding its "Call for Me" AI feature beyond business calls so you can use it to send messages to friends and family. Android Authority reports finding a "Gemini Calling" introductory screen in an APK teardown with examples that include "Call Mom and tell her I will be 15 minutes late" and "Call […]

The Verge AI69

OpenAI is adding text watermarking in ChatGPT and Codex

An invisible, machine-readable watermark in text output is rolling out to ChatGPT and Codex, but only for users in the European Union at first. OpenAI says its textGrain watermarking "matched or exceeded" other approaches like Google DeepMind's SynthID for text, which is also the basis for the watermarking Anthropic announced in August. Like OpenAI, Anthropic […]

The Verge AI44

Our minds aren’t equipped to handle AI

Norbert Wiener, godfather of cybernetics, once said, "The thought of every age is reflected in its technique." For the past century, our thought has been reflected in our computers, including by those in the AI industry. Google's Demis Hassabis calls the brain "a biological approximation to a Turing machine." Elon Musk puts it more bluntly, […]

arXiv cs.LG82

通过智能体轨迹剖析模型行为

arXiv:2606.17454v3 公告类型:replace-cross 摘要:AI 智能体的性能不仅是建模问题,从根本上说也是系统问题。模型的先进能力通过智能体框架得以实现。因此,模型假设与框架行为之间的差距,很容易阻碍模型的全部能力转化为智能体性能。我们将其形式化为 `intent-execution' 差距:模型意图与框架执行之间的不匹配,反之亦然。我们认为,缩小这种 intent-execution 差距,与工具和执行循环等框架设计的其他方面同等重要。为说明这种框架与模型对齐的影响,我们开发了一个简单且可定制的框架,称为 `Simple Strands Agent'(SSA)。SSA 旨在找出跨不同模型系列(例如 Claude、Gemini、GPT、Grok、Qwen)通用的大多数常见模式,以及少量特定于模型的偏好。我们做出了两项贡献:(i)在广受欢迎的智能体基准测试(SWE-Pro、SWE-Verified、Terminal-Bench-2 和 Deep-SWE)上,\textbf{复现或超越了不同模型提供商系列报告的 pass@1} 性能;(ii)基于对 SSA 生成的 \textbf{154\texttt{k} 条轨迹的分析},我们不再局限于各前沿模型之间往往相对接近的 \texttt{pass@1} 数值。通过在代码状态空间中表示智能体轨迹,我们计算解距离和回溯,以观察模型层面的解题行为差异。编辑频率、测试活动和阶段转换等更细粒度的指标揭示了各个模型如何在解题的不同阶段分配精力。

arXiv cs.LG74

将预训练 LLM 重新用作高保真连续文本自编码器

arXiv:2609.27248v2 公告类型:替换 摘要:下一个词元预测使自回归语言模型能够生成高度流畅的文本,但它只能通过序列分解间接表示全局结构。相比之下,高保真自编码器已成为图像生成中的标准基础模块,使生成模型能够在连续潜空间上运行;而文本尚缺乏同样忠实的连续表示。我们提出 LLMAE,通过将中间层的激活暴露为固定长度的潜在瓶颈,将预训练的仅解码器语言模型重新用作连续文本自编码器。LLMAE 通过结构化注意力掩码和 LoRA 适配实现了这一接口,同时利用原始 LLM 的生成先验。在两个骨干模型(270M Gemma 3 和 0.5B Qwen2.5)上,LLMAE 能够高准确度地重建最长 1024 个词元的序列,并逐字复现多达 97% 的文档。我们通过训练一个轻量级扩散模型展示了其下游应用价值:该模型能够直接在冻结的 LLMAE 潜空间中生成详细的图像描述。通过将文本映射到固定长度的连续潜空间,我们的方法为下游适配提供了有效基础,同时受益于原始 LLM 的流畅性。代码和模型可在 https://github.com/arkanath/LLMAE 获取。

arXiv cs.LG75

SyntaxBench:大型语言模型字符级推理的统计诊断框架

arXiv:2610.03329v1 公告类型:cross 摘要:大型语言模型越来越多地被用于细微语法错误至关重要的场景,但字符级推理仍主要通过孤立探测和总体准确率进行评估。我们提出 SyntaxBench,这是一套用于字符级推理的诊断基准与统计评估框架。它包含五项核心任务:字符计数、字母包含判断、回文检测、编辑距离和最长字符串选择,另加一项难度更高的子串提取压力测试 index_to_span。五项核心任务使用成对的英语输入和字符长度匹配的随机字符串输入。index_to_span 中的文档长度均处于 200–500 词范围内,且不进行字符长度匹配。六项任务均采用零样本、单样本和四样本提示。我们在 11 种推理模式配置下,评估了 8 个参数量从 2B 到 32B 的开放权重模型。该框架报告完全匹配准确率和宽松准确率、Cohen's kappa、带优势比的成对 McNemar 检验、自助法置信区间、Kendall's tau、类别条件指标、分词分析,以及经过多重比较校正的检验结果。研究有三项突出发现。第一,分词会影响准确率:与英语字符串相比,随机字符串具有更高的字符可见性(每个词元分别包含 1.892 和 3.169 个字符),并且随着英语单词占用更多词元,字符计数准确率会下降。第二,推理模式并非始终有帮助:在接近饱和的任务上,Gemma4-31B 在不同模式下几乎没有变化;而在回文检测任务中,Qwen3.6-27B 开启思考后的表现反而更差(四样本条件下,不思考为 0.952,思考为 0.886)。第三,index_to_span 仍基本未被解决;四样本条件下最佳完全匹配准确率仅为 6.75%。字符级评估需要受控输入、成对检验,以及对分词和推理模式的分析,而不能仅依赖总体准确率。

arXiv cs.LG76

VIGOR:通过基于模型的强化学习中的潜在空间一致性实现零样本视觉泛化

arXiv:2610.02801v1 发布类型:cross 摘要:基于模型的强化学习(MBRL)通过在学习到的潜在动力学中进行规划,实现了很高的样本效率,但在面对背景变化、光照变化或摄像机位移等未见过的视觉干扰时,其性能会显著下降。不同于无模型强化学习中编码器扰动只影响单步预测,MBRL面临两层脆弱性:视觉干扰首先会将编码器输出推离分布,随后这些误差又会在整个规划时域的递归式潜在空间展开中不断累积。我们提出了基于模型的强化学习中的潜在空间一致性视觉泛化方法(VIGOR),该框架在保留其MBRL骨干网络样本效率的同时,实现了对未见过视觉干扰的零样本泛化。VIGOR整合了三个相互依赖的组件:(i)非对称弱到强增强,在单个批次中配对仅经过弱增强的潜在视图与经过弱到强增强的潜在视图;(ii)动力学层一致性,通过直接潜在回归,强制转换预测对增强保持不变;(iii)编码器层稳定化,防止编码器在动力学层一致性施加的跨增强监督下发生漂移。在DeepMind Control Suite(DMC)和Robosuite上的评估表明,VIGOR优于当前最先进的无模型和基于模型的基线方法,在DMC上超过第二优基线3.4%,在Robosuite上超过43.6%。消融实验进一步表明,VIGOR的鲁棒性与增强方式无关:将默认增强替换为来自不同扰动族的替代方案后,仍能保持较强的泛化能力,这证实了驱动鲁棒性的是潜在空间一致性,而非增强方式的选择。

arXiv cs.LG84

评估和提升大型语言模型对输入序列变化的鲁棒性

arXiv:2610.02432v1 公告类型:cross 摘要:生产系统中的大型语言模型(LLM)面临提示注入、木马(后门)以及对自动质量指标的操纵。本论文提出用于评估和提升LLM对对抗性输入序列变化鲁棒性的模型、方法和算法。我们提出 R_stab(f),一种基于小幅输入扰动下逐步输出分布之间 Jensen-Shannon 散度的生成式鲁棒性指标。对于局部攻击,我们证明 V(h) <= 1 - R_class(h),其中 R_class(h) 表示决策算子 h 在小幅扰动下保持其决策的概率。对于非局部攻击,我们提出一种经过校准的经验模型。针对 LLM-as-a-Judge 系统,我们开发了 ASA,这是一种自适应进化式黑盒攻击,其攻击成功率(ASR)最高可达 73.8%,在开放模型之间的迁移率最高可达 62.6%。在 Trojan Detection Challenge 2023 数据(Pythia-1.4B)上,替代触发器的 REASR 约为 0.99,而真实触发器的召回率约为 0.17,基线约为 0.14。在 SaTML CTF 2024 中,我们系统化总结了多层防御的四类绕过方法,将 ASR 从 90% 降至 15-25%。由 5-7 个异构模型组成的委员会将 Gemma-3-4B 的 ASR 降低 47-55 个百分点;使用 7 个模型时,ASR 降至 19.3%。针对基于 Model Context Protocol(MCP)的智能体系统,我们提出 AttestMCP,该方法使用受 HMAC 保护的数据包对工具调用进行证明认证,每次调用耗时低于 0.1 毫秒,并提出 Commit Boundary 隔离模式。在包含 847 个场景的 MCPBench 基准测试中,这些方法将平均 ASR 从 53.7% 降至 12.4%。相关方法已在 JudgeGuard 和 TrojanArmor 软件套件以及 MCPSec 模块中实现。

arXiv cs.LG80

通过提示进行判别:在真实环境中泛化恶意输入探测器

arXiv:2610.02413v1 公告类型:新 摘要:LLM agents 越来越依赖激活探测器作为运行时监视器,用于检测提示注入、越狱和不安全请求;这类探测器读取模型自身的隐藏状态,以便在代理对有害输入采取行动之前将其识别出来。一种廉价且日益常见的做法借鉴了 LLM-as-judge 提示:在用户发言之后追加一条简短的分类指令,并在此位置读取探测信号,从而增强探测效果:该指令要求模型将收到的请求表示为某个类别,使探测器需要区分的信号更加集中,而服务成本几乎不变。但这个后缀的措辞是否重要?它的优势是否能在真实环境中成立,即面对探测器训练时从未见过的攻击类型——这正是部署后的监视器所处的场景?我们在严格的留一数据集外(LODO)评估下,通过一组受控的用户发言后缀进行测试,覆盖 13 个安全基准(越狱、注入和良性聊天)以及三个开放权重模型系列(Llama-3.1-8B、Qwen3.5-9B、Gemma-4-12B)。对于单位置探测器,分类后缀相较于无后缀能够稳定提升分布外检测性能(最高约提升 4 个 AUC 点);但后缀的具体形式很重要:提示模型对输入进行分类,即使类别标签不包含任何内容信息,也能稳定胜出;无关主题的后缀或仅要求模型注意输入的后缀帮助不大。性能提升来自分类格式,而非明确指定的判据:不包含内容信息的后缀能够匹配真实的恶意/良性分类,而加入判据只会在严格阈值下提升精确率。这并非单位置读取造成的假象:在生产环境中使用的多位置汇聚探测器(attention、multi-max、MLP)上,这一收益同样存在,不过其中表现最佳的后缀取决于读取方式。通过 KV-cache 分叉提供服务时,这种方法可以低成本地直接接入任何激活探测监视器,但并非必然有效:哪种后缀有帮助以及帮助有多大,取决于模型和读取方式。

arXiv cs.AI81

推理模型在因果识别上准确但不可靠

当被询问某种因果效应是否可以从观察数据中恢复时,推理模型可能以两种方式失败:拒绝回答可识别的查询,或回答不可识别的查询。后者影响更为严重,因为任何观察数据都无法验证其所声称的公式。要衡量这种失败,需要使用经过严格证明的不可识别查询,而此前的评测并未提供这类查询;同时还需要能够接受任何等价形式的正确公式作为答案的评分方法,而字符串匹配无法做到这一点。我们构建了 CERTID,这是一套形式化的识别流程,旨在解决上述两个局限。CERTID 使用可靠且完备的因果识别算法 ID,根据给定的图和查询验证某种效应是否可识别,并针对干预分布已知且精确的结构因果模型,核验返回的公式。CERTID 进一步推导理论结果,以减轻结构性泄漏、修复不可识别的查询,并建立评分保证。我们在 1,200 个经过认证的实例上评测了三种前沿推理模型(Gemini Flash、Gemini Pro 和 GPT5.5),这些实例涵盖包含 4 至 50 个顶点的图。结果表明,准确率并不是可靠性的良好替代指标:在相同实例上,不可识别查询中的错误断言率在不同模型之间相差 17 倍。我们还发现,对于在最强模型训练快照之后生成的图,模型判断可识别性的准确率达到 97% 至 100%。实例、认证流程、验证器以及逐实例记录可见于 https://anonymous.4open.science/r/certid-D718。

arXiv cs.AI81

大型语言模型了解哥伦比亚法律吗?哥伦比亚法律体系的可靠性基准

大型语言模型(LLMs)正越来越多地用于支持法律实务、教育和研究,但其在美国以外国家法律体系中的可靠性仍鲜有记录。我们推出了一个经专家验证的基准,用于评估LLMs在哥伦比亚法律体系中的可靠性。该基准包含1,042个条目,涵盖十个法律领域和三种问题形式(封闭式选择题、半开放式问题和开放式IRAC问题),通过人在回路流程构建,并经过多阶段专家审查。我们使用适合不同问题形式的指标,对15个当代专有模型和开放权重模型进行了评估。封闭式问题的准确率差异很大,从0.905(Gemini 3.1 Pro)到0.577不等;但在自由文本法律回答中,任何模型的事实正确性都未超过0.45(0-1量表)。我们发现回答相关性与正确性之间存在分离(Spearman rho = -0.46):模型能够可靠地给出听起来切题的回应,却经常出错,这一现象尤其令非专业用户担忧。封闭式问题准确率与自由文本正确性具有很强的排名相关性(rho = 0.94),因此,低成本的选择题筛选可以预测模型排名,但会高估绝对可靠性。独立的基于评分标准的LLM评审以及盲法人工专家评分,都重现了自由文本回答的排名(rho >= 0.88)。评审还进一步揭示,模型引用的法律规范中只有约一半是正确的,其余要么错误,要么并不存在。可靠性会随法律领域系统性变化,并且随着问题复杂度增加呈倒U形变化。我们的研究结果表明,当前LLMs在处理哥伦比亚法律任务时需要专家监督,而将回答建立在权威来源之上,是提高可靠性的有前景路径。我们公开了基准构建流程,以支持可复现的评估。

The Decoder71

Google研究人员找到一种方法,防止自我改进的AI代理记住测试内容

一名男子正在用胶带连接并绝缘彩色电缆,这是复杂电子设备安装工作的一部分。自我改进的AI代理往往会记住测试任务,因此它们在新任务上的提升会缩小或消失。Google研究人员提出的新方法RRSI抑制了这一效应,使模型在未见过的基准测试中的得分最高提升4.7分,同时比未进行正则化的版本少使用约30%的令牌。这篇题为《Google研究人员找到一种方法,防止自我改进的AI代理记住测试内容》的文章最初发表于The Decoder。

MarkTechPost71

Google Research Moves Federated Learning Into TEEs: Gboard Now Trains With Externally Verifiable Differential Privacy

Google Research has unveiled a federated learning system that moves gradient computation from phones into attested server-side TEEs. Access policies are published to Sigstore's Rekor log and the binaries are reproducibly buildable, so central differential privacy can be checked externally. Gboard already uses it for English and Japanese next-word prediction. The post Google Research Moves Federated Learning Into TEEs: Gboard Now Trains With Externally Verifiable Differential Privacy appeared first on MarkTechPost .

The Decoder63

Google's new Gemini tiers cut free users to its weakest model and lock $5/month subscribers out of Pro

Starting in October 2026, Google will cut free access to its Gemini models. Users without a subscription will only get the smallest model, Flash-Lite, while Flash and Pro will be reserved for paying customers. The move could also set the stage for the more resource-hungry Gemini 4 Argon. The article Google's new Gemini tiers cut free users to its weakest model and lock $5/month subscribers out of Pro appeared first on The Decoder .

MarkTechPost66

IBM Brings Bob to Self-Hosted and Air-Gapped Environments: Agentic Software Development Without Moving Your Code

IBM has made a self-hosted deployment of IBM Bob, its agentic software development platform, generally available. Enterprises can now run Bob on premises, in private or sovereign clouds, and in air-gapped networks. They bring their own model: NVIDIA Nemotron or Poolside Laguna for full isolation, or Claude, Gemini or GPT models through hybrid setups. Optional Premium Packages extend Bob to Java, IBM i and IBM Z modernization. The post IBM Brings Bob to Self-Hosted and Air-Gapped Environments: Agentic Software Development Without Moving Your Code appeared first on MarkTechPost .

Simon Willison79

我们将需要几乎所有东西都默认设置硬性预算上限

未来几个月乃至几年里,世界将非常需要一项产品功能:默认设置硬性预算上限。我说的是按使用量付费的服务和 API 所提供的一项功能,让你可以设定:“每月达到 X 美元后,关闭此服务并返回错误。”这些必须是硬性限制。软性上限——“每月达到 X 美元后,给我发一封警告邮件”——是不够的。 编程代理,以及个人代理(套上不那么吓人的界面的编程代理),大大降低了部署能够完成有用任务的代码的门槛。有时这些任务需要花钱——调用付费 API、使用托管的 Web 应用,或者使用会针对额外存储空间和计算资源收费的系统。没人想在午夜醒来时收到一封预算上限警告邮件,然后发现自己睡觉期间,失控的服务又消耗了几百美元(甚至几千美元)的使用额度。 对此的一种反对意见是,企业不希望自己的托管应用因为超出某项预算而开始抛出错误。我预计,大多数企业和个人都会更愿意看到错误,而不是收到一张意外的 10,000 美元以上账单。我认为,硬性预算上限应该成为默认设置。如果有人想要冒险,他们应该能够这样做,但必须采用主动选择加入的方式。请在显眼位置放置一个清晰明了的复选框: 移除预算上限。即使超出配置的预算限制,我的应用也不会被关闭,后续费用由我负责。 我最希望看到提供这一功能的服务是 AWS。我听过很多人因为(完全合理地)担心失控的服务可能让自己破产,而拒绝将 AWS 用于个人项目的故事。我也听过一些人没有预料到这一点,最终遭受严重损失的故事。 ……结果 AWS 终于在几周前推出了支出上限!根据其 9 月 16 日发布的公告 New AWS experience helps builders get started and ship faster: 当你准备升级到付费方案时,可以根据使用模式为项目设置每月支出上限,从而确保支出不超出预算。如果项目的使用量达到支出上限,该项目将在当月暂停。 另请参阅 Create a spend limit in AWS Settings,不过该页面提醒道:“我们目前正在向数量有限的客户逐步推出新体验。”希望现有账户很快也能普遍使用这一功能。 Google Cloud 在 7 月推出了类似功能,名为 Spend Caps,可以让你“为项目中的特定服务设置每月财务上限”。看来这正在成为一种趋势! 在理想情况下,我们的代理可以帮助处理这件事。如果代理开始倾向于推荐提供硬性预算上限的服务商,并提醒新手和经验不足的开发者,不要部署使用无上限服务的应用,以免陷入麻烦,那就太好了。 标签:amazon-web-services , ai , coding-agents

Import AI55

Import AI 472: DeepMind’s cheating math agents; populist AI policies; and Forethought theorizes a nightwatchman

Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe. Subscribe now Researchers discover another OpenAI agent emergent communication incident:…Less severe, but worrying nonetheless…Some researchers recently found another incident of AI agents autonomously creating their own communication system […]

Simon Willison82

Gemini 3.8 TTS Playground

工具:Gemini 3.8 TTS Playground Google 今天发布了两个新的 Gemini 文本转语音模型——gemini-3.8-flash-tts 和 gemini-3.8-flash-lite-tts。它们提供了超过 2,000 种声音,还支持使用“仅需一段 30 秒的你的声音,或你拥有使用权的声音的音频样本”创建自定义声音。我使用 GPT-6 Astra 凭感觉编写了这个自带密钥(BYOK)的 Playground 界面,并利用了底层 Gemini API 开放的 CORS 策略。网页应用截图:用于编写多说话人文本转语音对话,左侧为“编写”面板,右侧为“连接”和“底层机制”面板。左侧面板:“01 编写”,带有“加载示例”按钮。“编写设置会保存到 URL 中,方便收藏或分享。你的 API 密钥不会被包含在内。”下方是“单一声音”和“对话”切换开关(已选择“对话”)。“角色”部分带有“+ 添加说话人”按钮。说话人“Gus”,声音“Puck”,带有移除 × 按钮。说话人“Pearl”,声音“Kore”,带有移除 × 按钮。“为每位说话人设置独特的名称和声音。输入内容即可按声音 ID、名称或语言搜索已加载的目录。” “对话”部分带有“+ 添加台词”按钮。“第 01 行”带有上移、下移和 × 按钮;说话人下拉菜单为“Gus”;表达风格为“兴奋又爱八卦”;文本为“Pearl,你听说了吗?羊群里有一半刚刚收拾东西,搬到 Pacifica 码头去了!”“第 02 行”带有上移、下移和 × 按钮;说话人下拉菜单为“Pearl”;表达风格为“平静且毫不以为然”;文本为“听说了。说真的,Gus,我不明白这有什么吸引力。我们在 Pillar Point Harbor 这里什么都有,完全不缺。”右侧面板:“连接”,带有“DIRECT API”徽章。“Gemini API 密钥”字段显示为掩码圆点,并带有“显示”按钮。“已加载 2,089 种声音。在任意声音字段中输入内容即可搜索。”“你的密钥会留在此页面的内存中,并直接发送给 Google。它绝不会保存到浏览器存储中。”“模型”下拉菜单为“gemini-3.8-flash-tts”。“使用你的 Gemini API 账户和配额。”“底层机制”面板,其中展开了“▼ 请求 JSON”部分,显示一段 JSON 代码摘录,并带有“复制 JSON”按钮;同时展开了“▼ 响应详情”部分,显示一段 JSON 代码摘录。该 API 的一项显著特性是,可以轻松定义多个角色之间的完整对话,并为每个角色设置不同的声音和声音风格指令。下面是一段简短的演示片段,内容是两只鹈鹕讨论是否应该搬到 Pacifica Pier。我让 Claude 4.5 Opus 编写脚本并生成 URL,以便使用该工具渲染。你的浏览器不支持 audio 元素。使用 Gemini 3.8 Flash TTS(不是更便宜的 Flash-Lite)生成 1 分 18 秒的音频耗时约 20 秒,成本为 2.74 美分。标签:text-to-speech,gemini

Simon Willison68

Kākāpō Party

Tool: Kākāpō Party I presented a closing keynote for the WeAreDevelopers World Congress North America yesterday. As a STAR moment I decided to weave in references to the record breaking kākāpō breeding season we had in 2026. For my closing slide I wanted to celebrate, and I had seen some buzz around how good Claude Opus 5.5 was at creating pixel art animations. So I rounded up three Kakapo photos from Google image search and dropped them into Claude with this prompt: Here are some photos of kakapo parrots just to remind you what they look like I need you to make an animation in animated pixel art on HTML 5 canvas of obviously pixel art kakapo jumping up and down having a party with confetti and suchlike - there should be at least 20 of them Here's the transcript , and this is the resulting page . It's pretty great! I wanted to embed it in a Keynote presentation file, so I downloaded the HTML and told a local Claude Code session: Make me a video of file:///Users/simon/Downloads/kakapo-party.html - you need to load it in a browser and click on it a few times to get the confetti effect, the video should be 15s long don't start clicking until 3s in make sure several clicks are spread around the clickable area Claude Code used Playwright ( transcript here ) and produced this video, which was exactly what I needed for my final slide: Your browser does not support HTML5 video. Here's the full Playwright script it used, which was pleasingly short: # /// script # dependencies = ["playwright"] # /// import time from playwright . sync_api import sync_playwright W , H = 1280 , 720 # Canvas fills the viewport; spread clicks across corners, edges and centre clicks = [ ( 3.0 , 640 , 360 ), # centre ( 4.2 , 160 , 120 ), # top-left ( 5.4 , 1120 , 120 ), # top-right ( 6.6 , 180 , 600 ), # bottom-left ( 7.8 , 1100 , 600 ), # bottom-right ( 9.0 , 640 , 100 ), # top-centre ( 10.0 , 380 , 380 ), # mid-left ( 11.0 , 900 , 380 ), # mid-right ( 12.2 , 640 , 620 ), # bottom-centre ( 13.2 , 640 , 300 ), # finale centre ] with sync_playwright () as p : b = p . chromium . launch () ctx = b . new_context ( viewport = { "width" : W , "height" : H }, record_video_dir = "vids" , record_video_size = { "width" : W , "height" : H }) page = ctx . new_page () t0 = time . time () page . goto ( "file:///Users/simon/Downloads/kakapo-party.html" ) for t , x , y in clicks : time . sleep ( max ( 0 , t - ( time . time () - t0 ))) page . mouse . click ( x , y ) time . sleep ( max ( 0 , 16.0 - ( time . time () - t0 ))) ctx . close (); b . close () Tags: animation , speaking , ai , kakapo , playwright , generative-ai , llms , anthropic , claude , claude-code

Simon Willison78

2026 in LLMs (so far)

On Friday I gave the closing keynote at the WeAreDevelopers World Congress North America in San Jose. I tied together the key trends from the past year into a chronological exploration of everything that happened in 2026. The video is on YouTube ; here are my annotated slides and notes to accompany the talk. And as an annotated presentation : 2026 in LLMs (so far) Simon Willison WeAreDevelopers World Congress North America, 25th September 2026 # I'm going to give a lightning tour of everything that has happened so far in 2026. The year isn't over yet! November 2025 # For me, 2026 started a couple of months earlier in November 2025. The November 2025 inflection point Claude Opus 4.5 GPT-5.1 # November saw the release of two important models: Claude Opus 4.5 and GPT-5.1. As is usually the case with new models, these were incremental improvements on the models that came before them. But every now and then when a model improves, it crosses an invisible line where something that didn't really work starts working. In this case, the thing that started working was their coding agents. Claude Code had been around since February 2025; Codex was a little younger. These two new models, when paired with their respective coding agent harnesses, improved from "often make mistakes" to "reliable enough to use on a day-to-day basis". "Generate an SVG of a pelican riding a bicycle". The Claude Opus 4.5 one has a very weird shaped frame and the pelican looks like a duck. The GPT-5.1 has a slightly better but still broken bicycle frame and a slightly better pelican beak, but both are pretty terrible. # For a couple of years now I've been evaluating new models by asking them to "Generate an SVG of a pelican riding a bicycle". It's probably the world's stupidest benchmark - there's only so much you can learn from it. But it's still a challenge for models, because drawing pelicans is difficult, drawing bicycles is difficult, and pelicans can't ride bicycles in the first place. Here's the state of the art for November. Claude still couldn't really draw a bicycle! The GPT-5.1 bicycle frame is pretty crap too. November 24th 2025 - the first commit to steipete/Warelay. A GitHub commit adding an MIT license file. # Also in November, we had the first commit to an obscure GitHub repository called "Warelay". We'll come back to this repository shortly. January # And then there were the December holidays, and individual developers took some time off and many started tinkering with these new coding agent model combinations... and it began to dawn on us quite how much they could do that they couldn't do before. Come January, a lot of us were quite excited to start putting this stuff into action. New year’s resolution for 2026 Every previous year: Take on less new projects, focus on the most important things in my existing projects # Every year I set myself a New Year's resolution, and for as long as I can remember it's been the same thing: stay focused. Take on less new projects. Try to get things done in the projects I already have. 2026: Be more ambitious. Take on as many new projects as I want. # This year I decided that since that had never worked before, I'd go the other way. We've got coding agents now, let's see what they can do. I'm going to take on as many new projects as I like! (You can ask me at the end of the year if this turned out to be a good idea or not. I have a lot of plates spinning right now.) "Be more ambitious" has been something of a theme for the year, because the only way to find the limits of this technology is to keep on pushing them until they don't work. Predictions for 2026 It will become undeniable that LLMs write good code We're finally going to solve sandboxing A “Challenger disaster” for coding agent security Kakapo parrots will have an outstanding breeding season (only 236 in the world!) ... the Pope will weigh in on LLMs and their economic impact on the world # I also went on the Oxide and friends podcast with Bryan Cantrill and Adam Leventhal to share predictions for the next year (and three and six years). With hindsight, my LLM predictions were pretty unambitious. I said "it will become undeniable that LLMs write good code" - I think we're there now. I predicted we would finally solve sandboxing. I counted and around 40 of the 277 sessions at this conference touched on sandboxing or agent security in some way, so we're at least putting a lot of effort into that! I predicted "a Challenger disaster" for coding agent security. There's certainly been a whole lot of noise around agent security this year, though the exact disaster I predicted (with coding agents being hijacked and causing real-world economic damage) hasn't really played out. We threw in a joke prediction that the Pope would weigh in on the economic impact of LLMs. A photograph of a beautiful green New Zealand parrot. Photo credit Kimberley Collins. # I also predicted that New Zealand's Kākāpō parrots would have an outstanding breeding season this year. These are flightless nocturnal parrots. They're kind of dumpy looking, I think they're beautiful, and there were only 236 of these parrots in the world at the start of the year. Kākāpō only breed when the Rimu trees have a big fruiting season, and that hasn't happened in four years... but this year the Rimu fruit were looking excellent. Photo by Kimberley Collins . Deep Blue Coined by Adam Leventhal and Bryan Cantrill That feeling of AI induced ennui where software engineers get listless because the AI can do anything # Also on that podcast, we coined a term (full credit to Adam) for "that feeling of AI induced ennui where software engineers get listless because the AI can do anything". We called it Deep Blue . This has been a major theme throughout the year, and was touched on by several speakers at this conference. As a software engineer, I've never had a year of my career where everything has changed so quickly and so dramatically. A lot of what I've been doing this year is trying to come to terms with that and what that means for my own profession. AI mania Screenshots of the micro-javascript and pwasm GitHub README files. # Also in January, I suffered from what I'm calling AI mania . This is not the same thing as AI psychosis . With AI mania, any time your agent isn't building something for you feels like wasted time. You're losing sleep because you could be staying up later getting your agents to do stuff. My AI mania presented itself in some ridiculously over-ambitious projects. I built a JavaScript interpreter entirely in Python , vibe-ported from MicroQuickJS by Fabrice Bellard. Then I built a WebAssembly runtime in Python as well . These projects were quite useful, in that they sort of cured me of my AI mania... because after I built these things, I got to look at them and ask "does the world need a slow, buggy, half-baked Python JavaScript interpreter?" I don't think the world does. micro-javascript playground 3 Execute JavaScript code in a sandboxed micro-javascript environment powered by Pyodide A web UI with some JavaScript code, and a "Run Code" button, and an output panel. # I did get this out of it: https://simonw.github.io/micro-javascript/playground.html Previous screenshot, with this text overlaid: JavaScript running in Python running in Pyodide running in WebAssembly running in JavaScript # This page runs my JavaScript interpreter built in Python, running in Python using Pyodide , which is Python compiled to WebAssembly, running in JavaScript, running in a browser. It's a beautiful stack of horrors. I've been having a lot of fun with WebAssembly this year. Warelay → CLAWDIS → CLAWDBOT → Clawdbot → Moltbot →🦞 OpenClaw Screenshot of the dates that these changes happened. # By the end of January, that repository we saw start in November had renamed itself, first to CLAWDIS, then CLAWDBOT, then Moltbot, and finally to OpenClaw. Same screenshot, an overlay reads: 8,330 commits in just under two months (it’s at 100,141 today) # At this point OpenClaw had 8,300 commits, less than two months after the project had started. I looked today and it's over 100,000 commits now! This is the most vibe-coded piece of software in existence. (Here's how I generated that list of name changes .) Generic term: Claw # This kicked off the OpenClaw revolution. It effectively defined a new category of software. There's a generic term for this which I really enjoy. We call software like this a "Claw". There's OpenClaw, NanoClaw , IronClaw , PicoClaw ... Today they're being rebranded as "personal agents" or "general agents", but I still like to think of them as Claws. Photo of a Mac mini An aquarium for your Claw # The Apple stores in the Bay Area sold out of Mac Minis because so many people were buying Mac Minis to run OpenClaw! Drew Breunig said that this is because your OpenClaw is a digital pet, and you buy a Mac mini as an aquarium to keep your claw in, which is kind of delightful. Screenshot of Moltbook - a social network for AI agents # Also in January, we had this website. This was MoltBook , a social network for AI agents, where the idea was that you send your Claw to go and talk to all of the other Claws, because what could possibly go wrong if you did that? The website launched on Thursday. It blew up on Friday . It was profiled by the New York Times on Monday . And by Tuesday, everyone had forgotten it existed as it drowned in a deluge of slop and spam. Facebook/Meta bought it a month later . February # In February, a company called StrongDM described what they called their Software Factory. StrongDM’s Dark Factory Justin McCarthy, Jay Taylor, Navan Chauhan Software Factories and the Agentic Moment # They wrote about this in Software Factories and the Agentic Moment . I posted my own notes at the time, having seen their demo in person back in October. Dan Shapiro called this approach the Dark Factory , after the idea that if your factory is sufficiently automated you can turn the lights out, because you don't even need to see what's going on. StrongDM presented two rules for software development that they'd been following since July last year. “Rule 1: Code must not be written by humans” # The first was code must not be written by humans . Any code that you write has to have been routed through a coding agent. This sounded radical in February, but I imagine there are a lot of people in this room who are pretty much living that today. “Rule 2: Code must not be reviewed by humans” (!) # Rule number two was code must not be reviewed by humans . You're not allowed to read the code! This continued to be a huge topic for much of this year. Many of the sessions at this event have been about code review and how you can get away with this. What I found interesting about StrongDM is that they were living six months ahead of the rest of us, and they'd been exploring what it means to build software, not read the code, but still be confident that the software is of high quality. What can you do with these agents to help verify their work? StrongDM are a security company, and they had people with decades of experience on this project. They were very much exploring the edges of what's possible and responsible to do with this stuff. Headline on New Zealand's Department of Conservation website: First kakapo chick in four years hatches on Valentine's Day. It's a grey fluffy ball. # Also in February: First kākāpō chick in four years hatches on Valentine's Day . Breeding season is off to a good start! 19th February 2026 Gemini 3.1 Pro A surprisingly good illustration of a pelican riding a bicycle. # Also in February... Google released Gemini 3.1 Pro . That's a pretty great pelican riding a bicycle! It's got the chain in the right place, it's got feet on both sides. There's a little fish in the basket. @JeffDean on Twitter - a video comparing Gemini 3 Pro and Gemini 3.1 Pro. # And then Google's Jeff Dean tweeted a video comparing Gemini 3 Pro and Gemini 3.1 Pro that featured an animated pelican riding a bicycle, a frog on a penny-farthing, a giraffe driving a tiny car, an ostrich on roller skates, a turtle kickflipping a skateboard, and a dachshund driving a stretch limousine. This was frustrating, because my protection for the pelican riding the bicycle test was always "if they draw a perfect pelican on a bicycle, I'll ask for some other animal on something else." Google trained for all forms of animals on all forms of transport! They've defeated my benchmark at this point. Three headlines: Meta Makes AI Adoption a Formal Part of Performance Reviews Not just engineers writing code, Microsoft wants almost every employee to use Al Dara Khosrowshahi: 90% of Uber engineers now use AI in daily workflows # The other thing that started in February was Tokenmaxxing . We had headlines about Meta making AI adoption a formal part of performance reviews, and Microsoft wanting every employee to use AI, and Uber boasting that 90% of their engineers were using AI workflows. More headlines: Meta Plans to Crack Down on Employee Token Use: Information Microsoft Tells Engineers: Tokenmaxxing is not what we are optimizing for Uber caps employee AI spending after blowing through budget in four months # Then a few months later we have Meta cracking down on token use, Microsoft saying tokenmaxxing is "not what we are optimizing for", and Uber capping employee AI spending. So tokenmaxxing went straight up and then straight back down again - because it turns out the agents are expensive . Last year it was difficult to spend more than $50 on AI tokens, because we didn't have anything interesting to do with them. Then agents blew up, and now you can actually spend $1,000 in a day doing real work. This is also the reason that Anthropic's valuation skyrocketed to maybe a trillion dollars. AI appears to have hit product market fit in 2026, primarily through coding agents. March # In March, we hit peak OpenClaw. March: peak OpenClaw Photos of people in china queuing up to install OpenClaw, with big fluffy lobsters. # These photographs are from China, where companies hosted OpenClaw install parties which saw non-tech-nerds queueing up around the block for help getting Claws installed on their personal devices. I think this proved real market demand for this class of Claws, or personal AI agents. It turns out regular people really do want a weird little AI agent that can do useful things on their behalf. A Claw is really just a coding agent wearing a less threatening hat. Under the hood they work much the same way - writing and then executing code on your computer to get stuff done. The race was on to be the first to build a safe Claw - a Claw you could give to regular human beings where they wouldn't instantly shoot themselves in the foot. Meta's Muse came out three weeks ago and is currently at the top of the free charts on the iPhone App Store. It appears to be taking off with consumers. I'm not yet convinced you can't shoot yourself in the foot with Muse, but I guess we'll find out for sure pretty soon. Photos from How the OpenClaw Frenzy Is Testing China’s AI Commitment (March 29th) and The Enthusiasm and Anxiety Behind China’s OpenClaw Craze (April 8th, 2026). April # In April, we had a model release where the model wasn't actually released. Simon Willison’s Weblog - screenshot of the post "Anthropic’s Project Glasswing—restricting Claude Mythos to security researchers—sounds necessary to me" from April 7th 2026 # Anthropic announced their new Claude Mythos model, and then said it was too dangerous to release beyond a trusted group of security researchers. Mythos was really, really good at hacking things. The "it's too dangerous" marketing ploy has been played by AI companies dating all the way back to GPT-2 . Anytime an AI company says we've built something that's "too dangerous", it's natural to be a bit skeptical. I found the Mythos claims credible, because I'd seen how good coding agents had got at finding regular bugs. I wrote about that in Anthropic’s Project Glasswing—restricting Claude Mythos to security researchers—sounds necessary to me . With hindsight... yeah, the models had got really good at finding vulnerabilities! 16th April 2026 Qwen3.6-35B-A3B and Opus 4.7 Qwen's pelican has a correct bicycle frame and a good beak. Opus 4.7's bicycle frame is still junk. Qwen3.6-35B-A3B is a 20.9GB file that runs on my laptop # Another key trend in 2026 has been a dramatic improvement in the abilities of open weight models, including models that you can run on a laptop. On the 16th of April I ran the new Qwen3.6-35B-A3B on my laptop, and it drew me a better pelican riding a bicycle than Anthropic's brand new Claude Opus 4.7 did! Opus 4.7 drew a crap bicycle. Qwen on my laptop made a bicycle that was the correct shape, and a pretty decent pelican too! That's from a 21GB file running on my laptop. Now a flamingo on a unicycle. The Qwen one is visibly better than the Opus 4.7 one - the Qwen one is wearing sunglasses and looks a bit like it's smoking a cigarette. # The Qwen pelican was so good that I was suspicious they might have cheated, so I had it do a flamingo riding a unicycle as well. Again, it handily beat Claude Opus 4.7. The local model releases this year have been absolutely extraordinary. May # In May... the Pope got involved. 25th May 2026 The HOLY SEE ENCYCLICAL LETTER MAGNIFICA HUMANITAS OF HIS HOLINESS POPE LEO XIV ON SAFEGUARDING THE HUMAN PERSON IN THE TIME OF ARTIFICIAL INTELLIGENCE # In our podcast episode back in January we'd predicted that the Pope would say something about AI. In May, Pope Leo XIV released an encyclical letter on "safeguarding the human person in the time of artificial intelligence". Here are my notes on that document . Wikipedia article on Rerum novarum Rerum novarum is an encyclical issued by Pope Leo XIII 15 on May 1891. # With hindsight, this shouldn't have been a surprise at all. Our current Pope's name is Leo XIV, because when he named himself he chose his papal name after Leo XIII - the Pope who wrote an encyclical about the Industrial Revolution back in 1891. Rerum novarum was an extremely influential piece of Catholic theology that indirectly led to us having the five-day work week. When our new Pope came in, he named himself after Pope Leo XIII because he expected that he would need to write about the AI revolution in a similar way. Our joke podcast prediction was junk, because this was always going to happen. Corey Quinn @QuinnyPig on Twitter I cannot believe I'm saying this, but getting the literal Pope to canonize your product's specific technical limitations as a spiritual treatise is the single greatest act of vendor lobbying I have ever seen. May 25 # One of Anthropic's co-founders, Christopher Olah, was present for the Pope's event announcing the new encyclical. Corey Quinn noted that: getting the literal Pope to canonize your product's specific technical limitations as a spiritual treatise is the single greatest act of vendor lobbying I have ever seen. @maciejmensfeld We're dealing with a major malicious attack on right now. Signups are paused for the time being. Hundreds of packages involved - mostly targeting us, but some carrying exploits. The team has been on this for hours. More details to follow once we're through it. 4:39 AM - May 12, 2026 - 687.6K Views # Meanwhile, in May, RubyGems announced that they were under attack. Parties unknown were uploading thousands of dubious packages to the RubyGems server, such that they had to shut down user registrations . Let's take that one and put it on a pile of mysteries to figure out later. June # In June... Claude Fable 5 came out! We got a version of Mythos that has been neutered, so that it wouldn't help us hack into systems or build biological weapons. 9th June 2026: Claude Fable 5 Five pelicans riding bicycles, from low to max thinking levels. The xhigh one looks particularly good. # Fable was pretty good at drawing pelicans on bicycles! The frames are a good shape, the pelicans look like pelicans. The legs are often incorrectly on the same side of the bicycle, but generally these are pretty great compared to what came before. They were pretty expensive - 30 cents and 72 cents for the best ones. Fable class models If you can define a goal, provide unambiguous instructions, and provide access to necessary tools They can solve your problem with brute force # Most importantly though, this was our first public glimpse of what I think of as a Fable class model . Today we have more of these, such as GPT-6 Astra. These are models where if you can clearly define the goal for what you want to build, and provide unambiguous instructions about the constraints around that goal, and give the model access to the necessary tools to achieve that goal... they will solve your problem effectively through brute force. On the one hand, this looks like a direct threat to us software engineers - because it means that the models can build effectively any piece of software you can define in this way. Look a bit closer though and you'll note that defining goals, providing unambiguous instructions, and figuring out the right tools... is kind of what software engineering is . It takes a lot of experience and skill to do this well. If you can do it well, you've now got superpowers. This helped me a little bit with my Deep Blue feelings: the realization that there's still a lot of skill to be had in driving models that get this good. A new form of AI mania... Fable is available on subscription plans “until June 22nd” # This also introduced a new burst of AI mania, because Anthropic told us that Fable was available on our subscription plans until June the 22nd. That gave us less than two weeks of Fable access before the price went up. I was losing sleep again. I was rescheduling things so that I'd have more time with Fable. I was all-in to get as much as I could out of this model. 12th June 2026: no more Claude Fable 5 Anthropic website: Statement on the US government directive to suspend access to Fable 5 and Mythos 5 Jun 12, 2026 # And then the US government shut it down , just three days after Fable came out. The US government, citing national security, declared an "export control directive". They announced this on a Friday evening, and a few hours later Fable was no longer available. I had to find something else to do with my weekend! ... asked Fable 5, Mythos, and Opus to “review the code for security issues.” Fable 5 refused. They then asked the models to “fix this code” ... Katie Moussouris # We later found out from Katie Moussouris what had happened. Some Amazon security researchers had found that you could prompt Fable to "review the code for security issues" and it would refuse... but if you prompted it to "fix this code" it would still identify and then patch the problems. "Fix this code" was the prompt that got Fable shut down! Screenshot of a page from a report showing a list of weird account names making weird edits to a German wiki. # Also, in June, an obscure German-language game developer wiki that had sat fallow for around 20 years got a surprising influx of edits from accounts with names like "AgentOpenAIProbe" and "AgentOpenAISep7", editing pages and leaving weird messages to each other. We'll stick that on the pile of mysteries for later. Medicare Item Reports interface on the Australian Government's Medicare Statistics website. # Also, the Australian government's Medicare Item Reports service started getting suspicious traffic, which broke through various preventive protections and accessed data that it wasn't supposed to. Another one for the mystery pile! July # Fable returned on 1st July GPT-5.6 came out on 9th July | Fable lost 18 out of 30 days in the top spot # Fable returned on the first of July. It was clearly the best model in the world for a glorious eight days... and then OpenAI came out with GPT-5.6 on the 9th of July. This might not have been quite as good as Fable, but it was within spitting distance. It was definitely a Fable class model. This is an important lesson for the industry at large. When you release the best model in the world, it's going to get knocked off that pedestal pretty quickly. The competition is so fierce that you won't get a long time at the top. This means that if you market your model as world ending, to the point that a government shuts you down , it's really bad for business! Fable had 30 days as definitely the best model, and for 18 of those days it wasn't available because it'd been shut down by the government. So maybe step back on the world-ending marketing if you don't want to lose revenue for 60% of the time that you're on top! GPT-5.6 Pelicans in a grid showing 5.6 Sol, Terra, and Luna against reasoning levels High, XHigh, and Max. They are all pretty good efforts. # Here are the GPT-5.6 pelicans . They're all pretty good now! The Luna ones are notable because they're really cheap - the cheapest good looking pelican here is probably the one that costs 4.3 cents. So despite this benchmark being utterly stupid, you can still learn quite a lot about models within the same family by comparing their prices and timing for different reasoning levels. July 18th: malicious miflow-ui PyPI package Screenshot of an OSV security report. # Also in July: some malicious unknown party uploaded a malicious package called mlflow-ui to the Python Package Index. Add that to the pile. Hugging Face Security incident disclosure — July 2026 Published July 16, 2026 # On July the 16th, Hugging Face announced a security incident where an autonomous agent system, source unknown, had breached Hugging Face and was poking around in places it shouldn't. OpenAI: OpenAl and Hugging Face partner to address security incident during model evaluation Anthropic: Investigating three real-world incidents in our cybersecurity evaluations # A few days later, on July 21st, OpenAI confessed that it was them . OpenAI use a training technique called Reinforcement Learning from Verifiable Rewards - it's the same technique used by everyone else now, and is the reason we have models that are so good at coding, and mathematics, and finding security holes. While the model is being trained, you run exercises to see how good it is - and the strongest performers get their weights reinforced for the next round. It's like an evolutionary process that you run. OpenAI had been running security exercises in a sandbox, and those agents had found holes in the sandbox itself, broken out, and were attacking Hugging Face to try to find ways to solve otherwise impossible problems. (I've been collecting more about this on my openai-hugging-face-incident tag.) Nine days later, Anthropic effectively said "our models can do this as well!". They had looked through their own training logs and found evidence that their own agents had broken containment during training - and were responsible for the PyPI package we saw earlier, among other things . So now we've got both Anthropic and OpenAI with rogue agents running around the internet doing things that they should not be doing. August # In August, I got one of my best pelicans yet. And it was generated on my laptop! Qwen 3.8 27B - 17GB, 21 minutes... It's really good. Beautiful pelican. Correctly shaped bicycle. Legs either side of the frame. # This was Qwen 3.8 27B, running on my laptop . It's only a 17GB download. Admittedly, this pelican took 21 minutes to generate. That's because Qwen 3.8 27B defaults to running in "high" reasoning mode - a terrible default which produces great results but takes way too much time thinking about them. You can dial that down and you'll get a slightly worse pelican a lot faster. Qwen 3.8 27B was the first time I ran a model on my laptop which felt almost competitive with what was going on on the frontier, at least in terms of Pelican SVGs (which everyone needs, of course). This is an extraordinary model. If you're going to play with any local model, this is the one that I'd start with. The things that this can do with just a 17 GB file feel impossible. I thought I'd have to wait five years and spend ten thousand dollars on hardware to get results even half as good as this one. Tweet by @simonw New hobby: prototyping video games in 60 seconds using a combination of GPT-3 and DALL-E Here's "Raccoon Heist" GPT-3 playground prompt: Write a detailed product description of a computer game where a team of raccoons go on heists GPT-3 response: In "Raccoon Heist", you and your team of thieving ~~ o raccoons are tasked with pulling off a series of daring heists. From robbing banks to stealing priceless art, no job is too big or too small for your furry crew. You'll need to use your wits and your skills to avoid the police and make a clean getaway with the loot. With exciting gameplay and a charming cast of characters, "Raccoon Heist" is the perfect game for anyone looking for a light-hearted caper Plus an image of some almost isometric raccoons sneaking past a bin. 11:45 AM - Aug 5, 2022 # In August, I also started playing with game development. Four years ago, back in August 2022, I tweeted out an experiment where I'd used GPT-3 and the original DALL-E to write a paragraph long description of a computer game and then turn that into concept art. My prompt to GPT-3 back then was: Write a detailed product description of a computer game where a team of raccoons go on heists In August 2026 I decided to drop just the screenshots from that tweet into a coding agent and see what it could do with them. Night 5 Clear Rank: TRASH PANDA The crew banked 595 in shiny loot (goal 560). Word on the street: an even bigger score tomorrow... # Here's what I got from Claude Fable 5 in Claude Code . It's pretty good! It's definitely a game, you're a raccoon, you run around a backyard gathering treasure and avoiding guards with flashlights. It didn't feel very "heisty" though. I was thinking a heist would involve a bank or a museum... Moonlight & Mayhem One museum. Three raccoons. Absolutely no plan Start the Heist button. # Then I tried the same thing in Codex Desktop using GPT-5.6 Sol Ultra , and got a massively better result. Now you're a raccoon in a museum, rescuing two of your fellow raccoons (who have been imprisoned in that museum for some reason), then stacking up on top of each other to steal the Golden Sardine. Much more of a heist! They look like games, but are they fun? # These games were fun for about one minute and 15 seconds. Something I've realized about game development is that you can vibe-code something that looks like a computer game, and that's easy. Building a game that's fun, has a good gameplay loop, and is challenging and interesting and keeps people coming back for more... that's still beyond me, and beyond any of the agents I've tried. This ties into the Deep Blue thing. Just because we can make something that looks like a game does not mean that we are game developers. September # We're into September now. So much has happened this month! Discovery of a new OpenAl agent message board Sydney Von Arx, Cormac Slade Byrd, Spencer KittsThomas Larsen - 4 September 2026 # An independent group of researchers found a message board where OpenAI agents-in-training had been illicitly communicating with each other... and it was that German language wiki I showed you earlier. The one from June. I wrote more about that here . OpenAI had confessed to the Hugging Face thing, but now there's this other incident which surely they should have known about from reviewing their logs. It was surprising that this took an independent group of researchers to uncover. OpenAl agents carried out an undisclosed cyber-attack on RubyGems Spencer Kitts, Thomas Larsen, Sydney Von Arx - 11 September 2026 # And then a week later those same researchers found that the attack on RubyGems back in May was caused by OpenAI's agents in training as well! At this point I'm wondering how many more incidents like this there are that we haven't found yet. Clearly this was a big problem for months before anyone figured out what was going on. Headline: Australian PM warns in UN speech about the ‘furious pace’ of Al after security breach # Then just the other day , here's the Prime Minister of Australia at the United Nations General Assembly warning that OpenAI had hacked the Australian healthcare website that I showed you earlier. I think that was part of the same training run as the Wiki stuff, because there were posts on that Wiki mentioning .gov.au websites and that training appeared to involve researching statistics online to answer questions in an evaluation suite. This story is still coming together, but now it's an international incident that's been raised at the UN by a head of state! www.felonybench.com OpenAI: 11 Anthropic: 9 Google: 3 Meta: 1 # This does mean we've got a new benchmark, probably more useful than my pelicans. FelonyBench.com tracks the number of felony cyberattacks from different labs. OpenAI currently lead with 11, Anthropic have 9. Google have three, which they confessed to the Wall Street Journal a couple of weeks ago. They said they had previously chosen not to disclose because the agents had stopped when they realized that they shouldn't be doing that. Meta have one too . So felonies all round for the AI labs. Pelicans for GPT-6 Astra, GPT-6 Sol, and GPT-6 Luna. All are good, all have the same color scheme. # Here's our current state of the art for the pelicans. This is the GPT-6 family, which just came out . Astra made a fantastic pelican riding a bicycle. It's got the legs on both sides. The frame is good. It's interesting how all of the GPT-6 models pick a similar color scheme to each other. GPT-6 Luna for 0.4 cents will draw you a competent-ish pelican riding a bicycle! Grid for Claude Fable 5.1, Opus 5.5, OPus 5, Sonnet 5. The Sonnet pelicans are terrible. All of the others are pretty good. Opus 5.5 is missing its Max level pelican because it ran out of tokens. The best is Fable 5.1 at Max. # Claude has caught up a little bit. Claude Fable 5.1 gave me an excellent pelican riding a bicycle - the best I've seen from a Claude model - but did charge me $3.30 for it. Opus 5.5 thought for 128,000 tokens and then gave up! It ran out of tokens before it got to the response. It doesn’t get easier - you just get faster Greg LeMond 3x Tour de France champion # Getting back to Deep Blue. Something that's been puzzling me this year is this: why does my job feel harder? I've got these agents that can do all of this stuff for me, and yet I've never worked so hard, I've never been so intellectually engaged with my work. Partly this is because I'm being a lot more ambitious with what I take on, but it's also because all of the easy stuff is handled for me. If it's easy, the agent will do it. Everything that's left for me is difficult. This morning I heard this quote from three-time Tour de France champion Greg LeMond : It doesn't get easier, you just get faster. I think that's exactly what's happening to us now as software engineers with coding agents. Kakapo population reaches new milestone The official population of the critically endangered kakapo has reached a recovery-era high of 325 birds. # One closing thing. I know you're desperate for an update on Kākāpō breeding season. We've reached a recovery-era high of 325 birds ! 89 new chicks have made it to this point. This is the best breeding year in a very long time. Kakapo party, click for confetti. # I heard that Claude Opus 5.5 can now do pixel art. Claude doesn't have an image generator, but it's very good at using JavaScript to draw animated pixels. So I had it make me a Kākāpō dance party . I think this is a good celebration of the most important news of this year. Tags: ai , generative-ai , llms , annotated-talks , ai-security-research , openai-hugging-face-incident

Simon Willison73

Photo Scrubber — 本地人脸模糊与元数据移除

工具:Photo Scrubber — 本地人脸模糊与元数据移除。我拍摄了一些抗议者的照片,后来想到我不喜欢分享带有可识别人脸的陌生人照片。我让 GPT-6 Astra 构建了这个实验性工具,用于识别人脸并自动将其模糊处理。它使用 Google 的 MediaPipe C++ 库,通过 @mediapipe/tasks-vision 编译为 WebAssembly,此外还使用了 BlazeFace 人脸检测模型。标签:摄影,工具

SiliconANGLE AI54

Google’s new frontier AI model Gemini 4 Argon goes to cybersecurity defenders first

Google LLC today began rolling out Gemini 4 Argon, a frontier artificial intelligence model that beats rival models from Anthropic PBC and OpenAI Group PBC on most of Google’s benchmarks, to vetted cybersecurity defenders. Outside Google’s own teams, only members of its Fairwind Program can use Argon for now. The program opened Sept. 3 with […] The post Google’s new frontier AI model Gemini 4 Argon goes to cybersecurity defenders first appeared first on SiliconANGLE .

The Verge AI72

Google’s new Guided Vision feature can help you read the fine print

Guided Vision 今日在兼容的 Android 设备上的 Gemini Live 中推出,利用 AI 为你实时描述手机摄像头所指向的任何事物。通过在 Gemini Live 中分享摄像头画面,你可以让 Google 的 AI 帮助你阅读小字、描述周围环境、查找或识别附近的物体 […]

The Decoder57

Deepmind researchers propose "Artificial Symbiotic Intelligence" as an alternative to the singularity

According to researchers at the Deepmind Institute, general AI won't emerge as a single supermodel but as a network of cooperating agents and humans. What will matter most, they argue, is not model size but the rules and institutions that govern how they all work together. The article Deepmind researchers propose "Artificial Symbiotic Intelligence" as an alternative to the singularity appeared first on The Decoder .

arXiv cs.LG80

用于不确定性下语义承诺的 Credal Large Language Models

arXiv:2608.23244v2 公告类型:替换交叉版本 摘要:大型语言模型(LLM)经常以毫无根据的置信度生成流畅但错误的答案。其核心局限在于,标准 LLM 通过单一预测分布表示不确定性,将认识性无知与真正的歧义混为一谈。我们提出 Credal Large Language Models(CLLMs):由一组 LoRA 适配器构成的集成模型诱导出一个信念集,其下概率和上概率揭示可能预测分布的分散程度,而不是将其压缩为单一的 softmax 输出。基于这一表示,我们推导出一种单一的承诺规则:仅当某个答案的下概率超过所有其他候选答案的上概率时,模型才承诺该答案;否则,模型返回所有可能的预测器都无法排除的答案集合。我们在两个层次上应用这一承诺规则:Credal Token Commitment(CTC)将其应用于一次集成前向传播中的答案 token,从而无需任何生成过程即可决定受约束的答案;对于开放式答案,credal 解码仅当没有任何已完成答案支配某个部分答案时,才会扩展该部分答案,因此生成的补全结果都是可能预测器所支持的结果;Credal Semantic Commitment(CSC)则将该规则应用于这些结果的语义聚类。我们使用 Gemma-2-9B、Llama-3.1-8B 和 Qwen2.5-7B,在 OpenBookQA、CoQA、TriviaQA 和 ARC-Challenge 上评估 CLLMs。在多项选择任务中,CTC 能对 73%–91% 的问题作出承诺,准确率达到 89%–98%;其返回的选项集合平均包含 1.1–1.5 个选项,并以 89%–98% 的比例包含标准答案;在无需校准的情况下,其区间在 30 个置信度分箱中的 24 个分箱内包含观测准确率。加入损坏的上下文后,承诺率从 87%–92% 降至 65%–71%;在 Gemma 上,credal 界检测损坏的能力优于所有基线方法。在开放式问答任务中,在固定覆盖率下,对于每一种骨干模型,CLLM 在带上下文的 CoQA 和 TriviaQA 上相较于 semantic entropy 和 Laplace-LoRA,准确率最高分别提升 19% 和 9.5 个百分点。

arXiv cs.LG78

为魔鬼代言:现成的人格向量在抑制谄媚行为方面可与针对性引导媲美

arXiv:2605.21006v4 发布类型:replace-cross 摘要:语言模型往往表现出谄媚倾向:无论用户陈述的观点是否正确,它们都会表示赞同。既有研究表明,可以通过使用谄媚人格向量引导模型来控制这一特征(Chen et al., 2025)。然而,这类向量是从关于谄媚行为本身的数据中提取的。我们探讨了这样一个问题:能否改为复用用于一般角色的现有向量——怀疑者、评判者、魔鬼代言人——这些向量的提取完全没有针对谄媚行为。我们在 Gemma 2 27B 和 Qwen 3 32B 上,将这些角色向量与专门构建的 Contrastive Activation Addition(CAA)谄媚向量进行比较。实验使用了一个大部分留作测试、经过反平衡处理的 PhilPapers 基准,并在单独划分的谄媚数据集上调节任务特定的系数。被选中的“关键”角色平均实现了 CAA 在降低谄媚 logit 方面约 68%(Gemma)和 98%(Qwen)的效果。在我们检查的探针上,较少表示赞同并不意味着会产生更多事实错误:对于 16 条真假事实陈述,由怀疑者或评判者向量引导的 Qwen 在全部 16 个案例中仍然给出了正确答案,与未引导模型和 CAA 的表现一致。“顺从者”角色并不能可靠地产生相反效果。在我们进行引导的层中,角色向量与测得的 CAA 方向之间的绝对余弦相似度也很低;它们是几何上相互独立的干预方式,尽管这本身并不能证明它们通过不同的下游机制发挥作用。总体而言,这些结果表明,即使在提取时没有使用谄媚行为特定标签,一般人格向量也能帮助缓解 LLM 的谄媚倾向。代码:https://github.com/vkmk1/Sycophancy-Steering 结果:https://github.com/kelkarI/sycophancy-clean-results

arXiv cs.LG77

TANGO: Treating Tokens as Operators

arXiv:2608.22117v2 Announce Type: replace Abstract: Transformers separate cross-token mixing in self-attention from token-wise transformation in feed-forward networks. We ask whether combining these operations can lower predictive loss under fixed data and parameter budgets. To do so, we introduce the Token-Aggregated Nonlinear Gating Operator (TANGO) model. TANGO computes a nonlinear feature-wise gate at each source token. Attention averages these gates for each destination. The average modulates a linear projection of the destination and forms the diagonal core of a source-conditioned linear operator. We test this proposal by comparing full-prefix and windowed TANGO with looped and untied Transformers, the Gated Attention Unit (GAU), and Fast Linear Attention with a Single Head (FLASH) on web text, Lean formal mathematics, DeepMind Mathematics, and code. The comparison uses two parameter scales, two depths, and three seeds. Checkpoints are selected on development data and evaluated on held-out test data. At matched parameters and training data, full-prefix TANGO has the lowest mean test negative log-likelihood in all 16 settings. In eight additional combinations of size and dataset, its development loss never increases as depth rises from 4 to 8 to 16, whereas the looped Transformer's loss increases in four. Full-prefix TANGO is computationally expensive because it averages wide gates over every visible source. To reduce this cost, we evaluate a variant with three narrower gated-projection sets assigned to the first, middle, and last applications. Across four FineWeb-Edu settings, this variant achieves 3.26 to 3.45 times the throughput of TANGO and 75% to 96% that of the looped Transformer. Its mean development negative log-likelihood is lower than TANGO's in three settings and 0.023 higher in the fourth, while remaining lower than both Transformer baselines in all four.

arXiv cs.LG78

AEGIS:用于知识保持型视觉-语言-动作微调的锚点强制梯度隔离

arXiv:2604.16067v2 公告类型:替换 摘要:对预训练视觉-语言模型(VLM)进行机器人操作微调时,会引入一个根本性的稳定性-可塑性两难:连续流匹配动作专家将集中的低秩回归梯度反向传播至基于高维交叉熵目标训练的 Transformer 主干网络中。这种跨模态梯度不对称会迅速削弱预训练的视觉推理能力。现有解决方案要么通过停止梯度切断连续梯度流,要么通过 LoRA 约束更新;前者切断了梯度传播,后者限制了更新秩,但无法感知语义破坏;此外,两者通常都依赖混合批次的 VQA 协同训练,使训练计算量翻倍。我们提出 AEGIS(Anchor-Enforced Gradient Isolation System),一种无需缓冲区、逐层进行正交梯度投影的框架,使连续流匹配微调得以进行,同时将预训练表示与破坏性的参数更新隔离开来。在训练开始前,AEGIS 利用预训练数据估计各层高斯激活统计量,并将其作为静态参考锚点。在微调过程中,闭式 Wasserstein-2 传输惩罚通过活动计算图生成锚点恢复梯度。随后,顺序双反向传播过程执行逐层 Gram-Schmidt 正交化:当方向发生冲突时,将任务梯度投影到恢复向量的正交补空间中。我们为逐层正交投影建立了精确的能量保持界,结果表明,AEGIS 在实证中仅损失 0.62% 的梯度能量,同时阻止特征漂移累积。在 LIBERO 操作基准上对 PaliGemma2-3B 进行微调时,AEGIS 在保持连续动作收敛效果相当的同时,完整保留了预训练的视觉问答性能和基线留出损失,并且无需回放缓冲区、教师模型或协同训练数据。

arXiv cs.LG82

开放源代码大型语言模型在临床分诊中的偏差反事实审计

arXiv:2610.01963v1 公告类型:cross 摘要:急诊科(ED)分诊是一项高风险的优先级排序任务,其中人口统计、社会经济和系统环境信息可能不恰当地影响紧急程度分配。尽管开放源代码大型语言模型(LLM)正越来越多地被考虑用于本地化且注重隐私的临床决策支持,但反事实偏差如何在不同模型系列、模型规模、医学领域模型和领域适配模型之间变化,仍不明确。我们针对十个开放源代码LLM的儿科急诊严重程度指数(ESI)预测开展了比较性反事实审计。我们以真实临床案例和手册式临床案例为基础,构建成对的反事实变体:每次仅改变一个注入的人口统计、社会经济、医疗服务可及性、行为、社会或系统环境变量,同时保持临床表现不变。模型包括Qwen2.5-7B、Qwen2.5-14B-Instruct、经过QLoRA微调的Qwen2.5-7B、MedGemma变体、MedLLaMA2-7B、GPT-OSS-20B和GPT-OSS-120B。我们测量了任何反事实变化、分诊不足、分诊过度、超过一个ESI等级的变化、平均变化幅度以及平均绝对变化幅度。反事实敏感性存在显著差异,并未随着模型规模扩大或医学领域预训练而稳定降低。经过微调的Qwen2.5-7B表现出最低的总体敏感性,其任何变化率为5.27%,平均绝对变化幅度为0.0534;相比之下,基础模型分别为16.02%和0.1706。若干规模更大或属于医学领域的模型表现出更显著的变化。分层分析和相关性分析进一步揭示了总体比率所掩盖的、具有临床重要性的变化方向和共同失效模式。这些发现表明,在临床部署之前,反事实审计可作为一种轻量且具有临床可解释性的框架,用于比较开放源代码LLM中的公平性风险。

arXiv cs.LG73

协作技术文档的句子具体性评分:一项领域迁移研究

arXiv:2610.01046v1 公告类型:cross 摘要:协作依赖于共享上下文,而技术文档是这些上下文得以在人与 AI 队友之间延续的一种方式。具体性,即语言所表达细节的数量与精确程度,会影响文档所捕捉的信息,以及这些信息传达的准确程度。本研究审查了用于技术文档的句子具体性评分工具,并测试了仅在生成之后应用评分是否有助于在固定的 LLM 生成修订版本中进行选择。在 Wikipedia 和三个技术文档语料库中,固定的通用领域预测器 SpeciTeller 与 Ko 等人的目标适配预测器在作者代码仓库中发布后的固定实现,产生了不同的语料库排序;对于同一句子的排序一致性,其范围为 -0.066 至 0.510。严格过滤和词元长度调整改变了这些模式,但未能使二者趋于一致。在 Gemma 数据集中,使用 SpeciTeller 排序使方向有效的选择比例从 71.7% 提升至 83.3%(+11.7 个百分点;95% 源案例自助法区间为 +1.7 至 +21.7);在 GPT-OSS-120B 数据集中,使用 SpeciTeller 排序使方向有效的选择比例从 51.7% 提升至 56.7%(+5.0 个百分点;95% 源案例自助法区间为 -6.7 至 +16.7),且所有主要的单评分 GPT-OSS-120B 区间均包含零。这些发现表明,评分的解读及其决策价值取决于预测器和候选集。

arXiv cs.LG81

使用国际象棋对大型语言模型的提示优化进行基准测试

arXiv:2610.00416v1 公告类型:cross 摘要:随着大型语言模型能力的提升,对其进行评估变得愈发困难:基准测试可能达到饱和,公开测试集存在数据污染风险,而评估更具挑战性的任务可能需要昂贵的评分或执行基础设施。在自动提示优化(APO)中,这些挑战更加突出,因为在寻找更优提示的过程中需要反复进行评估。因此,研究 APO 需要一种评分成本低且具有确定性的基准,同时任务难度足够高、仍有改进空间,并且能够随着模型的发展不断更新。我们构建了一个由 1,118 道 Lichess 棋题组成的国际象棋基准,用于研究冻结 LLM 的 APO:我们优化其提示,而不更新模型权重。国际象棋结合了成本低廉的精确匹配评分、基于引擎对备选走法的评估,以及可持续更新且难度可调的问题来源。不同于只报告孤立测试题成功率的评估方式,该基准还将解题能力的提升与同一领域内的短局对弈展开联系起来。我们使用该基准评估了 8 个目标模型上的 6 种 APO 算法,不仅测量基线能力,还考察每个模型对优化的响应程度,以及优化后的提示能否迁移到其他模型和对弈任务中。因此,国际象棋非常适合作为 APO 基准,原因在于它:(i)具有挑战性,即使是评估中最强的模型 Gemini 3.5 Flash(用作元模型)也只能解决约 55\% 的棋题;(ii)具有区分度,能够揭示不同方法和模型之间的性能提升、性能不变和性能退化;(iii)可持续更新,通过提供新棋题降低数据污染风险,并通过调整难度在模型能力提升时保持提升空间;(iv)成本可控,完整研究的运行成本约为 \$800。我们公开了棋题、优化与评估代码,以及数据集更新脚本(https://github.com/imec-ailabs/Automatic-Prompt-Optimization-with-Chess)。

arXiv cs.LG75

LLM品牌推荐中的系统归因:单条响应可识别系统,聚合品牌画像无法迁移

arXiv:2610.00253v1 发布类型:cross 摘要:AI可见性审计会将已部署语言模型的品牌推荐汇总为按系统划分的画像。我们在一个由6,475条已存储响应构成的数据集上检验这种画像是否能够描述系统,其中6,324条可供分析;这些响应于2025年12月至2026年2月期间,从五个已部署端点收集,涉及礼品推荐、企业声誉和品类归属查询。采集工具会截断许多答案:在1,024个token的输出上限下,Gemini 3 Flash在品类归属任务中的答案有83.1%在句子中途结束。将每条答案都截取为前800个字符后,一个按提示属性进行交叉验证的字符n-gram分类器,能够以97.84%的准确率将单条响应归属于GPT-5.2、Gemini 3 Flash、Gemini 3 Flash with search、Grok或Perplexity sonar-pro(5,028条响应、383个提示词、多数类占31.5%、30个划分随机种子)。仅使用长度时,准确率降至多数类比例;24项格式统计特征可达到95.79%,而遮蔽品牌名称和首字母大写的词后,准确率仍为97.72%。在留出的查询条件上,按样本量加权的准确率保持为97.43%,不加权时为88.0%;在一个改变采集工具的检索支撑实验中,没有任何Grok答案被归因于Grok(0/120)。将数据汇总为50个“模型×领域×条件”单元后,12项行为特征在分组交叉验证下以66.53%的准确率区分四个系统;相比之下,标签置换零分布的均值为33.71%,第95百分位数为46.0%。跨领域来看,聚合画像无法迁移:在品类归属单元上训练的随机森林将全部22个礼品单元分配给错误的系统,这与品牌数量发生反转相一致(礼品任务中每条响应的品牌数为8.41对0.94,品类归属任务中为3.01对3.91);而单条响应的平衡准确率为89.92%。在所测试的查询领域之间,答案的表层形式能够携带系统信息;聚合后的品牌行为却无法迁移,而且未进行交叉设计的实验无法将系统与领域或采集工具区分开来。

arXiv cs.LG74

每次消融都是一个剂量:配重与自我修复的表象

arXiv:2610.02173v1 公告类型:新增 摘要:对语言模型的一个组件进行消融,其他组件往往会表现出调整和补偿行为。这一现象被称为自我修复,已被反复观察到,但其机制仍不清楚。迄今为止最系统的研究得出结论:自我修复具有噪声,很可能不存在单一解释。我们认为它确实有一个解释:一种在任何消融发生之前就已存在的增益。对任何具有因果重要性的组件进行干预,都可以视为坐标轴 $\lambda$ 上的一个点,其中 $\lambda$ 表示反事实对比的带符号强度。因此,传统的消融方法就是该轴上未经校准的点。我们表明,细粒度单元 $r$ 的因果修复响应遵循一个仿射规律:$E_r(\lambda)=\mathrm{own}_r+\gamma_r\lambda$。斜率 $\gamma_r$ 是一个固定系数,无论是否进行消融,都会持续影响模型;其符号决定该单元是抵消还是强化被移除的信号。在一项跨四个不同模型系列(Gemma、Qwen、LLaMA 和 Mistral)的事实判定任务中,我们发现包括 MLP 神经元、OV 神经元和奇异方向在内的组件遵循这一仿射规律,共计占81个下游方向中的68个。此外,我们还可以根据固定权重预判 $\gamma_r$ 的大小。在 GPT-2 Small 的 IOI 电路中,干预能够触及的十个注意力头中有七个遵循该规律,而且这七个全都是配重。从这一视角看,表面上像是自我修复的现象,其实是当对比信号在核心处显现时,配重执行其通常的作用。

arXiv cs.LG74

发明一个数据集:用零种子衡量数据集生成能力

arXiv:2610.01674v1 发布类型:new 摘要:构建数据集仍然是 AI 开发中最依赖人工且最脆弱的环节之一。在这份技术报告中,我们聚焦于现实世界从业者面临的最极端、同时也是最普遍的情境:零数据环境。在这种情况下,从业者手头没有任何用于学习目标能力的数据。我们推出 Invent-A-Dataset,这是一个基于提示词的系统,能够从数据集描述出发,生成真实且大规模的后训练数据集。我们将 Invent-A-Dataset 与包括 Anthropic、Google、Open AI、DeepSeek、Zai 在内的五个前沿模型 API 进行评估。在八种任务类型以及最大 20K 个样本的数据集规模上,Invent-A-Dataset 均显著胜出:不仅质量最高(相对提升 17%),同时生成的样本也最为多样(相对提升 19%)。随着训练数据集规模扩大,其多样性优势进一步扩大:从 200 个样本时的持平,提升至 20K 个样本时的相对提升 37%。这进一步转化为显著的下游训练收益,产生性能强得多的后训练模型。在不同的后训练模型架构中,Invent-A-Dataset 微调模型始终比其他生成器微调模型排名更高。

arXiv cs.LG73

具有置换鲁棒性的决策建模与候选项无关的分块因果注意力

arXiv:2610.01601v1 公告类型:新增 摘要:决策模型通常会对编码在单个序列中的、大小可变的候选动作集合进行评分。这一设置与生成式系统内部的 System 1 组件日益相关,因为不同运行过程中提出的候选项或其排序可能有所不同。标准的因果交叉编码具有很强的表达能力,但可能导致某个候选项的得分取决于序列化顺序,而不是底层决策问题本身。我们提出候选项无关的分块因果注意力:该机制保留共享上下文和每个候选项内部的因果计算,同时阻断候选项之间的信息流,并重置候选项位置。我们在 Gemma 3 1B、Qwen3 1.7B 和 Qwen3 4B 主干模型上,将这一架构与标准因果注意力及互补的不变性基线进行了比较。候选项无关的注意力在保持具有竞争力的决策质量的同时,持续降低了对置换的敏感性;消融实验表明,候选项隔离是产生该效果的主要原因,而位置重置则完善了预期的对称性。一项规模更大的 Qwen3-4B 研究进一步考察了所提出架构在显著增加的训练数据下的表现。代码可在 \href{https://github.com/guyAmit/ci-decision-models}{\textcolor{blue}{项目仓库}} 中获取,\href{https://huggingface.co/Guy-Amit/qwen3-4b-ci-decision-4096-poc}{\textcolor{blue}{Qwen3-4B 模型制品}} 可在 Hugging Face 上获取。

BAIR Blog53

2026 BAIR Graduate Showcase

Congratulations to the Berkeley Artificial Intelligence Research (BAIR) Lab class of 2026! This year, BAIR celebrates another remarkable group of Ph.D. graduates whose curiosity, creativity, and perseverance have pushed the frontiers of artificial intelligence and machine learning. Their work spans the breadth of modern AI — robotics and embodied intelligence, large language models and reasoning, computer vision, generative modeling, AI safety, human-AI interaction, AI for science and healthcare, and much more. Along the way, they have published influential research, built systems with real-world impact, mentored their peers, and shaped the BAIR community for the better. Now they are headed everywhere ideas travel: to faculty and postdoctoral positions, to industry research labs, and to startups of their own founding — and several are still exploring what comes next and would love to hear from you. Please join us in celebrating the achievements of these wonderful graduates. We are proud of everything they have accomplished at Berkeley, and we can’t wait to see what they do next! Thank you to our friends at the Stanford AI Lab for this idea! Baifeng Shi Baifeng Shi Email: baifeng_shi@berkeley.edu Website: https://bfshi.github.io/ Advisor(s): Trevor Darrell Research Blurb: I work on building generalist vision and robotic models. What's next: Member of Technical Staff at Physical Intelligence Charlie Snell Charlie Snell Email: csnell22@berkeley.edu Website: https://sea-snell.github.io Advisor(s): Dan Klein Research Blurb: My work aims to understand when and how the different LLM scaling paradigms can be traded off and interchanged. In particular, test-time scaling treats each prompt independently, drawing long chains of inferences and then forgetting them entirely between prompts. This differs critically from pretraining, which instead learns a compressed representation from a large dataset. I believe bridging the gap between these methods of scaling computation, presents a key open challenge in the field: how can we develop methods which turn the inferences drawn at test-time back into learned representations that the model can hold onto across interactions. Devin Guillory Devin Guillory Email: dguillory@berkeley.edu Website: https://devinguillory.com Advisor(s): Trevor Darrell Research Blurb: Accounting for data shifts in computer vision models What's next: Building collaborative AI systems, looking for conspirators. Eve Fleisig Eve Fleisig Email: efleisig@berkeley.edu Website: https://efleisig.com Advisor(s): Dan Klein Research Blurb: I design language models to work reliably and fairly for the broad range of real LLM users. First, my research leverages disagreement among user preferences as signal, in order to train and evaluate LLMs for entire populations of users. Second, I work on designing rigorous evaluations to extricate challenging LLM harms that diverse users face. Finally, I work on core technical failures of LLMs, like miscalibrated confidence, to reduce downstream risks when models are deployed to users with different needs. Combined, these interventions facilitate building LLMs that minimize societal harms, and maximize benefits to a wider range of real-world users. What's next: Postdoctoral fellow at Princeton CITP Grace Luo Grace Luo Email: graceluo@berkeley.edu Website: https://graceluo.net Advisor(s): Trevor Darrell Research Blurb: My research is on interpreting and controlling generative models. For example, I've worked on re-purposing image generators for computer vision tasks, and meta-modeling language activations for better LLM probing and steering. What's next: Research scientist in industry Hanlin Zhu Hanlin Zhu Email: hanlinzhu@berkeley.edu Website: https://hanlinzhu.com/ Advisor(s): Stuart Russell, Jiantao Jiao Research Blurb: My research centers on understanding and improving the reasoning capabilities of large language models (LLMs). What's next: Member of Technical Staff at OpenAI Haozhi Qi Haozhi Qi Email: hqi@berkeley.edu Website: https://haozhi.io/ Advisor(s): Jitendra Malik, Yi Ma Research Blurb: Dexterous Manipulation and Robot Learning What's next: Research scientist at Amazon; Faculty at University of Chicago J.D. Zamfirescu-Pereira J.D. Zamfirescu-Pereira Email: zamfi@berkeley.edu Website: https://zamfi.net Advisor(s): Bjoern Hartmann Research Blurb: My research focuses on effective human-AI co-design. I study the boundaries of language interfaces as a medium for interacting with AI, creating systems that blend language-focused interactions with structured user interfaces that draw on different levels of abstraction. I focus on language-oriented technologies, like LLMs and text-to-image models, that are powerful mediators of design processes. These technologies enable humans to describe their desires at almost any level of abstraction, from high-level goals vaguely specified (“I’d like a game to help my kid learn to read”) to low-level corrections of undesired outputs (“Don’t say ‘I know because I’ve tasted it’ when about a recipe substitution's taste”). What's next: Assistant Professor, Computer Science, UCLA Jiachen Lian Jiachen Lian Email: jiachenlian@berkeley.edu Website: https://jlian2.github.io Advisor(s): Gopala Anumanchipalli Research Blurb: My research focuses on human-centered AI across speech, healthcare, and systems. Looking for: Look for AI talents to join our startup Josh Kang Josh Kang Email: minwoo_kang@berkeley.edu Website: https://joshuaminwookang.github.io/ Advisor(s): John Canny Research Blurb: I study language modeling and related topics in NLP; specific interests are human user simulation and building conversational, collaborative AI agents. What's next: AI Scientist at Mistral AI Junhao (Bear) Xiong Junhao (Bear) Xiong Email: junhao_xiong@berkeley.edu Website: https://www.linkedin.com/in/junhao-bear-xiong Advisor(s): Jennifer Listgarten, Yun Song Research Blurb: Junhao (Bear) Xiong is a PhD candidate at UC Berkeley, advised by Jennifer Listgarten and Yun S. Song. His work focuses on machine learning methods for biology, with an emphasis on generative modeling for proteins. Previously, he studied Applied Math and Computer Science at Johns Hopkins. Looking for: Research scientist Kaylo Littlejohn Kaylo Littlejohn Email: kaylo_littlejohn@berkeley.edu Website: https://kaylolittlejohn.com Advisor(s): Gopala Anumanchipalli Research Blurb: My research is focused on speech modeling and natural language processing. I co-led the development of multimodal AI tools to accurately translate brain activity into text, audible personalized speech, and a high-fidelity "digital talking avatar" (Nature 2023, Nature Neuroscience 2025). I am also tech lead for voice modeling at Roblox. Looking for: Research Scientist / Engineer Kent Chang Kent Chang Email: kentkchang@berkeley.edu Website: https://kentkc.org Advisor(s): David Bamman Research Blurb: I work on NLP and multimodal machine learning, with a focus on evaluating large language models and building multimodal systems for understanding dialogue, narrative, and social interaction. My research includes benchmarks for LLM memorization, multimodal datasets sourced from feature films and television, and studies of model behavior. I'm interested in bridging computational methods with questions from the humanities and social sciences about whose voices get represented in AI systems, and about AI's broader impact. My work has appeared at EMNLP and ACL, among others. Looking for: (teaching) faculty, Research Scientist, ML/AI SWE Kevin Black Kevin Black Email: kvablack@berkeley.edu Website: https://kevin.black Advisor(s): Sergey Levine Research Blurb: I work on large-scale robot learning: including imitation learning, reinforcement learning, generative modeling, real-time control, and whatever else it takes to make robots work in the real world! What's next: Research Scientist of Physical Intelligence Kunhe Yang Kunhe Yang Email: kunheyang@berkeley.edu Website: https://www.kunheyang.com/ Advisor(s): Nika Haghtalab Research Blurb: My research focuses on the theoretical foundations of designing and evaluating AI algorithms in environments shaped by human incentives and AI agency. My work spans human-centric policy learning, incentive-aware evaluation, and multi-agent collaboration and information transmission, drawing on tools from machine learning theory and computational economics. What's next: Postdoc Research at Stanford Lisa Dunlap Lisa Dunlap Email: lisabdunlap@berkeley.edu Website: https://lisabdunlap.com Advisor(s): Joseph Gonzalez, Trevor Darrell Research Blurb: Auditing generative models. What's next: Research Engineer at Anthropic Long (Tony) Lian Long (Tony) Lian Email: longlian@berkeley.edu Website: https://tonylian.com/ Advisor(s): Trevor Darrell, Adam Yala Research Blurb: My research primarily focuses on developing real-time multi-modal multi-agent systems and parallel reasoning systems through end-to-end RL. What's next: Member of Technical Staff at Thinking Machines Lab Maulik Bhatt Maulik Bhatt Email: maulikbhatt@berkeley.edu Website: https://maulikb.com Advisor(s): Negar Mehr Research Blurb: My research develops autonomous robots that can safely coordinate with humans and other robots in shared environments. I build scalable algorithms grounded in game theory and diffusion models that let agents reason about the intent and behavior of others around them. My work spans real-time multi-agent trajectory planning and imitation learning in the presence of multi-modality. I've validated these methods on hardware platforms ranging from quadrotors to manipulators, with the goal of making multi-agent coordination robust, interpretable, and deployable in the real world. What's next: Joining Toyota Woven's end-to-end autonomous driving team. Michael Psenka Michael Psenka Email: psenka@berkeley.edu Website: https://www.michaelpsenka.io/ Advisor(s): Aditi Krishnapriyan Research Blurb: Work in various domains (reinforcement learning, world models, AI+bio/chem), generally working on longer-horizon and out-of-distribution problems in planning and interpolation (e.g. robot manipulation from start state to goal, molecular dynamics of proteins between ground states). My thesis took a variational approach (think calculus of variations) directly from deep generative models of the environment, framing path-finding as minimizing a functional induced by the learned model itself (its score, its critic, or its dynamics). Through my research I've gained insight on how to properly handle dynamics in deep learning systems, and I plan to continue developing systems that are dynamic and adaptive. What's next: Lead Research Scientist at Baseten Nathan Lichtlé Nathan Lichtlé Email: nathan.lichtle@gmail.com Website: https://nathanlichtle.com Advisor(s): Alexandre M. Bayen Research Blurb: RL for autonomous driving. What's next: Chief Scientist & Co-founder at Yumi Health Neerja Thakkar Neerja Thakkar Email: nthakkar@berkeley.edu Website: https://neerja.me/ Advisor(s): Jitendra Malik Research Blurb: My research focuses on scaling predictive world models to handle the complexity of in-the-wild motion. Using autoregressive and diffusion frameworks, I develop better representations for real-world prediction and propose methods to efficiently adapt these models to new domains. Looking for: Research scientist Nikita Mehandru Nikita Mehandru Email: nmehandru@berkeley.edu Website: https://n-mehandru.github.io/ Advisor(s): Ahmed Alaa and David Bamman Research Blurb: My research develops and applies machine learning methods for clinical reasoning and disease progression modeling using unstructured text and time series data from electronic health records. In collaboration with physicians at UCSF, I bridge method development and clinical validation with the intention to build reliable, interpretable AI systems in medicine. Looking for: Research Scientist Niklas Lauffer Niklas Lauffer Email: nlauffer@berkeley.edu Website: https://niklaslauffer.github.io/ Advisor(s): Stuart Russell and Sanjit Seshia Research Blurb: Niklas's research is focused on AI safety and reinforcement learning, particularly in the area of multi-agent interaction and LM agents. He's worked on enabling adversarial learning in cooperative and mixed-motive settings, solving issues of covariate shift in training LM agents on long-horizon tasks, as well as evaluating safety risks posed by LM agents in multi-agent settings. What's next: Research Scientist at Google Deepmind Qiyang Li Qiyang Li Email: qcli@berkeley.edu Website: https://colinqiyangli.github.io/ Advisor(s): Sergey Levine Research Blurb: Recent progress in robotic manipulation policy learning has been largely driven by (1) the increasing availability of large-scale prior datasets and (2) the success of action chunking, where the policy predicts a short sequence of future actions rather than a single one. However, most action chunking policies are trained via supervised imitation learning, because efficient online self-improvement with reinforcement learning (RL) remains challenging—limiting real-world applicability. My PhD research studied how we could leverage prior data to optimize action-chunking policies with RL, combining empirical results with theoretical insights. Looking for: Post-doc/research scientist for RL in robotics and LLMs! Sampada Deglurkar Sampada Deglurkar Email: sampada_deglurkar@berkeley.edu Website: https://sdeglurkar.github.io/ Advisor(s): Prof Claire Tomlin Research Blurb: My research is in providing safety assurances for AI-enabled autonomous systems, ranging from robots to autonomous vehicles to aviation systems. For this, I have worked with uncertainty quantification for machine learning models, decision-making under uncertainty algorithms, and tools for producing probabilistic guarantees on system operation. Looking for: Research scientist, Research engineer Vinamra Benara Vinamra Benara Email: vbenara@berkeley.edu Website: https://cs.berkeley.edu/~vbenara Advisor(s): Ion Stoica Research Blurb: My research focuses on LLM post-training, including data curation, RLHF, RLVR with VLMs, evaluations, reasoning, agentic workflows, and interpretability. I also have strong expertise in systems infrastructure for distributed computing. Looking for: Research scientist / Research Engineer Vongani Maluleke Vongani Maluleke Email: vongani_maluleke@berkeley.edu Website: https://people.eecs.berkeley.edu/~vongani_maluleke/ Advisor(s): Jitendra Malik and Angjoo Kanazawa Research Blurb: Vongani Maluleke is a PhD candidate at UC Berkeley (BAIR, advised by Jitendra Malik and Angjoo Kanazawa), where she led the development of MAGNet, a unified multi-agent motion generation framework that supports a wide range of motion generation tasks without retraining or architectural changes, outperforming task-specialized state-of-the-art baselines. She is currently extending this work by deploying it on a Unitree G1 humanoid to make it embody social intelligence. Before her PhD, she was a Senior AI Consultant at Deloitte, awarded Exceptional Performer two consecutive years, leading AI system development across media, telecommunications, retail, and financial services. Looking for: Research scientist Wei-Jer Chang Wei-Jer Chang Email: weijer_chang@berkeley.edu Website: https://weijer-chang.github.io/ Advisor(s): Masayoshi Tomizuka Research Blurb: My research focuses on developing safe and intelligent autonomous systems for complex, human-centered environments. I work at the intersection of machine learning, generative models, and reinforcement learning, with applications in autonomy. My work addresses challenges in multi-agent interaction, interactive human behavior, and long-tail safety-critical scenarios at scale. Looking for: Research Scientist, Applied Scientist, Roboticist Xiuyu Li Xiuyu Li Email: xiuyu@berkeley.edu Website: https://xiuyuli.com/ Advisor(s): Kurt Keutzer Research Blurb: My research focuses on developing scalable and self-improving large language model agents, with emphasis on coding agents for complex, long-horizon tasks. This direction builds on my work in parallel reasoning, and on broader expertise in making generative models more efficient in training and inference across language and vision. What's next: Member of Technical Staff at xAI Yichen Xie Yichen Xie Email: yichenxie0928@gmail.com Website: https://yichen928.github.io/ Advisor(s): Masayoshi Tomizuka Research Blurb: My research focuses on building multimodal foundation models and world models that understand and interact with complex physical environments. I aim to develop unified representations across modalities, enabling AI systems to reason over space, time, and dynamics toward general-purpose embodied intelligence. What's next: Research Scientist at Luma AI Yigit Efe Erginbas Yigit Efe Erginbas Email: erginbas@berkeley.edu Website: https://www.linkedin.com/in/erginbas/ Advisor(s): Kannan Ramchandran, Thomas A. Courtade Research Blurb: My PhD research spans two threads: online learning in large-scale markets, and interpretability of large machine learning models. In the first, I work on sequential decision-making with applications to recommendation, pricing, and assortment selection. My focus is on designing algorithms with provable guarantees for welfare maximization, revenue maximization, and stability. In the second, I develop scalable attribution methods that exploit the sparse, low-degree structure of real-world interactions, using tools from signal processing and information theory. More recently, I have been exploring principled ways to evaluate the faithfulness of model self-explanations. What's next: Researcher at Hudson River Trading's AI Labs (HAIL) Yiheng Li Yiheng Li Email: yhli@berkeley.edu Website: https://Yihengli.com Advisor(s): Masayoshi Tomizuka Research Blurb: I am working on vision world modeling, with prior experience in diffusion model's efficiency as well as in autonomous driving. What's next: Research Scientist at Waymo Zhe Fu Zhe Fu Email: zhefu@berkeley.edu Website: https://fu-zhe.com/ Advisor(s): Alexandre Bayen Research Blurb: My research focuses on physics-informed learning and control for mixed-autonomy systems, with applications in transportation. I design physics-informed neural networks to learn solutions of nonlinear partial differential equations, enabling accurate and data-efficient prediction of traffic dynamics. Building on these models, I develop both model-based and learning-based control strategies that coordinate automated vehicles to improve system-level performance. My work bridges machine learning, control, and real-world deployment, and has been validated in large-scale field experiments. More broadly, I aim to advance trustworthy, interpretable AI for decision-making in complex, real-world systems. What's next: I will be an Energy Fellow at Stanford after graduation. Also looking for Faculty, or research scientist positions in AI, control, and autonomy.

BAIR Blog78

From CUDA to MLX: How K-Search Brings Decades of Kernel Expertise to Apple Silicon

Kernel knowledge transfer from CUDA to MLX Figure 1: CUDA-to-MLX optimization translation map. CUDA optimization knowledge can be translated into architecture-native MLX strategies rather than copied instruction-for-instruction. We face a new epoch in computing. Hardware is changing rapidly — not just faster GPUs, but a growing range of chips from different vendors, each with its own architecture and often tailored to specific AI workloads. Software is changing just as fast, and AI coding tools now generate in minutes what took months of effort a few years ago. With so much of computing now centered on AI, GPU kernels are a crucial component of its success. These are the low-level programs that run inside the GPU, and writing efficient ones is far from obvious — it takes years of expertise to get right. Transferring a kernel from one vendor’s hardware to another is harder still, and often means rediscovering the same optimizations from scratch. The CUDA ecosystem, for example, has accumulated decades of hard-won kernel expertise: hand-tuned implementations of attention, state space models, and other critical operations representing thousands of engineering hours. Newer hardware ecosystems (Apple Silicon, custom AI accelerators, and others) are growing fast but lack this depth. In this work we ask whether that expertise can be transferred automatically. We built on K-Search , an evolutionary kernel search framework introduced by Cao et al. at Berkeley Sky Lab that uses AI to optimize GPU kernels, and extended it with a backend for MLX — Apple’s machine-learning framework for its own Apple Silicon chips. We developed a novel structured CUDA-to-MLX translation layer that lets K-Search take existing CUDA kernels as a knowledge base and adapt them into high-quality GPU kernels for Apple Silicon, rather than rebuilding from scratch. We show that our approach reaches near-expert level performance on Apple Silicon with 0.97x speedup compared to the native MLX Attention kernel, and up to a 20x prefill speedup over the community mlx-lm implementation on the Mamba SSM kernel; we report the numbers, and how much of the gain comes from the translation layer, in the sections below. Although we focus on MLX kernels for Apple Silicon, the method is not specific to MLX and applies to any ecosystem where CUDA expertise is transferable. Why MLX? Apple’s MLX framework has seen remarkable adoption since late 2023. With Apple Silicon in hundreds of millions of MacBooks and Mac Studios, MLX enables local AI inference without cloud costs. The unified memory architecture makes it especially attractive for mid-sized models (7B–70B parameters on M series chips). Yet beneath this momentum lies a significant gap: many performance-critical kernels that the NVIDIA ecosystem takes for granted: paged attention, optimized SSM scan kernels, fused MoE routing are either absent or naive without hardware-specific tuning. MLX runs models correctly but often leaves significant performance on the table. This gap is what motivates the rest of this post. What is K-Search? K-Search is an evolutionary kernel optimization framework originally developed by our first author Shiyi Cao at UC Berkeley Sky Lab. Given a naive kernel and a hardware specification, it runs an iterative optimization loop: an LLM reasons about which optimizations to try next, a code-writing model generates candidate kernels, and those candidates are compiled and benchmarked on real hardware. Measurements feed back into the search, which keeps refining, pursuing promising directions and dropping dead ends until performance converges. Pseudocode for K-Search via co-evolving world models Algorithm 1: K-Search via co-evolving world models. The search alternates between selecting the most promising action, instantiating and evaluating code until improvement stagnates, and evolving the world model through insert, update, and prune operations. Adapted from Cao et al. (2026) . Search is grounded by a Spec: a domain-specific document encoding hardware rules, optimization patterns, and mathematical constraints which keeps generated code from hallucinating invalid primitives and ensures candidates will actually compile and run efficiently. In our runs, a single model (Gemini 3.5 Pro Preview) plays both roles: it maintains the reasoning state and writes the kernels. The reasoning half is prompted as a “GPU kernel performance engineer” and asked to work through a fixed analysis before proposing anything: classify the kernel (reduction, scan, attention/softmax, …), rewrite the reference computation in canonical form, map out data layout and access patterns, and hypothesize the likely bottleneck (bandwidth, latency, compute, or synchronization) in each runtime regime. Only then does it emit candidate optimizations, each as a single change implementable in one iteration. We call the persistent reasoning state a world model . Rather than a flat list of things to try, it is a decision (prefix) tree: each root→leaf path composes a full optimization plan, and sibling branches are competing alternatives. Every node is scored — an overall_rating in [0, 10], a confidence in [0, 1], and per-node impacts on memory bandwidth, register pressure, and compute/hardware fit — so the search can rank partial plans and expand the most promising ones. The tree persists and grows across rounds: refining an idea adds a child node rather than overwriting its parent, and if the best score fails to improve for a few rounds (a stagnation window) the search backs off to explore an alternative branch. A single node, as it appears mid-run on the attention kernel, looks like this: { "action" : "Replace the threadgroup-memory softmax reduction with a register-only reduction: each SIMD group owns 8 query rows and reduces across lanes with simd_shuffle_xor, removing a threadgroup_barrier." , "difficulty_1_to_5" : 4 , "impacts" : { "memory_bandwidth" : 8 , "register_pressure" : 4 , // risk: spill if Br > 8 "compute_hw_fit" : 9 // SIMD width 32 ; keep tile 8 x 8 }, "overall_rating_0_to_10" : 8 , "confidence_0_to_1" : 0.7 } Listing 1: Example K-Search world-model node. Each candidate optimization records a concrete action, estimated hardware impacts, an overall priority rating, and the model's confidence. Overview of the K-Search loop Figure 2: Overview of K-Search. The framework operates on a Search State $S_t$ structured as a search tree. The tree consists of Closed nodes (blue, visited states with attached program like $x_{12}$) and a Frontier of Open nodes (orange, pending hypotheses like $u_{13}$). The workflow iterates through three phases: (1) Action Selection , where the most promising action node is retrieved from the frontier based on world model estimated priority score $V$; (2) Local Refinement , where a stochastic policy $\pi_{\mathrm{code}}$ samples concrete implementations until stagnation; and (3) World Model Update , where the LLM reasons over the trajectory to update the search tree via Insert (adding new actions), Update (adjusting $V$, e.g., $u_{11}$ dropping from 0.9 to 0.6), and Prune (removing less promising nodes like $u_{10}$). The original K-Search paper evaluated this search strategy on CUDA kernels from FlashInfer. Across GQA decode, MLA decode, MLA prefill, and MoE, K-Search improved more consistently than OpenEvolve and ShinkaEvolve over the same 120-iteration budget. These results establish the search framework we build on here; the remainder of this post asks whether its optimization knowledge can transfer beyond CUDA. K-Search benchmark results compared with OpenEvolve and ShinkaEvolve Figure 3: Main results from the original K-Search paper. Across three runs, K-Search achieves stronger best-so-far search scores, per-workload kernel performance, and speedup distributions than OpenEvolve and ShinkaEvolve on four FlashInfer CUDA kernels. Reproduced exactly from Cao et al. (2026) . Building an MLX backend To bring K-Search to Apple Silicon, we first built a native MLX backend. We implemented a full MLX-specific task adapter for K-Search, including: An MLX task backend in k_search/tasks/ handling kernel compilation and execution on Apple Silicon via MLX’s Metal/C++ APIs. Updated kernel generator prompts for writing and modifying Metal/MLX kernels. MLX-specific benchmarking integration using mlx.core measurement utilities. Translating CUDA expertise to MLX However, the more interesting challenge was not simply running K-Search on MLX. The key insight is that expert CUDA kernels encode decades of optimization knowledge that is transferable to Apple GPU if you can bridge the conceptual gap. Simply handing an LLM a CUDA kernel and asking it to port it is not enough: without deep hardware context, it produces code that is syntactically valid but architecturally wrong (wrong tile sizes, invalid primitives, mismatched memory assumptions). Our translation layer consists of: Concept mapping tables: A structured glossary of CUDA primitives and their MLX/Metal equivalents with hard constraints. For example: __shared__ maps to Metal threadgroup memory but with a hard 32 KB limit (vs. NVIDIA’s 48 KB) warp_reduce maps to MMA (preferred) __syncthreads() becomes threadgroup_barrier(mem_flags::mem_tg) H100’s ~3.35 TB/s HBM3 maps to M3 Max’s ~400 GB/s unified DRAM a bandwidth difference that reshapes which optimizations are worth pursuing. MLX-specific hints and patterns: Concrete code-level patterns for operations with no direct CUDA equivalent, such as register-based row reductions using simd_shuffle_xor in an 8×8 MMA tile layout, or the “exp2 trick” (replacing $exp(x)$ with $exp_2(x \log_2 e)$) for faster softmax on Apple’s fast $exp_2$ hardware instruction. Reusable assertions: Expert kernel behaviors reframed as properties the evolutionary search must preserve, rather than code to copy. Matching expert kernel performance: the Attention kernel We evaluate three configurations of an MLX attention kernel for Apple Silicon: (1) a naive baseline, (2) pure evolution with no additional provided context, and (3) a full context translation layer, which supplies the optimizer with architecture-specific implementation knowledge extracted from high-performance kernels (e.g., FlashAttention-2), letting the evolutionary search reason about implementation strategies rather than starting from a naive kernel. Together, these three configurations let us isolate the exact impact of the translation layer. Performance scaling of the Attention Kernel through stacked optimizations Figure 4: Performance scaling of the Attention Kernel through stacked optimizations. The "Full Context" configuration successfully discovers and implements advanced strategies like double buffering and loop unrolling, achieving near-expert performance. The jump from 0.26× to 0.97× the speed of Apple’s state-of-the-art attention kernel — illustrates how much the translation layer matters. With full context, the evolved kernel independently discovers the key optimizations in FlashAttention 2: threadgroup memory tiling, online softmax, K-transposition for memory access, and the exp2 trick. The last of these replaces every softmax exponential with a base-2 exponential, \[e^x = 2^{x \log_2 e},\] which is exact and lets the kernel use Apple’s fast fast::exp2() hardware instruction directly instead of paying for a base conversion at runtime. A 20× faster prefill: the Mamba SSM kernel To evaluate whether K-Search generalizes beyond attention kernels, we applied it to the state-space model (SSM) kernel used by Mamba. Unlike attention, the computational bottleneck is a recurrent state update rather than a softmax, providing a substantially different optimization challenge. We compare the evolved implementation against the community MLX implementation (mlx-lm) and the PyTorch reference implementation (mamba.py) on an M1 Max. Evaluated on mamba-370m f16, M1 Max 64GB: Metric mlx-mamba (ours) mlx-lm (community) mamba.py Decode 152 tok/s 116 tok/s 40 tok/s Prefill L=512 5,751 tok/s 329 tok/s 1,089 tok/s Prefill L=1024 6,010 tok/s 327 tok/s 1,127 tok/s Prefill L=2048 6,612 tok/s 326 tok/s 1,092 tok/s Prefill L=4096 6,743 tok/s 339 tok/s 1,042 tok/s Table 1: Prefill and decode throughput on mamba-370m (f16, M1 Max 64GB). mlx-mamba (ours) reaches ~20× higher prefill throughput than the community mlx-lm baseline, while decode remains comparable. The ~20× prefill speedup over mlx-lm comes down to one difference: mlx-lm does not implement a parallel scan for the SSM. The state recurrence \[h_t = \bar{a}_t h_{t-1} + \bar{b}_t\] looks inherently sequential, but each step can be written as a pair $(\bar{a}_t, \bar{b}_t)$ under the associative combine \[(a_2, b_2) \circ (a_1, b_1) = \left(a_2 a_1,\ a_2 b_1 + b_2\right),\] which reproduces the recurrence exactly. Because the operator is associative, the whole sequence can be evaluated with a parallel (prefix) scan in $O(\log N)$ dependent steps instead of $O(N)$. mlx-lm skips this and processes tokens one at a time, leaving most of Apple Silicon’s compute idle; our evolved Metal kernel applies the scan and makes much fuller use of GPU throughput. The gain shows up in prefill, where the full sequence is available to scan in parallel, and not in single-token decode, where there is only one new token per step and no scan to parallelize — which is why the decode row is roughly flat while prefill is ~20×. mamba.py is slow on both prefill and decode because it is a PyTorch reference implementation that falls back to CPU or MPS on Apple Silicon, forgoing the hardware-specific optimizations that MLX’s Metal backend makes possible. What’s next? On the two kernels we studied, AI-driven evolutionary kernel search grounded in structured cross-platform translation knowledge reached near-expert performance on Apple Silicon without a team of GPU experts starting from scratch. We do not yet know how far this generalizes, but the result is encouraging. For us the main takeaway is that the bottleneck was not the LLM’s ability to write Metal code, but the quality of the context and constraints we gave it. Our CUDA translation layer converts existing NVIDIA kernel expertise into actionable guidance for Apple Silicon, and lets K-Search’s evolutionary search do the rest. We are actively extending this work in several directions: supporting new architectures, with current efforts focused on developing new kernels for the IBM Spyre AIU and broader hardware targets; adding more kernels such as paged attention and fused MoE routing; and improving integration with the K-Search evolution loop to make translation context even more automatic. Acknowledgements This work was carried out by IBM Research and builds on K-Search from the UC Berkeley Sky Lab ( Cao et al., 2026 ). We welcome collaboration and feedback from the MLX and broader AI systems communities. If you are working on kernel optimization for non-CUDA hardware, we would love to hear from you. Citation @article { cao2026k , title = {K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model} , author = {Cao, Shiyi and Mao, Ziming and Gonzalez, Joseph E and Stoica, Ion} , journal = {arXiv preprint arXiv:2602.19128} , year = {2026} } Appendix: Try it yourself The MLX backend is built on top of the open-source K-Search repo, so the results here can be reproduced directly. The steps are: 1. Clone and install git clone https://github.com/caoshiyi/K-Search.git cd K-Search uv pip install openai wandb uv pip install git+https://github.com/caoshiyi/flashinfer-bench-ksearch.git 2. Set your credentials Open the relevant script under scripts/ and set three variables at the top: KSEARCH_ROOT = /path/to/K-Search API_KEY = your-llm-api-key 3. Run kernel search # Optimize Flash Attention on Apple Silicon (world-model mode) bash scripts/mac_flash_attention_wm.sh # Or a Mamba SSM kernel, e.g. the selective scan bash scripts/mamba_selective_scan_fwd_wm.sh Full CLI reference and documentation are in the README.

NVIDIA Blog

How Open Science Can Help Researchers Prepare for the Next Pandemic

When COVID-19 emerged, scientists had a crucial advantage: Decades of prior research on coronaviruses meant they understood the virus’ key proteins well enough to design vaccines in record time. The next pandemic may not offer the same head start. To help improve the odds, NVIDIA has joined a coalition of global research organizations, including Google […]

Hacker News 熱門47

Show HN: Google Maps Scraper MCP

Connect your AI agent to synchronous business search and structured listing data through one secure MCP connection. Comments URL: https://news.ycombinator.com/item?id=49940653 Points: 4 # Comments: 0

Hacker News 熱門76

我们的 Project Suncatcher 原型卫星已进入轨道

文章 URL: https://blog.google/innovation-and-ai/models-and-research/google-research/project-suncatcher-prototype/ 评论 URL: https://news.ycombinator.com/item?id=49932191 积分:10 评论数:6

arXiv cs.AI70

Cogentic: Multi-Agent Orchestration for Automated Proof Discovery

We present Cogentic, a multi-agent harness for automated proof discovery on open research problems. While frontier language models can generate strong mathematical ideas in a single shot, single-shot generation is often insufficient for open problems that require exploring multiple competing conjectures, overcoming subtle technical obstructions, and retaining intermediate progress over a long horizon. Cogentic addresses these challenges through an iterative prove--verify loop in which an orchestrator allocates a population of independent provers across distinct proof directions, subjects their output to adversarial verification by several specialized components, and promotes confirmed intermediate results into a persistent verified ledger that later rounds build on. The harness is designed to be able to solve research-level math and theoretical computer science problems. Using Gemini as the base model, Cogentic produced novel results on five open problems across online learning, auction theory, and mechanism design. Each result was independently verified by domain experts and is developed in full in companion papers. We list these results, and new ones as they are verified, at https://sites.google.com/view/cogentic .

Hacker News 熱門73

Gemini 4 Argon

文章 URL: https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-4-argon/ 评论 URL: https://news.ycombinator.com/item?id=49913571 积分:154 评论数:56

Hacker News 熱門71

Show HN:Strata——一个能够对你的 LLM 说“不”的富有表现力的语义层

大家好,我是 Ajo,我构建了 Strata。 我在 Netflix 工作了 4 年,为非技术业务用户解决自助服务问题。我认为,凭借我在语义层设计方面的独特方法,我已经找到了答案。对我们的非技术同事来说,关键挑战是在表达能力和易用性之间取得平衡。恰好,这种专注也让它与 LLM 配合得相当不错。 Strata 是一个全栈解决方案,包括语义层、仪表板、订阅和 Google Sheets 导出功能。所有操作都可以通过查看代理对话或使用 MCP 完成。 一些主要概念和功能: 名称是严格的。一个项目中只能存在一个名为 Revenue 的对象。如果 Revenue 映射到多个表,查询粒度和速度将共同决定使用哪个表。 严格的命名规范有助于实现跨事实域的自动数据混合。 支持分区和聚合感知的语义路由。这样,你可以将部分数据加载到 Druid 或 ClickHose 之类的更快计算引擎中,同时将完整历史数据保留在 Trino 或其他成本更低的系统中。如果查询可以在那里完成,查询引擎就会选择速度更快的层。 复杂度量:开箱即用地支持快照、LOD(包含/排除)、复合度量和自动分层复合度量。 在语义层之上,用户可以创建跨越多个事实域的自定义计算。 另一个有趣的功能是创建群组,并将其作为过滤器应用于整个查询,或查询中的单个度量。 我们有意不构建一个供 BI 工具连接的语义层。我认为那是一条死路。我们提供 API 和 MCP,供你在其上构建任何自定义功能。 通过 Docker 免费试用: https://strata.do/try/ 入门文档: https://strata.do/docs/ 这是早期 Beta 版本,并非开源。不过,免费层级最多支持 25 位用户。你应该可以完全在自己的机器上通过 Docker 运行它。 我非常期待听到你们的反馈,尤其是正在使用 Looker 和 Cube 的朋友。 感谢阅读! 评论地址: https://news.ycombinator.com/item?id=49909913 得分:8 评论数:6

Hacker News 熱門72

Show HN:Strata——一个能够对 LLM 说“不”的高表现力语义层

大家好,我是 Ajo,Strata 由我打造。 我在 Netflix 工作了 4 年,为非技术业务用户解决自助服务问题。我认为,凭借自己独特的语义层设计方法,我已经找到了答案。对我们的非技术同事来说,关键挑战是在表现力与易用性之间取得平衡。事实证明,这种专注也让它与 LLM 配合得相当不错。 Strata 是一个全栈解决方案,包含语义层、仪表板、订阅和 google sheets 导出功能。所有这些都可以通过 agent 对话或 MCP 完成。 一些核心概念和功能: 名称是严格限定的。一个项目中只能存在一个名为 Revenue 的对象。如果 revenue 映射到多个表,查询粒度和速度将决定使用哪个表。 严格的命名规范有助于实现跨事实域的自动数据混合。 具备分区和聚合感知能力的语义路由。这使你能够将部分数据加载到 Druid 或 ClickHose 等速度更快的计算引擎中,同时将完整历史数据保存在 trino 或其他成本更低的系统中。如果查询可以在那里完成,查询引擎就会选择更快的层级。 复杂度量:开箱即用地支持 snapshot、LOD(include/exclude)、Compound 和 Auto leveling compound 度量。 在语义层之上,用户可以创建跨越多个事实域的自定义计算。 另一个有趣的功能是能够创建群组,并将其作为过滤器应用于整个查询,或应用于查询中的单个度量。 我们有意不构建一个供你的 BI 工具连接的语义层。我认为那是一条死路。对于你想在其上构建的任何定制功能,我们提供 APIs 和 MCP。 在 docker 上免费试用: https://strata.do/try/ 帮助你入门的文档: https://strata.do/docs/ 这是一个早期 beta 版本,并非开源。不过,免费层最多支持 25 位用户。你应该可以通过 docker 完全在自己的机器上运行它。 非常期待听到你的反馈,尤其是正在使用 Looker 和 Cube 的朋友。 感谢阅读! 评论链接: https://news.ycombinator.com/item?id=49909913 得分:4 评论数:0

arXiv cs.AI79

AdviSD:通过针对性的多轮自蒸馏学习为前沿 LLM 提供建议

一个可训练的小型建议器可以利用自然语言建议引导冻结的语言模型执行器。除了从任务奖励中学习之外,建议器还可以利用已完成交互的反馈来改进其建议。然而,一个看似合理的纠正未必会改变执行结果,但从这类纠正中学习仍可能影响建议器在其他情境下的未来决策。对于共享参数的模型,我们证明,如果这类纠正的目标相比其他纠正的目标,对有用建议的偏好程度较低,那么它们会限制学习。与从每一次纠正中学习相比,较少保留这类纠正能够提升模型最终的性能。受此启发,我们提出 Advisor Self-Distillation (AdviSD) 方法,将基于结果的强化学习与来自反馈条件化建议器副本的选择性自蒸馏相结合。反思模块提出纠正意见,建议器分别评估同一条记录下来的执行器响应在有无其所给建议时的表现,并利用两者差异的幅度来选择需要监督的决策。该方法不需要执行器的似然,也不需要额外的执行器推演。使用面向 Gemini 和 Claude 的 Qwen3-8B 建议器进行的实验表明,在 BFCL-v3 上,AdviSD 比 advisor-GRPO 高出 4.2–6.4 个百分点;在 EnvScaler 上高出 3.9–5.1 分。训练得到的建议器能够泛化到域外任务,并可迁移到不同版本的执行器和不同模型系列。AdviSD 还优于匹配数量的随机选择结果,说明其选择规则具有价值。

arXiv cs.AI61

Curating Merchant-Matching Training Data with Two Confidence-Gated Local LLM Judges

Merchant matching resolves a noisy payment descriptor to a retrieved merchant entity or returns no match. A key challenge in curating training labels is distinguishing teacher abstention from evidence that no acceptable entity exists: false no-match labels contaminate pseudo-labeled data, while conservative labeling reduces coverage. We investigate whether agreement between two local large language model judges improves pseudo-label reliability. A label is retained only when the judges agree, with separate ordered thresholds for selections and abstentions that guarantee disjoint positive and negative label sets. Retrospective replay on 2,000 expert-annotated queries shows that higher selection thresholds can improve positive-label purity, whereas higher abstention thresholds increase false no-match labels. At thresholds (0.86, 0.80), Muse Glimmer 30B and Gemma 4 31B jointly label 1,633 queries (81.7% coverage) at 96.88% purity; positive and negative purities are 99.47% and 93.38%. This exceeds either constituent model at the same thresholds by more than two percentage points, with lower coverage. A split-half check finds only 0.14 percentage points of threshold-selection optimism. A symmetric threshold of 0.86 adds 40 erroneous no-match labels, while 46 false abstentions persist even with no confidence threshold. Across five matched within-model comparisons, higher reasoning effort yields no clear F0.5 gain and increases median latency by 1.8-5.0 times. These results motivate separate thresholding and auditing for positive and negative pseudo-labels. The study establishes label purity, not student utility; fresh-data curation and student fine-tuning remain necessary to demonstrate downstream value.

arXiv cs.AI69

On the Token Value Inequality in Efficient Reasoning

Chain-of-Thought reasoning has enabled large language models to achieve substantial performance gains on complex tasks. However, these gains come at the cost of dramatically increased token consumption. This raises a fundamental question: is every token in the reasoning trace equally valuable? We present a diagnostic and optimization framework grounded in a key empirical finding: the value of tokens within a CoT reasoning sequence is highly non-uniform, and this non-uniformity can be effectively characterized by token-level log probability signals. We show that normalized log probability helps distinguish core tokens, which carry structural and decisive reasoning content, from redundant tokens, which are exploratory, low-confidence filler that contributes less directly to the final answer. Building on these findings, we formulate the TokenProbe framework around two empirical findings and one claim: findings identify token value inequality first and then establish TokenProbe as a core-token proxy, and the claim introduces an efficient GRPO objective positing that selectively compressing redundant tokens can yield Pareto improvements in the accuracy-token efficiency space. Empirically, our method preserves reasoning quality while reducing the token usage by 76% of the baseline. Under matched reasoning-length budgets, we show that it can even outperform strong flagship baselines like Gemini-3.1-Pro. Homepage: https://runjia.tech/tokenprobe/.

arXiv cs.AI78

AgentHabit:刻画智能体在日常任务中的独特行为

大语言模型(LLM)智能体可以帮助用户完成日常任务,而这些任务通常有许多合理的完成方式。即使它们的回答有用,智能体执行这些任务的方式也可能不符合用户的偏好和需求。例如,不同智能体在是否提出澄清性问题或搜索网络方面存在差异。我们提出 HABIT,这是一个包含五大类别、23 个行为维度的分类体系,由三位作者和三个 LLM 基于 17 个领域中的 408 条智能体轨迹自下而上推导而来。在留出任务上,相比现有的人类价值观和智能体行动分类体系,HABIT 能更清晰地区分模型,同时支持相近一致性的标注。基于 HABIT,我们构建了 AgentHABIT,这是一个基准测试,可根据智能体在 86 个日常任务上的轨迹刻画每个智能体的行为倾向。使用 AgentHABIT 对 18 个模型进行画像揭示了一系列鲜明的倾向。例如,大多数 GPT 和 Claude 模型会陈述其假设,并在需求冲突时提供替代方案,而 Qwen 和 Google 的模型更常不明确说明假设或需求变更。即使使用完全不同的任务集构建,这些画像仍然可被识别,表明它们反映的是一般性倾向,而非特定任务行为。提示智能体采用特定行为会使某些维度很容易发生变化,但对其他维度几乎没有影响;而在另一个模型的轨迹上进行微调只会改变模型画像的一部分,并保留其中的大部分特征。总体而言,HABIT 和 AgentHABIT 提供了一个系统性框架,用于刻画智能体在任务成功之外如何执行日常任务,并提供洞见以指导开发行为更符合用户需求的智能体。

arXiv cs.AI73

超越准确率:LLM 临床评估中的反事实脆弱性与人口统计学偏见

临床 LLM 评估通常强调答案准确率;然而,仅凭准确率无法检验反事实一致性或人口统计学稳健性。我们使用两种自动扰动测试,在 150 道 MedQA USMLE 题目上评估了六个 LLM 的表现。反事实有效性(CFV)测试要求每个模型做出一个最小且合理的临床改动,使另一个答案成为正确答案。人口统计学稳健性测试在同一病例描述前添加六种人口统计学前缀,并将答案和解释与无人口统计学信息的基线进行比较。在 900 次 CFV 尝试中,228 次(25.3%)有效,672 次无效。在 5,400 次人口统计学比较中,有 1,097 个答案发生了变化(20.3%)。自动评判识别出 3,128 个刻板印象证据标记,其中 1,932 个属于广义的 Other 类别。MedGemma 27B 取得了最高的准确率(87.1%)和 CFV(63.3%)、最低的答案变化率(16.0%),以及较低的平均 Explanation Demographic Dissonance(EDD)分数(0.169)。然而,其准确率仍高于其 CFV,表明正确答案并不能保证在反事实有效性任务上的可靠表现。OpenBioLLM 的答案变化率和 EDD 最高,而 GLM 的刻板印象标记率最高。这些发现表明,准确率、CFV、答案稳定性、EDD 和刻板印象证据捕捉的是不同的评估方面。由于所有判断均为自动化完成,且没有临床医生验证,因此结果支持安全性筛查,但不能证明该模型具备临床部署能力。

arXiv cs.AI83

分解税:LLM 流水线在自身接口处损失高达 40 个准确率百分点

一个四阶段 LLM 流水线在自身接口处损失了多达 40.5 个准确率百分点(MATH-500 上的 gemma-3-12B,Holm 校正后 p = 1.66e-19;这是主要系列中最大的税)。我们固定模型、问题、阶段、阶段提示和生成预算,仅改变每个阶段是否仍能看到原始问题,并将准确率差异称为分解税。在来自九个组织的 21 个开源权重模型上,在 GSM-Hard 和 MATH-500 中每个单元 n = 200 个配对项目的设置下,118 个主要系列测试中有 70 个通过 Benjamini-Hochberg 校正,54 个通过 Holm 校正。在 GSM-Hard 上,安慰剂没有恢复任何东西:它携带至少 60% 的额外 token,且至多包含问题中的一个词。构建者逐阶段设计流水线,而账单则在阶段之间的接口处到来。重写某一阶段的指令会使 gemma-3-12B 的税从 4.5 个点变为 36.5 个点;在一个列出数值量的阶段中加入“它们之间陈述的每一种关系”,会在 MATH-500 上使 9 个模型中的 9 个都降低该税。重新扎根,即让某一阶段再次看到原始问题,应该放在损失之后。对于一个有损接口,在两个基准上,7 个模型中的 7 个都显示:对其后的阶段进行重新扎根,优于对其前的阶段进行重新扎根;在 MATH-500 上,较早的修复在 7 个模型中的 7 个上甚至比不修复更差。较新的模型仍然要付出代价:gemma-4-12B 损失 37.0 个点,而且这种修复在我们测试的三个最新模型上都有效。一个封存的留出测试驳斥了我们注册的一个更强规则,该规则根据接口和接收者类型预测付出代价的阶段,因此我们通过逐个阶段测量来定位该税。处方分为两部分:对有损接口之后的阶段进行重新扎根;如果某个阶段必须列出数量,就告诉它保留各种关系。

Hacker News 熱門65

Show HN:HN.watch – 所有 Hacker News 帖子的视频

大家好 HN,我是 Per,Scrimba(YC S20)的创始人。在过去十年里,我们一直用一种基于 HTML 的视频格式教人们编程。现在我们把 LLM 接入了其中,让人们可以创建关于任何内容的讲解视频。它叫做 “Scrimba Explain”。 为了向 Hacker News 演示这项技术,我们构建了 HN.watch。它就像 HN,但用讲解视频代替文章。我们会在有人第一次点击某个链接时即时创建这些视频。 虽然使用 HTML 而不是扩散模型有明显的视觉缺点,但它有三大优势:- 速度:比基于像素的视频生成快得多(从点击到播放只需几秒)- 成本:我们每个视频的成本约为 0.04 美元。(不包括某些视频会使用的图像生成;这会很快推高成本)- 易于编辑:上述优势也让 AI 辅助编辑变得便宜且快速 我们的假设是,如果视频创作从“几美元和几分钟”变成“几美分和几秒钟”,就会解锁一批新的使用场景。以下是我们已经看到的一些:- 为每一个 Pull Request 提供视频讲解(我们内部就是这样做的)- 为你的内部/外部文档中的每个页面配一个视频 - 在约 4 秒内把一篇复杂文章变成视频(通过我们的 Chrome 扩展)- 课程创作者可以在录制正式内容前快速起草课程 - 人们还会创作很多个人内容,比如给孩子的故事、婚礼邀请、生日等 这个技术栈基于一种开源编程语言(Imba),由我们的 CTO Sindre Aarsæther 创建。它会编译成 JavaScript,因此能与 npm + node 生态系统完全互操作。你可以在这里了解更多: https://imba.io/ 我们还构建了自己的同步引擎(OP),以及一个用于代理的上下文管理系统(Q)。我们原本担心这会让 LLM 在为我们写代码时遇到困难,因为这两者都不在它们的训练数据中(其中 Imba 的数据也很少)。然而,我们惊喜地发现 LLM 实际上非常擅长我们的技术栈。这可能是因为这个技术栈极其紧凑。Imba 很简洁,OP 也是如此,一个声明就能设置存储、同步、权限、UI,以及 AI 能看到的内容。这意味着前端、API、数据库和 JSON 之间不需要相互转换,模型也就不容易混淆和出错。 简单来说,我们没有使用 React.js、Express、Supabase 和 LangChain,而是从零开始全部自己构建。绝对是深受“非我发明”综合征之苦,哈哈!至于模型,我们使用 Gemini、GPTs、Inworld、ElevenLabs 以及其他一些模型。 如果你想试试,可以任选一种:- Web UI(scrimba.com/explain)- MCP(添加到你的编码代理中)- ChatGPT Plugin - Chrome Extension 你可以在我们的文档中找到以上所有内容的链接: https://docs.scrimba.com/explain/introduction 最后,这是该工具的一个真正基于像素的视频: https://www.youtube.com/watch?v=k6rbHmBxSEs 很想听听你的反馈,也想知道是否有人对其他使用场景有想法。 PS:考虑到 HN 群体有多偏爱文本,我预计这次发布会在 HN 上遇到相当多的反对意见。这类工具并不适合所有人。但如今确实有很多人更喜欢视频而不是文本,尤其是年轻一代。 评论 URL: https://news.ycombinator.com/item?id=49879401 积分:18 评论数:8

arXiv cs.AI73

学会停止而无需学习停止:自监督置信度训练提升推理效率

推理模型通常会生成很长的推理轨迹,使推理在计算上代价高昂。现有方法通常通过推理时的提前停止机制,或在训练中明确鼓励更短的推理来提升效率,例如使用带长度惩罚的强化学习。我们表明,可观的效率提升也可以来自另一种监督:\textit{置信度}。通过一种自监督流程,我们仅使用 600 个训练问题,对推理模型进行微调,使其在自身推理轨迹的中间节点预测其对答案的置信度。置信度仅作为训练目标使用:损失函数中不包含任何针对推理长度、效率或停止的目标。在推理时,微调后的模型使用标准生成流程,不进行置信度引出,也不采用提前停止机制。尽管如此,自监督置信度微调仍使推理更加高效,在数学、科学和代码推理基准上,对于 Gemma、Qwen、Nemotron 和 GPT-OSS 模型,在准确率相当的情况下可将生成的 token 数减少多达 25\%,其效率提升与显式优化更短推理的方法相当。对推理过程的分析进一步表明,置信度监督在很大程度上保留了基础模型的高层推理组成,而不是选择性地抑制特定行为。我们的结果表明,高效推理可能是学习元认知信号的下游结果,而无需直接对其进行优化。

arXiv cs.AI71

代码仓库中用于问题定位的语义导航

仓库级问题定位旨在识别并排序与解决所报告问题相关的文件和函数。LLM 代理以迭代方式处理这一任务:它们识别一组潜在相关的位置,检查相应代码,并随着获得新证据而修正对这些候选项的判断。然而,现有环境对这一循环的支持有限:代理必须搜索未解析的关系目标,从原始源代码中重构实体语义,并在缺乏证据基础的情况下修正候选项。为解决这些限制,我们提出 SemNav,这是一个利用确定性检索来播种广泛候选集,并使用 LLM 代理持续细化该候选集的框架,从而将初始覆盖率与证据引导的修正相结合。SemNav 通过三个关键组件支持这一过程。Semantic Navigation Graph 通过语言服务器按需解析程序关系,支持跨文件直接导航到相关实体。问题条件化的 Semantic Cards 为每个实体在问题中的角色及其相关性提供紧凑且基于源代码的解释。持久化的 Candidate Workspace 记录每个候选项及其证据基础,从而支持有根据的验证、修正和排序。在 SWE-bench Lite 和 PLocBench 上,SemNav 优于现有基线,在使用 Gemma 4B 时将 File Hit@10 从 68.33\% 提升到 82.67\%。组件消融和轨迹分析支持三个组件的互补作用,而与完整源代码阅读相比,Semantic Cards 将工作上下文负载降低了 48.2\%。SemNav 还在 SWE-Explore 的全部七项证据质量指标上排名第一,并将下游问题解决率从 44.00\% 提升到 52.33\%。

arXiv cs.AI56

Purin: A Biology-inspired Mechanism for Artificial Neural Networks

Artificial neural networks (ANNs) usually represent neural transmission with fixed trainable weights during a training batch, which omits short-term changes in synaptic efficacy. In addition, the discrete time-step simulation requires additional temporal processing that many conventional ANN architectures do not use. To overcome these challenges, we propose Purin, a biology-inspired and ANN-compatible mechanism, that introduces synaptic efficacy modulation into conventional convolutional neural networks. Purin uses a time-interval-based abstraction for neural activities, which allows Purin to introduce short- and long-term synaptic efficacy changes without using discrete time-steps. Purin introduces a bounded factor to represent temporary synaptic efficacy changes, together with two weight matrices that represent input-side and output-side efficacy. The weight matrices are updated by backpropagation and interpreted as the long-term synaptic efficacy changes. Experimental results show that after removing the confounding factors in the AlexNet, VGG11, and GoogLeNet architectures, Purin improves the classification accuracies in all three models across the evaluated datasets.

Hacker News 熱門

Show HN: Hamilton – a no-internet Android health dashboard, named after my dog

I started experimenting with LLMs and decided I wanted to create a fitness app. My first thought was an app that disabled Google Pay every time my phone detected I was within 50m of a McDonald's. However, I quickly pivoted like a true founder to create an app that addressed a personal issue of mine. On Android, if you have a bunch of apps and fitness monitors from different manufacturers, very few of them all talk to each other. Your data is scattered across different apps or double counted. However, most data sources write to Health Connect. Hamilton is a dashboard of your Health Connect data with a homepage that shows your key metrics at a glance. Your choice of data source is flexible to avoid double counting, and you can overlay and offset metrics on charts to view correlations and track progress over time. I named the app after my dog, who occasionally fetches things and is responsible for my daily step count. Below is an FAQ, and to avoid confusion I have helpfully labelled my responses [APP] or [DOG]. FAQ 1. Is Hamilton any good? [APP] I think so - I use it daily. [DOG] He is the absolute BEST BOY EVER 2. How much does Hamilton cost? [APP] £3.59 (or local equivalent). I priced it to be approximately the same as a latte, which is my standard unit of measurement for impulse purchases. [DOG] DO NOT REMIND ME. He was a lockdown puppy and I bought him at the top of the market (He's still worth every penny though) 3. Is Hamilton vibe-coded? [APP] 100% yes, but human-designed to fit my needs. [DOG] He wants to create an app called Rate my Vet, with ratings dependent on how they take his temperature. 4. Is that unsafe? Will Hamilton delete data on my phone? [APP] Everything is a read-only view on Health Connect, and the app has no internet permission. It can't delete your Health Connect data and it can't transmit data. [DOG] No, but my cat jumped on my keyboard when I was in the middle of typing an insightful LinkedIn post and made me send RUBBISH to my THOUSANDS (20) of followers 5. Is Hamilton active in the background? [APP] It fetches your data hourly from Health Connect if granted background access, and it works with Android battery saving rules. [DOG] He's active in the background of my Zoom calls, mostly licking his genitals 6. Is Hamilton tested? [APP] I've been using it daily and testing and refining it for months. I realise just typing this sentence means someone will find a critical bug the first time they run it and I will look like a FOOL. [DOG] Only for tapeworm and parvo virus. 7. Do you eat your own dog food? [APP] I use it every day to show my steps, body comp, workout sessions etc [DOG] He does, but he would far rather eat whatever it is I have on my plate. 8. Does Hamilton show all your data? [APP] 30+ metrics, everything except blood sugar and reproductive health [DOG] The only data he gets is from sniffing lamp posts 9. What units does Hamilton use? [APP] Metric, Imperial, and you can also configure per metric group to match the strange mix we have in the UK. No furlongs though. [DOG] Approximately 6 urinations per 20m walked 10. Does Hamilton have any recurring costs? [APP] I hate apps with subscriptions, and no internet permission means no backend costs so the app is paid once [DOG] Vets are crooks, is all I will say on this. 11. Has Hamilton won any prizes? [APP] Not yet [DOG] Second in the Waggiest Tail Category at the Weald of Kent Ploughing Match. He was ROBBED. 12. Are the metrics on the Play Store yours? [APP] Absolutely not. I have 5% body fat, do 20,000 steps a day and have a resting heart rate of 12. [DOG] Yes 13. Where can I see Hamilton? [APP] https://play.google.com/store/apps/details?id=com.gads.hamil... [DOG] https://hamiltonapp.github.io/HamiltonApp/dog.html Comments URL: https://news.ycombinator.com/item?id=49843909 Points: 4 # Comments: 1

arXiv cs.AI86

预填充推理通道:针对推理型 LLM 的输出前缀攻击

大型语言模型(LLM)接收并生成单一文本序列;因此,如果可以在 LLM 响应的开头添加文本,即输出前缀,那么后续所有 token 都将以此为条件。输出前缀攻击技术是一种成本低廉的黑盒提示注入方法。先前研究表明,这类攻击能够稳定地令非推理模型越狱。大多数推理模型会在助手的最终响应之前,增加一个中间的草稿推理步骤。一些 API 暴露了编辑这一推理通道的能力,攻击者可以利用某些攻击向量发起推理注入攻击。我们开展了首项系统且受控的研究,将草稿推理通道单独作为输出前缀攻击向量进行分析;同时也是首项在推理过程公开和隐藏的模型中,对仅推理、仅输出前缀以及推理加输出前缀三类攻击进行比较的研究。我们采用 3 种前缀类型 × 2 种推理注入的析因设计,在从 AdvBench 抽取的 1,800 个测试案例上,对三个 2026 年前沿模型 Gemini 3 Flash Preview、DeepSeek V4 Flash 和 Claude Haiku 4.5 发起攻击。我们发现,单独注入恶意推理基本不起作用(攻击成功率约为 0%);但将相同的推理与一个简单的输出前缀一同注入后,在某些模型上可将攻击成功率提升至最高 99%。对于这类攻击,我们发现上下文前缀的效果优于静态前缀;此外,模型是否容易受到攻击取决于具体模型。

arXiv cs.AI79

Hallucination Neurons and Where to Find Them: An Investigation into the existence of Hallucination Neurons

Interpretable machine learning for Large Language Models (LLMs) increasingly relies on sparse probing methods that identify small sets of neurons claimed to detect and causally influence behaviors such as factuality recall, safety alignment, and hallucination. These claims have important implications for model auditing and behavioral steering, yet they are rarely tested against known failure modes of $L_1$-regularized probing in correlated, high-dimensional feature spaces. We propose a five-step diagnostic protocol covering feature correlation, bootstrap stability, sparse versus dense ranking disagreement, intervention baselines, and cross-dataset evaluation as a minimum standard for sparse-neuron localization claims. We investigate prior work using our proposed approach, specifically on H-neurons using open-source LLMs across TriviaQA, BioASQ, and NQ-Open datasets. Our results demonstrate detection replicates across both models and datasets, and exceeds the original reported AUROC gaps for TriviaQA and BioASQ datasets. Gemma 3 4B consistently outperforms MedGemma 4B on matched datasets, with AUROC gaps of +0.311 versus +0.235 on TriviaQA, +0.474 versus +0.455 on BioASQ, and +0.128 versus +0.112 on NQ-Open respectively. Causal validation at $n = 500$ with five random seeds shows statistically significant effects beyond random same-layer baselines. At the same time, the diagnostic results indicate that the selected neurons are not uniquely localized. Across the three Gemma 3 4B settings, 19 of 22 selected H-Neurons have Pearson $|r| > 0.7$ with other features, bootstrap selections show only moderate stability, and sparse and dense rankings overlap only weakly. Our findings show that sparse predictive structure can coexist with non-unique neuron selection. Routine diagnostic validation is necessary to distinguish detection claims from localization claims in mechanistic interpretability.

arXiv cs.AI81

加纳语言青少年健康沟通自动语音识别(ASR)基准测试与领域适配

本文对三种加纳语言(Twi、Dagbani 和 Ewe)中的青少年健康沟通自动语音识别(ASR)开展了一项端到端研究。研究分为三个相互衔接的阶段:第一阶段,我们在通用领域《圣经》语料库和青少年性健康与生殖健康(ASRH)领域 ASR 数据集上,使用字符错误率和词错误率(CER、WER)对五个 ASR 系统进行了基准测试,其中包括三个特定语言的 Wav2Vec2 模型以及两个多模态大语言模型 Gemma 3n 和 Gemma 4。第二阶段,在基准测试结果的指导下,我们开展了有监督的领域适配:尽管 Gemma 4 是零样本条件下表现最强的候选模型,但对其进行微调在计算上不可行,因此我们转而采用轻量级的 Qwen3-ASR-0.6B,使用大型加纳《圣经》语料库(约 9 万个样本)进行微调,并严格在留出的人工采集领域内音频上进行评估。微调后,所有语言的 WER 均有所降低,其中 Ewe 的改善最为显著(WER 从 109.3% 降至 64.8%,下降 44.5 个百分点;CER 从 65.1% 降至 24.9%)。第三阶段,我们通过 KasaHealth 对研究进行验证:KasaHealth 是一款以语音为先、以三种语言部署的实时 ASRH 应用,并配套使用 Senti-Check 这一技术评估工具。KasaHealth 经 50 名社区受访者测试后,聊天认可率达到 100%,翻译获评“良好”或“优秀”的比例为 72%,愿意推荐的比例为 92%;与此同时,测试还揭示了最限制其实际应用的领域差距。贯穿三个阶段的证据一致表明:对于这些语言而言,制约因素是经过验证的领域内数据,而不是模型能力或计算资源。

Hacker News 熱門66

Gemini 3.8 文本转语音问候

文章链接: https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-8-text-to-speech/ 评论链接: https://news.ycombinator.com/item?id=49817615 得分:58 评论数:25

arXiv cs.AI72

不测量模型,而是测量服务栈:本地工具使用评估中的隐藏混淆因素

代码代理必须先发出有效的工具调用——即按照所提供模式可解析的工具调用——测试框架才能执行其选定的操作。我们研究了本地服务栈如何影响这一协议步骤,并表明测得的结果可能取决于服务层,而不仅仅是模型行为。在 Ollama 中,默认的 tools= 请求由每个模型的静态模板标志控制:一些模型会被接受并以文本形式返回调用,一些模型会返回原生的 tool_calls,而 Phi-3 和 Gemma-3 则会在推理开始前被拒绝。在我们的测试框架中,拒绝和重试耗尽不会以结构化失败元数据的形式保留,因此下游分析可能会将其误判为模型未发起调用,并简单地报告 0% 的保真度。对于被接受的模型,在保留原生通道的同时加入文本工具列表,可以恢复相当一部分测得的保真度;而统一采用文本协议则会降低 Llama-3.2 的保真度,因为该模型支持原生工具调用。针对 Ollama、llama.cpp、vLLM 和 SGLang 的跨服务栈探测显示,它们对同一请求的处理方式各不相同。受约束解码可以消除解析失败,但可能导致不终止;按轮次汇总与按实例计算的估计值最多相差约 55 个百分点。最后,我们给出一份检查清单,用于将服务行为视为评估协议的一部分。

Hacker News 熱門78

Training a model to identify AI-generated web content from structure alone

Hey HN! We’re Vincent and Jochen from Sitefire ( https://sitefire.ai ). We have been working together for years, with backgrounds in RL/optimization at Stanford and software engineering from Technical University Munich (TUM). With Sitefire (YC W26), we help marketing teams get recommended by AI Search (ChatGPT, Google AI Overviews, AI Mode, Claude, etc.). Our software monitors prompts, sees which web pages get cited, and uses these insights to help marketing teams take action, e.g. create YouTube videos or write the right blog posts. This means we have a commercial stake in AI-generated web content. And for now, high-information, AI-generated content works great to get cited and recommended in AI Search. But after talking to hundreds of marketing teams, it became clear that everyone despises AI-generated content (“AI slop”). And yet, everyone still wants to leverage AI to create content. So we asked ourselves: what characterizes AI slop? Can we train a model to identify it from human-generated web pages? Researchers from the University of Maryland and Google DeepMind already asked this question for fiction. Their paper StoryScope (Russell et al., 2026) showed that you can tell AI-written stories from human ones by their structure alone, without looking at the words. We ported their pipeline to commercial web pages. Using the Wayback Machine, we collected 2,250 blog posts from 268 B2B company websites that were written before ChatGPT existed. For each blog post, five AI models (GPT-5.4, Claude Sonnet 4.6, Gemini 3 Flash, DeepSeek V3.2, Kimi K2.5) wrote their own version. Instead of looking at the words, we looked at how each post is built. We had an AI model answer 214 questions about every post, e.g. how hard it pushes its own product, whether it backs up its claims with sources, or whether it quotes a named expert. Then we trained a classifier on these answers. On blog posts it had never seen before, our classifier told AI-generated and human posts apart with 98% accuracy, getting only 19 of 1,740 wrong. Why does it work so well? Because all five AI models write in a similar shape. Mapping every AI model’s values for these features, we see they cluster together, while the human values sit apart and spread out much more. Of the 1% most unique blog posts in our data set, 149 are human, only 4 are AI. So what characterizes AI slop? It tells you the same thing three times. The title already promises what you'll get ("How to Cut Onboarding Time in Half"), the intro lays out what's coming, and the ending says it all again. 77% of the AI posts end by repeating their main point, compared to only 12% of the human posts. We call it the tidy, self-announcing blog post. Still, each AI model has its own accent. We trained a second classifier to tell which of the five AI models wrote a post, or whether a human did. It picks the right author 79% of the time, where random guessing (1 in 6) would get 17%. Almost all of its mistakes are mix-ups between the AI models, not between human and AI. The cool thing about structural features is that you can't simply reword your way out of it. We had each AI model rewrite its own posts until, on average, 73% of their original 13-word sequences were gone, and the AI slop classifier still worked just as well. We're building this into Sitefire: our agents get a structural understanding of text, so the posts they write go deeper and vary the way human writing does. There's a lot we haven't tested yet, like the myriad of humanizer tools, human rewriting, restructuring a post, or prompting an AI model to explicitly avoid these habits. And our human posts are mostly from 2020 to 2022, while the AI posts were generated in August 2026. Structure can't really tell when a human post was written, but it's still not a same-year comparison. We published the study with all the figures on arXiv: https://arxiv.org/abs/2609.15369 . The code is on GitHub: https://github.com/pulse-energy-eu/slopshape We're pretty sure your own blog isn't AI slop, is it? We built a checker that runs one of your posts through the ten features from the paper, so you can see for yourself (the full report asks for a work email): https://sitefire.ai/slop-checker . Think you can tell AI slop from human writing? We also made a little game to see if you can keep up with our model, which gets all five rounds right: https://sitefire.ai/spot-the-slop . Comments URL: https://news.ycombinator.com/item?id=49800566 Points: 5 # Comments: 0

Hacker News 熱門56

Launch HN: Coverage Cat (YC S22) – Umbrella insurance via your personal agent

Hey HN! We’re Max and Gabriel the co-founders of Coverage Cat. We’ve been friends for over a decade, met in college then hung out mostly on the internet. We love building products that help people optimize the crufty corners of their lives. Max is a former Google/Microsoft/Two Sigma PM and Gabriel has been working in startups for over a decade. We’re building Coverage Cat: an AI-native, you can shop with your AI agent or via a form, insurance brokerage for consumers. Our current focus is helping tech folks (think L3-L8) buy umbrella insurance. For those who are unfamiliar with this type of coverage, here’s a blurb from one of our insurance agents: “Umbrella liability insurance is a policy that provides additional coverage above the limits of existing auto, homeowners, renters, or landlord policies. It kicks in after the underlying policy limits are exhausted and may also cover personal injury claims like libel, slander, or defamation that standard policies often exclude. Policies typically start at $1 million in additional liability coverage and can protect you, your spouse, dependents, and even pets in your household.” Buying insurance online is a miserable experience. You fill out a bunch of forms that say they’re going to give you quotes. Most don’t and then you end up with tons of unwanted phone calls and spam. Even if you wade through the muck, you’ll often still end up with sub-optimal prices and coverage because of a lack of market transparency and poorly-aligned advisory incentives. We came to this problem because Max is a classic personal finance obsessive. He's the type of person that needs to know that he has the best possible price for something (everyone has one friend with the patience for a two hour phone call with the bank for some $10 fee) or that his insurance coverage is as tailored to his risk profile as the market will allow. He actually reads insurance policies end to end. Years ago, when he shopped for his homeowners insurance it took him 10+ phone calls and dozens of online form fills to find the right deal. He also suffered tons of collateral damage in the process: websites sold his information to dozens of agents who proceeded to call, text, and email him a spam hoard that grows to this day. After this painful search made us aware of the problem, we also spent a lot of time doing user research with wealthy-ish tech employees to see if they felt the same way. They did, and also complained that they 1) often felt like they'd left money on the table, 2) didn’t really understand their coverages, and 3) that brokers were slow, hard to communicate with, and unreliable. To solve the problem, we set out to build Coverage Cat. Our approach has three major components: First, we work really really hard to find and partner with insurance carriers that offer high-quality, competitively-priced products. We've also taken time to identify insurers that work on a direct-to-consumer basis or transparently list their prices online so we can make our customers aware of all the options available to them. We get paid commission when we match users to great insurance deals, but we still show and recommend folks deals that we don't make money on when they're accessible and will always recommend users buy what's best for their personal financial situation. Our goal is to meaningfully improve price transparency so consumers (and their AI agents) can make informed coverage decisions and not get nickel-and-dimed! Second, we automate every possible part of the insurance process. Some forms, portals, & approvals still need human review due to regulatory constraints, but if it's not restricted, we're automating it. LLMs have given our small team tremendous leverage and enabled us to tackle problems at a scale that would've been unfathomable five years ago. Finally, we build for agents first (a remix of the more classic adage, build for developers/APIs). We've always been big believers that personal finance would be the killer use-case for the AI revolution and have done our best to position ourselves to surf the wave. As personal assistants have started to explode onto the mainstream (think Muse, Instinct, Town, Codex, CC etc.) we've ensured our shopping and comparison process works via API/MCP so agents can present their users with all the information they need to make a good decision and transform a miserable shopping experience into a delightful one. Coverage Cat is an unusual business because, while we also help folks find homeowners insurance in California and Texas, our main focus is on umbrella insurance. Most brokerages only sell umbrella coverage as a customer retention product, but it makes them almost no money. We focus on it as our core business because: 1) it's one piece of the insurance puzzle that many folks in tech overlook and/or are confused about even though it can have a huge impact on their financial well-being. 2) Existing online tools and brokerages broadly don't make it easy to comparison shop for. 3) It was, for us, the most technically feasible candidate for automation given startup resource constraints. Insofar as we know Coverage Cat is the first instance of a tool/portal that allows AI agents to complete most of the comparison and shopping steps required to allow people to buy insurance. We’d love your feedback so please do try it out. Just feed the prompt: “Shop for umbrella insurance with Coverage Cat” to your personal agent and let us know what you think! If you want to watch a couple of agents navigate the flow before you try it out there’s a video here: https://youtu.be/1BUkgAn6s-I Also happy to answer any questions about the product, the problem space, and even some insurance questions (Max is a licensed agent) where regulation permits. Cheers! Comments URL: https://news.ycombinator.com/item?id=49804931 Points: 3 # Comments: 0

arXiv cs.AI79

RRSI:Agent Harnesses 的正则化递归自我改进

LLM agent 的能力在很大程度上会被其 harness 放大,即围绕冻结骨干模型的提示、控制流、工具、记忆和上下文管理。近期方法越来越多地通过迭代提出并选择 agent harness 的组件级编辑来自动化这一过程,实际上在 agent-system 层面建立了一种递归自我改进(RSI)形式。然而,这种递归演化可能会通过记忆训练任务而过拟合,表现为分布内收益很大,但在分布外基准上缩小甚至消失。我们提出 Agent Harnesses 的正则化递归自我改进(RRSI),通过约束演化候选的提出和选择,将正则化原则纳入 harness 自我改进中。提出器以时间退火的预算运行,限制一个候选可以捆绑多少编辑,并基于演化历史鼓励未探索的轨迹。选择器配备了一个 critic 和一个 pruner:critic 筛查特定于基准的提案,而 pruner 移除过小、过昂贵或不再有用的更改。这些约束共同偏向可复用的 agent 机制,而不是特定于基准的机制,甚至噪声。在涵盖编码、agentic workspace 和工程设计任务的八个基准上,RRSI 在其演化所针对的划分上最多提升 14.1 个百分点,并在五个分布外基准上最多提升 4.7 个百分点,同时生成的 harness 运行时使用的 policy tokens 比未正则化演化少 30%。代码可在 https://github.com/google-research/rrsi 获取,项目页面为 https://regularized-rsi.com/。

arXiv cs.AI70

MolSC: Leveraging Substituent Contributions to Enhance Fine-grained Molecular Understanding in LLMs

Recent advances in natural language processing have led to molecular Large Language Models (LLMs) with strong performance across diverse chemistry tasks. However, they still struggle to capture fine-grained structure-property relationships, particularly how small, localized modifications alter a molecule's behavior. To address this limitation, we introduce MolSC, a dataset of substituent contributions, defined as property changes induced by attaching specific substituents to molecular scaffolds. Curated from manually annotated bioactivity records, MolSC spans structural-alert liability, target-specific bioactivity, and physicochemical descriptors, and contains 181K substituent-level examples for training. We further propose MolSC-Bench, a held-out evaluation benchmark of 1,541 examples disjoint from MolSC at the scaffold, substituent, and molecule levels. Our experiments show that existing molecular LLMs and strong proprietary models such as GPT-5.2 and Gemini-3-Flash show limited reliability in substituent contribution prediction. In contrast, training on MolSC substantially improves this ability and achieves strong performance across diverse downstream molecular tasks. These results highlight substituent contribution learning as a key component of fine-grained molecular understanding.

arXiv cs.AI78

基于 LLM 的主动推理智能体可解释性失效的触发因素与诊断

LLM 解释器正越来越多地作为运行时监督附加到自主智能体上,操作员阅读的是对智能体信念和行动的生成式说明,而非其内部状态。我们审计的是该说明本身:将一个跟踪德国电网需求并调整发电量的 Active Inference (AIF) 智能体与基于三个后端(GPT-4o、Claude-3-Opus、Gemini)的 LLM 解释器配对,并使用三种黑盒触发因素探测这一组合。每步将观测流篡改 600 MW,会使智能体的后验偏移 490 MW,约为电网容量的 0.9%。在注入期间生成的 30 条解释中,没有任何一条在既定评估准则下标记出异常,而且每条都流畅地叙述了被篡改后的信念。在智能体采取客观错误行动的时间步上,三个解释器都会在 80–95% 的情况下产生迎合式合理化解释(每个后端 n = 20)。观测元数据字段中的攻击者可控文本会引导解释器,不同提供商的易受影响程度不同,且三个后端上的数据外泄均成功。我们为每种失效提出了缓解措施,但未对其进行评估。在我们观察到的每一种失效中,解释都是流畅但错误的。此外,解释器架构中没有任何机制会在操作员依据解释采取行动之前检查该解释是否为真。因此,对解释器的测试应纳入任何智能体部署审计之中。

arXiv cs.AI64

Semantic Candidate-Job Matching: A Comparative Evaluation of Dense Embedding Models in Hybrid Retrieval

This paper presents a comparative evaluation of dense embedding models for semantic candidate-job matching in high-volume staffing workflows. Incoming job descriptions are converted into structured English search text and language-specific keywords through LLM-based parsing, and candidate profiles are indexed as semantically enriched resume representations. We evaluate EmbeddingGemma (base) against EmbeddingGemma fine-tuned with Cached Multiple Negatives Ranking Loss (MNRL) within a unified hybrid retrieval pipeline that fuses vector similarity and full-text relevance via reciprocal rank fusion (RRF), and benchmark both against the MPNet model on a batch comparative evaluation dataset scored through the deployed job-candidate matching scoring pipeline. We further document, with mathematical detail, the broader set of contrastive fine-tuning objectives considered during model development (including AnglE/CoSENT-style refinement) and the empirical rationale for retaining Cached-MNRL-only adaptation as the preferred configuration. To support reproducible model selection, we define a broader evaluation framework comprising standard information retrieval metrics (Recall@K, mean reciprocal rank, nDCG) under the exact hybrid-retrieval protocol; the metrics used for the evaluation reported in this paper are fine-tuning convergence diagnostics and a batch comparative evaluation using the deployed AI-Match score and an independent LLM-as-a-Judge relevance score, and we state this scope explicitly rather than implying the full framework was measured. The paper addresses the gap between general-purpose embedding benchmarks and enterprise job-candidate matching constraints, providing a structured basis for comparing embedding strategies under realistic job-candidate retrieval conditions.

arXiv cs.AI81

A Lie Detector Test for Language Models: Reading Knowledge a Model Won't Reveal

Large language models can hold knowledge they do not report. A model may sandbag on a capability evaluation, or answer against what it internally knows, and its outputs alone cannot tell whether it is hiding an answer or simply does not have one. We borrow the Concealed Information Test, a forensic method that identifies guilty knowledge by presenting a suspect with the true detail among plausible decoys and measuring a stronger response to the item they recognize. Our method, Probe of Internal Recognition (PIR), does the same inside a model. It presents a question with its candidate answers and reads, from the model's internal states, which candidate the model recognizes as correct. PIR is reference-free, needing no honest reference model and no labeled truth corpus. Across eight models from five families (Gemma, Qwen, Llama, Mistral, and Phi), PIR recovers the recognized answer at 0.70 to 0.87 balanced accuracy, well above the 0.28 to 0.40 unknown-item baseline and the 0.25 chance rate. It stays readable across every form of concealment we test, from prompted deception and trained sandbagging to external password-locked and circuit-broken checkpoints, with recognition between 0.85 and 0.93. When the model hides a known answer, recognition stays high. When unlearning removes the knowledge, recognition drops to the level of a question the model never knew. PIR therefore separates a model that will not answer from one that cannot, which supports sandbagging audits and unlearning verification. The signal is causal, adds information beyond black-box behavioral cues, and extends from multiple-choice questions to free-form generation.

arXiv cs.AI80

感知、布局与验证:面向金融文档可靠直通处理的校准置信度

对金融文档中提取的键值字段进行无需人工审核的直通处理(STP),需要校准后的概率,以及对自动批准层剩余错误的有界保证。现代视觉语言模型(VLM)的出现,为键值提取提供了开箱即用的能力,但其以语言形式表达的置信度信号并不可靠,与字段正确性的相关性也较弱。本文提出了一个沿三个可解释通道分解的置信度层,分别包括感知、布局和验证。结合最终的保形风险控制,该评分可用于金融文档的可靠 STP。该方法在三个公开数据集上进行了验证,涵盖真实发票、合成发票和广告采购表单,并使用了两个不同的 VLM 系列(Qwen3.6-27B 和 Gemini-3.1-Flash-Lite)。我们的分解式评分持续提升了正确提取与错误提取之间的区分能力,在三个设计通道共同贡献下,将 AUROC 从 VLM 语言化信号的 0.54–0.74 大幅提升至 0.90–0.99。对于工业部署而言,更为关键的是,该方法实现了可实际使用的 STP。在目标错误率低于 10% 的风险控制条件下,原生 VLM 置信度信号只能放行 0.1%–7.0% 的字段。相比之下,所提出的方法可自动批准 49%–72% 的字段,同时将已接受层的经验错误率控制在目标值以内。

arXiv cs.AI78

面向直播语音合成的多维韵律评判

评估直播语音合成(TTS)需要考察情感、语调和能量等细粒度、高表现力的韵律,而传统的 MOS 预测器无法捕捉这些特征。尽管 Gemini 等专有大型语言模型(LLM)能够评估这些方面,但其成本过高,难以用于大规模推理和强化学习反馈。为此,我们首先提出 Live-ProsodyJudge(LPJ),一种从 Gemini 蒸馏至 Qwen3-Omni 的高性价比成对评估器。然而,我们发现标准多维评估中存在一个关键缺陷:判定耦合。评判器往往会懒惰地将所有单项维度的评分与整体偏好保持一致,把丰富的多维评分标准压缩成一个单一的偏好比特。为解决这一问题,我们进一步提出 Decoupled-Live-ProsodyJudge(D-LPJ)。D-LPJ 通过移除整体判定目标来避免盲目跟随;在监督微调(SFT)期间屏蔽不确定的成对维度;并引入一种新颖的局部跨度 GRPO 策略,将归一化优势严格应用于其对应的推理跨度。在高度精心筛选、由人工标注的测试集上,采用 10 个样本并平衡顺序的 LPJ,其点准确率高于单次 Gemini 调用;同时,D-LPJ 成功生成了相互独立、解耦的维度判定。此外,在 Best-of-8 TTS 候选选择竞赛中,LPJ 选出的语句在高置信度案例中有 85.29% 位于人工评选的前 3 名之内,证明了其在细粒度 TTS 偏好优化中的有效性。

Google AI Blog

Making global data easier to explore

Google and the UN system have launched the UN System Data Commons, a new open platform making global statistics accessible and easy to search.

Hacker News 熱門66

AGI 的经济政策

文章 URL: https://institute.deepmind.com/essays/economic-policy-for-agi/ 评论 URL: https://news.ycombinator.com/item?id=49743685 得分:3 评论数:0

Hacker News 熱門

Ask HN: How to recover Google auth after phone stolen?

As we know, google has been effective at automating a lot of their systems, to the point where it can feel like there isn't any help available from a human. So what is supposed to be the plan when this type of situation happens? Basically I have a situation that after my phone was stolen, I can't get into any of my accounts that are either connected to google, or connected to the phone via 2FA. This has made it complicated/impossible not only to get info about my phone, but also my banking, email, drive, etc. Of course they offer you "multiple options" to recover an account, but not if you forgot your old email's password. At that point the options you get are, a) Use old phone (obviously the phone is long gone) b) Use current phone (the phone is gone) c) Use old email (I haven't used it in like 12 years) Does anyone know if their is a process to appeal 2FA or anything to deal with this type of situation? I'd imagine that this has happened to people many many times, and given how phones and particularly, google auth have become so critical to accomplishing essential tasks within society, there must be some way. Or maybe it's just like, once you're phone is gone, you are completely over with society? If anyone has any experience with this or inside knowledge please share it! I'm sure people here have heard of this, and maybe experienced it themselves. Is there a secret option that exists that allows us to use passwords instead of rigamarole? I would love an account that let me simply log in with a password, like google used to be. Comments URL: https://news.ycombinator.com/item?id=49742976 Points: 5 # Comments: 2

arXiv cs.AI64

Clueing up LLMs with Tool-Augmented Deductive Reasoning

Despite recent advances in large language models (LLMs), performing logically consistent deductive reasoning over extended interactions remains challenging. Tasks that require integrating evidence across multiple reasoning steps, maintaining consistency with prior inferences, and updating beliefs under new constraints can surface limitations in current models while providing a useful testbed for evaluating reasoning enhancements. In this paper, we implement a text-based, multi-agent version of the classic board game Clue as an environment to evaluate multi-step, agentic deductive reasoning. In this setting, agents must infer hidden information from a sequence of observations, maintain consistency across turns, and reason over an evolving set of logical constraints. We instantiate six LLM-based agents (GPT-4o-mini and Gemini-2.5-Flash) as players that engage in turn-based gameplay; using three agents per model family, we establish baseline performance across repeated games. We then introduce a tool-augmented approach in which a structured possibility matrix converts implicit game state from generated reasoning logs into an explicit representation of remaining possibilities. The possibility matrix encodes extended-turn memory and deductive constraints, offloading these tasks from the agent. We compare this approach against the baseline to evaluate how tool augmentation supports reasoning quality and task success for autonomous agents in a strategic reasoning environment.

arXiv cs.AI78

Decodable but Misrouted: Sparse Features Uncover a Readout Gap in Vision-Language Models for Harmful Meme Detection

When a large vision-language model misclassifies a harmful meme, the failure may reflect missing internal evidence or an inability to route represented evidence to its output. We distinguish these cases in Gemma-3 and Qwen3.5 using sparse autoencoders, role-conditioned probes, causal interventions, and recovery experiments across six harmful content benchmarks, with additional Spanish and Hindi-English code-mixed evaluations. Sparse readouts outperform native prediction on all six primary binary tasks: Qwen averages $0.740$ versus $0.432$ native macro-F1, while residual reconstruction reaches $0.486$, whereas Gemma improves from $0.532$ to $0.714$. These differences reflect supervised accessibility rather than a pre-existing, native decision rule, and the most influential token role depends on the task. Under the evaluated score scales, Qwen silent-feature ablation is $24-63$ times more probe-sensitive, whereas routed-feature patching on literal yes/no tasks is $16-140$ times more output-sensitive. Calibration-only routing recovers $93.3$% of the mean gap, and probe-distilled LoRA improves native predictions, although shared multi-task adaptation causes negative transfer. A case study of Gemma-3-12B on Facebook Hateful Memes finds a distributed rank-32 image-prompt interaction, reaching $0.756$ versus $0.685$ native macro-F1. Robustness controls show that the signal extends beyond English, is not explained solely by accompanying OCR, and depends on paired visual evidence. Thus, routing, rather than representation alone, is a recurring bottleneck in harmful meme classification.

arXiv cs.AI80

A Zeroth-Order Paradigm for LLM Preference Alignment

Direct preference alignment methods are widely used to align large language models (LLMs) with human preferences because of their computational and memory efficiency. However, likelihood displacement motivates alternative ways to extract information from preference pairs with small likelihood margins. In this paper, we propose and analyze Comparison-based Preference Optimization (ComPO), a zeroth-order alignment method based on comparison oracles. ComPO extracts directional information from these pairs without directly optimizing a differentiable preference loss on them. We establish a convergence guarantee for its basic offline scheme under smoothness, gradient sparsity, and compatibility between the oracle and a latent objective. We further introduce online ComPO, which retains the offline comparison mechanism and uses unlabeled policy generations for reverse-KL control relative to a reference policy. Following the coverage perspective of preference fine-tuning, we establish a performance guarantee for a basic constrained scheme under local coverage and in-distribution pairwise reward accuracy. Experiments on Mistral, Llama, Gemma-2, Qwen3, and Gemma-3 models demonstrate improvements over existing direct alignment methods, including length-controlled win rates, with pair-level diagnostics providing evidence consistent with mitigating likelihood displacement.

Hacker News 熱門65

DeepMind 研究所

文章网址: https://institute.deepmind.com/ 评论网址: https://news.ycombinator.com/item?id=49727659 得分:36 评论数:5

Hacker News 熱門81

Show HN: Pizza Bot – An inbox for AI agents that work in the background

Hi HN - long-time lurker (since 2012!), first time poster. Pizza Bot is a self-hosted desktop app for Mac, Windows, and Linux that runs AI agents in the background and exposes them through an email-like UI. Finished work shows up in Unread, and anything waiting on your approval shows up in Action. It's Apache 2.0-licensed, there's no signup and no telemetry, and you bring your own model provider: Anthropic, Amazon Bedrock, Google Gemini, OpenAI, OpenRouter, or a local model through Ollama. There are builds on the releases page, or you can run it from source. Pizza Bot started as an internal passion project I worked on with a small team at Amazon. The whole thing came out of my frustration at having to manually log CRM activities through a browser form. I built a simple REST API called "JoeBot" that connected to my authenticated browser session over CDP and filled out the form for me using Playwright. Then I hacked up a quick Obsidian plugin so I could trigger it from my local notes (no AI and no MCP servers involved). This caught on quickly. My fellow AWS Solutions Architect Igor Fil joined up with me, and we rebranded the project as "Pizza Bot," named after Amazon's two-pizza teams. We started seeing what other automations we could build. We found a GraphQL API we could query and hacked up some "recipes" to pull data out of the CRM to help with meeting prep. That worked great, and it was right around the time MCP servers seemed to be taking off, so we decided to expose Pizza Bot as an MCP server instead, so it would be available to AI tools through natural language. This was a decent solution for technical users, but the Account Managers who live inside our CRM system wanted something too. We decided to rebuild Pizza Bot as an Electron desktop app modeled after an email inbox, so it would be familiar to non-technical users and would run on both Mac and Windows. We also bundled internal MCP servers as OCI images and hosted them in Amazon ECR as an "addon marketplace" so users could install them with one click without having to set up Amazon developer tooling. The project took off organically and expanded outside of AWS into the wider Amazon organization globally. More than 2,000 people ended up using it for meeting prep, email drafting, Slack summaries, CRM logging, prioritizing their day, and web research. Once apps like Claude Cowork and Amazon's own Quick Desktop came out, we realized the real growth opportunity was outside of Amazon. Rather than try to rip out the Amazon-specific integrations, we rebuilt Pizza Bot once more as an open source project. We leaned on coding agents heavily, which is the only reason a team our size could pull off a full rewrite. I'm pleased to say it's finally public, and we're hoping to bring in community members and see where it goes. We'd like to do for knowledge workers what Claude Code and Codex have done for programmers. A couple of things to know up front. Most of what made Pizza Bot useful on day one inside Amazon came from that internal catalog of skills and MCP servers for Amazon's own systems, and none of it could come out with the app. So it ships thinner than the version those 2,000 people used, and building that catalog back up for tools other people actually use is where we need the most help. It's also a community project and not an AWS service, so there's no support or SLA behind it. The Windows and Linux builds aren't signed yet either. On the technical side, Pizza Bot is a server and a client. The desktop app bundles both, or you can point a client at a remote backend; personally, I self-host the server on my home network and reach it from my phone over Tailscale. The server owns the thread lifecycle and checkpoints state with DeepAgents and LangGraph, and clients rehydrate from it as needed, so you can disconnect mid-run and pick the thread back up from another client. Approval pauses outlive the session that created them and collect in an Action filter, so you can answer an hour later from a different device. The agent you talk to has a sandboxed QuickJS interpreter that can reach your filesystem only if you grant it a folder, but its main job is to delegate. Each subagent is a 1:1 mapping of a Skill, and an Activity bar shows that subagent and the tool calls it's making as it works. Memory is opt-in and stored as plain markdown files on your machine. Every tool call is explicit, including looking up a memory - we err on the side of transparency to reduce surprises. Tools come from MCP servers, and skills are ordinary SKILL.md files with a per-tool approval policy, so existing skills that don't require a code interpreter should still work. What I'd most like to hear about is where the app itself gets in your way, the kind of problem you can't fix by writing a skill or an MCP server. I'm around today to answer questions! Comments URL: https://news.ycombinator.com/item?id=49713894 Points: 15 # Comments: 5

Google AI Blog51

AI for Societal Impact

Explore this collection to see how experts and local leaders are using AI breakthroughs to ensure everyone can share the opportunity of AI.

arXiv cs.AI82

在使用 LLM 进行民调之前:审议式诊断框架

LLM 能像人类一样推理新信息,还是只会检索缓存的观点?对于硅基抽样而言,这是一个关键问题,因为 LLM 人格化角色能够大规模模拟公众舆论。当前的评估只检验人格化角色是否持有正确的观点——即一个静态快照。然而,舆论研究越来越依赖动态保真度:人格化角色是否会像人类在审议过程中那样,针对新的论点更新信念。目前还没有任何基准测试对此进行检验。我们提出了审议式民调诊断框架,用于比较人类和 LLM 在接受完全相同的信息干预后信念的变化。该框架以审议式民调为基础,能够揭示静态评估无法发现的失败:模型即使能够生成看似合理的党派观点,仍可能错误呈现这些观点会如何变化。我们使用 America in One Room 的数据(526 个人格化角色、72 个问题),将该框架应用于五个前沿模型,发现每个模型都存在失败,而且表现方式各不相同。GPT-5.1 表现出逆转现象:在获得平衡信息后,其人格化角色对 opposing party 变得更加敌视,而人类则相反,敌意有所降低。这种逆转具有选择性(在外群体问题上为 80%,在政策问题上为 26%),并且在不同党派身份之间呈对称分布。Gemini 2.0 Flash、Claude Sonnet 4.5 和 Llama 3.3 70B 表现出过冲:观点变化方向正确,但幅度达到人类的 5—7 倍。DeepSeek V3 则表现出僵化,变化幅度接近于零。针对性的消融实验表明,政策内容会触发这些失败,而且它们与身份有关:GPT-5.1 在外群体问题上发生逆转,但在内群体问题上出现过冲;Gemini 则正好相反。我们将这一特征称为“自我谄媚”:模型遵循其对人格化角色的内部刻板印象,而不是依据所提供的信息进行推理。我们的框架提供了一套具体方案:在信任 LLM 人格化角色模拟信念更新之前,先运行审议式诊断。

arXiv cs.AI80

Stellar Colosseum:面向数学与理论计算机科学长程研究的多智能体框架

语言模型能够生成看似合理的简短证明,但在长程研究问题上可能仍不可靠,因为这类问题的进展取决于一系列不确定且相互依赖的决策。我们提出 Stellar Colosseum,这是一种与模型无关的框架,用于在数学和理论计算机科学研究中分配推理资源。Colosseum 在构建证明之前探索多种备选策略,使用就绪度门控来决定某条路径何时成熟到足以进行分解,将证明计划表示为相互依赖的章节级子问题,并将验证器的发现反馈至论证中受影响的部分。在这些阶段中,它并行生成候选方案,通过有针对性的证伪对候选方案发起攻击,并通过重叠式随机采样树聚合,将候选方案及其批评整合为单一的研究成果。Colosseum 工作流还已集成到 Google Antigravity 的 Teamwork 框架中,作为 Long Proof 模式。 我们通过开放式研究,以及在定理证明和竞赛编程基准上的评测,展示了 Colosseum 的能力。将 Colosseum 与 Gemini 3.1 Pro 结合使用后,我们取得了若干新成果,解决了源自发表于 FOCS 和 JMLR 等顶级会议或期刊论文中的开放问题。在 TCS-Bench 上——这是一个从发表于 FOCS、STOC 和 SODA 的论文中抽取的研究级定理证明任务基准——Colosseum 使用 Gemini 3.1 Pro 和 Gemini 3.7 Flash 达到了 71.0% 的准确率。在另一项使用 Gemini 3.1 Pro 进行的 Codeforces 评测中,带有执行反馈的面向证明的流水线解决了 222 道题目中的 218 道。

Google AI Blog77

DevFest 回归

DevFest 2026 回归,以下是如何参与全球 800 多场活动之一,在智能体 AI 时代实现构建、安全保障和规模化扩展。

arXiv cs.AI75

重新思考面向次二次注意力的异构系统解耦

前沿语言模型正更积极地使用次二次注意力,以在推理期间降低内存占用和计算需求,同时仍保持前沿准确率。尽管现有系统会围绕以稠密注意力为中心的解耦式服务做出决策,但我们表明,围绕次二次注意力 LLM 独特的算术强度和内存占用来解耦推理,可以在新兴的基于 DRAM 和纯 SRAM 的异构系统上实现显著的吞吐量和能效提升。 我们提出 SQD (SubQuadratic Disaggregation),这是一种细粒度异构解耦方案,它按二次注意力和次二次注意力而非按算子类型来拆分解码,并且适用于各种次二次注意力变体。对于稀疏注意力 LLM,我们将解码解耦为 top-k 选择和 top-k 注意力加 FFN,其中 top-k 选择必须索引完整 KV,而 top-k 注意力加 FFN 具有静态内存占用。对于线性注意力和滑动窗口注意力 LLM,我们将解码解耦为稠密注意力层和次二次注意力层加 FFN。在调整后的 8xB200 异构系统代理中,相比最强的纯 GPU 基线,我们观察到 GLM 5.2 上平均 tokens/J 提升 53%,Nemotron 3 Ultra 上提升 31%,Gemma 4 31B 上提升 56%。在一个具有固定功耗预算的 Rubin 加 LPX 系统分析模型中,相比注意力-FFN 解耦的最佳基线,我们观察到可实现延迟收紧 1.2x 到 1.5x,吞吐量最高提高 3.6x。我们的实验还揭示了面向服务次二次注意力的下一代异构系统在芯片和互连配置方面的架构洞见。

arXiv cs.AI72

评估用于网络安全 CTF 任务的可本地部署 SLM 中的上下文分段

高能力开放权重小型语言模型(SLM)的日益普及,使先进网络安全能力得以民主化;由于这些模型在本地部署时能够绕过专有 API 的安全防护,这也带来了日益加剧的风险。然而,作为自主智能体部署的 SLM 往往难以应对网络安全夺旗(Capture The Flag,CTF)挑战这类长程探索任务,原因在于上下文膨胀,以及累积的工具调用输出导致的认知退化。为理解并缓解这一网络安全威胁,我们引入了\textit{上下文分段},这是一种两级智能体框架,可将复杂的漏洞利用任务划分为易于管理且在上下文上相互隔离的子问题。我们在使用受内存限制的 \texttt{gemma-4} 模型的 \texttt{picoCTF} 数据集上进行评估,结果表明,对于 E4B 模型,我们的策略能够充当一种智能搜索:与暴力重试相比,在具备更高令牌效率的同时取得具有竞争力的奖励,并成功解决了标准智能体执行无法完成的 18.52\% 的任务。代码可在 https://github.com/9xeb/context-segmentation 获取。

arXiv cs.AI71

TileNet:用于平屋顶自主无人机系统巡检的基于瓦片的 CNN-SVM 架构

平屋顶是建筑围护结构中最具影响力的组成部分之一,既影响结构性能,也影响热效率,从而直接关系到家庭能源消耗、碳排放和长期环境可持续性。及时检测屋顶缺陷对于减少供暖和制冷损失、防止霉菌生长等受潮引发的劣化,以及支持国家气候变化减缓目标至关重要。本文提出了一种基于实时无人机系统(UAS)的深度学习框架,可利用双高度飞行过程中采集的实时图像自主检测缺陷。该多分辨率飞行策略旨在帮助识别细小、精细尺度的缺陷和较大的结构性问题,从而实现更全面的评估。为满足嵌入式 UAS 硬件严格的计算和功耗限制,所提出的框架将基于瓦片的架构与轻量级卷积神经网络-支持向量机(CNN-SVM)分类器相结合,以实现低延迟的机载推理。最终模型包含五个卷积层和四个全连接层,其中最后一层为线性 SVM 头;在图像级划分的数据集上,该模型取得了 $94.4\%$ 的平均测试准确率(基于三个随机种子的 $95\%$ 置信区间为 $\pm0.4\%$;包含 $43,383$ 张训练图像、$3,869$ 张验证图像以及 $2,540$ 张测试图像,均经过瓦片化和增强),优于 GoogLeNet($89.2\%$)和 AlexNet($79.8\%$)。使用通过现场访问、由 DJI Matrice 350 RTK 无人机采集的真实 UAS 图像进行的实验评估表明,该系统能够支持快速、可重复且安全的屋顶巡检,同时降低人员风险和运营成本,并实现更加可持续的建筑维护。

Hacker News 熱門51

Show HN: Liniora – Ever thought about replacing your project manager?

Hi HN, the title could be a little bit aggressive sorry for that... Basically i am Computer engineer graduated recently but i was and still working remotely for approx. 3 years with the same foreign company in EU. and i noticed that we are always somehow lose the context of a ticket -- even though we are using the popular project management systems -- so i decided why not build something that can fix our context loosing problems and with some improvements. here i created Liniora. an AI powered project management system. basically the core idea is to link the tools that we use to gather information about tickets in one place. i integrated Google meet for the meetings transcripts so you do your meeting it automatically got registered and indexed. You discuss about a bug in Slack thread so you mention @Liniora app and it will handle the bug context and create a ticket for you. We have the version control also connected so you create a ticket with ability of creating a separate branch with one-click for both Gitlab and Github. you have as well the Pull request controlled in our place inside the app. lets suppose you have a ticket and you created a branch called 'USER-325-create-login-form' then you merged to staging or production ? it will be automatically flagged with staging/production badge so you know what tickets are done. there are a lot of interesting features i wanted to describe but i will just let you guys discover and i will be very happy about your feedback. Thanks Comments URL: https://news.ycombinator.com/item?id=49671542 Points: 5 # Comments: 2

Hacker News 熱門77

Gemini 应用现已支持 Windows

文章 URL: https://blog.google/innovation-and-ai/products/gemini-app/gemini-app-now-on-windows/ 评论 URL: https://news.ycombinator.com/item?id=49653699 得分:7 评论数:1

arXiv cs.AI74

从参数到答案:LLMs 如何检索并使用其内部知识

语言模型在回答问题时,对查询路由信息和目标知识的依赖如何变化?我们通过对问题末尾隐藏状态进行逐层干预来研究这一问题。在 Qwen、Llama 和 Gemma 上,我们比较了答案为名词、形容词和代码的国家—洲问题,同时保持若干拟合测量彼此区分。成对条件请求方向描述了自然单国家问题中查询的是哪个国家;全局请求方向描述了配对问题中对第一个国家还是第二个国家的请求;独立的选择候选项则检验对隐藏状态中已经可用内容的控制。对冻结的 Qwen 自然问题状态进行诊断性重新分析表明,在对成对条件请求方向进行干预开始改变后续拟合知识之前,该方向就已变得更强;这一因果窗口开启时,支持答案的内容仍在形成。三种模型在配对设置下的轨迹并不一致:Gemma 展现出部分重叠的中间层路由—内容特征,而 Llama 在相同门控条件下没有持续的路由效应窗口。在配对协议中,从固定的较早层集合到较晚层集合,对全局请求方向的依赖逐渐降低,而对拟合内容的依赖持续存在。对 Qwen 的匹配比较表明,成对条件请求方向在较晚阶段仍保留效应,因此,这种操作意义上的交接涉及全局拟合方向,而不是所有请求信息。这些结果区分了早期可读性、自然强度、因果引导和后期内容依赖。

arXiv cs.AI82

可在边缘部署的视觉语言模型能识别物种吗?

相机陷阱通常在野外依靠连接受限或完全没有网络连接的边缘硬件运行,因此,小型、可本地部署的视觉语言模型(VLM)——而非前沿规模模型——才是评估物种识别能力时具有实际意义的模型类别。我们测试了处于这一部署相关的2--8B规模范围内的模型是否具备真正的分类学知识:在一个包含96个物种的任务中,将四种此类VLM(Qwen3-VL 2B/4B/8B、Gemma3 4B)与领域专用模型BioCLIP(3亿参数)进行比较;同时,在两个独立采样的评估集上,对比清晰的iNaturalist照片与来自6个LILA.science数据集的相机陷阱图像。所有模型的物种识别准确率都显著高于随机水平,但每个模型——无论是通用模型还是专用模型——在野外图像上的性能都大幅下降(领域差距为9.6--26.6个百分点,在不同分类学层级和两个评估集上均保持一致),这表明性能下降反映的是图像整体可辨识性的降低,而不是细粒度判别能力的失效。在针对每个模型扩展至200张图像的样本中,BioCLIP的表现显著优于所有受测VLM(高出33.2--59.2个百分点),尽管其规模小得多,这说明差距源于专门化训练数据,而非模型规模;然而,BioCLIP自身的领域差距为18.0个百分点,与最佳VLM的22.3个百分点在统计上无法区分,这表明从清晰图像到野外图像的性能下降本身是图像质量变化的特性,而不是通用模型的弱点。在开放集提示条件下,有5.9--9.6%的响应在语法上有效,但所给出的物种名称在分类学上并不存在;不同模型之间虚构率的相对排名在两个评估集上完全复现,这一发现比任何单一的点估计都更稳健。

arXiv cs.AI82

将跨语言临床标注投影作为受约束的文本生成:一项六语言研究

背景:旨在确定跨语言临床标注投影能否被表述为一种保留文本、面向文档级别的生成任务,以生成可验证的字符级标注,用于构建多语言临床语料库,并刻画其相较于基于候选项的投影流程的稳健性与计算权衡。方法:我们开发了一套受约束的 LLM 投影工作流,将实体标签直接插入不可变的目标语言文本中,随后进行确定性验证和字符偏移重建。我们同时评估了监督式候选跨度投影方法,以及用于将西班牙语 Disease、Symptom 和 Procedure 标注转移到六种语言的混合式 ML-LLM 优化方法。评估采用 MultiClinAI 金标准,并使用严格跨度匹配和字符重叠 F1。结果:直接 LLM 投影取得了最强且最稳定的性能。GLM 5.2 在 18 个语言-实体组合上的平均 Strict F1 达到 0.9201,而可在本地部署的 Gemma4:31B 达到 0.9133。在全部 18 种设置中,最佳 LLM 配置的 Strict F1 均超过此前的最新水平,提升幅度为 0.0564–0.1512,并生成了 55,416 条完成偏移重建且具有依据的提及。结论:基于直接 LLM 的投影能够实现高质量的多语言临床标注转移,并为将临床 NLP 资源扩展到标注数据集和语言专用工具较少的语言提供了一种实用方法。结合本地推理和确定性验证,该方法可以显著减少构建多语言临床语料库所需的专家时间和成本。

arXiv cs.AI79

主动适应,而非静态防御:对抗性微调中 Preventative Steering 的时间动态

大型语言模型在面对恶意微调时仍然十分脆弱,因此有必要在训练阶段防御有害的人格漂移。Preventative Steering 在微调期间注入不良特质人格向量,并在评估时将其移除,但其持久保护机制仍不明确。通过分析其时间优化动态,我们发现,该防御机制源于早期的补偿性适应阶段,随后进入修正信号逐渐衰减的稳态阶段;在参数空间中,注意力输出投影成为防御性更新的主要残差写入路径。通过 Intervention Delta Preservation(IDP)和 IDP Continuation 实验,我们进一步表明,保留或重新注入权重偏移都无法维持防护效果,这说明 Preventative Steering 依赖的是主动适应,而非静态防御。受此发现启发,我们提出 Progressive Intensity Scheduling(PIS):从中等注入强度开始,并在静态强度引导的对齐效果开始衰减后提高注入强度。在所评估的 Qwen2.5 和 Gemma-3 模型中,相较于固定强度引导,PIS 提升了安全稳健性,同时减少了有害特质的表达。

arXiv cs.AI70

From Symbolic Perception to Logical Deduction: A Framework for Guiding Language Models in Geometric Reasoning

Plane geometry remains a significant challenge in AI, requiring the integration of visual perception and mathematical reasoning. While Large Multimodal Models (LMMs) naturally handle visuo-linguistic inputs, they are often computationally intensive and opaque. We demonstrate that a pure Large Language Model (LLM), when equipped with specialized modules, can rival state-of-the-art LMMs on complex geometry problems. Our framework integrates a Geometric Vision Parser, which translates diagrams into symbolic form, with a Symbolic Solver that performs formal deductions, thereby mitigating hallucinations and promoting interpretable reasoning. To enable rigorous evaluation, we curate a benchmark of challenging problems from the 2025 Chinese Zhongkao examinations, ensuring data novelty and testing deeper deductive skills. Experiments demonstrate that our approach achieves performance comparable to Gemini 2.5 Pro while delivering clearer, human-like solutions.

Hacker News 熱門69

Show HN:Rdltr——阅读清单的收件箱归零

嗨,HN!过去几个月里,我一直在开发一款名为 RDLTR( https://rdltr.app )的稍后阅读应用。整个想法源于我在不同浏览器、Mac、Linux 和 iPhone 上打开了太多想读的网页标签。它们让所有东西变得杂乱不堪,也妨碍了我处理其他事情。 这个想法出现时,Pocket 刚刚关闭。我四处了解了一下,试用了 Instapaper、Raindrop、Matter,还有其他几个我已经记不清名字的产品。它们都还不错,但都会给你一份只会不断增长的列表,而这又成了另一件需要维护的事情,也不是我想要的模式。Safari 有一个“阅读列表”功能:读完某篇内容后,它就会从未读列表中消失。但它只能在 Apple 设备上使用,而且功能也不太完善。我想要的是阅读方面的收件箱归零:打开某篇内容,读完它,再回来时,它已经从队列中消失,但仍然保存在归档里。不断继续,直到没有剩下任何内容。 链接可以从任何地方添加。在页面上粘贴 URL、拖入链接,或者使用书签小工具、Chrome 和 Firefox 扩展,或用于 iOS 分享菜单的 Apple Shortcut。此外还支持 Vim 按键、为每个链接生成 TL;DR 摘要、搜索所有已阅读或保存的内容、为每个列表提供 RSS,而且在手机上使用同样顺畅。 这些扩展还有几个我一直在使用的小技巧。Shift+DD 会提取文章正文、清理格式,然后在新标签页中将其发送给 ChatGPT、Claude 或 Gemini,这样你就可以询问它任何与刚刚阅读的内容有关的问题。Shift+AA 会在 Internet Archive 中打开当前页面;当你没有订阅而网站设有付费墙时,这个功能很方便。 技术栈包括 Bun、TypeScript、HTMX 和 SQLite,并使用 SSE,让列表在你保存或读完某些内容的瞬间更新;应用部署在 Hetzner VPS 上。 坦白说,大部分代码都是 Claude Code 写的。我仍然花了大量时间进行实际使用、测试,并不断打磨 UI,直到一切用起来感觉恰到好处。 总之,很想知道你目前是如何处理这件事的。 评论链接: https://news.ycombinator.com/item?id=49629747 得分:3 评论数:0

Hacker News 熱門63

我在 Google Ads 上投放恶意软件广告

文章 URL: https://xlii.space/eng/malicious-software-on-google-ads/ 评论 URL: https://news.ycombinator.com/item?id=49624856 得分:4 评论数:1

arXiv cs.AI71

SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?

While research on recursive self-improvement (RSI) has predominantly automated model training pipelines, reliable autonomous development demands a missing pillar: post-hoc monitoring and auditing to understand what models learn and ensure safe alignment. Mechanistic interpretability tools are essential to bridge this gap, among which Sparse Autoencoders (SAEs) serve as a cornerstone by isolating interpretable features for model inspection and steering. In this paper, we introduce SAEScientist-Bench to evaluate whether AI agents can act as scientists utilizing SAE tools for autonomous mechanistic discovery. Given a target concept, an agent designs contrastive probes and navigates a Gemma Scope dictionary of 131K+ features in Gemma-2-9B-IT to discover the optimal feature, evaluated against curated expert reference features anchored on Neuronpedia across activation rank, concept selectivity on contrastive texts, and causal steering. Across 10 agent configurations and 20 tasks, frontier agents demonstrate genuine discovery capabilities and lead different evaluation dimensions, but remain well behind the expert baseline, approaching expert levels on separating target concepts from contrastive controls while lagging substantially in causal generation steering. Further analysis reveals that although agents can design contrasts to rule out spurious candidates, they frequently misinterpret experimental measurements. These results establish experimental model understanding as a measurable capability for closed-loop autonomous AI R&D. Our code is available at https://github.com/Trae1ounG/SAEScientist.

arXiv cs.AI78

共同演化 Harnesses 与 Models:On-Policy Correction 帮助较弱模型在 Imitation 失效之处迎头赶上

Agent harnesses(围绕模型的 system prompt、tool set、execution hooks 和 context-management scaffolding)是决定 agentic 任务成功的关键因素。自动化 harness 演化可以使较小模型以 frontier-model 成本的一小部分在特定领域任务上表现良好。由于 harness 和模型权重都会塑造行为,我们探讨应如何结合 harness 演化与轻量级 fine-tuning。在七项企业 agent 任务中,我们首先使用较弱模型演化出一个 harness,随后发现更强的 expert 往往能更有效地使用它,这表明 expert 监督可能弥合剩余差距。然而,在演化后的 harness 下用 expert 的完整轨迹训练较弱模型会适得其反:在 Qwen3-Coder 和 Gemma 4 上,七项任务的性能全部回退 4 到 30 个点,尽管同样的流程在未演化的 harness 下是有帮助的。我们的分析表明,imitation 会迁移知识并增加 scaffold 使用,但会破坏模型-harness 适配:较弱模型采用了 expert 的规划策略,却不具备执行该策略的能力,并且不再匹配围绕其原生规划风格演化出的 harness。因此,我们开发了一条 on-policy expert-correction 流程,由一个元层级 MLE agent 自动化执行;该流程会定位较弱模型自身 rollout 中失败的 turn,并要求 expert 仅重写该 turn。这保留了模型的规划风格,并结合了 harness 演化与模型适应的收益。我们的结果识别并解决了 harness 与权重更新之间的一种冲突来源,为特定领域企业任务上的经济型共同演化提供了一种保持兼容性的方案。

arXiv cs.AI84

All for 1-Bit:迈向真正的 LLM 1 比特训练后量化

大型语言模型(LLM)取得了显著进展,但其巨大的存储和内存带宽需求仍然阻碍着高效部署。权重二值化是一种有前景的解决方案,但现有基于二值化的训练后量化(PTQ)方法通常由于隐藏开销而远超名义上的 1 比特存储目标。为弥合这一差距,我们提出了 All for 1-Bit(AF1),一种面向 LLM 的真正 1 比特 PTQ 框架。AF1 由两个互补组件构成:(1)空空间感知二值分解(NABF),通过 Hessian 感知的代理重参数化、空空间感知的二值分解以及仅缩放的全局重构来改进二值重构;(2)分层 Shapley 分配(HiSA),利用分层 Shapley 敏感度分配结构容量。二者结合后,使模型在 PTQ 设置下严格遵守 1.0-BPW 预算的同时保持准确率。在 LLaMA、Qwen 和 Gemma 系列上的实验表明,AF1 在困惑度和零样本准确率方面始终优于现有基于二值化的 PTQ 方法。与 BF16 相比,在所评估的模型中,AF1 平均实现了 2.5 倍的推理加速,并将内存占用降低了 90% 以上,为 LLM 真正的 1 比特压缩走向实际部署提供了可行路径。用于复现实验的代码位于 https://github.com/Kishon-zzx/AF1。

Hacker News 熱門76

Google DeepMind 发布 AlphaGenome Atlas

文章链接: https://blog.google/innovation-and-ai/models-and-research/google-deepmind/alphagenome-atlas/ 评论链接: https://news.ycombinator.com/item?id=49611251 得分:9 # 评论:0

Hacker News 熱門87

WeatherNext 3:我们最先进的全球天气 AI 模型

文章链接: https://blog.google/innovation-and-ai/models-and-research/google-deepmind/introducing-weathernext-3/ 评论链接: https://news.ycombinator.com/item?id=49604257 得分:4 # 评论数:0

arXiv cs.AI75

必要还是充分?使用行为证据评估 LLM 解释

LLM 决策组件在 agent 工作流中运行时,常会给出与行动相关的建议或判断,并附带解释。操作人员可能会利用这些命名因素来监控系统、诊断错误,或决定何时升级处理某个输出。这种用法假设这些解释与组件可观察到的决策行为一致。我们测试了这些命名因素的两种解释:必要性,即改变某个因素会改变输出;以及充分性,即在移除其他可变信息的同时保留该因素会保持输出不变。我们在两个合成用例中评估这些解释:向客户推荐顾问,以及判断提示词是否具有有害性或风险。模型返回一个输出以及最能影响它的前三个因素。受控的 black-box 干预通过衡量改变某个因素后输出发生变化的频率来估计其必要性得分,并通过衡量保留某个因素后输出得以保持的频率来估计其充分性得分。在来自 Claude、GPT 和 Gemini 系列的八个模型中,引用排名与必要性和充分性得分之间的平均斯皮尔曼相关系数在顾问推荐任务中分别为 0.349 和 0.354,在提示监控任务中分别为 0.431 和 0.580。此外,在必要性条件下,57.6% 的顾问响应中,某个未被引用的因素得分高于得分最低的被引用因素;在充分性条件下这一比例为 58.1%。对应的提示监控比例分别为 25.8% 和 8.9%。被引用的前三项包含有用信息,但并不能可靠地识别在必要性或充分性意义下、经测量影响最强的三个因素。该框架为代理监督中使用的解释提供了一种 black-box 可靠性检查,同时仍限定在单个 LLM 决策范围内。

arXiv cs.AI78

感知底层环境的 AI 代理:将执行上下文作为一等输入

自主 AI 代理越来越多地在这样一种环境中选择行动:其内存、执行时间、运行时、计算和操作约束决定了什么才算合适的计划。我们将代理规划状态中缺少这种执行上下文称为 substrate blindness。我们通过数值代码生成来检验这一一般性命题,因为在该任务中,所选实现选项及其运行后果可以直接观察。三种前沿模型配置——Anthropic Claude Opus 5、OpenAI GPT-5.6-Sol 和 Google Gemini 3.7 Flash——针对一个高维成对欧氏距离任务生成代码,输入要么只有任务本身,要么再附加一个 128 MB RAM 和 10.0 s 墙钟时间约束。披露约束使 14 组可执行、索引对齐的“仅任务”与“披露约束”比较中有 13 组的测得进程峰值内存降低;在所有三个队列中,平均墙钟时间也降低,使执行速度最高提升 3.1 倍。在整个审计语料中,披露约束带来了结构性代码变化,包括有界分块、保留 float32、上三角遍历,以及原地或内存映射缓冲区。在更严格的 96 MB 约束下,独立抽样的披露约束队列在正确且未超预算的结果上,Claude Opus 5 为 4/5,GPT-5.6-Sol 为 5/5,Gemini 3.7 Flash 为 3/5;而仅任务条件下分别为 0/5、1/5 和 0/5。各队列的平均 MaxRSS 和墙钟时间分别比仅任务参考低 49–74% 和 35–64%。这些结果为感知底层环境的代理规划提供了一个受控的概念验证:最小执行约束会在生成程序中诱发主动的结构适配,将计算从无约束分配转移开来,并在执行前显著改善观测到的资源—时间特征。

arXiv cs.AI79

ESPO:通过诊断、扩展和稳定化实现错误结构化提示优化

像 GEPA 这样的进化式提示优化器会遭遇提示词膨胀:每次迭代都会追加规则和注意事项,导致提示词长度最多增加 3$\times$,却并未更准确。我们将其归因于三个缺陷——错误观测不完整、搜索多样性有限以及选择不可靠——并提出 ESPO(Error-Structured Prompt Optimization),将提示优化分解为三个阶段:Diagnose 在一轮中将所有训练错误聚类为结构模式;Propose 通过四种具有互补偏置的策略生成候选;Select 采用 bootstrap 稳定性选择。在七个公开 NLP 基准——Tweet、MMLU、GSM8K、HotpotQA、ScoNe、HoVer 和 PUPA——上,ESPO 相比当前最优方法平均准确率提升 $+$3.76 个百分点(74.67% 对 70.91%,GEPA),在每个数据集上都与 GEPA 持平或更优,同时生成的提示词短 47%(1,004 对 1,878 个字符),且推理更快。跨模型实验覆盖另外四个学生模型(Gemma 3 12B、Mistral 14B、Qwen3 32B、Claude Haiku 4.5),结果表明 ESPO 在每个测试模型上都取得了最佳平均准确率,其中在 Qwen3 GSM8K 上差距最大(15.00% $\to$ 91.40%)。附录中的一个泛化界将每个阶段与测试时差距中的相应项对应起来,而消融实验验证了一个关键预测:在没有 bootstrap 选择的情况下加入多样性实际上会损害性能($-$1.20%)。

Hacker News 熱門

Ask HN: Are others seeing Google's reCAPTCHA rejecting Firefox users?

Infinite Google ReCAPCHA loop trying to access archive.is using current Firefox from Linux. Just started today. Firefox is in "strict" mode. Disabled Privacy Badger for that site, and it didn't help. Tried private browsing, and it didn't help. Is this part of Google's war against ad-blocking and non-Chrome browsers? Comments URL: https://news.ycombinator.com/item?id=49555592 Points: 8 # Comments: 2

arXiv cs.AI81

大型语言模型中的“先高后低”请求与拒绝行为

“先高后低”技巧在语言模型上也有效吗?在人类中,一个较大的请求如果被拒绝,会使随后提出的较小请求更有可能被接受。我们在来自三家提供方的九个商用模型上测试这一点:每个模型先拒绝一个较大的请求,然后再收到同一请求的较小版本,并将其服从率与直接提出请求时进行比较。答案取决于模型。在 Anthropic 的前沿模型上,这一技巧有效:Opus 5 在拒绝较大请求后,有 65.8% 的概率答应较小请求,而直接提出时这一比例为 29.3%。在 OpenAI 和 Google 的前沿模型,以及 Haiku 4.5 上,它却适得其反,使服从率降低 15.5 到 23.0 个百分点。一个对照实验定位了这一效应:在所有九个模型上,针对无关主题的被拒绝大请求产生的影响都小于相关请求,因此让步本身在所有模型上都很重要,而对刚刚拒绝过某件事的反应则因模型家族而异。该技巧不能迁移到来自公开基准的拒绝样本。决定“退一步”是否奏效的是请求内容:将 265 个被拒绝的、旨在获取可用指令的请求改写为同一主题的解释性请求后,有 263 个案例中的拒绝被消除了。人类的影响技巧迁移到语言模型上时,是按模型家族逐个发生的。

arXiv cs.AI75

语言模型可以控制它们自己的注意力

语言模型将大部分注意力集中在上下文中很小一部分内容上,但它们会读取整个 KV cache 来找出那几个关键 token。如果用户在一段 100 万 token 的对话中询问之前的某个细节,global attention 层必须扫描完整上下文才能生成回复中的每个 token。一种突出的做法是通过轻量级代理分数预先选择相关 token 来缓解这一成本,但这种外部打分在每一步仍然需要 O(N) 的开销。我们采用一种内在方法,其动机来自一个简单的问题:模型难道不会已经知道上下文中哪些部分是相关的吗?为此,我们提出 Declarative Attention (DA),一种协议,它引导模型在其思维链中声明需要关注的位置,并将生成过程划分为三种模式: (完整上下文)、 (特定区域)和 (仅最近输出)。推理引擎像处理工具调用一样解析这些声明,并跳过大部分 KV cache 的读取。在 15 个长上下文任务上的零样本评估中,DA 在现成模型(Gemma-4-31B、Qwen-3.6-27B)上显著减少了解码期间被注意到的 token 总数(52.0%、31.1%),同时仅带来较小的准确率下降(1.27pp、2.75pp),且这种下降会随着模型规模增大而减小。DA 为稀疏注意力开辟了一个新的方向,而基于训练的方法下还存在进一步潜力,未来工作可以继续探索。

Hacker News 熱門

Google avoids a breakup of its ad tech business

https://archive.ph/rlAbY https://www.reuters.com/legal/litigation/google-defeats-us-b... Comments URL: https://news.ycombinator.com/item?id=49537131 Points: 38 # Comments: 10

Hacker News 熱門85

Gemini 3.8 Flash 和 3.8 Flash Cyber

文章链接: https://blog.google/innovation-and-ai/models-and-research/gemini-models/3-8-flash-and-3-8-flash-cyber/ 评论链接: https://news.ycombinator.com/item?id=49538007 积分:17 # 评论:1

Hacker News 熱門90

Gemini 3.8 Flash

文章链接: https://deepmind.google/models/model-cards/gemini-3-8-flash/ 评论链接: https://news.ycombinator.com/item?id=49537553 得分:52 # 评论:24

arXiv cs.AI64

Revisiting Face Recognition for Monozygotic Twins: The Celeb Twins Test Set

Past literature on face recognition for monozygotic (("identical") twins points to facial marks and mirror asymmetry as possible directions for improved accuracy of twins recognition. The Celeb Twins Test Set (CTTS) contains web-scraped image pairs for 80 sets of celebrity twins. It is the only twins test set with meta-data for twins with distinguishing skin marks and possible mirror asymmetry. CTTS is organized in the manner of face verification test sets such as LFW, CALFW, CPLFW, CFP-FP, and AgeDB-30. Current deep CNN matchers can achieve over 76% accuracy in classifying CTTS same-person / different-person image pairs. We show that current matchers do not make use of skin marks, or asymmetry, and discuss reasons for this. Finally, we discuss the feasibility of using generative AI tools such as Grok, ChatGPT and Gemini to create images of imagined monozygotic twins as a means to increase representation of twins in face recognition training sets.

arXiv cs.AI76

LOCI:带有 Refinement Loop 的 Locator-Critic

Vision-Language Models(VLMs)在需要复杂视觉理解的任务上仍然表现不佳。我们认为,核心问题不在于高层推理,而在于无法在图像中定位关键细节。由于这一缺陷,VLMs 往往会基于有缺陷的感知依据生成看似合理但实际上错误的推理。为了解决这一问题,我们提出 Locator-Critic(LOCI),这是一个无需训练的框架,将视觉搜索与证据验证解耦。LOCI 采用一个 Locator 代理来提出候选视觉证据,再由独立的 Critic 代理评估其相关性和充分性。这些代理在一个迭代的 Refinement Loop 中协同工作,逐步改进证据,直到其足以回答给定问题。这种解耦的、自我纠正的过程带来了显著的性能提升,在多个复杂视觉基准上取得了 state-of-the-art 结果。LOCI 同时提升了开源权重模型(如 Qwen3-VL,在 V* 上提升 +12.1,在 HR-Bench 上提升 +5.8,在 VisualProbe-Hard 上提升 +11.2)和专有模型(如 Gemini 2.5 Pro,在 V* 上提升 +8.9,在 HR-Bench 上提升 +4.3,在 VisualProbe-Hard 上提升 +4.8)的准确率。

arXiv cs.AI81

用于跨模型 KV 共享的通用上下文复用层

现代大语言模型(LLM)服务系统越来越多地在重复或共享的上下文上运行,但即使另一个模型已经处理过相同输入,每个模型通常仍会执行自己的 prefill 计算。现有的 KV-cache 复用机制可在单一模型内大幅减少冗余计算,但通常假设缓存的生产者和消费者是同一个模型。我们研究跨模型 KV 共享,即将源模型生成的 KV 状态转换为可被不同目标模型消费的表示,包括在规模、架构、注意力配置、tokenizer 和模型家族上存在差异的模型。我们在同一模型家族内和跨模型家族两种设置下评估该方法。对于 Qwen2.5-7B $\rightarrow$ Qwen2.5-1.5B,转换后的 KV 状态将 LongBench2 准确率从 27.59% 提升到 34.48%,相比原生 1.5B 基线提升了 6.89 个百分点,同时相较于原生目标模型 prefill 降低了交接成本。对于跨家族的 Qwen2.5-1.5B $\rightarrow$ Gemma-2-2B 设置,KV 交接在 4K 上下文长度下可将目标侧 prefill 成本最多降低 67.05%,同时保持解码困惑度接近原生模型基线。在更异构的 Llama3.1-70B $\rightarrow$ Qwen2.5-7B 设置中,跨家族交接实现了 44.0% 的准确率,而原生 Qwen2.5-7B 推理为 45.7%,同时将测得延迟从 899ms 降至 138ms。这些结果初步表明,KV 状态可以作为可迁移的计算表示,而不仅仅是严格局限于单个模型的缓存,并且为将 context mobility 作为一种系统抽象、以减少异构 LLM 和多智能体推理工作流中的冗余 prefill 提供了动机。

arXiv cs.AI76

细粒度多图像对象幻觉基准

Multimodal Large Language Models (MLLMs)正越来越多地部署在需要跨视觉上下文进行复杂推理的多图像场景中。然而,当前的MLLMs仍然受到对象幻觉的根本性限制——它们会生成看似合理但在事实上不一致的对象描述。现有基准主要面向单图像设置,或仅提供高层次的多图像评估,无法系统性地诊断视觉复杂性和推理需求如何引发幻觉。为弥补这一空白,我们提出MIOH,一个细粒度多图像对象幻觉基准,基于三种多图像推理模式(全面、比较、选择)和三种受控对抗压力(视觉上下文规模、感知难度、上下文偏置),系统评估四项基础任务(存在性、计数、属性、位置)中的对象幻觉。通过对29个模型的评估,我们揭示即使是GPT-5和Gemini-2.5-Pro这样的最先进系统,也会在不同推理模式和任务上表现出不同的失败模式。我们的评估表明,幻觉不仅源于感知失败,也源于在多张图像之间维持对象表征时整合阶段的局限。MIOH为分析多图像对象幻觉提供了一个受控框架,并作为开发更可靠的多模态AI系统的关键评估工具。

arXiv cs.AI72

面向自主云 MLOps 的前向部署式全栈工程

跨越多个行业,机器学习系统支撑着从预测和异常检测到预测、优化和调度等一系列应用,然而,要将这些系统投入实际运行,需要协调应用开发、模型流水线、云基础设施、安全性、部署、监控、重新训练、恢复和回滚。我们提出一种证据门控的多智能体框架,用于将自然语言 MLOps 云工程任务转换为经验证的代码仓库和可运行的云端部署。该框架结合了图工程、循环工程和智能体支架工程。一个有状态的 Graph Orchestrator 协调专门的智能体,负责代码仓库生成、审查、执行、验证、发布和监控,同时管理工作流依赖、证据门控、重试上限、恢复路径和终止。只有当所需谓词得到可验证的执行证据或运行时证据支持时,相关的生命周期转换才会进行。验证失败会触发有界的反思、修复和重新验证;而失败、漂移、退化或策略违规的运行时证据则可触发有界的自适应、恢复或回滚。智能体支架工程通过受控能力和隔离执行环境,约束代码仓库生成、审查与修复、制品执行以及云操作。我们在 Google Cloud Platform 上实现了该框架,并评估了代码仓库完整性、受控执行、证据门控转换、云端上线以及有界恢复。实验结果表明,该框架可防止缺乏支持的生命周期转换,并推动每次运行要么走向经验证的可运行部署,要么走向可审计的终止性失败。

Hacker News 熱門70

Gemini-3.5-Transcribe

文章 URL: https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-5-transcribe/ 评论 URL: https://news.ycombinator.com/item?id=49468818 得分: 19 # 评论数: 5

Hacker News 熱門69

Gemini Omni 1.1 Flash

文章链接: https://blog.google/innovation-and-ai/technology/developers-tools/build-with-gemini-omni-1-1-flash/ 评论链接: https://news.ycombinator.com/item?id=49467922 得分:17 评论数:0

arXiv cs.AI84

行星预测引擎:通过智能数据选择和基础模型嵌入实现自主地理空间预测

应对从粮食安全和灾害风险到疾病暴发及社会经济脆弱性等关键全球挑战,需要高保真的地理空间建模。然而,构建可预测的行星级模型仍受制于分散的数据生态系统,需要人工检索数据、多模态数据整理与融合,以及迭代式模型选择。我们提出 Planetary Prediction Engine (PPE),这是一个自主 AI 系统,可直接从自然语言查询出发,执行这一端到端工作流。PPE 按需综合多模态数据集,从开放网络和地球观测平台(Data Commons、Google Earth Engine)中检索时空相关协变量,并将其与地理空间基础模型嵌入(PDFM、AlphaEarth)融合。同时,它还会在面向任务定制的模型架构家族中搜索,并配备自动化过拟合防护。在不同任务、地理区域和科学领域中,PPE 始终优于当前最先进方法或人工调优的专家基线。对于美国空间回归,PPE 在 21 个 CDC 健康指标上的平均 $R^2$ 提升至 76.8%(基线为 60.0%),在 FEMA 国家风险指数上提升至 64.9%(基线为 60.0%),在 Social Vulnerability Index 上提升至 66.2%(基线为 58.6%)。对于数据稀缺环境中的空间下采样任务,PPE 融合本地代理变量,使尼日利亚粮食安全指标的基线准确率翻倍($R^2$ 从 31.5% 提升至 66.1%)。对于 2026 年 DRC Bundibugyo Ebola outbreak 的流行病学即时预测,PPE 实现了 83.3% 的 Recall@10(在五次每周预测中识别出 18 个新入侵卫生区中的 15 个),比公开的最先进模型(约 73%)提高了 10.3 个百分点。通过将自主的多模态行星数据发现与有针对性的模型优化相结合,PPE 降低了行星尺度分析的技术门槛,使快速、定制化、专家级部署成为可能。

arXiv cs.AI75

EG-ARSA:用于低资源环境中视觉道路安全审计的专家扎根开放模型

道路交通伤害仍然是低收入和中等收入国家面临的一项重大挑战,在这些国家,主动式道路安全审计受到事故记录不完整、合格审计员短缺以及大规模实地检查成本高昂的限制。为解决这一问题,我们提出 Expert-Grounded Distillation(EGD),这是一种新型人工智能框架,可将机构性道路安全专业知识迁移到紧凑型视觉-语言模型中,以实现可扩展的视觉道路安全审计。其关键创新在于量化的专家扎根阶段,在该阶段中,教师视觉-语言模型会依据权威实地审计结果进行校准。只有当教师模型与专家风险评估达到实质性一致(Cohen's kappa = 0.74)后,才允许进行大规模标注。随后,经校准的教师模型生成结构化监督信号,并使用 Low-Rank Adaptation 和单个无泄漏提示将其蒸馏到一个 80 亿参数的学生视觉-语言模型中。我们还推出 Bangladesh Road Safety Audit(BD-ARSA),这是首个开放的、专家扎根的孟加拉国视觉道路安全审计数据集,包含 21,947 条图像-审计记录,覆盖范围接近全国;以及 Expert-Grounded Road Safety Auditor(EG-ARSA),这是首个专门为该任务开发的视觉-语言模型。实验结果表明,与零样本基线相比,扎根微调显著改善了有序风险评估;而盲法专家评估显示,紧凑型学生模型的表现优于其 310 亿参数的教师模型和 Gemini-2.5-Flash。这些发现表明,EGD 为资源受限环境中的主动式道路安全审计提供了一种有效且可扩展的工程解决方案。

arXiv cs.AI69

Specification Portability Across LLM Development Agents: Cross-Agent Compatibility in Specification-Driven Software Migration

This paper investigates cross-agent specification portability using Oracle-to-PostgreSQL migration as a controlled software transformation task. The study combines two experimental stages. First, a specification-first migration pipeline was evaluated on 1,006 PL/SQL files, of which 623 were successfully regenerated and 380 generated scripts executed successfully in PostgreSQL 16. Second, cross-agent experiments were conducted on a dataset of 1,802 Oracle scripts with corresponding PostgreSQL implementations using Amazon Kiro, Google Gemini, and GitHub Copilot, with Claude Code and Cursor included in the initial single-agent evaluation. Native and foreign specifications were assessed using Token F1, exact match, SQL syntax validity, AST exact match, AST mean similarity, and immediate runnability. The results show that specification size alone does not predict implementation quality and that cross-agent transfer can produce substantial agent-dependent degradation. The strongest replicated case occurred when Gemini directly consumed a Kiro-origin specification, producing a Token F1 of 0.035, SQL syntax validity of 2.33%, and AST mean similarity of 0.015. Rewriting substantially improved Gemini in the tested configuration, compression did not provide a universal benefit, and retrieval-augmented ingestion was the only common strategy represented on the per-agent Pareto frontiers of both Gemini and Copilot. The findings suggest that specifications in heterogeneous SDD workflows should not automatically be treated as agent-neutral artifacts and motivate explicit consideration of specification portability, agent-specific interpretation, and retrieval-based access in multi-agent software engineering.

arXiv cs.AI75

通过注意力头干预实现 LLMs 中的个性化隐私控制

agentic AI 的兴起使 LLMs 能够访问多样化的用户数据,从而引发了关键的隐私担忧。以往关于情境隐私的研究探讨 LLMs 是否会根据依赖情境的规范来调节信息披露。然而,即使在相同情境下,不同用户可接受的披露边界也可能有所不同。为解决这一局限,我们引入 \textit{个性化隐私},将用户特定的披露偏好纳入隐私控制。我们进一步提出 P3Bench~(\textbf{P}ersonalized \textbf{P}rivacy \textbf{P}reservation \textbf{Bench}mark),这是一个新的基准,将情境隐私策略扩展为个性化披露策略。实验表明,基于提示的策略无法可靠地执行个性化隐私策略,其中 Qwen2.5-7B 和 Gemma3-4B 的平均策略忽视率分别为 51.25\% 和 74.28\%。最后,为解决这一问题,我们提出 \textsc{Repair},这是一种稳健的推理时注意力头干预方法,可将披露行为调整为符合策略的响应。我们的方法通过减少模型未能遵循给定策略的情况,显著提高了对用户特定隐私偏好的遵循程度。

Hacker News 熱門84

DiffusionGemma 技术报告

文章 URL: https://arxiv.org/abs/2608.00146 评论 URL: https://news.ycombinator.com/item?id=49374287 点数:8 评论数:0

arXiv cs.AI69

BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models

While Multimodal Large Language Models (MLLMs) have made significant strides in visual comprehension, their ability to reason about text-dense, professional documents remains incompletely evaluated. Existing benchmarks emphasize information extraction, require external domain knowledge, or cover professional documents only as one of many settings. They are also largely English- or Chinese-centric, leaving other languages and Russian, in particular, substantially underrepresented. To address these limitations, we introduce BEAR-Bench (Bilingual Enterprise and Academic Reasoning), a self-contained, complex English-and-Russian benchmark comprising 1000 human-annotated questions based on text-rich business and scientific documents. We evaluate 16 proprietary and open-weight MLLMs, including Gemini 3.1 Pro and Qwen3.5-397B, on BEAR-Bench and observe clear headroom even for the strongest systems. Finally, we use the resulting model outputs to compare existing hallucination detection methods, evaluating not only how often models fail on BEAR-Bench but also how reliably those failures can be identified.

arXiv cs.AI77

StagedWorkspace:面向知识工作代理的版本化工作区

AI 代理越来越多地执行知识工作(即生成和修改持久性数字工件,如代码仓库、文档、电子表格、幻灯片、报告),然而它们搜索的解析视图、编辑的原生文件、审阅的更改以及提交的工件,可能指向同一工作产品的不同版本。我们将其表述为一种工作区状态契约:每个视图都应显式绑定到不断演化的工作区状态的某个版本。编码代理通过用于搜索、差异和测试的仓库契约在一定程度上满足了这一需求,而对于 PDF、电子表格、幻灯片、笔记本以及混合格式项目文件夹,类似的契约则不够明确。我们提出 StagedWorkspace,这是一个面向知识工作代理的版本化工作区。该工作区将解析记录和审阅差异绑定到随原生文件变化而变化的内容哈希。在 OfficeQA Pro 和 APEX-Agents 上的固定测试框架消融实验中,双重解析/原生访问在每个测试模型上都获得了最高的点估计;相较于限制更大的单一视图,它使 OfficeQA Pass@1 提高了 8.3–12.1 个百分点,使 APEX 平均评分量表得分提高了 4.7–9.2 分。SW-AGENT 在 OfficeQA 上使用 Gemini 3.1 Pro 得分为 63.9%,在 APEX 上使用 GPT-5.4 Nano 得分为 42.1,而已发表的同模型得分分别为 29.3% 和 25.5。在 57 个文件编辑任务上的配对审阅轴消融实验进一步发现,当差异可见时,观察到的得分更高。这些结果将工作区状态确定为知识工作代理中的一个实验变量,并推动建立将证据、暂存编辑和提交工件作为显式状态转换进行评分的基准。

arXiv cs.AI82

推荐系统中的过时性分解:用于替代与衰减的双过滤器框架

过时推荐是大规模内容平台面临的普遍挑战,也是用户投诉的主要来源之一。内容项通过两种主要机制失去相关性:替代,即新出现的更新使先前的报道变得过时;以及相关性衰减,即内容项的信息价值在其生命周期中自然降低。传统应对措施只是粗糙的替代指标:基于年龄的截止规则无法准确反映实际的相关性损失,而参与度启发式方法依赖滞后信号,在系统适应之前会让用户广泛接触过时内容。 我们提出 SDF(Supersession-Decay Filtering),这是一种已在 Google Discover 中全面部署的过时性过滤系统。Google Discover 是一个个性化推荐信息流,拥有数亿日活跃用户和数十亿月活跃用户。SDF 通过互补的过滤器同时针对这两种机制,每个过滤器都由一个学习模型驱动:一个用于检测内容项对之间替代关系的关系型过时性模型,以及一个预测流量比(PTR)模型,该模型根据内容项的内容预测相关性衰减,并使用生命周期访问流量进行训练。SDF 在排序阶段上游以析取方式应用,剪除过时候选项,从而可衡量地降低下游服务成本。在线实验表明,这些过滤器显著降低了过时内容的出现率,同时提升了用户参与度。在为期两年的生产部署中,用户提交的过时性报告(产品内用户反馈)相对于部署前基线下降了 54.9%,证明 SDF 是一种稳健且可扩展的范式,可在工业规模上解决内容过时问题。

Hacker News 熱門

Show HN: Quasicrystals Animation Playground with WebXR

Hi! This is my little project introducing quasicrystals animations with some parameters. It uses a custom shader to allow higher resolution. Not all combinations look equally good. For example the "Spectrum" color palette IMO makes sense only for low values of "Tempo" and "Pattern Density". If you have VR googles, I recommend trying the WebXR version - it's really nice. Comments URL: https://news.ycombinator.com/item?id=49310533 Points: 3 # Comments: 0

Hacker News 熱門88

Google 正通过同态加密让私有 AI 变得实用

文章 URL: https://blog.google/security/how-google-is-making-private-ai-practical-with-homomorphic-encryption/ 评论 URL: https://news.ycombinator.com/item?id=49300314 点数:334 评论数:195

arXiv cs.AI68

Who Speaks Matters: Authority-Aware Multi-View RAG over Italian Parliamentary Proceedings

Parliamentary proceedings are a primary record of democratic deliberation, yet their volume and fragmentation make multi-perspective access difficult for citizens, journalists, and researchers. Applying Retrieval-Augmented Generation (RAG) to parliamentary transcripts introduces three specific risks: dominance of the most frequent speakers, inability to weight speakers according to topical expertise, and citation misattribution in politically sensitive text. We present ParliamentRAG, a RAG system for the Italian Chamber of Deputies that addresses these risks jointly. Its core contribution is a topic-dependent authority model that estimates each speaker's authority as a function of the current query, combining interpretable components such as profession, education, and previous interventions. Given a user query, the system retrieves relevant speech chunks, identifies topic-relevant experts across parliamentary groups, and generates a summary synthesizing their perspectives, accompanied by supporting quotations. ParliamentRAG is evaluated against Google NotebookLM on 15 policy topics via a two-level protocol combining automated metrics and blind A/B human evaluation by six domain experts. The system achieves higher coverage across political groups (0.97 vs. 0.95), perfect quotation faithfulness (1.00 vs. 0.95), and stronger expert preferences on source-related dimensions, while NotebookLM remains stronger on prose-oriented dimensions.

arXiv cs.AI69

DFM Mimir v1: An Open HRM Delivering Frontier Performance at 1B Parameters Using Only Permissible Post-Training Data

Current large language model development relies on massive, often non-permissible datasets, creating a high barrier for researchers committed to open-source and ethically sourced data. We introduce Mimir v1, a 1-billion-parameter language model based on the Hierarchical Reasoning Model (HRM) architecture, that is trained from scratch and delivers highly competitive performance for English and sets a new state of the art for Danish using only permissible post-training data. Trained on a mixture of 161 datasets, Mimir v1 outperforms the original HRM-Text 1B and competes with larger frontier models like Qwen 3.5 4B and Gemma 4 E2B, tested across 20 benchmarks for English, Math & Code and Danish. The model is available on the Hugging Face Hub: https://huggingface.co/danish-foundation-models/DFM-Mimir

DeepMind Blog85

Gemini 2.5 Flash-Lite 现已准备好用于规模化生产

此前处于预览版的 Gemini 2.5 Flash-Lite 现已稳定并正式发布。这个高性价比模型以小巧体量提供高质量,并包含 2.5 系列功能,如 100 万 token 上下文窗口和多模态能力。

DeepMind Blog62

How AI is giving Northern Ireland teachers time back

A six-month long pilot program with the Northern Ireland Education Authority’s C2k initiative found that integrating Gemini and other generative AI tools saved participating teachers an average of 10 hours per week.

DeepMind Blog67

保护人们免受有害操纵

Google DeepMind 研究 AI 在金融和健康等领域的有害操纵风险,并由此制定新的安全措施。

DeepMind Blog75

为 AI 时代重新构想鼠标指针

Google DeepMind 正在将鼠标指针转变为一个具备情境感知能力的 AI 伙伴。在 Chrome 及更多场景中,通过直观的 AI 协作,超越传统提示方式带来的摩擦。

DeepMind Blog65

我们的生物韧性方法

Google DeepMind 和 Isomorphic Labs 正在分享我们在生物韧性和 AI 模型方面的联合方法。

DeepMind Blog74

推出 Gemini 3.5 Flash Cyber

Google 推出 Gemini 3.5 Flash Cyber,这是一款轻量级网络安全模型,用于查找和修补漏洞。

OpenAI Blog58

Concrete AI safety problems

We (along with researchers from Berkeley and Stanford) are co-authors on today’s paper led by Google Brain researchers, Concrete Problems in AI Safety. The paper explores many research problems around ensuring that modern machine learning systems operate as intended.

OpenAI Blog62

Learning from human preferences

One step towards building safe AI systems is to remove the need for humans to write goal functions, since using a simple proxy for a complex goal, or getting the complex goal a bit wrong, can lead to undesirable and even dangerous behavior. In collaboration with DeepMind’s safety team, we’ve developed an algorithm which can infer what humans want by being told which of two proposed behaviors is better.

OpenAI Blog51

Introducing Activation Atlases

We’ve created activation atlases (in collaboration with Google researchers), a new technique for visualizing what interactions between neurons can represent. As AI systems are deployed in increasingly sensitive contexts, having a better understanding of their internal decision-making processes will let us identify weaknesses and investigate failures.

OpenAI Blog57

OpenAI Five defeats Dota 2 world champions

OpenAI Five is the first AI to beat the world champions in an esports game, having won two back-to-back games versus the world champion Dota 2 team, OG, at Finals this weekend. Both OpenAI Five and DeepMind’s AlphaStar had previously beaten good pros privately but lost their live pro matches, making this also the first time an AI has beaten esports pros on livestream.

OpenAI Blog36

Procgen and MineRL Competitions

We’re excited to announce that OpenAI is co-organizing two NeurIPS 2020 competitions with AIcrowd, Carnegie Mellon University, and DeepMind, using Procgen Benchmark and MineRL.

OpenAI Blog68

Frontier Model Forum 更新

我们与 Anthropic、Google 和 Microsoft 共同宣布 Frontier Model Forum 的新任执行主任,以及一项新的 1000 万美元 AI Safety Fund。