NVIDIA 与英伟达在 GPU、CUDA 与模型生态上的动态。
Keeping a city running means making decisions that affect residents every day — from when to repair a road to how to use energy more efficiently. Across the EMEA region, cities and local software, infrastructure and integration partners are using NVIDIA accelerated computing to build physical AI agents and digital twins that help teams make […]
Across industries, the U.K. is putting the full stack to work — and the scale of deployment is growing fast, from enterprises transforming their businesses to startups solving some of the planet’s hardest problems. NVIDIA is putting that principle into practice, including through a five-year, $250 million compute commitment to strengthen the U.K.’s agentic safety […]
Americans have big concerns about artificial intelligence, but that hasn’t stopped OpenAI Inc., Nvidia Corp., IBM Corp. and others from charging ahead. Last week saw shares of Nvidia, CoreWeave Inc., Oracle Corp. and Nebius Group N.V. decline amid reports that OpenAI’s annualized net revenue was approximately $50 billion, roughly $20 billion below previously reported figures. […] The post On theCUBE Pod: AI doubters get loud, and IBM seeks competitive advantage appeared first on SiliconANGLE .
When Nvidia originally showed off DLSS 5 earlier this year, the technology immediately got branded as gen AI for video games. To gamers, Nvidia had lost the plot. I personally couldn't get over how faces were transformed using DLSS 5. The 15-year-old student from Hogwarts Legacy suddenly looked much older, and Grace from Resident Evil […]
NVIDIA RTX Spark 为 PC 开启全新篇章——通过革新，将游戏 PC、创意工作室和个人 AI 超级计算机的能力融于一颗超级芯片之中。凭借高达 128GB 的统一内存和每秒 1 千万亿次浮点运算的 AI 算力，RTX Spark 提供 NVIDIA RTX 的全栈能力以及 […]
法国企业正在航天、金融服务、公共部门和科学研究等各行业部署智能体计算，展现出该国生态系统如何从雄心迈向实际应用。法国的进展建立在精心布局的基础之上。2025 年 2 月举行的人工智能行动峰会，以及 2025 年和 2026 年举行的“选择法国”峰会，促成了重大的算力投入承诺 […]
随着 AI 工厂从兆瓦级发展到吉瓦级，其建设所需已不止于先进的计算技术。开放标准正帮助业界更快速、更高效、更可靠地扩展 AI 基础设施。在 San Jose McEnery Convention Center 举行的 Open Compute Project（OCP）全球峰会上，NVIDIA 宣布了新的合作伙伴关系及贡献，旨在 […]
Microsoft 今天宣布推出第三代自研 AI 芯片 Maia，旨在成为 Microsoft 产品和服务的“主要 AI 推理系统”。Maia 300 不仅提高了在云端运行 AI 工作负载的成本效益和效率，还更擅长处理最新的推理模型和智能体任务。Maia 300 将 […]
arXiv:2610.13325v1 公告类型：cross 摘要：智能体式工作负载会在多次调用之间重复发送几乎不变的长上下文。如今，前缀缓存可以跳过其中大部分预填充阶段，但每个请求仍必须先完成全部分词，之后才能查询缓存。分词这一长期以来被视为无关紧要的 CPU 阶段，如今却成为首 token 生成时间（TTFT）中占比最大的部分；而为整个 GPU 集群执行分词的前端，则限制了其能够供给的 GPU 数量。我们提出 GraphTok，这是一种 GPU 分词器，可将从预分词、字节对编码（BPE）合并到 token ID 组装的完整编码流程，作为一张可重放的 CUDA 图运行。GraphTok 在 GPT-2、Llama 3、Qwen 2.5、DeepSeek-V3 和 Gemma 3 上逐位复现参考 token ID；对于 GPU 无法占优的短输入，则将其交给 CPU 引擎处理；同时，它还可以接入未经修改的 vLLM 和 NVIDIA Dynamo。GraphTok 在包含数据传输的情况下，可在 1.7–3.2 毫秒内编码一百万个 GPT-2 token；在 Llama 3 上，其速度比最强的 CPU 分词器 gigatoken 快 4.7–6.0 倍。在 TokTier 自有的单请求基准测试中，其速度比 gigatoken 快 2.6–3.9 倍，最高比 TokTier 快 18.1 倍。在使用缓存前缀的 vLLM 中，它将平均 TTFT 降低了 2.1–2.4 倍。最大的提升体现在容量方面。在 2 秒 P99 目标下，配备 GraphTok 的单个 Dynamo 前端可持续处理 518 个请求/秒、每个请求包含 12.1 万个 token 的流量，相当于高峰时段 56 个 GPU 的负载；相比之下，使用 gigatoken 时为 296 个请求/秒，使用默认分词器时仅为 18.5 个请求/秒。在 12.1 万至一百万个 token 的范围内，其可承载负载是 gigatoken 的 1.75–2.5 倍；在 gigatoken 的尾延迟失控的情况下，P99 最多可降低 64 倍。
arXiv:2610.02626v2 公告类型：交叉发布 摘要：视觉-语言-动作（VLA）模型越来越多地引入中间推理来改进机器人操作，但现有方法主要针对已观测状态进行推理，并未显式预判未来场景的演变。然而，若将此类推理扩展为在每个推理步骤中显式推演未来，则会引入大量计算开销。我们提出了 IG-VLA，这是一种使模型能够想象未来并内化要义的 VLA 推理框架。我们的 Latent Spatiotemporal Reasoning 直接在视觉表征空间中学习想象与任务相关的未来场景演变，无需高成本的像素级视频生成即可指导动作预测。为进一步降低推理开销，我们引入了 Scene Gist Memory，将推理得到的场景-行为关联内化为紧凑的 Scene Gist Token，在保留未来推理优势的同时，绕过推理阶段的显式未来想象。在 LIBERO、LIBERO-Plus 和 VLABench 上开展的大量实验验证了 IG-VLA 的有效性和效率。在 LIBERO-Plus Language 测试套件上，推理策略与要义策略的成功率均比最强基线高出近 6%。要义策略相较于基线还实现了最高 6.38 倍的加速，在单张 NVIDIA A6000 GPU 上，将每个动作块的推理延迟从 1081 毫秒降至 169.5 毫秒。这些结果表明，未来时空推理能够被有效内化，从而实现高效的 VLA 部署。
Nvidia Corp. is in early talks to acquire open-weight model developer Reflection AI Inc. or invest more in the startup, according to the Financial Times. People familiar with the matter told the paper that an agreement could come within weeks. The talks could still collapse. The FT described more than one way a deal could take […] The post Nvidia reportedly in early talks to buy, acquihire or invest more in Reflection AI appeared first on SiliconANGLE .
Hello! I built Flopper to make it easier to compare GPUs and understand the tradeoffs for workloads for AI and beyond. When researching GPUs from say Nvidia, AMD, and Huawei, I kept running into the same problem: specs were scattered across datasheets, and matching them to available configurations, vendors, and prices was cumbersome and a bit weird and frankly annoyingggg! AMD made me make an account :( So I started a place to bring that information together. Flopper lets you compare GPU specs, including TDP and compute throughput in terms of flops/tops, alongside estimates for AI workloads or other compute workloads (say, ~FP32 based) I recently added pricing data from dozens of providers to help track how prices change over time. Getting reliable quotes for systems like the NVL72 is still particularly difficult tbh, not sure how I can tell you how much that costs easily due to them being behind large procurement contracts and big multi-nationals. One thing that's become fairlyyyy clear is the interest in consumer cards and local LLMs. The bigger challenge is connecting advertised specs to real-world performance. In our testing so far, I’ve seen roughly 85% of advertised throughput, but that’s a ballpark observation for the workloads I’ve tested, not a general rule and so variable (power, cooling, workload etc). I'm considering a CLI that lets people run benchmarks and submit results, potentially using MLPerf as an automation? That way, we can all compare quoted performance with the workload, configuration etc. I'd love feedback: what information is missing, how can I improve it? And what would you need to see to trust a community-submitted benchmark? Corrections and additions also very welcome - some of the measurements are best efforts (and simple multiplications) but I hope it's of some use. Thanks for looking :) Comments URL: https://news.ycombinator.com/item?id=50035826 Points: 2 # Comments: 0
arXiv:2610.11317v1 公告类型：cross 摘要：混合专家（MoE）架构已成为扩展大型语言模型的关键技术，近期研究表明，细粒度专家设计具有显著优势。从头开始训练此类模型成本高昂，而利用预训练稠密模型进行稀疏化升级是一种颇具吸引力的替代方案。然而，我们发现了细粒度升级中的一种结构性缺陷：当细粒度专家源自同一个源模型时，朴素路由会发生坍缩，导致下游准确率降至接近随机水平（例如，在 Qwen3-1.7B 上，Drop-Upcycling-fine-grained 在 15 个基准测试中的平均准确率仅为 23.2%，与从头训练的 22.2% 基本相当，而同一方法的粗粒度版本则达到 50.2%）。我们提出 DivMoE，这是首个能够实现具备结构均衡路由的细粒度 MoE 增量升级框架。DivMoE 引入了领域专用的细粒度专家初始化方法：从经过领域自适应持续预训练的稠密模型中构建专家；同时引入多样性约束路由，通过硬性结构约束确保每个 token 激活来自不同领域组的专家。在两个基础模型和 15 个基准测试中，DivMoE 始终优于六种升级基线（在 Qwen3-1.7B 上达到 55.6%，而最强基线为 51.6%），并且在 Stage 2 持续预训练后，在每项基准测试中都严格优于稠密基础模型，从而弥合了困扰以往细粒度升级方法的性能回退差距。在公开推理混合数据上进行监督微调后，我们的 12B 参数 DivMoE 模型以 64.5% 的平均准确率达到了 Moonlight-MoE（16B）的水平，同时比受控的 NVIDIA-Upcycling 基线高出 6.1 个百分点。
arXiv:2610.11245v1 Announce Type: new Abstract: KV-cache entries are stored before their future queries are known, but each decoding query needs precision in different places. We study this mismatch using separate budgets for retained bits and bits fetched per query. ReadKV stores each key and value in a progressive code whose prefixes support different reconstruction precisions. For each query, it allocates key-channel prefixes using the query, computes attention from the reconstructed keys, and then allocates value-token prefixes using that attention. Stored entries remain unchanged. Each stage optimizes a calibrated distortion objective under a fixed budget; we prove exact allocation under diminishing refinement gains and relate these objectives to attention-output error. We also exhibit a finite-dimensional attention family where query-dependent access strictly outperforms every query-independent reader at the same read budget, even with unrestricted competing encoders and decoders. Across six base models, reading four bits on average from an eight-bit cache increases C4 perplexity by at most 0.66%, using about one quarter of the logical reads and half the retained capacity of a 16-bit cache. It is consistently more accurate than storing and fully reading four bits at the same payload-read budget. Retaining more bits than each query fetches is aimed at long-context decoding, where the cache bytes moved per step, rather than the weights, dominate cost. Long-context question answering and retrieval on two instruction-tuned models provide additional quality evidence. On the tested 8K-token, batch-one, single-layer workload on an NVIDIA A10G, a restricted eight-bit ReadKV reader with a two-bit mean payload-read budget has 39% lower latency than the tested TurboQuant codec.
将仿真创意转化为可运行的应用程序，意味着整合资产、连接物理与渲染，并检查场景是否按预期运行。开发者正将 Frontier AI 模型与 NVIDIA Omniverse 库结合起来，以协助完成这项工作——构建用于探索各种场景、调查故障和改进设计的应用程序。开发者通过 […] 引导 AI agents。
Full-stack safety solution for physical AI is being used by robotics companies.
NVIDIA researchers introduced PivotOPD, an on-policy distillation method that trains multi-turn LLM agents to avoid early pivotal mistakes and recover from them, posting the best average against 13 baselines on 3 agent benchmarks. The post NVIDIA PivotOPD Teaches Multi-Turn AI Agents to Recover From Pivotal Mistakes appeared first on MarkTechPost .
arXiv:2610.09778v1 公告类型：交叉收录 摘要：大语言模型（LLM）推理的可复现基准测试面临挑战，因为重复测量结果可能随执行状态和系统状态而变化。我们提出了顺序隔离方法，这是一种受控的基准测试与回归测试协议，旨在有意改变工作负载并发度的同时，降低不同运行之间的测量方差。我们使用 vLLM 0.9.1，在 NVIDIA A100 80GB GPU 上评估了三个具有代表性的开源 LLM，覆盖六种上下文长度和八个并发级别，每种配置重复测试五次。最终协议将平均变异系数（CV）从控制最少的方法阶段的 15.2% 降至 2.2%；根据每种配置五次重复测试各自的首 token 延迟（TTFT）中位数（P50）计算 CV，144 种配置中有 113 种（78.5%）的 CV 低于 3%。测量结果还表明，在受测技术栈上，并发用户数从 200 增至 500 时出现了明显的延迟跃变，且三个模型的 P99 延迟呈现出描述性差异。此外，我们还提供了一个显式的成本盈亏平衡模型，并分析其对 API 定价的敏感性。该协议旨在为可复现比较和回归测试提供稳定的参考，而非预测不受控生产流量下的绝对表现。基础设施即代码和基准测试脚本支持复现实验环境。
Google LLC is making its SynthID identification tools for artificial intelligence-created and AI-edited videos and images more widely available with the launch of a new service today that everyone can use. The web-based SynthID Detector can identify images, video and audio clips that have been created using AI models from Google, OpenAI Group PBC, Nvidia […] The post Google debuts SynthID Detector tool for flagging AI-generated content, but it’s far from perfect appeared first on SiliconANGLE .
Nous Research Inc., the developer of the popular Hermes artificial intelligence agent, today announced that it has raised $90 million in funding. Early-stage startup fund Robot Ventures led the Series B round. It was joined by Nvidia Corp., Samsung Electronics Co., Y Combinator and several others. Nous is now valued at $1.5 billion. The company […] The post Nvidia, Samsung back $90M round for AI agent startup Nous Research appeared first on SiliconANGLE .
Microsoft revealed the specs and price for its Surface Laptop Ultra, AI PCs that run on Nvidia chips that are designed to run AI models and agents.
Microsoft just wrapped up a big Windows and Surface-focused keynote in San Francisco. The biggest announcement was arguably the release details about the Surface Laptop Ultra, its new laptop that’s powered by Nvidia’s RTX Spark Arm-based chip. The machine will start at $2,599 for a configuration with an 8-core CPU, 24GB of RAM, and 512GB […]
在周三于旧金山举行的一场 Microsoft 活动上，Jensen Huang 和 Satya Nadella 概述了 NVIDIA 与 Microsoft 如何联合开发硬件和软件，让 AI 智能体在 Windows PC 上运行。Huang 表示，NVIDIA 因 Windows 而创立。如今，AI 智能体正走向 Windows。“如果你回顾整个历程 […]
Microsoft's Nvidia-powered Surface RTX Spark Dev Box is available for preorder now directly, and slated to ship in November for just about $6,000. It's pricier than the DGX Spark mini PC Nvidia launched last year, but PC prices have been climbing due to shortages of RAM and other components. The Dev Box's flat, 3D-printed anodized […]
Google 的 AI 水印检测工具 SynthID 现已向公众开放。任何人都可以检查图片、视频或音频是否由 Google 的 AI 或 OpenAI、Nvidia 等合作伙伴的 AI 创建。目前已有超过 1800 亿项内容带有该水印，但该工具只能识别带有 SynthID 标记的内容，无法普遍识别 AI 生成的媒体内容。本文《Google 表示，已有 1800 亿张图片和段视频带有 SynthID 水印，检测工具现已向公众开放》首发于 The Decoder。
From Nvidia and Chime to Obvious Ventures and Anthropic, explore the entire roundtable agenda at TechCrunch Disrupt 2026. Register now to save up to $100 on your pass and get a second pass at 50% off.
arXiv:2609.30059v2 公告类型：替换并交叉列出 摘要：深度学习推理和训练的性能在很大程度上取决于 GPU 内核的效率。PyTorch Inductor 等现代编译器能够从高层模型代码自动生成 GPU 内核，但其性能往往远逊于专家编写的实现。近期基于大语言模型（LLM）辅助的内核优化器能够缩小独立内核上的这一差距，但它们将编译后的模型视为黑箱，通常仅优化单个独立内核，既不遵循编译器的结构性决策，也不对模型进行端到端验证。我们提出 KernelOPT，这是一个将编译后的模型视为结构化产物的多智能体系统。它保留供应商库调用（cuBLAS、cuDNN），并使用五个由性能剖析引导的 LLM 智能体，专门针对生成的 Triton 子内核进行优化。四道关卡组成的级联验证流程依次执行静态验证、多随机种子正确性检查、模型级 float64 回退验证以及性能门控（$\gamma{=}1.03$），以筛选候选方案，并对重新拼接的模型进行端到端验证。当候选方案未通过验证时，系统保留编译器基线实现。该系统支持输入 PyTorch nn Modules、独立 Triton 内核和 Helion 内核。在 NVIDIA H200 上对 250 道 KernelBench 问题（100 道 Level 1、100 道 Level 2 和 50 道 Level 3）进行评估时，KernelOPT 在所有内核（包括回退案例）上相较于 torch compile 实现的几何平均加速比分别为 1.40$\times$（L1）、1.15$\times$（L2）和 1.07$\times$（L3）。仅统计成功优化案例（不包括验证关卡决定保留编译器基线实现的案例）时，几何平均加速比显著更高：2.54$\times$（L1：36/100）、1.84$\times$（L2：23/100）和 1.37$\times$（L3：11/50），体现了优化器能够取得实质性收益的场景。
arXiv:2609.39082v2 公告类型：替换 摘要：随着新证据的到来，序列模型必须更新其记忆内容以及记忆影响预测的方式。Transformers 的计算与缓存开销随上下文长度增长，而固定状态的循环模型可实现恒定内存开销的推理。然而，传统的线性递归和谱递归依赖静态状态转移，无法动态调整已存储表征的衰减或旋转方式。尽管近期的选择性架构引入了依赖输入的状态转移，但它们为每个记忆模态分配独立的控制，从而将控制成本与状态容量绑定。我们表明，高维谱记忆并不需要高维控制，并提出 Shared Phase and Retention Control for Efficient Adaptive Spectral Recurrence（SPARC）。SPARC 仅使用两个依赖输入的标量信号，就能协调异质复数模态的记忆保持与相位旋转，同时保留各模态特有的基准时间尺度和频率。其对角仿射递归既支持用于序列级 BPTT 的并行结合扫描，也支持用于在线信用分配的精确结构化实时循环学习（RTRL）。在部分可观测连续控制、POPGym 和序列分类任务中，相较于排名第二的方法，SPARC 在 Walker-P 上实现了 9.09% 的回报相对提升，在 FordA 上实现了 1.36% 的准确率相对提升。在 NVIDIA Blackwell GPU 上，相较于经过优化的 RG-LRU 基线，我们的实现在固定词元数量的工作负载下将循环混合器的训练延迟降低了 18.2%–34.2%，并将扫描速度提升至 3.1–4.7 倍。这些结果表明，两个共享控制信号就能在在线和完整序列场景中高效调控自适应谱记忆。代码已发布于 https://github.com/Botwwt/sparc。
arXiv:2609.33051v2 公告类型：替换 摘要：混合注意力模型用线性注意力替换大多数 softmax 注意力层，从而减少 KV 缓存的增长，并支持更大的解码批次，此时循环状态访问成为主要瓶颈。ReplaySSM 通过缓冲键和值来摊销状态更新开销，但每个新查询仍需读取完整状态，即使状态在两次更新之间保持不变。我们观察到，采用状态加权的低秩查询近似能够准确保留状态读取的输出。尽管未来的查询未知，用于近似这些查询的基向量却可以离线确定。基于这一观察，我们提出了 SketchSSM，它在保留完整状态更新的同时，对读取进行近似。在每次状态更新时，SketchSSM 读取一次完整状态，预先计算这些基向量对应的输出，并将其存储在紧凑草图中。随后的每个解码步骤将草图向量与依赖于查询的系数组合起来，无需读取完整状态即可重建输出。在四个基于 Mamba-2、GDN 和 KDA 的模型上，当草图的平均秩为 8 时，SketchSSM 将状态访问流量降低至原来的约十分之一，同时在四项解码基准测试中达到与 FP32 完整状态基线相当的平均准确率，并在四项 RULER 检索任务中保持召回率。在此秩下，使用单张 NVIDIA B300 时，相对于 Standard vLLM 基线，Mamba-2、GDN 和 KDA 的线性注意力内核分别实现了 7.30 倍、5.02 倍和 5.24 倍的加速，Nemotron 3 Super 的解码吞吐量最高达到基线的 2.77 倍。
arXiv:2610.07094v1 Announce Type: cross Abstract: LLM deployment is shifting from single-turn completion to agentic trajectories in which a model plans, calls tools, reads results and reasons at test time before acting. This inverts the economics of inference hardware: chat serving amortises weight reads across large batches, whereas agent trajectories are sequentially dependent, run at effective batch one, and make per-token decode latency (TPOT) the dominant term in task completion time. Using a roofline analysis and a closed-form episode-latency model, we show why this regime favours accelerators that keep weights in on-die SRAM (Cerebras WSE-3/3T, Groq/NVIDIA LPU) or compiler-managed tiered memory (SambaNova SN40L/SN50), and why three vendor ecosystems converged in 2026 on disaggregated prefill/decode serving. We show that per-step reliability compounds exponentially in trajectory length-a 2% per-step failure rate erases a 2x decode advantage for a 20-step agent-so determinism and tail latency are first-order performance variables. We then propose a four-layer evaluation framework (silicon, serving system, agent episode, enterprise) with a metric set built on goodput at an agentic SLO and cost per successful episode, a six-axis benchmark protocol over six task families, a paired-bootstrap statistical design, an attestation protocol for vendor-run benchmarks, and TCO, availability and adoption-timing models with explicit break-even conditions. All performance figures are public and labelled by evidence class; we state seven falsifiable hypotheses and the experiments that test them, and argue that the most likely original result is that token-throughput rankings diverge from cost-per-successful-task rankings on long-horizon work.
推出 Mistral Large 4：Le chonk。Mistral 重返赛场。今天，他们发布了 Mistral Large 4 的预览版：这是一个拥有 1 万亿参数、其中 490 亿为活跃参数的模型，训练使用了他们自有集群中的 3,800 块 NVIDIA Grace Blackwell GPU。该预览版可通过其 API 使用。他们承诺将在“本月底”发布开放权重模型。该模型通过 Mistral API 仅支持两个推理等级——“none”和“high”。下面是两只鹈鹕——“high”版本看起来更好，不过令人意外的是，它只使用了 2,717 个输出 token，而“none”版本使用了 3,275 个：表现不错。鹈鹕的喉囊很棒，自行车车架尺寸合适，脚踩在脚踏板上。不过，两只脚踏板似乎都位于车架前方。渐变效果不错。在 Artificial Analysis 上，它的得分为 38，仅落后于 DeepSeek 4.1 Flash；后者是一个拥有 5,520 亿参数的模型。这比去年 12 月的 Mistral Large 3 有了巨大提升，后者画出了这只糟糕的鹈鹕，并在 AA 上只得到 9 分。它当然还不是 Fable 级别的模型，但很高兴看到 Mistral 发布了一个重新回到大约落后前沿模型 6 个月水平的模型。来源：Hacker News 标签：ai、generative-ai、llms、mistral、pelican-riding-a-bicycle、llm-release
Nvidia-backed Lambda is raising up to $4 billion at a $14.5 billion pre-money valuation ahead of a planned 2027 IPO, led by Coatue and Blackstone.
Mistral AI has released Mistral Large 4, nicknamed Le Chonk, as a public preview. It is a 1.05 trillion parameter Mixture of Experts model with 49 billion active parameters, native image input, and a 1 million token context window, trained on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own European datacenters. The API is live now; open weights ship end of October 2026. The post Mistral AI Releases Mistral Large 4 (Le Chonk): A 1.05T Parameter Multimodal MoE Model appeared first on MarkTechPost .
电信运营商正日益基于开放模型构建其 AI 战略——原因远不止成本因素。开放模型让电信公司能够在最关键的工作负载中信任、控制并定制 AI，涵盖从自治网络到客户服务等领域。NVIDIA 最新发布的《State of AI in Telecommunications》报告反映了这一转变，其中 […]
Reflection AI Inc. today introduced Beam, an open-source large language model with 501 billion parameters. The launch comes a few months after the startup raised funding at a $25 billion valuation. Around the same time, Reflection AI reportedly inked a $6.3 billion deal with SpaceX Corp. to rent Nvidia GB300 NVL72 appliances. It used those […] The post Reflection AI debuts open-source Beam model with 501B parameters appeared first on SiliconANGLE .
Discover how to construct an end-to-end streaming robotics learning pipeline using the NVIDIA Cosmos3-DROID dataset without local downloads, leveraging byte-range Parquet reads, behavior cloning, and temporal ensembling. The post Building a Streaming Robotics Learning Pipeline Using NVIDIA Cosmos3-DROID appeared first on MarkTechPost .
Nvidia-backed Reflection AI unveils Beam, its first open-weight model, which it says rivals GLM-5.2 on reasoning with far less inference compute. Weights are due this month.
arXiv:2609.20744v3 公告类型：替换 摘要：视频扩散模型在去噪过程中会反复处理长时空令牌序列，使注意力机制成为主要的计算瓶颈。线性注意力提供了颇具吸引力的替代方案，并已广泛应用于近期的大型语言模型中，但将其直接应用于视频模型，往往无法保留高质量生成所需的细粒度交互。我们提出 Video DeltaNet（VDN），将局部 Softmax 注意力与双向线性记忆相结合，以建模长程视频上下文。其线性分支引入了 Video Delta Attention（VDA），通过联合整合每一帧的空间令牌，每帧更新一次记忆。独立的输出投影和可学习门控机制用于校准两个分支，同时，分阶段的教师对齐方案逐步将这一新路径引入预训练模型。我们在 MiniMax H3 上实例化了 VDN，将这种混合机制应用于视频到视频的交互，同时保留 Softmax 以处理涉及文本或音频的交互。借助八步蒸馏和优化后的 SGLang 服务栈，VDN-H3 在八块 NVIDIA B200 GPU 上，仅用 6.70 秒即可完成 14.3 秒、768p 视频的 DiT 去噪；相比相同 GPU 数量下采用 50 步密集 H3 基线，实现了 14.5 倍加速。GitHub 代码见：https://github.com/OpenVDN/vdn-minimax-h3。模型权重见：https://huggingface.co/OpenVDN/vdn-minimax-h3
arXiv:2610.02241v1 公告类型：cross 摘要：混合专家（MoE）架构使前沿语言模型能够扩展至万亿级参数规模，但其部署受到庞大内存占用和内存带宽限制的制约。尽管现代加速器提供了稀疏张量核心（SpTCs），可通过低精度半结构化稀疏性减少权重存储并提升吞吐量，但由于模型质量显著下降以及缺乏分组稀疏 GEMM 原语，将其用于 MoE 仍然面临挑战。我们提出了一种端到端的软硬件协同设计框架，将专家权重压缩为硬件原生的低精度稀疏表示，并在 SpTCs 上加速其执行。在算法层面，我们通过连续重参数化放松离散的半结构化支持集选择，从而在路由器加权重构目标和可扩展专家并行压缩下，实现与量化权重的可微联合优化。在系统层面，我们开发了定制的分组稀疏 GEMM 内核，专为 SpTCs 上的低精度稀疏 MoE 推理而设计。在参数规模从 300 亿到 1 万亿的 MoE 模型上，我们的框架将当前最先进的联合稀疏量化精度最高提升 4.35 个百分点，同时保持原始模型 96.09% 的性能。在 NVIDIA B200 GPU 上，我们的内核相较厂商基线最高实现 1.65 倍的性能，服务吞吐量提升 1.18 倍，端到端延迟最高降低 4.03 倍。这些结果表明，软硬件协同设计是实现可扩展、高效 MoE 部署的一条切实可行的路径。
arXiv:2610.03226v1 公告类型：new 摘要：大型语言模型（LLM）代理生成的 GPU 内核，其效率可能仍低于专家实现，但仅凭运行时间无法揭示这一差距与设计发现和实现之间的关系。我们推出 D2K-Bench，这是一个包含 26 项任务和 85 个工作负载的诊断基准，用于衡量代理将专家设计指导转化为高效 GPU 内核的能力。设计指导涵盖 L1：高层次算法洞见，L2：数据流设计，以及 L3：低层次优化技巧，同时还包括这些层级之间的依赖关系。在成对运行的对照实验中，有无设计指导的两组实验共享任务描述、工作负载、工具、硬件和 350 轮的预算。补充评估则考察独立提出的设计，以及生成代码中实现的设计属性。在 NVIDIA B200 GPU 上对五个模型进行测试后发现，设计指导使 130 个以上模型—任务组合的正确率从 93.1% 提升至 98.5%，并使全部 26 项任务的性能分数从 1.46 提升至 1.95。对于在两组实验中均在全部 26 项任务上提交正确结果的三个前沿模型（GPT-6-Astra、Claude-Opus-4.8 和 GPT-5.6-Sol），几何平均加速比从 $1.69\times$ 提升至 $2.49\times$。在全部五个模型中，平均综合实现得分从 100 分中的 57 分提升至 70 分。这些结果表明，专家设计指导具有重要价值，同时也揭示出一些尚未得到实现的设计属性。
大型语言模型（LLM）智能体生成的 GPU 内核可能仍不如专家实现高效，但仅凭运行时性能无法揭示这一差距与设计发现和实现之间的关系。我们推出 D2K-Bench，这是一个包含 26 个任务和 85 个工作负载的诊断基准，用于衡量智能体将专家设计指导转化为高效 GPU 内核的能力。该指导涵盖 L1：高层次算法洞见，L2：数据流设计，以及 L3：底层优化技巧，并包括这些层级之间的依赖关系。在有无指导的成对运行中，任务描述、工作负载、工具、硬件和 350 轮预算均保持一致。补充评估还考察了独立提出的设计，以及生成代码中实际实现的设计属性。在 NVIDIA B200 GPU 上对五个模型进行测试后发现，指导使 130 多个模型-任务组合的正确率从 93.1% 提升至 98.5%，并使全部 26 个任务的 Performance Score 从 1.46 提升至 1.95。对于在两种运行中均在全部 26 个任务上提交正确结果的三个前沿模型（GPT-6-Astra、Claude-Opus-4.8 和 GPT-5.6-Sol），几何平均加速比从 $1.69\times$ 提升至 $2.49\times$。在全部五个模型上，综合实现得分均值从 100 分中的 57 分提升至 70 分。这些结果表明专家设计指导具有重要价值，同时也揭示出一些仍未被实现的设计属性。
NVIDIA announced a new 64GB configuration of DGX Spark — from Acer, ASUS, Dell, Gigabyte, HP and MSI — its GB10-powered desktop AI system. It gives developers a way to start with one system for local models and agents, then cluster two 64GB units for 128GB of memory across the cluster and more compute when […] The post NVIDIA Announces DGX Spark 64GB: A 1-PetaFLOP Grace Blackwell Desktop for Local AI Agents, Fine-Tuning, and Inference appeared first on MarkTechPost .
Prime Intellect has launched Prime Inference, an OpenAI-compatible platform for serving frontier open models on NVIDIA Blackwell. Its GLM-5.3 deployment uses Dynamo, vLLM and NVFP4 KV compression to serve 66 sessions per prefill group at 101 tok/s per user. The post Prime Intellect Launches Prime Inference: Serverless and Reserved Serving for Frontier Open Models appeared first on MarkTechPost .
IBM has made a self-hosted deployment of IBM Bob, its agentic software development platform, generally available. Enterprises can now run Bob on premises, in private or sovereign clouds, and in air-gapped networks. They bring their own model: NVIDIA Nemotron or Poolside Laguna for full isolation, or Claude, Gemini or GPT models through hybrid setups. Optional Premium Packages extend Bob to Java, IBM i and IBM Z modernization. The post IBM Brings Bob to Self-Hosted and Air-Gapped Environments: Agentic Software Development Without Moving Your Code appeared first on MarkTechPost .
Learn how NVIDIA IsaacTeleop turns XR hand tracking and motion controller input into robot commands using a pure Python retargeting engine and NumPy. The post Inside NVIDIA’s IsaacTeleop: From Hand and Controller Tracking to Robot Actions with the Graph-Based Retargeting Engine appeared first on MarkTechPost .
The price of anything with memory is skyrocketing thanks to AI. Aging streaming devices are no exception.
Artificial intelligence factory economics increasingly depend on more than access to high-performance graphics processing units. As agentic systems draw on multiple models, databases and tools, the entire data center must work as one computing system. That transition is shifting attention from individual chips to the infrastructure that turns computing capacity into useful intelligence. Networking, storage, […] The post Nvidia ties AI factory economics to tokens and power efficiency appeared first on SiliconANGLE .
Storage architecture is being rewritten for artificial intelligence factories. Traditional enterprise storage was designed around workloads that scaled in relatively predictable ways. AI changes that equation by combining heavy data movement with transactional metadata activity, often on shared infrastructure. NetApp Inc. is addressing that pressure through its work with Nvidia Corp., a partnership that dates […] The post NetApp and Nvidia rethink storage for AI factories appeared first on SiliconANGLE .
China is reportedly mulling letting ByteDance, Alibaba buy banned Nvidia chips.
The deal, which is expected to close by year's end, is worth $8.2 billion.
Initially launched in 2019, the Nvidia Shield TV Pro now retails for $299.99.
Nvidia’s chip-smuggling problem won’t go away as arrests continue.
arXiv:2609.23900v2 公告类型：替换 摘要：混合语言模型的树式推测解码必须在循环、卷积和注意力状态之间保持一个连贯的延续。我们提出了 LumoTree，这是一种验证器，能够并行执行循环路径，复用每条路径内的状态分块，并通过共享的逻辑树协调原生循环重放、卷积历史收集和注意力缓存重映射。融合式候选选择、驻留 GPU 的接受判定以及分组 split-K 注意力共同支持验证流程。组件实验显示，候选选择结果完全一致，循环计算结果也在成对误差界内保持一致。在单台 NVIDIA DGX Spark（GB10）上进行的探索性 Qwen3.8-27B NVFP4 部署中，针对十个 SWE-bench Verified Astropy 任务，记录到 25.63 pooled tokens/s 的吞吐量。结果刻画了组件级数值一致性和编码代理部署情况；完整模型延续能力和受控的应用级加速效果仍有待研究。
arXiv:2610.00687v1 公告类型：cross 摘要：硬件可运行故障（HOF）会中断大型语言模型（LLM）训练，但无需重置、维修或更换硬件，即可在同一硬件上恢复。现有恢复系统通常仍需重新加载检查点、重新计算丢失的训练进度并重建进程状态，使原本可以继续训练的 GPU 处于空闲状态。我们提出 Leto，一种容错训练系统，利用幸存硬件实现高效的原位恢复。我们的关键洞见是：恢复训练所需的状态可以在活动训练进程之外保留或准备，同时仍驻留在同一硬件上。Leto 保留工作模型状态和可复用的进程状态，并在影子训练器中预先初始化其余状态。我们设计了两级纠删保护和块级事务更新，以确保保留的模型状态可恢复且保持一致；当活动训练需要 GPU 内存时，则回收影子状态。在 6-GPU 和 72-GPU NVIDIA A100 集群上的评估表明，与性能最佳的检查点基线相比，Leto 的恢复速度提高了 3.6--6.5 倍，并将有效训练时间最多提升 13.7 个百分点。大规模模拟显示，在 131,072-GPU 集群上，有效训练时间超过 95%。
arXiv:2610.00465v1 公告类型：cross 摘要：下一代大型语言模型（LLM）正从云端扩展到无处不在的边缘设备。然而，边缘设备通常要么没有足够的内存来存储日益庞大的 LLM 权重，要么即使拥有足够内存，也会因加载权重而消耗难以承受的能量。这引出了我们的问题：边缘设备能否在不存储或加载权重的情况下运行 LLM，而是通过无线方式接收权重并即时使用？受无线广播启发，我们提出了 AIR-LLM，这是一种面向边缘设备的 LLM 推理架构，由以下部分组成：（i）中央无线电设备（例如 5G 基站），将 LLM 权重广播到空中；（ii）边缘用户接收这些权重，并利用射频混频器，直接在射频（RF）域中完成 LLM 推理的一般矩阵-向量乘法（GEMV）。为进一步缩短空中传输时间，AIR-LLM 利用 MIMO 空间复用，并在边缘侧提出一对节能的预编码器和后编码器，以校准其自身的无线信道。由于中央无线电设备无需感知用户，AIR-LLM 具备用户可扩展性，因此一次广播即可服务覆盖范围内的无限用户。我们在两个真实城市场景的 NVIDIA Sionna 射线追踪信道上实现了 AIR-LLM，并结合真实射频混频器的性能分析进行了评估。在 LLaMA-3.1-8B 上，WikiText-2 困惑度仅下降 4.0%，与 FP16 和仅权重量化基线相比，AIR-LLM 分别节省了 157.7 倍和 40.4 倍的能耗；在 20 个用户的情况下，其空中传输时间分别缩短了 104.1 倍和 26.0 倍。
arXiv:2610.02144v1 公告类型：新 摘要：我们推出了 Faynt，这是一系列面向 Super Smash Bros. Melee 的 1000 万参数和 7500 万参数 Transformer 策略，每个策略都能通过单个 checkpoint 控制全部 26 个角色。经过强化学习（RL）后，10M 在由十四个专精角色或多角色版本所支持的角色阵容上进行的同角色对局中赢下了 244 场中的 240 场（98.4%），并且对每个版本都保持胜绩。这些对手仍保留 21 帧或 24 帧的动作延迟；Faynt 不使用额外延迟，而我们尚未单独分析这一差异的影响。在另一项针对私下提供的零延迟 Slippi-AI 模型的评估中，10M 在两种条件设置下的全部 68 场对局中均获胜。我们研究了架构、优化、规模扩展和超参数迁移，以指导在约 840,000 场人类回放上进行预训练。后训练结合了基于排名和结果的课程学习、75M 到 10M 的知识蒸馏，以及仅限于 Fox 镜像对局的强化学习。在最初的 152 场基准测试中，监督式 10M 赢下了 69.7% 的对局，而预训练 75M 的胜率为 45.4%，尽管后者整体留出集控制器预测损失更高。用于选择监督式 checkpoint 的加权验证损失，与全部四个预训练和监督式策略的胜率排序一致。监督式后训练完成后，两个模型每分钟承受的伤害都更少，在前期建立的领先优势更大，并且在丢失第一个生命后获胜的频率更高。在记录的游戏状态上进行优化推理时，10M 在 NVIDIA T4 上每次决策平均耗时 5.2 毫秒，75M 为 8.7 毫秒，不包括模拟器执行和通信时间。我们将权重、两套基准测试以及用于自动化模型锦标赛的平台开源。
arXiv:2610.00049v1 发布类型：新 摘要：GPU 各代产品以不同速率扩展矩阵、特殊函数和内存流水线，因此随着硬件演进，内核瓶颈也会转移。FlashAttention-4 暴露了 NVIDIA Blackwell 注意力计算中的这种不平衡。我们测试短小的多项式程序能否加速大语言模型（LLMs）中的其他特殊函数单元（SFU）操作。首先，我们在独立的 IEEE binary16（FP16）测试中，将原生 PyTorch 计算与打包融合乘加（FMA）程序进行比较；该测试覆盖驻留于 L2 和高带宽内存（HBM）中的工作集。随后，在四项 GB200 集成任务中，我们用三阶或四阶 bfloat16（BF16）程序替换原生 sigmoid、tanh 和 sigmoid 线性单元（SiLU）：稠密 SiLU、tanh 软上限注意力、sigmoid 注意力，以及路由专家 Swish 门控线性单元（SwiGLU）。这些程序在使用它们的内核内部结合了解析对称性、目标格式舍入和打包算术。独立路径在 L2 中的性能提升为 1.19–2.19 倍，在 HBM 中为 1.00–1.70 倍。稠密 SiLU、tanh 软上限注意力和路由专家替换分别使完整训练步骤吞吐量提升 2.7%、2.9% 和 8.0%。sigmoid 注意力替换使完整注意力前向计算性能提升 7.4%，使完整 GPU 步骤性能提升 0.3%。采用相同检查点的开放权重消融实验，以及每项任务一次的配对预训练比较，将评估扩展到了模型行为。在接近 1000 亿 tokens 的常见训练时长处，四项任务中最终平滑训练损失的差异（多项式减原生）介于 $-0.107$ 和 $+0.079$ 之间。
Kernel knowledge transfer from CUDA to MLX Figure 1: CUDA-to-MLX optimization translation map. CUDA optimization knowledge can be translated into architecture-native MLX strategies rather than copied instruction-for-instruction. We face a new epoch in computing. Hardware is changing rapidly — not just faster GPUs, but a growing range of chips from different vendors, each with its own architecture and often tailored to specific AI workloads. Software is changing just as fast, and AI coding tools now generate in minutes what took months of effort a few years ago. With so much of computing now centered on AI, GPU kernels are a crucial component of its success. These are the low-level programs that run inside the GPU, and writing efficient ones is far from obvious — it takes years of expertise to get right. Transferring a kernel from one vendor’s hardware to another is harder still, and often means rediscovering the same optimizations from scratch. The CUDA ecosystem, for example, has accumulated decades of hard-won kernel expertise: hand-tuned implementations of attention, state space models, and other critical operations representing thousands of engineering hours. Newer hardware ecosystems (Apple Silicon, custom AI accelerators, and others) are growing fast but lack this depth. In this work we ask whether that expertise can be transferred automatically. We built on K-Search , an evolutionary kernel search framework introduced by Cao et al. at Berkeley Sky Lab that uses AI to optimize GPU kernels, and extended it with a backend for MLX — Apple’s machine-learning framework for its own Apple Silicon chips. We developed a novel structured CUDA-to-MLX translation layer that lets K-Search take existing CUDA kernels as a knowledge base and adapt them into high-quality GPU kernels for Apple Silicon, rather than rebuilding from scratch. We show that our approach reaches near-expert level performance on Apple Silicon with 0.97x speedup compared to the native MLX Attention kernel, and up to a 20x prefill speedup over the community mlx-lm implementation on the Mamba SSM kernel; we report the numbers, and how much of the gain comes from the translation layer, in the sections below. Although we focus on MLX kernels for Apple Silicon, the method is not specific to MLX and applies to any ecosystem where CUDA expertise is transferable. Why MLX? Apple’s MLX framework has seen remarkable adoption since late 2023. With Apple Silicon in hundreds of millions of MacBooks and Mac Studios, MLX enables local AI inference without cloud costs. The unified memory architecture makes it especially attractive for mid-sized models (7B–70B parameters on M series chips). Yet beneath this momentum lies a significant gap: many performance-critical kernels that the NVIDIA ecosystem takes for granted: paged attention, optimized SSM scan kernels, fused MoE routing are either absent or naive without hardware-specific tuning. MLX runs models correctly but often leaves significant performance on the table. This gap is what motivates the rest of this post. What is K-Search? K-Search is an evolutionary kernel optimization framework originally developed by our first author Shiyi Cao at UC Berkeley Sky Lab. Given a naive kernel and a hardware specification, it runs an iterative optimization loop: an LLM reasons about which optimizations to try next, a code-writing model generates candidate kernels, and those candidates are compiled and benchmarked on real hardware. Measurements feed back into the search, which keeps refining, pursuing promising directions and dropping dead ends until performance converges. Pseudocode for K-Search via co-evolving world models Algorithm 1: K-Search via co-evolving world models. The search alternates between selecting the most promising action, instantiating and evaluating code until improvement stagnates, and evolving the world model through insert, update, and prune operations. Adapted from Cao et al. (2026) . Search is grounded by a Spec: a domain-specific document encoding hardware rules, optimization patterns, and mathematical constraints which keeps generated code from hallucinating invalid primitives and ensures candidates will actually compile and run efficiently. In our runs, a single model (Gemini 3.5 Pro Preview) plays both roles: it maintains the reasoning state and writes the kernels. The reasoning half is prompted as a “GPU kernel performance engineer” and asked to work through a fixed analysis before proposing anything: classify the kernel (reduction, scan, attention/softmax, …), rewrite the reference computation in canonical form, map out data layout and access patterns, and hypothesize the likely bottleneck (bandwidth, latency, compute, or synchronization) in each runtime regime. Only then does it emit candidate optimizations, each as a single change implementable in one iteration. We call the persistent reasoning state a world model . Rather than a flat list of things to try, it is a decision (prefix) tree: each root→leaf path composes a full optimization plan, and sibling branches are competing alternatives. Every node is scored — an overall_rating in [0, 10], a confidence in [0, 1], and per-node impacts on memory bandwidth, register pressure, and compute/hardware fit — so the search can rank partial plans and expand the most promising ones. The tree persists and grows across rounds: refining an idea adds a child node rather than overwriting its parent, and if the best score fails to improve for a few rounds (a stagnation window) the search backs off to explore an alternative branch. A single node, as it appears mid-run on the attention kernel, looks like this: { "action" : "Replace the threadgroup-memory softmax reduction with a register-only reduction: each SIMD group owns 8 query rows and reduces across lanes with simd_shuffle_xor, removing a threadgroup_barrier." , "difficulty_1_to_5" : 4 , "impacts" : { "memory_bandwidth" : 8 , "register_pressure" : 4 , // risk: spill if Br > 8 "compute_hw_fit" : 9 // SIMD width 32 ; keep tile 8 x 8 }, "overall_rating_0_to_10" : 8 , "confidence_0_to_1" : 0.7 } Listing 1: Example K-Search world-model node. Each candidate optimization records a concrete action, estimated hardware impacts, an overall priority rating, and the model's confidence. Overview of the K-Search loop Figure 2: Overview of K-Search. The framework operates on a Search State $S_t$ structured as a search tree. The tree consists of Closed nodes (blue, visited states with attached program like $x_{12}$) and a Frontier of Open nodes (orange, pending hypotheses like $u_{13}$). The workflow iterates through three phases: (1) Action Selection , where the most promising action node is retrieved from the frontier based on world model estimated priority score $V$; (2) Local Refinement , where a stochastic policy $\pi_{\mathrm{code}}$ samples concrete implementations until stagnation; and (3) World Model Update , where the LLM reasons over the trajectory to update the search tree via Insert (adding new actions), Update (adjusting $V$, e.g., $u_{11}$ dropping from 0.9 to 0.6), and Prune (removing less promising nodes like $u_{10}$). The original K-Search paper evaluated this search strategy on CUDA kernels from FlashInfer. Across GQA decode, MLA decode, MLA prefill, and MoE, K-Search improved more consistently than OpenEvolve and ShinkaEvolve over the same 120-iteration budget. These results establish the search framework we build on here; the remainder of this post asks whether its optimization knowledge can transfer beyond CUDA. K-Search benchmark results compared with OpenEvolve and ShinkaEvolve Figure 3: Main results from the original K-Search paper. Across three runs, K-Search achieves stronger best-so-far search scores, per-workload kernel performance, and speedup distributions than OpenEvolve and ShinkaEvolve on four FlashInfer CUDA kernels. Reproduced exactly from Cao et al. (2026) . Building an MLX backend To bring K-Search to Apple Silicon, we first built a native MLX backend. We implemented a full MLX-specific task adapter for K-Search, including: An MLX task backend in k_search/tasks/ handling kernel compilation and execution on Apple Silicon via MLX’s Metal/C++ APIs. Updated kernel generator prompts for writing and modifying Metal/MLX kernels. MLX-specific benchmarking integration using mlx.core measurement utilities. Translating CUDA expertise to MLX However, the more interesting challenge was not simply running K-Search on MLX. The key insight is that expert CUDA kernels encode decades of optimization knowledge that is transferable to Apple GPU if you can bridge the conceptual gap. Simply handing an LLM a CUDA kernel and asking it to port it is not enough: without deep hardware context, it produces code that is syntactically valid but architecturally wrong (wrong tile sizes, invalid primitives, mismatched memory assumptions). Our translation layer consists of: Concept mapping tables: A structured glossary of CUDA primitives and their MLX/Metal equivalents with hard constraints. For example: __shared__ maps to Metal threadgroup memory but with a hard 32 KB limit (vs. NVIDIA’s 48 KB) warp_reduce maps to MMA (preferred) __syncthreads() becomes threadgroup_barrier(mem_flags::mem_tg) H100’s ~3.35 TB/s HBM3 maps to M3 Max’s ~400 GB/s unified DRAM a bandwidth difference that reshapes which optimizations are worth pursuing. MLX-specific hints and patterns: Concrete code-level patterns for operations with no direct CUDA equivalent, such as register-based row reductions using simd_shuffle_xor in an 8×8 MMA tile layout, or the “exp2 trick” (replacing $exp(x)$ with $exp_2(x \log_2 e)$) for faster softmax on Apple’s fast $exp_2$ hardware instruction. Reusable assertions: Expert kernel behaviors reframed as properties the evolutionary search must preserve, rather than code to copy. Matching expert kernel performance: the Attention kernel We evaluate three configurations of an MLX attention kernel for Apple Silicon: (1) a naive baseline, (2) pure evolution with no additional provided context, and (3) a full context translation layer, which supplies the optimizer with architecture-specific implementation knowledge extracted from high-performance kernels (e.g., FlashAttention-2), letting the evolutionary search reason about implementation strategies rather than starting from a naive kernel. Together, these three configurations let us isolate the exact impact of the translation layer. Performance scaling of the Attention Kernel through stacked optimizations Figure 4: Performance scaling of the Attention Kernel through stacked optimizations. The "Full Context" configuration successfully discovers and implements advanced strategies like double buffering and loop unrolling, achieving near-expert performance. The jump from 0.26× to 0.97× the speed of Apple’s state-of-the-art attention kernel — illustrates how much the translation layer matters. With full context, the evolved kernel independently discovers the key optimizations in FlashAttention 2: threadgroup memory tiling, online softmax, K-transposition for memory access, and the exp2 trick. The last of these replaces every softmax exponential with a base-2 exponential, \[e^x = 2^{x \log_2 e},\] which is exact and lets the kernel use Apple’s fast fast::exp2() hardware instruction directly instead of paying for a base conversion at runtime. A 20× faster prefill: the Mamba SSM kernel To evaluate whether K-Search generalizes beyond attention kernels, we applied it to the state-space model (SSM) kernel used by Mamba. Unlike attention, the computational bottleneck is a recurrent state update rather than a softmax, providing a substantially different optimization challenge. We compare the evolved implementation against the community MLX implementation (mlx-lm) and the PyTorch reference implementation (mamba.py) on an M1 Max. Evaluated on mamba-370m f16, M1 Max 64GB: Metric mlx-mamba (ours) mlx-lm (community) mamba.py Decode 152 tok/s 116 tok/s 40 tok/s Prefill L=512 5,751 tok/s 329 tok/s 1,089 tok/s Prefill L=1024 6,010 tok/s 327 tok/s 1,127 tok/s Prefill L=2048 6,612 tok/s 326 tok/s 1,092 tok/s Prefill L=4096 6,743 tok/s 339 tok/s 1,042 tok/s Table 1: Prefill and decode throughput on mamba-370m (f16, M1 Max 64GB). mlx-mamba (ours) reaches ~20× higher prefill throughput than the community mlx-lm baseline, while decode remains comparable. The ~20× prefill speedup over mlx-lm comes down to one difference: mlx-lm does not implement a parallel scan for the SSM. The state recurrence \[h_t = \bar{a}_t h_{t-1} + \bar{b}_t\] looks inherently sequential, but each step can be written as a pair $(\bar{a}_t, \bar{b}_t)$ under the associative combine \[(a_2, b_2) \circ (a_1, b_1) = \left(a_2 a_1,\ a_2 b_1 + b_2\right),\] which reproduces the recurrence exactly. Because the operator is associative, the whole sequence can be evaluated with a parallel (prefix) scan in $O(\log N)$ dependent steps instead of $O(N)$. mlx-lm skips this and processes tokens one at a time, leaving most of Apple Silicon’s compute idle; our evolved Metal kernel applies the scan and makes much fuller use of GPU throughput. The gain shows up in prefill, where the full sequence is available to scan in parallel, and not in single-token decode, where there is only one new token per step and no scan to parallelize — which is why the decode row is roughly flat while prefill is ~20×. mamba.py is slow on both prefill and decode because it is a PyTorch reference implementation that falls back to CPU or MPS on Apple Silicon, forgoing the hardware-specific optimizations that MLX’s Metal backend makes possible. What’s next? On the two kernels we studied, AI-driven evolutionary kernel search grounded in structured cross-platform translation knowledge reached near-expert performance on Apple Silicon without a team of GPU experts starting from scratch. We do not yet know how far this generalizes, but the result is encouraging. For us the main takeaway is that the bottleneck was not the LLM’s ability to write Metal code, but the quality of the context and constraints we gave it. Our CUDA translation layer converts existing NVIDIA kernel expertise into actionable guidance for Apple Silicon, and lets K-Search’s evolutionary search do the rest. We are actively extending this work in several directions: supporting new architectures, with current efforts focused on developing new kernels for the IBM Spyre AIU and broader hardware targets; adding more kernels such as paged attention and fused MoE routing; and improving integration with the K-Search evolution loop to make translation context even more automatic. Acknowledgements This work was carried out by IBM Research and builds on K-Search from the UC Berkeley Sky Lab ( Cao et al., 2026 ). We welcome collaboration and feedback from the MLX and broader AI systems communities. If you are working on kernel optimization for non-CUDA hardware, we would love to hear from you. Citation @article { cao2026k , title = {K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model} , author = {Cao, Shiyi and Mao, Ziming and Gonzalez, Joseph E and Stoica, Ion} , journal = {arXiv preprint arXiv:2602.19128} , year = {2026} } Appendix: Try it yourself The MLX backend is built on top of the open-source K-Search repo, so the results here can be reproduced directly. The steps are: 1. Clone and install git clone https://github.com/caoshiyi/K-Search.git cd K-Search uv pip install openai wandb uv pip install git+https://github.com/caoshiyi/flashinfer-bench-ksearch.git 2. Set your credentials Open the relevant script under scripts/ and set three variables at the top: KSEARCH_ROOT = /path/to/K-Search API_KEY = your-llm-api-key 3. Run kernel search # Optimize Flash Attention on Apple Silicon (world-model mode) bash scripts/mac_flash_attention_wm.sh # Or a Mamba SSM kernel, e.g. the selective scan bash scripts/mamba_selective_scan_fwd_wm.sh Full CLI reference and documentation are in the README.
了解如何在部署于 Amazon Elastic Kubernetes Service（Amazon EKS）上的 NVIDIA NeMo Agent Toolkit（NAT）中使用 Amazon S3 Vectors 作为持久化记忆层。本文介绍 NAT 的记忆子系统工作原理，以及如何使用多智能体投资研究用例，将 Amazon S3 Vectors 实现为自定义记忆提供程序。
系统性能、高效的基础设施扩展能力和持续的软件优化，是决定 AI 推理经济效益的关键杠杆。更高的系统性能意味着能够生成更多词元，从而带来更高收入。高效扩展意味着随着硬件增加，吞吐量按比例增长，以更少的资源实现大规模用户服务。持续优化意味着从基础设施投资中创造更多价值。 […]
清洁能源并不难获得，但由于基础设施过时、研发周期过长以及前期成本门槛等瓶颈，大规模采用的步伐历来较为缓慢。在 New York Climate Week，NVIDIA 正重点介绍五家开创清洁能源项目的公司——它们将 AI 融入基础架构，加速从研究到项目落地的进程 […]
Today, Egypt’s AI builders gathered in the Grand Egyptian Museum for a reception that highlighted the nation’s rapidly growing AI ecosystem — spanning AI natives, developers, researchers, startups and enterprises — building applications across industries. The event included a keynote from Paolo Guglielmini, vice president of EMEA at NVIDIA. Ahmed Mostafa, regional AI adoption lead […]
每座 AI 工厂都需要与其计算架构相匹配的电力和冷却系统。随着 AI 基础设施不断扩展，电力、冷却、水资源、场地和电网限制正影响着建设者能够部署的方案。选择符合完整工厂设计的产品，有助于建设者将计算容量转化为有价值的 AI 输出。为帮助建设者做出这些决策，NVIDIA 正在推出 […]
为了构建和部署能够在动态环境中感知、推理并采取行动的复杂机器人应用，开发者需要全新的物理 AI 模型和工具。ROS 开放框架是 Open Robotics 推出的一个项目，旨在帮助人类构建机器人。NVIDIA Isaac ROS 5.0——一套基于 ROS 构建的 GPU 加速软件包集合，于今日在 […]
NVIDIA AI Day Singapore将于9月22日至23日在Raffles City Convention Centre举行，为与会者提供探索实践培训、专家主导的课程以及先进工具的机会，以加速其在AI和高性能计算领域的工作。在此次活动中，NVIDIA及其合作伙伴将展示东南亚地区取得的突破性AI进展[…]
When Sakeena Fiza describes her work as a validation engineer at NVIDIA, she does so in terms more befitting a detective story than a world-class engineering lab. “Validation engineers look in the shadows and shine a light into every corner,” Fiza said. “Every time we get a system, our first thought is: how can it […]
When COVID-19 emerged, scientists had a crucial advantage: Decades of prior research on coronaviruses meant they understood the virus’ key proteins well enough to design vaccines in record time. The next pandemic may not offer the same head start. To help improve the odds, NVIDIA has joined a coalition of global research organizations, including Google […]
Building on nearly a decade of co-engineering, CoreWeave has built NVIDIA compute, networking and software into a cloud purpose-built for AI that’s still returning on investment across multiple generations of deployment. Now, CoreWeave is bringing the next generation of NVIDIA infrastructure to production. At CoreWeave Fully Connected, running this week in San Francisco, CoreWeave announced […]
汇聚全球最杰出的人才与最新的加速计算技术，能够带来强大的突破，助力解决一些最重大的研究难题。为促进此类创新，NVIDIA Graduate Fellowship Program 为开展卓越研究且研究方向与 NVIDIA 技术相关的博士生提供资助、导师指导和技术支持。该项目现已进入第 26 届 […]
AI 工厂的建设规模以兆瓦计，甚至达到吉瓦级。每座兆瓦级工厂的成本约为 6000 万美元，而 AI 工厂运营商只有在清晰了解投资回报的情况下，才会投入如此规模的资本。三项关键因素决定了 AI 工厂的回报：盈利能力：工厂一年可能赚取的收入 […]
GPT-6 Astra Ultrafast 目前已在 OpenAI API 中，以及面向符合条件的 ChatGPT Work 和 Codex 用户开放。借助 OpenAI 模型中的推理优化，并利用 NVIDIA Blackwell 架构的能力，Ultrafast 的令牌生成速度最高可达 Astra Standard 模式的 8 倍。对于开发者，[…]
本地 AI 正变得越来越实用。随着 AI 智能体从实验走向日常开发，功能日益强大的开放模型正逐渐缩小到适配更多设备的规模，为开发者提供更多本地运行的选择。本月，NVIDIA DGX Spark 将由顶级制造商合作伙伴提供 64GB 统一内存版本——Acer、[…]
文章链接： https://www.reuters.com/business/retail-consumer/amazon-seeks-offload-8-billion-nvidia-chips-investors-ft-reports-2026-10-02/ 评论链接： https://news.ycombinator.com/item?id=49933958 得分：9 # 评论数：2
文章 URL： https://github.com/Vibra-Ingenn/Janus 评论 URL： https://news.ycombinator.com/item?id=49926773 得分：5 评论数：0
在策略蒸馏（OPD）是一种很有前景的语言智能体训练方法，它能够针对学生模型生成的轨迹提供密集的教师监督。然而，在多轮交互中，一个错误动作会改变学生模型之后所遇到的状态，因此错误会跨轮次累积。在针对三个 Qwen3 模型（8B 至 235B）的初步实验中，我们发现，超过半数的失败轨迹都包含一个关键错误，即使智能体进一步偏离任务完成目标的动作，而且这一错误通常很早就会发生。这些关键错误往往仍然可以恢复：在关键错误发生后的几轮中引导模型，就能重新取得任务成功。因此，我们提出 PivotOPD，一种在策略蒸馏框架，联合训练学生模型以避免关键错误，并从这些错误所导致的状态中恢复。在每次关键错误发生时，教师模型会提供一个标准动作，并在接下来的几轮中分别指定一个恢复动作。预防性蒸馏使用标准动作结合反向 KL 散度，引导学生模型避开关键错误；恢复性蒸馏则使用恢复动作结合正向 KL 散度，将学生模型很少采样到的恢复行为传递给它。在 ALFWorld、WebShop 和基于搜索的问答任务上与 13 个基线方法的比较中，PivotOPD 在 Qwen3-1.7B 和 Qwen3-8B 学生模型上都取得了最强的平均性能；对于 1.7B 学生模型，其在 ALFWorld 上的性能较最强基线提升了 5.5%。这些增益也能迁移到软件工程领域的另一模型系列：在 SWE-Bench Verified 上，PivotOPD 将 Nemotron-3.5 学生模型的解决率提高了 3.2%。项目主页：https://research.nvidia.com/labs/lpr/pivotopd/
近期的 LLM 越来越多地采用混合架构，以线性注意力取代标准注意力，例如 Gated DeltaNet（GDN）和 Kimi Delta Attention（KDA）。尽管这类方法将上下文压缩为固定大小的循环状态，并显著降低了长上下文处理的成本，但反复读取和更新该状态仍然是推理过程中的主要瓶颈。量化为降低这一成本提供了一种自然的方法，但由于舍入误差的累积，以及状态中存在离群行和离群列，量化可能会严重降低模型质量。为应对这些挑战，我们提出了 LeapQuant，这是一种无需训练的方法，可在 8 位循环状态量化下实现近乎无损的性能。首先，为减轻误差累积，我们提出按窗口量化，在跨越一个 token 窗口后，仅在窗口末尾对状态进行一次量化。在窗口内部，输出由固定的低比特状态与高精度的缓冲更新共同计算。其次，为减少每次量化引入的误差，LeapQuant 保留状态中幅值最大的离群项，将其作为少量高精度的 Compensator Tokens，并使其与真实 token 共享更新路径。随后，我们在量化前对剩余残差进行平滑，以进一步降低误差。在 Qwen、Kimi 和 GLM 模型系列上的综合实验表明，LeapQuant 能够显著降低推理期间的内存和计算成本。在精度与 FP32 基线相当的情况下，该方法在 NVIDIA B200、RTX PRO 6000 和 RTX 5090 GPU 上实现了内核级平均 2.05--3.70$\times$ 的加速，以及端到端推理 1.47$\times$ 的加速。
Writing fast GPU code for physical simulation is difficult: implementations must preserve numerical accuracy while handling irregular data access, synchronization, and iterative solvers. We introduce GPUPhysBench, a benchmark of 50 tasks testing whether coding agents can meet these demands. Tasks cover fluids, deformable solids, and granular materials, from individual simulation operators to complete simulators. Agents write, compile, test, and optimize GPU code with access to a NVIDIA GPU under fixed time budgets. We report pass rates and runtime performance relative to expert-optimized reference implementations. In a single-attempt evaluation of six frontier model-harness pairs, the two strongest pass all 50 tasks, but even the fastest reaches at least 0.9 the reference speed on only 22% of them, and no submission is more than 5% faster than the reference. The largest gaps arise in collision detection, constraint solving, and iterative solvers. GPUPhysBench brings physical simulation workloads to coding-agent evaluation, testing both the ability to implement numerical methods correctly and the ability to make them run efficiently.
文章 URL: https://www.cnbc.com/2026/09/28/nvidia-releases.html 评论 URL: https://news.ycombinator.com/item?id=49879883 点数: 41 评论数: 74
文章 URL： https://www.cnbc.com/2026/09/28/nvidias-jensen-huang-ai-distillation-china.html 评论 URL： https://news.ycombinator.com/item?id=49879032 点数：10 评论数：2
Hey HN, Guanming here, cofounder of General Instinct. We just released InstinctFlash, a high-performance serving framework for robotics models. It’s licensed under AGPL-3.0. On Jetson Thor, we see speedups about 1.2x to 7.9x from runtime optimizations alone and up to 33.78x for LingBot-VA when we combine those runtime optimizations with a distilled few-step diffusion scheduler, going from the original 25 visual / 50 action steps to 2 / 4 steps. Across 50 Robotwin2.0 tasks, we evaluated 1,153 episodes per configuration, LingBot-VA with InstinctFlash at 2 visual / 4 action steps achieved a 90.5% success rate, compared with 92.1% for the baseline at 25 visual / 50 action steps. Here’s an optimized 5B world action model, running in real time on a Jetson Thor: https://youtu.be/nku65iyL5Fw InstinctFlash currently supports 8 VLA / world-action model families, including pi0.5 and NVIDIA Cosmos Policy, across RTX 4090 / 5090 and Jetson Thor. Just give it your fine-tuned checkpoint and InstinctFlash handles the rest, exposing the accelerated model through a Python runtime or an OpenPI-compatible WebSocket server. We started working on this because we kept running into the same problem while deploying robot policies, the models were getting much better, but inference was often way too slow for the control loop we actually wanted. For pi0.5, mixed-precision GEMMs and CUDA graphs speed up computation and reduce launch overhead. For Cosmos, caching avoids redundant computation across diffusion steps. World-action models’ diffusion denoising step depends on the previous one which motivated our work on few-step distillation. Right now, InstinctFlash contains 6 aspects of optimization. - Graph: CUDA graph capture, memory planning and separating prefill from repeated execution. - Cache: Reusing KV and conditioning state across diffusion steps and prediction calls. - Attention: Specialized attention paths for different model architectures. - Kernels: Fused operations and kernels tailored to specific backends and tensor layouts. - Precision: FP8 and mixed-precision execution. - Model: Few-step distillation for diffusion and action generation. Teams at Samsung, Siemens, and other robotics startups have used InstinctFlash for model acceleration on VLAs, WAMs, and diffusion-based world models. Now we are opening up access to you. Feel free to try it here: https://github.com/General-Instinct/InstinctFlash More implementation details and benchmarks: https://general-instinct.com/blog/instinctflash-edge-inferen... Would love to hear your feedback! Comments URL: https://news.ycombinator.com/item?id=49802789 Points: 4 # Comments: 0
Safe motion planning in automated driving requires anticipating evolving traffic risks and deciding when to revise the current planned trajectory. We introduce RiskWorld, a risk-aware world modeling framework for shared occupancy forecasting and selective trajectory replacement. Spatial risk fields and temporal actor context are fused with visual bird's-eye-view features. Flow-guided evolution transports occupancy and scene features, while signed residuals correct occupancy after transport. One forecast is generated per planning step and reused across candidates. Each candidate is compared with a current-state persistence reference, yielding a nonnegative collision-score correction. The trajectory selected by current-world evaluation serves as the planning anchor and is replaced only when additional predicted risk triggers intervention and an alternative satisfies component-wise constraints on predicted risk and trajectory error. Candidate geometries remain unchanged. We evaluate RiskWorld for open-loop planning on nuScenes using camera features, annotation-derived current and historical actor states, and dataset-provided map context. RiskWorld achieves the lowest collision rate at a long evaluation horizon of 3 s, and the second-best average L2 error among various state-of-the-art baselines, while running at 11.5 FPS on a single NVIDIA RTX 4090 with 90.81 M parameters. Within-setting ablations show that RiskWorld achieves lower collision rates than the current-state rescoring baseline, while forecast reuse enables additional candidates to be evaluated at low marginal computational cost.
文章链接： https://developer.nvidia.com/blog/introducing-cuda-rust-two-tracks-for-writing-gpu-kernels/ 评论链接： https://news.ycombinator.com/item?id=49724881 得分：25 评论数：4
Model serving is one of the largest cost drivers in production recommender systems. Maximizing its throughput requires navigating a deeply layered hierarchy: GPU kernels, the ML framework computation graph, the model server, and on-demand feature processing -- each demanding specialized domain expertise. Such cross-layer expertise is inherently difficult to acquire, and does not scale with a workload that continuously grows and evolves, leaving significant cost efficiency gains unrealized. While recent AI agents have demonstrated human expert level efficiency in standalone GPU kernel optimization, automated tuning and optimization for the rest of the serving stack remain largely unexplored. We present FlashVector, an agentic system that optimizes performance across all layers of the model serving stack. The key contribution is an extensible framework to generalize the single kernel optimization agent paradigm to heterogeneous technical stacks, and to deliver performance improvements holistically. After deployment in Unity's Vector advertising platform, FlashVector achieved up to 2x throughput increase and up to 1.98x latency speedup on model server, and up to 1.6x throughput increase on feature store. These optimizations were discovered not only at the GPU kernel and computation graph levels, but also across the other components of the model serving stack, such as the model server (NVIDIA Triton's C++ codebase) and the on-demand feature transformation service (Python codebase), demonstrating the extensibility of the framework to more complex system architectures.
文章链接： https://nvidia.github.io/OpenShell-Research/dev-notes/posts/2026-09-10-learning-formal-methods-agent-policy-prover/ 评论链接： https://news.ycombinator.com/item?id=49713261 积分：4 # 评论数：0
Hi everyone, Been working on Otis, an open-source ai agent that gives you one minimal experience across local and hosted open-weight models, privacy-focused by design. On setup it recommends a local model based on the hardware Otis is running on, downloads it and runs it through llama.cpp for you. Ollama, LM Studio and Nvidia PAIR are supported too. Excited for everyone to try it and all feedback is welcome! Comments URL: https://news.ycombinator.com/item?id=49696084 Points: 3 # Comments: 0
文章 URL： https://invezz.com/news/2026/09/11/nvidia-says-every-1-it-invests-brings-back-100-so-why-does-the-stock-keep-falling/ 评论 URL： https://news.ycombinator.com/item?id=49682319 点数：14 评论数：7
文章链接： https://www.economist.com/interactive/briefing/2026/09/03/nvidia-is-the-central-bank-of-ai 评论链接： https://news.ycombinator.com/item?id=49673098 积分：30 # 评论数：9
https://blogs.nvidia.com/blog/nvidia-to-acquire-hugging-face... 评论链接： https://news.ycombinator.com/item?id=49548952 得分：256 评论数：78
文章链接: https://blogs.nvidia.com/blog/nvidia-to-acquire-hugging-face/ 评论链接: https://news.ycombinator.com/item?id=49548952 积分: 19 # 评论数: 3
We propose a lightweight two-stage framework for real-time video anomaly detection. The first stage employs YOLO v11n-pose to detect persons and extract seventeen skeletal keypoints in a single forward pass. The second stage encodes each cropped person region through CLIP ViT-B/32 and computes cosine similarity against predefined textual descriptions of anomalous behaviors. This architecture eliminates the need for optical flow, standalone pose estimators, and density-based scoring modules. Experiments on CUHK Avenue, ShanghaiTech Campus, and a custom indoor dataset collected at Chulalongkorn University demonstrate an end-to-end throughput of approximately 51 FPS on an NVIDIA Titan XP GPU, a 3.36x speedup over the multi-feature baseline, while maintaining frame-level AUROC values of 89.26%, 70.26%, and 84.13%, respectively.
文章链接： https://techcrunch.com/2026/08/29/nvidias-ai-advantage-is-moving-beyond-the-gpu/ 评论链接： https://news.ycombinator.com/item?id=49497235 得分：5 评论数：0
文章网址： https://news.bgov.com/bloomberg-government-news/nvidia-starts-a-pac-as-ai-chip-maker-buids-influence-force-in-dc 评论网址： https://news.ycombinator.com/item?id=49469249 得分：14 # 评论数：3
https://www.theinformation.com/articles/nvidia-agrees-buy-op... （付费墙） https://techcrunch.com/2026/08/24/hugging-face-reportedly-in... 评论链接: https://news.ycombinator.com/item?id=49458161 得分: 19 # 评论数: 3
NVIDIA teams use ChatGPT Work to reduce manual tasks, connect fast-moving signals, and scale successful workflows globally.
Hi HN, I built PantheonGPU because I wanted a better way to answer a simple question: is this GPU actually healthy and performing the way it should? A GPU can show normal temperatures and utilization and still be underperforming, unstable under certain workloads, or have memory, PCIe, or configuration issues. PantheonGPU actively tests the GPU instead of only monitoring telemetry. It currently includes 45+ tests covering compute, tensor workloads, memory, cache, PCIe, thermals, stability, and AI/LLM inference. It supports both NVIDIA CUDA and AMD ROCm. I’m also exploring a larger use case: running Pantheon across GPU fleets to identify individual GPUs that behave differently from the rest of a server or cluster. I’d especially appreciate feedback from people running AI infrastructure, multi-GPU systems, local LLMs, or GPU clouds. Comments URL: https://news.ycombinator.com/item?id=49350637 Points: 5 # Comments: 0
文章 URL： https://www.reuters.com/business/nvidia-scales-back-250-billion-openai-data-center-guarantee-wsj-reports-2026-08-14/ 评论 URL： https://news.ycombinator.com/item?id=49323686 得分：5 评论数：0
Embodied intelligent virtual agents are expected to operate as persistent, adaptive, and context-aware entities within complex virtual and Metaverse worlds. However, implementing cognitively capable agents in such environments is conceptually and technologically challenging. Among a range of blueprints and development approaches, the Cognitive Embodied Agent Architecture (CEAA) has been developed as an implementation-oriented framework for architecting components of perception, memory, reasoning, planning, and embodied action. Considering the recent advances in edge computing and generative AI language models, this paper explores the use of Small Language Models (SLMs) to support edge-based operation of selected CEAA components, focusing on "Think" and "Memory" as processes central to cognitive orchestration and persistence of virtual agents in interactive virtual worlds. An edge-based virtual agent gateway system was developed and evaluated on an NVIDIA Jetson Orin NX using Qwen2.5 models of different sizes, exploring the system's capability to process service requests and handle memory-driven conversations. A series of simulation experiments evaluated routing accuracy, memory-read performance, and latency, demonstrating an SLM-driven prototype agent system that partially implements selected CEAA processes to support the development of embodied agents whose cognitive "brain" can operate efficiently and contextually for interactive experiences in immersive virtual worlds.
基于 Transformer 的语言模型性能强大，但由于反复进行高维矩阵乘法，推理成本很高。我们提出 Reduced Matrix Multiplication (RMM)，这是一种无需训练、输入自适应的推理方法，通过沿收缩维度选择信息量较高的切片来减少 Transformer 矩阵乘积，而不修改模型权重。在简单的保留比例控制下，RMM 提供了平滑且可预测的准确率—效率权衡。针对参数规模从 1B 到 70B 的语言模型，我们发现简化容忍度取决于模型家族、任务、组件和保留比例，尽管它通常会随模型规模增大而提高。在中等程度的简化下，RMM 在所评估的判别式、自回归生成和长上下文设置中保持稳健。我们进一步表明，同一原则可扩展到多模态视觉—语言推理。机制性消融揭示了 Transformer 内部的一种结构性不对称：attention 侧计算比 MLP 组件更容易被简化。最后，在 NVIDIA A100 上使用自定义 kernel 的实际耗时基准测试表明，这些计算节省可以转化为实际运行时间收益，尤其是在更长序列长度下。综上，这些结果将 RMM 定位为输入自适应推理时优化的一个可扩展方向。
OpenAI 和 NVIDIA 宣布建立战略合作伙伴关系，将部署由 NVIDIA 系统驱动的 10 吉瓦 AI 数据中心，第一阶段将于 2026 年启动。
今天，我们宣布以 $730B 的投前估值获得 $110B 的新投资。其中包括来自 SoftBank 的 $30B、来自 NVIDIA 的 $30B，以及来自 Amazon 的 $50B。
Teams use Codex with GPT-5.5 to ship production systems and turn research ideas into runnable experiments.