与「BENCH」相关的搜索结果

Introducing OpenDesign Go The first benchmark-curated design model plan. $8 first month—20% cheaper than OpenCode Go and Higher usage limits. Up to 300K+ calls/mo. 10 models: GPT-6 Luna, DeepSeek V4.1 Flash and more. Supports API keys. Use Go with OpenCode, Hermes and more.

Raven 0.2.0 — The Harness of Harnesses, built for RSI. 🐦‍⬛ One harness can't be best at everything. Raven combines its own specialist harnesses (Research, Code, Design, Oncall) with the agents you already use (Claude Code, Codex and more) into one team. And it's built for RSI, and not just at the skill level. The whole harness can be rewritten by AI: prompts, policies, strategy code, playbooks. Every sub-harness, including the orchestration layer itself, is its own instance that can be improved. With Raven you can: 1. Orchestrate many agents as one team. Raven's sub-harnesses and external agents work in one task graph with shared memory across sub-agents, powered by leading orchestration (0.963 Node F1 on the Multi-Agent Orchestration Benchmark). 2. Run long, complex tasks. Oncall and proactive execution keep work going for days, from scientific research loops to shipping a full Godot game. 3. Build vertical agents with RSI. Use Raven's RSI to develop and refine an agent for your domain, and we'll optimize it with you. Experimental for now; reach out to the Raven team(Discord: More in the video and slides below. Open source, Apache-2.0. (lots of work made with Raven lives there, and much of this launch's material was made with Raven too)

Anthropic 放出了 Sonnet 5.5 的完整跑分，最抢眼的一行是 Terminal-Bench 4.0：Sonnet 5 是 10.3%，Sonnet 5.5 是 70.6%，一代之内涨了将近七倍，还超过了 Opus 5.5 的 66.4%。 几个关键数字： OSWorld 2.1 电脑操控：57.0% 到 80.1%，Opus 5.5 是 81.8%。 GDPval-AA 职场知识任务：1844，Opus 5.5 是 1846，GPT-6 Sol 是 1487。 图表识别 Chartography：15.6% 到 61.6%，Sol 是 53.6%。 价格 $2/$10，和 Sonnet 5 一样，是 Opus 5.5 的一半。 很多人看这张表最大的误区，是以为 Sonnet 5.5 已经全面追平 Opus 5.5。脚注里有三处要读： Terminal-Bench 上 Sonnet 用的是 max 档，Opus 用的是 xhigh 档，档位不一样。 独立机构 Artificial Analysis 自己跑出来是 64%，比官方低。 FrontierCode 上 Sonnet 是 46.2%，Opus 5.5 是 54.4%，Sol 是 49.3%，Sonnet 在这一项排在两者后面。 这张表真正讲透的一件事是：便宜的 token 不等于便宜的任务。max 档下，Sonnet 5.5 每个任务要输出约 19 万 token，比 Opus 5.5 还多约六成。独立测下来，最高档的单任务成本已经超过 Opus 5.5。而且拉满也不一定更强，FrontierCode 上 xhigh 是 52.1%，max 反而掉到 46.2%。 所以同样用 Sonnet 5.5，一律拉到 max 的人，账单可能比用 Opus 还高；按任务选档的人，才吃到它便宜的那一半。 你上一次选模型，是看跑分表，还是看自己任务的账单？挑一个真实任务，high 和 max 各跑一遍，对比 token 和结果，再决定。

Higgsfield is the most untold story in tech. $1BN in ARR in 18 months. Faster than everyone other than OpenAI and Anthropic. They spend $4M a month on models. They expect this to be $100K per person per month. They have 150 people working in a content machine. They will breed more millionaires than any other company in Kazakh history. For the first time, @alexmashrabov on the journey to $1BN in ARR. (below) 1. The Power of the Immigrant Founder Coming from Uzbekistan, Alex was pushed into competitive programming at age eight as his single path to reach the United States. For international founders, placing top in global competitions serves as the ultimate social elevator, instilling the relentless work ethic required to build breakout companies. 2. My Biggest Lessons in the Journey to Finding Product-Market Fit @higgsfield burned over $10 million of its $16 million seed round chasing hype and narrative rather than product quality. With under $5 million left, the team pivoted to product-led growth, solving camera control for creative directors, which immediately triggered organic hypergrowth without paid ads. 3. The 150-Person Content Team Powering Higgsfield's Billion in ARR Nearly half of Higgsfield's workforce consists of 150 in-house creative professionals producing tutorials, ads, and cinematic projects. Generating 90 minutes of TV-quality AI video requires 100 hours of raw output, proving human taste and curation remain the primary drivers of distribution. 4. We Spend $4 Million per Month on Models Higgsfield spends $4 million monthly on internal model usage, averaging $10,000 per employee so teams can freely vibe code and test workflows. Uncapped inference compute acts as a force multiplier, allowing top talent to discover breakthroughs at maximum velocity. 5. Why Chasing Benchmarks Is Bullshit and the Corporate Misalignment Occurring Public benchmarks have devolved into corporate psyops where lab researchers overfit test data to secure bonuses before job-hopping. Text-to-video benchmarks ignore real production workflows requiring 3,000-word prompts, proving direct customer iteration beats artificial leaderboards. 6. Why Team Sizes Won't Be Impacted as Much as People Think While AI handles over 60% of basic support requests, complex B2B environments cannot eliminate human teams. High product velocity constantly shifts rules and context, requiring smart, coordinated operators across legal and customer success. 7. Americans Are Way More Promiscuous When It Comes to Leaving Companies Silicon Valley workers routinely jump jobs every two years, prioritizing short-term trends over deep commitment. This transactional market gives international hubs an advantage, where cultural loyalty and team stability build compounding technical moats. (links in comments)

Fomo @fomo这收入，确实有点夸张。 1、近30天手续费：3553万美元 RH链：1749万，占约49% Solana：1177万，占约33% BSC：355万，占约10% 其他链：约272万 2、近30天协议收入：3194万美元 相当于手续费的约90%转化为协议收入 日均收入约106万 3、24小时数据 手续费：104万美元 协议收入：98.5万美元 活跃地址：102999 4、推荐返佣 Q3向推荐人支付手续费分成：503万美元 说明激励机制有用，就是看我fomo那里，推荐人可以拿到被推荐用户他们手续费的 25%，另外用户可以省10%。 5、融资 累计融资：9400万美元 A轮：Benchmark B轮：包括Index Ventures、USV等机构 跟朋友们聊，使用的老师都在称赞fomo把看别人买什么、发现代币和自己交易放在了一起，省掉了来回切工具的麻烦。当然对于普通用户，可能连背后走哪条链都不用太在意，知道想买什么、能方便买到就够了，除了手续费贵点。 发币？空投？ 在公开信息里，没有找到Fomo 任何关于Token、积分或空投计划，但是我想他是有发币逻辑的，他的收入已经超过pump，并且Pump没有发空投，如果Fomo反过来玩一次，把一部分Token分给真实交易用户、推荐人和早期贡献者，那产品与市场的好感度，确实可能直接拉满。当然不止于空投，产品本身很多值得学习的地方 在没有积分、没有任务、甚至没有明确空投预期的时候，我也觉得确实是一个值得探索实践的空投好标的。 fomo之前完成9400万融资，最近又宣布完成新一轮融资本轮共有140 名投资人参投，绑定了这么多人上车，肯定要搞把大的，你说呢？

NVIDIA 发布 Skill2Env：用“集体技能”强化智能体 NVIDIA 研究者们把社区公开的 Agent Skills 编译成可执行 RL 训练环境的数据流水线：3.4k 个 Skills 变成 8k 个带程序化测试和行为量规的终端任务；仅 300 步 RL 训练就让 Qwen3.8-27B 在 Terminal-Bench 2.1 上提升 4.7 个百分点，且模型行为显著向源 Skills 的方法论对齐。 开源项目： 核心洞察：公开 Agent Skills 是一个被忽视的监督来源 Agent Skills 是“教智能体做某件事”的文件夹：一个 SKILL.md 加上可选的脚本、参考资料和资产。论文指出，把公开 Skill 语料当作数据来读，它同时提供三样东西： · 任务分布的采样：人们真正想让智能体处理的任务分布（有人愿意花时间写下工作流，说明这活儿值得自动化）； · 真实世界的锚点：指向真实的仓库、数据集、工具和工件； · 结果测试表达不了的质量标准：领域专长、默认参数、常见坑、“好结果长什么样”。 # 数据流水线：四阶段编译，验证靠构造 1. Plan（分解）：容器化的 Codex 规划器读取完整 Skill 包、联网调研相关公共资产，把 Skill 拆解成若干可验证的 workflow，每个附带元计划（场景、初始世界、预埋缺陷、难点来源、解法草案、验证策略）、资产建议和“任务轴池”（任务原型 × 验证器模式 × 人物画像）。 2. Diversify（多样化）：宿主从轴池采样一组组合，加上复杂度、指令语气、请求者专业水平。关键设计是轴池以 workflow 为条件：研究型 workflow 配“证据可追溯”验证和研究者画像，而不是从全轴乘积空间乱抽，这让多样化保持 sensible。 3. Create（构造）：全新创建者 Codex agent 在 Docker 内工作，尽可能用真实素材（钉在特定 commit 的开源仓库、真实版本化文档、官方 API 规范）；需要联网服务的场景改造成本地替身（stub 服务器、录制回放 fixture、PATH 上的假 CLI、种子数据库），求解时绝不依赖网络。创建顺序被严格固定：先建世界 → 写指令 → 写测试 → 写量规 → 最后才写参考解，测试先于解法冻结，保证解法必须迁就评分契约而非反过来。 4. Verify（验证）：宿主端无模型参与的接收门：静态检查（布局、符号链接、Dockerfile 安全、基础镜像按内容摘要钉死）+ 两个容器内试跑：Oracle（参考解）必须全指标满分，NOP（什么都不做的 agent）必须全指标零分。任一失败即拒绝。 值得注意的一个反直觉选择：不做 teacher 模型预验证（不像部分工作用强模型试解、解不出就丢弃任务）。理由有二：这会把任务难度上限压到验证器能力，且成本翻倍；而 group-based RL 的在线动态过滤（rollout 无优势的 prompt 自动不产生梯度）天然淘汰过难/过易任务。 # 数据画像：广、贵、且忠实于源 规模与成本：7,971 个任务，用 GPT-5.6 Sol（xhigh 推理档）生成，API 花费超 9 万美元。（脚注：出于法律原因，公开发布的数据集改用 Kimi-K3-max 在同一流水线下生成。） 领域分布：13 个领域中，软件工程仅占 22.5%，AI/ML 10.5%，商业/金融/法律/HR 10.5%，营销 9.3%……论文对比了 TMax-15K、Terminal-Bench、DeepSWE 等，Skill2Env 是唯一全覆盖 13 域、且非技术知识工作占大头的语料。 忠实度探针（很聪明的设计）：用任务指令+量规作查询、对 3.4k 个 SKILL.md 做 TF-IDF 检索，73.2% 的任务 top-1 命中真实源 Skill，94.6% 进 top-10（随机 0.03%）。单用量规也有 68.5% top-1，证明量规携带的是 Skill 专属方法论而非泛泛建议。 SFT 数据：用 GLM-5.3 对每个任务 rollout 两次，得到 15,968 条轨迹，平均奖励 0.74，中位轨迹 19 次模型调用 + 23 次工具调用。 S2EBench：考虑到公开基准饱和，从 SkillHub 另外生成、逐条人工审核（指令无歧义、忠实于源 Skill、测试公允）后的 79 任务私有 held-out 基准。 # RL 实验：基础设施 + 极简配方 基础设施（论文明确说“现代 agentic RL 首先是基础设施挑战”）：Molt（PyTorch 原生全异步训练，Ray + vLLM + FSDP2）+ Polar（agent rollout 层：rootless Apptainer 沙箱、代理回传 token ID 和采样时 log-prob、prefix merging 把 harness 的多次补全缝合成训练轨迹）。 配方（刻意走“简单路线”）：GRPO 组归一优势 + DPPO 的 binary-KL 信任域掩码（δ=0.05，超出阈值的 token 直接丢弃，无需参考模型，还能防训练-推理失配）；G=8 rollouts/组，批 64，lr 1e-6 恒定，无 KL 惩罚、无熵奖励、无 SFT 热启动，每任务 65k 上下文。 量规校准奖励：开量规时，额外由 GPT-6 Astra 做 LLM-as-Judge（带“宪法”：惩罚无脑循环、reward hacking、答非所问；hacking 实证 = -5 分），总奖励 r = r_V + λs/5（λ=0.2），即 judge 最多把程序化奖励拉动 ±0.2。量规是校准可执行结果奖励，而非取代它，这是与“Rubrics as Rewards”一系的定位差异。 # 四项发现（论文最有信息量的部分） 发现 1：小规模 RL 即有跨域迁移。 仅 300 步、只用 2,400 任务子集训一个 epoch：S2EBench pass@1 +4.3（均分 +18.5），Terminal-Bench 2.1 +4.7（49.4→54.1）。训练集与 TB 无重叠（13-gram Jaccard < 0.8），且训练集从未针对 TB 调过，论文将其解读为规划、工具使用、收尾能力的通用提升而非任务族记忆。这让 27B 本地模型显著缩小了与云端前沿模型的差距。 发现 2：量规校准 RL 在基准上落后于纯结果 RL，一个诚实的负结果。 量规版在 TB 2.1 只有 50.1（纯结果版 54.1）；训练中量规版的程序化奖励长期停在 0.5–0.6，judge 分项从头到尾无上升趋势，两个奖励在训练分布上互相拉扯。论文不把它当作对量规奖励的终审判决（两者优化不同目标，而基准只考结果那一半），并给出两个疑因：λ=0.2 的加性形式让失败任务仍能拿正奖励、judge 看不到文件系统等设定均未调优；以及更本质的，Skill 写下的方法论可能本来就不是最大化基准通过率的分布。 发现 3：行为确实向 Skill 对齐，量规的价值所在。 200 个任务的成对偏好测试（judge 拿源 SKILL.md 当标准，比较匿名化的 base 与 RL 轨迹）：纯结果 RL 已被偏好 54.5% vs 33.5%；量规版被偏好 73.0% vs 24.0%。这说明量规奖励买到的东西在结果基准上看不见，但对“怎么做事”影响实质，对网页开发、报告综合、开放研究这类难验证任务尤其重要。 发现 4：GLM-5.3 蒸馏 SFT 反而伤害 Qwen。 在 GLM-5.3 轨迹上做 SFT：27B 上 TB 2.1 掉到 45.8；4B 上直接崩塌（TB 18.7→3.4，出现思维/工具调用死循环）。归因：教师的 interleaved-thinking + 工具调用风格与学生自身 post-training 不兼容，模仿覆盖了学生依赖的行为模式却带不来教师的能力。与 TMax 报告的“SFT 混合数据劣化已后训练的 Qwen”互相印证。因此论文所有 RL 结果都从未修改的原始 checkpoint 出发。

Your RWUSD just got a little more rewarding. APR is now 3.61%. RWUSD: yield benchmarked against tokenized U.S. Treasury Bills, with rewards paid daily. Put your RWUSD to work 👉

阿里把团队内部用了两年的官方 AI Code Review Skills 开源了，采用 “确定性工程 pipeline + AI Agent” 的混合架构，专门解决通用 Agent 做代码审查时 “漏审、定位漂移、质量不稳” 的老问题。 40.5K ✨ 开源项目 OpenCodeReview： # 核心设计：确定性工程 pipeline × Agent 各司其职 确定性工程负责硬约束： · 精确文件选择：用代码决定哪些文件必须审、哪些要过滤，不依赖模型自觉； · 智能文件捆绑：把相关文件合成一个审查单元（例如 message_en.properties 和 message_zh.properties 捆绑），每个单元以上下文隔离的 sub-agent 运行，分治策略让超大变更集也稳，且天然支持并发（默认 8 个文件 worker）； · 细粒度规则匹配：内置约 54 个按语言/文件类型的规则文档（Java、Go、TS/JS、Python、Rust、SQL/XML mapper、properties 等），用模板引擎而非自然语言把规则匹配到文件特征上，从源头消除信息噪声； · 外部定位与反思模块：评论的“落点”和“内容”分别由独立的 re-location 和 reflection 模块系统性校正，这正对“位置漂移”痛点。 Agent 负责动态决策： · 深度优化的场景 prompt（内部分为 plan → grouping → main → memory_compression → re_location → review_filter 多个任务模板，可在 internal/config/template/prompts/ 看到）； · 从海量生产环境的 tool-call 轨迹（调用频率分布、单工具重复率、新工具对调用链的影响）反向蒸馏出的专用工具集，包括全文件读取、代码搜索、其他变更文件查阅等，比通用 agent 工具箱更小更稳。 # 能力面与生态集成 功能上覆盖：workspace/分支区间/单 commit 审查、断点恢复（ocr session）、全文件 scan（无 git 历史也能审计陌生代码库）、本地 Session Viewer 网页查看与回放、SARIF/JSON 输出、OpenTelemetry 可观测性、MCP Server 扩展。 作为 “Skills 生态” 级项目，它的形态相当完整：既提供 npm 全局 CLI，也提供可移植的 Agent Skill（skills/open-code-review/SKILL.md，带标准 frontmatter，可直接被兼容 skill 的 agent 加载），还有面向 Claude Code、Codex、Cursor、Kimi Code、OpenCode 等平台的插件，每种都封装成斜杠命令或可调用 skill。LLM 侧兼容 OpenAI、Anthropic、AWS Bedrock 三类协议，并可直接复用 Claude Code 的 ANTHROPIC_* 环境变量。 其中一个设计很巧妙：Delegation 模式（ocr delegate preview/rule）。此时 OCR 只做自己擅长的确定性部分（文件选择和规则解析）审查本身交给宿主 coding agent 的 LLM 执行，用户无需给 OCR 配任何 API key。这实际上是把“harness 能力”与“模型能力”彻底解耦。 # 工程质量：超出平均水准的部分 · 安全有正式的 Assurance Case（ASSURANCE_CASE.md）：完整的威胁模型、四条信任边界、T1–T7 威胁逐条给出缓解措施，并按 Saltzer & Schroeder 设计原则和 OWASP Top 10 做了映射。细节经得起推敲：所有外部进程调用只限 git 且子命令硬编码、--end-of-options 防 flag 注入；Agent 读文件路径经 pathutil.WithinBase() 在符号链接解析前后双重校验；本地 Viewer 有 Host 白名单防 DNS rebinding + 严格 CSP。这类文档在一般开源项目里非常罕见。 · 贡献规范近乎严苛（AGENTS.md）：使用 AI 必须在 issue/PR 中披露工具与模型、必须逐行理解 AI 生成的代码、禁止“AI 生成→反复修复→再修复”的循环、禁止把 commit 署名给 AI。源码强制英文（CI 有 english-check，连全角标点都查）、90% 测试覆盖率门槛、-race 与 govulncheck 每次 push 都跑、SPDX 头与 LF 行尾强制。 # Benchmark：数据情况 官方基准 AACR-Bench（已在 Hugging Face 开放）规模不小：50 个流行开源仓库、200 个真实 PR、10 种语言、80+ 资深工程师交叉验证出 1505 条标注问题。结论是同模型对比 Claude Code：Precision 和 F1 显著更高、token 消耗约为 1/9、速度更快。 需要指出两点：其一，Recall 低于通用 agent，README 自己承认这是“以精度换噪声”的刻意权衡，如果你最怕漏问题而非误报，可能不适合；其二，该基准由阿里自建，虽开放了数据集供社区复核，但独立第三方的复现结论目前还少，可以把它当作“有披露的、方向可信的参考”。

here's a prompt to improve your agent harness based on what we've learned at cursor. enjoy # Improve this agent harness's token efficiency You're working on an LLM agent harness: the system prompt, tool definitions, request assembly, context caching, compaction, and retrieval, and how work is split across agents. Make the agent's runs cheaper without making it worse at its job. - Objective: lower price-weighted token cost per completed task. - Constraint: no measurable drop in task quality. Measure per task, not per request. Every turn resends the prefix (tools, instructions, setup, and the conversation so far), so a change that shrinks each request but adds turns can cost more. Weight tokens by billing type: output, uncached input, and cached input are priced very differently. Work in this order: map the harness and measure the baseline, rank the opportunities, make the changes that are safe to make directly, put the rest behind flags or in proposals, then report. Figures below come from one team's production coding agent and its multi-agent experiments. Use them to gauge magnitude, not as targets. One round of these changes (prompt trimming, tool offloading, cache layout, sparse line numbers, subagent tuning) cut that team's overall token cost about 7% with no loss in quality. The larger percentages apply only to the part of the request each change touched. ## Principles 1. Change what the harness sends, not how hard the model tries. Don't ask the model to conserve tokens. A harness that told its model to "take care to preserve tokens and not be wasteful" found it grew reluctant to take on ambitious tasks and sometimes quit, saying it wasn't supposed to waste tokens. 2. Capable models need definitions, not commands. Lists of "DO NOT", "You must", and "Important", and guards against older models' habits, can usually be replaced with plain descriptions of what each tool does. One team cut about two-thirds of its system prompt this way, and the shorter prompt worked across model families. Instruct only on what the model can't know (the product, the environment, the user's processes) and on quirks you've seen in transcripts. 3. Static context is for what most turns need. Everything else should be discoverable when needed. Less up-front context also means less confusing or contradictory information. 4. Expect removals to win. Guardrails written for weaker models, coordination steps that became bottlenecks, and prompting for behavior the model now does on its own all cost tokens. 5. Real usage decides. Evals are a fast proxy, but they skew toward hard problems and miss the real mix of requests. ## 1. Map the harness and measure the baseline Find: - Where requests are assembled, the system prompt, and tool schemas. If a framework or SDK builds requests, find its hooks for message order, cache control, and tool loading. - How tool results are formatted, and how history is kept, trimmed, or summarized. - How subagents or parallel agents are spawned, if any. - Which models and provider APIs are used. From the provider's docs, get the prompt caching behavior (automatic or explicit breakpoints, TTL, minimum cacheable length) and the prices for output, uncached input, and cached input. - Existing logging, token accounting, and evals. If the harness doesn't record per-request token usage by billing type and cache hits, add that first. Everything later depends on it. Then render a few real requests (from logs, or by running representative tasks) and count tokens per section with the model's tokenizer or the API's usage fields. Produce: - Cost share by source × billing type. Sources: system prompt, tool definitions, skill/rule/integration descriptions, user messages, file reads, search results, command and other tool output, history, summaries, subagents. - Static tokens per request, cache hit rate, and turns per task. - Per tool: the share of runs that call it at least once, and its error rate. Read the rendered requests, not just the templates. Duplication, leaked volatile values, and misordered blocks only show up there. Rank opportunities by share of spend × fraction removable ÷ quality risk. ## 2. System prompt and injected context Label every instruction: - Keep: product or environment knowledge the model can't infer, fixes for quirks seen in this model's transcripts, and rules a mode depends on. - Rewrite: commands and emphasis into plain descriptions. Reminders into constraints: "No TODOs, no partial implementations" works better than "remember to finish implementations." Vague quantities into ranges: "generate 20–100 tasks" gets far more ambitious behavior than "generate many tasks." - Delete: things capable models do by default, guards against behavior you haven't seen from this model, text that repeats tool descriptions, and lines that could contradict a user request. Models trained to rank system instructions above user messages will side with the system prompt. - Move: anything per-user or per-request (date, environment, repo state, lists of skills or subagents, user rules) into a user-role setup message after the cache boundary. Audit other injected context the same way. As models improved, the team behind these figures dropped directory trees, pre-retrieved snippets, compressed copies of attached files, lint errors injected after every edit, forced expansion of short file reads, and caps on tool calls per turn. They kept small, high-value facts: OS, repo status, and open or recently viewed files. Skip checklists for open-ended work. The model optimizes the listed items and deprioritizes everything else. ## 3. Tool definitions Tool schemas ride along on every request. Most tools beyond the core set were each needed in under 20% of conversations, and moving them out of static context cut tool-description tokens 60%. Doing the same for integration tools (such as MCP servers), with names in context and full schemas in one folder per server that the agent can search with grep or jq, cut total tokens 46.9% in sessions that used them. - Keep in static context: high-frequency tools (for a coding agent: read, search, edit, shell), tools the model tries to call even when they're absent, and tools a mode depends on. - Offload the rest: leave a name or one-line pointer and make the full schema discoverable on demand. Group related tools so they load together, and put status (such as "needs re-authentication") where the agent will see it. - Tighten what remains: describe behavior and arguments, and drop usage lectures. - Pick the split by testing a few configurations and tracking tokens, cost, latency, tool-call errors, and task success. ## 4. Cache layout Order each request so the reusable prefix is as long as possible: `tool definitions → system instructions → [breakpoint] → setup message (skills, subagents, rules, environment) → [breakpoint] → conversation` - Keep the prefix byte-identical across turns. Use deterministic tool order and serialization, put timestamps and IDs after the boundary, and don't rewrite earlier messages except when compacting. - Use explicit breakpoints if the provider supports them. Otherwise rely on automatic prefix caching with the stable part first. Respect TTL and minimum-length rules. - Switching models mid-conversation throws away the cache (caches are per model and provider) and hands the new model a history it didn't write. When a different model is needed, run it as a subagent with fresh context. Explicit breakpoints plus moving per-request setup after them cut cold cache misses 20%. ## 5. Tool results and other context added during a run - Large outputs (commands, integrations, logs): write them to a file and return the path, size, and a short tail. The agent can tail, grep, or read ranges for more. Truncating loses data, and inlining bloats every later request. Treat long-running terminal sessions the same way. - High-volume formats: look for overhead repeated on every line or item. Numbering every 10th line of a file read instead of every line cut cache-read tokens 1.6% without hurting citation accuracy. Each number costs 3–5 tokens, and agents read tens of thousands of lines per session. Also check repeated absolute paths, verbose JSON keys, ANSI codes, progress bars, and repeated headers. - Good retrieval saves exploration turns. Adding semantic search alongside grep raised codebase question-answering accuracy 12.5% on average and cut the iterations users needed. - Tool errors waste tokens and leave confusing debris in context. Classify expected errors (invalid arguments, unexpected environment, provider error, timeout, user abort), treat unknown errors as harness bugs, and track rates per tool and per model. One focused effort along these lines cut unexpected tool errors 10×. ## 6. Long runs: compaction, subagents, and model mix - Compaction: keep the summarization prompt short and the summary compact, carry forward plan state and remaining tasks, and save the full history to a file the agent can search for details the summary dropped. A model trained to self-summarize from a one-line prompt wrote ~1k-token summaries with half the compaction error of a multi-thousand-token prompt that produced 5k+ token summaries. Untrained models may need more guidance, so test how short you can go. A more expensive summarization model made a negligible difference. - Scratchpads and running notes: rewrite them instead of appending. For repeated work in one environment, a small agent-maintained notes file with a line budget, loaded at start, is a promising way to shorten later runs. - Subagents: fresh context keeps the parent lean, but isolation adds coordination cost (duplicate or stale work). If the model already delegates on its own, remove prompting that pushes it to. Have subagents return short handoffs: what was done, findings, concerns, and deviations. A subagent should use a different model only when the user or harness says so. - Model mix: in large multi-agent runs, workers used at least 69% of tokens, and over 90% in most runs. A frontier planner with cheap workers matched a frontier model doing everything at about one-eighth the cost. Planner choice still changes worker spend. One planner that cost less on its own saw its workers use several times more tokens, and the run cost more overall. Measure the whole tree. - Routing and reasoning effort: send simple turns to a cheaper model or lower effort, and upgrade only when a stronger model is clearly better. A router built this way matched or beat single frontier models on user satisfaction at 41–68% lower cost. - Reasoning continuity: if the API returns reasoning items (including encrypted ones), pass them back on later turns and alert when they go missing. Dropping them cost one reasoning model 30% on a coding benchmark, and it burned tokens reconstructing its plan. ## 7. Fit the harness to each model Adapt to what each model was trained on instead of forcing one shape on all of them. If you've tuned the harness for a similar model, start from that version. - Edit format: use the one the model was trained on (for example, patch-style or search-and-replace). An unfamiliar format costs extra reasoning tokens and causes more mistakes. - Shell or tools: shell-first models fall back to `cat` or inline scripts. Name tools after their shell equivalents (such as `rg`), and if needed add: "If a tool exists for an action, prefer to use the tool instead of shell commands (e.g. read_file over `cat`)." - Literalness: some model families follow instructions literally and others tolerate imprecision. Some spiral on emphasized wording. Strip caps and emphasis for literal models. - Triggers: some models ignore a tool until told when to use it. A literal trigger works: "After substantive edits, use the to check recently edited files for linter errors. If you've introduced any, fix them if you can easily figure out how." - Progress updates: if a model reports progress through reasoning summaries, keep them to 1–2 sentences that note new findings or a change of tactic, and remove instructions about messaging mid-turn. - Quirks worth a targeted line: hedging or refusing as context fills ("context anxiety"), declaring completion early, stopping to ask permission, and calling tools that don't exist. Tie each added instruction to the transcript behavior it fixes. Re-audit when models change, since guidance one version needed can be dead weight for the next. ## 8. Validate - Offline: run a fixed set of realistic tasks before and after, ideally drawn from real usage and phrased the way users actually write (short and ambiguous). Compare task success, tokens, cost per task, turns, and tool errors. Don't ship a change that lowers success. - Online, if you have users: A/B test each change or small bundle. The primary metric is cost per completed task. Guardrails are task success signals, tool-call errors, latency, turns per task, and cache hit rate. For a coding agent, a good success signal is how much agent-written code survives over time. In general, check whether the user's next message moves on or reports a problem. - Ship only when cost drops and no guardrail regresses beyond noise. Record null results. ## What to change directly and what to propose - Change directly, each in its own revertible commit: token and cache telemetry, deterministic serialization and tool order, moving volatile content out of the cached prefix, explicit cache breakpoints, writing large outputs to files instead of truncating, passing back reasoning items that are being dropped, and fixes for recurring tool errors. - Change behind a flag so it can be tested: system prompt edits, tool offloading, output format changes, compaction changes, and subagent prompting. - Propose only: changes to which models run, routing, reasoning-effort defaults, or how work is split across agents. ## Traps - Asking the model to use fewer tokens or do less. - Truncating tool output. - Dropping reasoning items to save input tokens. - Volatile content in the cached prefix, or tool order that changes between requests. - Offloading a tool the model needs on the first turn or tries to call when it's missing. - Emphasis-heavy prompts (MUST, NEVER, IMPORTANT, all caps), especially with literal models. - Forcing a terser output format than the model was trained on. Fewer output tokens can mean less thinking and worse results. - Optimizing raw token counts instead of cost, per request instead of per task, or evals instead of real usage. - Switching models mid-conversation to save money. - Adding coordination layers that become bottlenecks. ## Report back with 1. The harness map and baseline: cost by source × billing type, with the biggest sources called out. 2. A ranked list of changes: layer, what changes, estimated savings and how you estimated them, quality risk, how to validate, and how to roll back. 3. The changes you made, including a system prompt diff with a keep, rewrite, delete, or move reason for each line. 4. A test plan for the flagged changes. 5. Gaps: anything you couldn't find or measure.

A judge delivered a quick verdict in Tomasz Kosowski’s bench trial, where he faced charges of first-degree murder in the death of Steven Cozzi. Kosowski, who acted as his own attorney, was convicted of killing Cozzi, an attorney representing a client who was suing him.