El sistema AKV de código abierto de Tsinghua reduce los costos de los agentes LLM en un 27,7 % y mejora el rendimiento al reutilizar inteligentemente las entradas de caché KV en lugar de resumir el historial.
Resumen generado con IA. Consulta el original para comprobar el contexto y la precisión.
Este es el extracto disponible del feed. Puede estar abreviado o en el idioma original.
Tsinghua University and CUHK researchers open-sourced AKV, a serving system for KV cache engineering in LLM agents. Instead of summarizing old history, agents drop selected KV cache entries and keep reusing the rest. Across 3 models and 4 agent benchmarks, average cost falls 27.7%. Average task score rises 1.7 points. The code is Apache-2.0 licensed and builds on SGLang. TL;DR What it is: Serving system plus policy library, no weights. Tested on GPT-OSS-120B (5.1B active), MiniMax-M2.7 (230B total, 10B active) and MiroThinker-1.7-Mini (30B total, 3B active). Runs on: NVIDIA GPUs via SGLang. Paper used 8 A800-80GB GPUs in BF16. Quick start serves GPT-OSS-120B on 2 GPUs (tensor parallel 2). Performance: 27.7% lower average cost and +1.7 points average score versus threshold summarization. Best: GPT-OSS-120B on SWE-bench Verified: cost per task 20.0 to 9.7 cents, score 40.0 to 43.2. Worst: MiniMax-M2.7 on BrowseComp-Plus scores 58.9, 3.4 points below Text Engineering’s 62.3. What is cache engineering, and how is it different from context engineering? AKV is a Tsinghua serving system that lets LLM agents drop and reposition their own KV cache. Context engineering trims or summarizes the text history. Editing that prefix invalidates every later KV state. The retained suffix must then be re-prefilled. Harnesses therefore summarize late and aggressively, which can lose details from middle rounds. Cache engineering leaves the text trajectory untouched. It only changes which KV entries stay visible to attention. Retained entries are reused without recomputation. They can still carry information from dropped messages. In the research paper’s cost analysis, KV cache reads made up 82.4% of total API cost. How does AKV work? AKV defines 3 cache operations. Append is the standard update for new tokens. Drop removes selected entries from future attention without changing positions. Repos rewrites retained keys to positions 0 through r-1 using a RoPE rotation. Values st
Una ayuda automática para ordenar, basada en fuentes, palabras clave y entidades, en una escala de 1–12. Alta: 9–12; media: 6–8; baja: 1–5. No mide urgencia, verifica noticias ni representa respaldo editorial.
Nanbeige4.3-33B es un lanzamiento de código abierto significativo porque ofrece un modelo de 33B de alto rendimiento, con licencia Apache 2.0, optimizado específicamente para tareas de codificación agéntica y de contexto largo.
Carbon-X democratiza el análisis genómico al ofrecer un modelo de código abierto y gratuito que traduce el ADN crudo a un inglés claro, permitiendo a los laboratorios procesar datos sensibles de forma privada.
TokenGate acelera el procesamiento de LLMs de contexto largo hasta 11,3 veces sin pérdida de precisión, reduciendo significativamente la latencia y el uso de memoria para aplicaciones a gran escala.