El nuevo método de atención de Zyphra acelera el procesamiento de IA mediante el uso de una estructura de árbol para omitir texto irrelevante, manteniendo un alto rendimiento en tareas complejas.
Resumen generado con IA. Consulta el original para comprobar el contexto y la precisión.
Este es el extracto disponible del feed. Puede estar abreviado o en el idioma original.
Zyphra has released Zyphra Hierarchical Attention (ZHA), a sparse attention method built around a tree-structured indexer. ZHA targets a known gap in sparse attention: the indexer that picks tokens still scans the whole context. Zyphra’s technical paper shows ZHA can be distilled into frozen pretrained models with about 4B tokens. On Qwen3.6-35B-A3B it stays close to dense attention on reasoning, agentic coding and 256K retrieval benchmarks. TL;DR Size: It is a method that adds 81,920 trainable parameters to Qwen3.6-35B-A3B (about 0.00024%). Tested up to 256K context. Runs on: Retrofits frozen pretrained models after a 4B-token KL warmup at 32K context. Speedups come from a modeled H100 BF16 roofline. No optimized kernels released. An FP4 router variant was also tested. Performance: Within a few points of dense attention on 7 reasoning, coding and retrieval benchmarks. Best: LongBench v2 hits 63.4% (deep tree) vs 58.0% dense, using about 1.1% of keys. Worst: On NoPE-based Nemotron-3.5-Lightning-30B-A3B, deep-tree RULER-13 drops 25.3 points below dense. What is Zyphra Hierarchical Attention (ZHA)? ZHA is a sparse attention method from Zyphra that finds relevant keys by searching a tree. It keeps the full KV cache. The tree only decides which cached keys each query reads. Fixed-state methods like SSMs and sliding windows saturate at long contexts. Fine-grained sparse methods like DeepSeek Sparse Attention score every past token with an indexer. The final attention is sparse, but the indexer still grows with the full context. Near 1M tokens, cited work shows indexing takes 60 to 90% of layer time. ZHA makes the search itself sparse. How does ZHA’s tree indexer work? Building the tree ZHA splits keys into contiguous chunks and compresses each chunk into a summary key. Summaries are grouped and compressed again, level by level. With chunk size 4, 6 levels cover 4,096 tokens per top node. Summaries come from a small set of learned reference queries. Each ref
Una ayuda automática para ordenar, basada en fuentes, palabras clave y entidades, en una escala de 1–12. Alta: 9–12; media: 6–8; baja: 1–5. No mide urgencia, verifica noticias ni representa respaldo editorial.
El backend Pando de OpenAI reduce la latencia mediana de push de Git en un 80 % mientras gestiona casi un petabyte de datos diarios, acelerando significativamente los flujos de trabajo de desarrollo para ingenieros y agentes de IA.
Letterpress-1 automatiza las mejoras de sitios web analizando mercados y generando cambios de código, ofreciendo una solución alojada que elimina la necesidad de hardware local.
Estos contratos definen los estándares técnicos y operativos para la integración de modelos de IA transformers, garantizando una implementación fluida, segura y eficiente en diversos sistemas.