Together Compiler reoptimiza continuamente la inferencia de IA a medida que cambia el tráfico, logrando mejoras de velocidad de hasta 2,6 veces y evitando la degradación del rendimiento causada por la configuración estática.
Resumen generado con IA. Consulta el original para comprobar el contexto y la precisión.
Este es el extracto disponible del feed. Puede estar abreviado o en el idioma original.
Together AI has introduced Together Compiler (TKCC), an agent-driven system that keeps re-optimizing inference as production traffic changes. Together Compiler already runs on several of Together’s serverless endpoints. The core claim for developers: an engine tuned once goes stale, so tuning should never stop. TL;DR System: It is an optimization system, tested on Kimi K3, GLM 5.3, MiniMax M3.1 Flash, DeepSeek v4.1 Flash and Llama 1B. Runs on: NVIDIA B200, B300, GB300, H100, AMD MI355X and AWS Trainium2. Each run used at most 2 nodes per model. Performance: Up to 2.6x higher TPS than the best published InferenceX configurations, in under 24 hours per setting. Best: 1,111 P90 TPS per user on MiniMax M3.1 Flash (B200, batch size 1), 1.8x over the 633 TPS baseline. Worst: GLM 5.3 on MI355X moved only from 463 to 480 TPS (about 4%). What is Together Compiler? Together Compiler is a continual compilation loop from Together AI that uses coding agents to rewrite a model’s forward pass. It takes a server launch command, a traffic profile and a hardware configuration. Agents propose faster forward passes, and a persistent engine measures each one. The winning code ships into the serving engine. When traffic drifts, the loop runs again. Why does inference need continual compilation? Together AI team reports that its traffic shape never sits still. Prompt lengths in one country ran about 97K tokens versus 70K elsewhere. One customer’s P90 input length grew from 18K to 33K tokens in 3 days. Kernels tuned on random data can also mislead. Random inputs activated 13 to 38% more MoE experts than real traffic. One agent-written top-k kernel took 20 µs on random data. On real data at 70K context, it took up to 62 ms. How does the persistent engine work? Most agent loops restart vLLM, SGLang or TensorRT-LLM for every experiment. The research team measures 761 seconds of cold startup for MiniMax M3.1 Flash on TensorRT-LLM. Together Compiler starts the engine once. Agents s
Una ayuda automática para ordenar, basada en fuentes, palabras clave y entidades, en una escala de 1–12. Alta: 9–12; media: 6–8; baja: 1–5. No mide urgencia, verifica noticias ni representa respaldo editorial.
GPT-Image-2.6 de OpenAI acelera la generación en un 50% y añade soporte para GIF/SVG, mejorando significativamente las capacidades de los desarrolladores y la eficiencia del flujo de trabajo creativo.
El marco de Deliberación Ágil de Google permite a los usuarios definir conceptos de imágenes subjetivos sin necesidad de ajuste fino, utilizando las API de Gemini en un entorno de cuaderno sencillo.
El nuevo sistema de memoria del MIT corrige objetos etiquetados incorrectamente en la base de datos de un robot, asegurando que pueda localizar con precisión artículos críticos como desfibriladores durante emergencias.