Baidu’s ERNIE Evolve Automated Research System Beats NanoChat and NanoGPT Records

This is the available feed excerpt, which may be shortened or in the publisher’s original language.

Baidu has just released ERNIE Evolve, an automated research system that improves AI training and inference code. Given a repository, an objective and evaluation criteria, it picks research directions, edits code and runs experiments. Only changes that pass a hidden evaluator and a separate reviewer are kept. Baidu team tested the ERNIE Evolve automated research system on three public benchmarks. It then optimized inference for ERNIE-5.5-Flash, still in development. TL;DR Size: It is an agent system; Baidu has not disclosed which LLMs power its agents. Runs on: Benchmarks used 1 NVIDIA B200 (NanoChat, SOL-ExecBench) and 8×H100 (Speedrun Track 1). Production tests ran on Kunlunxin P800 and GPUs (GPU model not disclosed). Performance: Beat the best public result on 3 of 4 public metrics, and cleared production checks on ERNIE-5.5-Flash. Best: Generation speed on Kunlunxin P800 reached 623.6% of baseline after 4 research rounds. Worst: SOL-ExecBench mean SOL of 0.819, behind the 0.827 leaderboard best. Bottom line (best): Real production inference gains: +33.7% deterministic throughput, +14.9% decode speed. Bottom line (worst): Self-reported, system not released, and public-benchmark margins are small. What is ERNIE Evolve? ERNIE Evolve is an automated research system from Baidu that optimizes code against a fixed, measurable objective. Baidu announced it on X as a practical step toward recursive self-improvement (RSI). In RSI, AI systems improve the software that builds and serves AI. How does ERNIE Evolve work? The system runs 2 nested loops. The outer loop decides what to investigate. The inner loop develops and tests improvements inside each direction. Director: picks the next research direction and a starting code version. It reads an Experience Bank of past results and a Knowledge Base of papers and code. Selector: chooses a candidate from a lane’s tree and names the next hypothesis to test. Executor: implements that plan in an isolated environment