Import AI 479: Spacewar; LLM-এর তুলনায় নিম্ন-ল্যাটেন্সি কাজ; Genie 4

এটি ফিডে পাওয়া অংশবিশেষ। এটি সংক্ষিপ্ত বা প্রকাশকের মূল ভাষায় হতে পারে।

Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe.Subscribe nowLLMs are very bad at low-latency tasks:…Realtime-Bench says it’ll be a while till AI systems can play games or do online auctions for you…Researchers have built Realtime-Bench, “a suite of carefully curated dynamic tasks where delayed actions directly affect success”. The test helps evaluate how well modern, general-purpose AI systems can perform tasks where decisions need to be made in real-time. The benchmark consists of 50 interactive game and web tasks, split across 30 web tasks in four categories (travel, e-commerce, service, and security), and 20 game tasks (arcade, puzzle, strategy, and simulation).Who did the research: The benchmark was built by people at the University of Texas at Austin, Ohio State University, University of Illinois at Urbana-Champaign, NVIDIA, Northwestern University, and Steadyworks.What it measures: The test is designed to probe for three qualities: temporal state awareness (”RT-Perceive”), where models can use observations to track changes in the world; adaptive deliberation (”RT-Deliberate”), where models alter their reasoning length and depth according to available time; and time-aware action selection (”RT-Act”) where models need to appropriately time their actions for the situation. “For example, an agent purchasing a ticket must track the sale status and assess whether observed availability is still current (RT-Perceive), limit further deliberation when the purchase window is closing (RT-Deliberate), and choose between waiting for sales to open and purchasing while tickets remain available (RT-Act).” They have three different conditions: Wall clock, which is designed to test out performance under real-world latency. Paused clock, which lets you arbitrarily pause things so you can get a sense of baseline capability, and Token clock, which “advances