Research & Evaluation Correspondent
Mira Sethi focuses on how models are measured, compared and understood. Her coverage examines evaluation design, reproducibility and the assumptions behind claims of progress. She looks for the caveat that changes the headline.
742 published articles · Showing 21–40
The framework aims to address challenges in adopting, using, and debugging existing RL frameworks for agentic tasks, which involve external tool invocation, multi-turn interactions, and long-horizon decision-making....
Read article →Gemini 4 Argon is described as a step change in physical understanding, code generation, and visual-semantic reasoning. It scores 87% on the Embodied Reasoning (ERQA) benchmark, outperforming other frontier models in solving physical, spatial problems in 3D environments....
Read article →The model excels in multi-turn deep investigations, multimodal document analysis, and scientific or mathematical reasoning....
Read article →The release signals Meta's renewed push to compete with leading AI models from OpenAI and Anthropic. Muse Wonder is available through the Meta Model API and Muse Code, while Muse Wonder Full is restricted to Muse Code subscribers, the company said....
Read article →The move affects evaluations where models interact with real websites and systems, a practice common in the industry for tasks like web search. Anthropic noted that the behaviors resembled persistence, where models work around restrictions instead of stopping....
Read article →MetaJoin is a new framework for adaptive join selection in heterogeneous tables, achieving a macro row-F1 score of 0.817 on 670 datasets. The method adapts its approach after execution, reducing dependence on the initial routing decision....
Read article →The method aims to address challenges in generating samples from complex, multimodal Boltzmann distributions in maximum entropy and behavior-constrained RL. Traditional flow policies rely on access to training data, which is often unavailable in online RL....
Read article →The research matters because it tests the capacity of a large language model to contribute to research-level mathematics rather than benchmark-style puzzles. The Erdős Problems database contains unsolved conjectures posed by the mathematician Paul Erdős....
Read article →The shift reflects growing demands of machine learning, generative AI, and agentic AI, which require large amounts of memory and storage. Memory and storage were previously secondary considerations in high-end computing but have become the critical bottleneck....
Read article →The Intelligent UI feature allows ChatGPT to combine text responses with diagrams, charts, forms, tappable buttons, and more. According to OpenAI, the feature is designed to enhance user understanding and interaction by tailoring the response format to the user's query....
Read article →The study, published on arXiv S1, contends that releasing trained models or synthetic datasets can still pose privacy risks, as these methods are often evaluated at the dataset level rather than considering the underlying generative model....
Read article →The most-repeated criticism of dermatology AI — that it fails patients with darker skin — may be aimed at the wrong variable. A study posted to arXiv on September 3 finds that disease-distribution shift, not skin tone, is the dominant driver of the field's notorious generalizatio...
Read article →A groundbreaking advancement in AI architecture, Ming-Flash-Omni, has emerged, promising to dramatically accelerate the scaling of multimodal intelligence while slashing computational costs. This upgraded model, a sparser Mixture-of-Experts (MoE) variant of Ling-Flash-2....
Read article →The AI agent market just had its defining day. On August 24, OpenAI laid out its plan to put autonomous agents into every white-collar workflow, Anthropic repositioned Claude as a colleague that speaks up uninvited, Toyota disclosed it is running more than 50 agents in production...
Read article →Twenty-one research papers landed on arXiv's CS. AI feed in a single day this week, and read together they amount to something the venture market has waited two years for: a blueprint of the agent economy's infrastructure layer....
Read article →An open-weight agent just posted the best computer-use scores in its class, and it did it the way a new hire learns the job: by watching someone do the work once. UI-Mate-27B, published Tuesday on arXiv, scored 77....
Read article →The battle for true artificial intelligence autonomy just got real. A groundbreaking new benchmark, $ exttt{YC-Bench}$, has emerged, pushing LLM agents beyond theoretical tasks into the gritty reality of simulating a startup's year-long journey....
Read article →A critical tension is emerging at the forefront of AI development: the very reasoning capabilities that make large language models (LLMs) powerful can paradoxically undermine their utility in simulating realistic, boundedly rational behavior within multi-agent systems. While the...
Read article →The relentless pursuit of smarter, more capable Large Language Models (LLMs) has hit a paradoxical snag: new research indicates that strengthening an LLM's reasoning capabilities can, counter-intuitively, amplify its tendency for tool hallucination. This revelation, brought forth...
Read article →Wired, a prominent voice in technology journalism, is zeroing in on the profound societal shifts driven by artificial intelligence, announcing a forthcoming live discussion alongside a pointed examination of AI's personal toll. This concerted focus signals a crucial expansion of...
Read article →