Resources

Understanding the crucial role that third-party evaluators play in promoting trusted adoption of AI, the International Network for Advanced AI Measurement, Evaluation and Science (NAAIMES) has released its first best practice guidance document, which aims to support the growing ecosystem of third-party evaluators. The guidance document builds on and complements existing best practice publications, while providing recommendations to third-party evaluators, such as capability elicitation.

While much of AI evaluation research focuses on models, users experience AI through complete systems—applications layered with retrieval, guardrails, tools, and autonomous capabilities. This changes how systems should be tested. In this post, part of a multi-blog series from the NAAIMES Network, rigorous AI system evaluation is explored: realistic testing environments, representative datasets, appropriate metrics, and meaningful thresholds to assess whether AI systems are truly fit-for-purpose and safe for real-world deployment. This blogpost introduces the technical, operational, and domain expertise that evaluators need to develop to conduct effective AI system assessments.

This blogpost by Scale AI outlines its collaboration with Singapore’s IMDA on AI evaluation research, beginning with a large-scale legal benchmark for Singapore law. The benchmark will assess whether AI models understand Singapore law, identify missing legally relevant facts, ask useful clarification questions, and respond to realistic legal scenarios grounded in Singapore law. The post also notes that Scale AI and IMDA are exploring broader technical research and information sharing, including model evaluation methods, tools, benchmarks, and multilingual AI safety.

The Singapore AI Safety Red Teaming Challenge 2026 convened participants from 14 countries across Asia-Pacific to test Gen AI apps for data leakage risks in English and regional languages, to understand how these risks manifest across linguistic contexts. The report summarises key challenge takeaways, including inconsistent security behaviour and inadequate multilingual protections across Gen AI apps.

An outcome of the 2026 International Scientific Exchange on AI Safety event, the 2026 Singapore Consensus builds on the 2025 report to present an updated global understanding on scientific research problems that are of top priority to advance AI safety research. The report outlines key developments in the AI risk landscape over the past year, such as the growth of misuse and incidents. Recognising the increasing deployment and capabilities of autonomous AI agents, the 2026 Singapore Consensus also provides a deep dive into risk management for the development and deployment of autonomous agents, which is detailed in the Companion Report on Agentic Risk Management.

AI agents are increasingly granted access to sensitive information and external tools to perform tasks on behalf of users, increasing data leakage risks. This report presents a joint evaluation by the SG and KR AISI examining whether agents can complete routine workflows without leaking data. Across 12 realistic tasks spanning areas such as enterprise productivity and customer service, the evaluation finds that agents may successfully complete tasks while still exhibiting data-handling failures, suggesting that task correctness alone is insufficient to assess agent safety. This report is a more detailed follow-up to an earlier blog post on this topic.