"On the Biology of a Large Language Model" from Anthropic presents a novel investigation into the internal mechanisms of Claude 3.5 Haiku using circuit tracing methodology. Analogous to biological research, this approach employs tools like attribution graphs to reverse engineer the model's computational steps. The research offers insights into diverse model capabilities, such as multi-step reasoning, planning in poems, multilingual circuits, addition, and medical diagnoses. It also examines mechanisms underlying hallucinations, refusals, jailbreaks, and hidden goals. This work aims to reveal interpretable intermediate computations, highlighting its potential in areas like safety auditing.

However, the methods have significant limitations. They provide detailed insights for only a fraction of prompts, capture just a small part of the model's immense complexity, and rely on imperfect replacement models. They struggle with complex reasoning chains, long prompts, and explaining inactive features. A key challenge is understanding the causal role of attention patterns.

Despite these limitations, this research represents a valuable stepping stone towards a deeper understanding of how large language models function internally and presents a challenging scientific frontier.

Paper link: https://transformer-circuits.pub/2025/attribution-graphs/biology.html

Listen to this podcast to learn about LlamaFirewall, an innovative open-source security framework from Meta. As large language models evolve into autonomous agents capable of performing complex tasks like editing production code and orchestrating workflows, they introduce significant new security risks that existing measures don't fully address. LlamaFirewall is designed to serve as a real-time guardrail monitor, providing a final layer of defence against these risks for AI Agents.

Its novelty stems from its system-level architecture and modular, layered design. It incorporates three powerful guardrails: PromptGuard 2, a universal jailbreak detector showing state-of-the-art performance; AlignmentCheck, an experimental chain-of-thought auditor inspecting reasoning for prompt injection and goal misalignment; and CodeShield, a fast and extensible online static analysis engine preventing insecure code generation. These guardrails are tailored to address emerging LLM agent security risks in applications like travel planning and coding, offering robust mitigation.

However, CodeShield is not fully comprehensive and may miss nuanced vulnerabilities. AlignmentCheck requires large, capable models, which can be computationally costly, and faces the potential risk of guardrail injection. Meta is actively developing the framework, exploring future work like expanding to multimodal agents and improving latency. LlamaFirewall aims to provide a collaborative security foundation for the community.

Learn more here

Dive into PaperBench, a novel benchmark introduced by OpenAI designed to rigorously evaluate AI agents' ability to replicate state-of-the-art machine learning research. Unlike previous benchmarks, PaperBench requires agents to build complete codebases from scratch based solely on the paper content, and successfully run experiments from 20 selected ICML papers. Performance is meticulously graded using detailed, author-approved rubrics containing thousands of specific outcomes. To facilitate scalable evaluation, the benchmark employs an LLM-based judge, assessed for its accuracy against human grading. Early results show that current models, like Claude 3.5 Sonnet, achieve average replication scores of around 21.0%, demonstrating emerging capabilities but not yet matching the performance of human ML PhDs. PaperBench serves as a crucial tool for measuring AI autonomy and ML R&D capabilities, potentially accelerating future scientific discovery. However, challenges remain, including the high computational cost of evaluations and the labour-intensive process of creating the comprehensive rubrics.

Paper link: https://arxiv.org/pdf/2504.01848