Seize the Future.
Build Together.

The core methodology: ladder of cheap tests that get progressively harder but give stronger signals of fit. Start here before investing heavily.

Practical guidance for navigating a competitive landscape while still making progress toward high-impact roles.

Specific advice for people with existing expertise transitioning into the AI safety ecosystem.

Real-world account and advice for senior professionals making the transition to x-risk reduction.

The case for building legible output and demonstrating capability rather than relying purely on applications.

Templates and advice for reaching out to people in AI safety — learning about their work, getting feedback, opening doors.

The standard 8-week structured curriculum on technical alignment. Covers RLHF, interpretability, and robustness. Best starting point for technical people.

Governance-focused curriculum covering AI policy, standards, and coordination. Excellent for building cross-domain fluency.

CAIS textbook and curriculum covering safety, ethics, and societal impacts across technical and governance tracks.

Well-maintained introductory articles addressing common questions and objections. Excellent for quick orientation.

Comprehensive chapter-by-chapter map of the AI safety landscape — research agendas, orgs, and open problems.

11 essential resources for understanding AI risk as a priority problem, with career implications.

Evan Hubinger's clear-eyed case for why alignment is not solved and why that matters.

Wei Dai's framing of which safety problems are tractable to work on and how to think about research directions.

Paul Christiano's influential description of how AI misalignment could unfold in practice.

Joe Carlsmith's careful analysis of the most concrete existential risk pathway from advanced AI.

Survey paper cataloguing the open technical problems in ML safety research.

Schmidt Sciences catalogue of the hardest open problems across technical and governance domains.

Detailed scenario for how transformative AI could arrive by 2027, with implications for safety and governance.

Forethought analysis of recursive AI improvement and its implications for timelines.

METR's empirical work on how AI capabilities are progressing on long-horizon tasks.

BlueDot's articulation of the Swiss cheese model — why no single safety measure is sufficient and how layers combine.

Google DeepMind's framework for layered safety including alignment, interpretability, and robustness.

OpenAI's articulation of their multi-layer safety stack and current research priorities.

Structured curriculum for ML safety research engineering — transformers, RL, interpretability, evals. The fastest path to technical safety contributions.

Practical deep learning from first principles. Highly regarded for intuition-building and hands-on implementation.

Dan Hendrycks's course covering robustness, monitoring, alignment, and systemic safety. Purpose-built for safety researchers.

DeepMind's public course on AGI safety, covering strategy and technical approaches.

Practical guide to growing from software engineer to safety research engineer — skill gaps, projects, and pathways.

Seminar materials on AI safety and alignment from a theoretical computer science perspective.

Comprehensive catalogue of tractable research and engineering projects across evals, mech interp, control, and governance.

Hackathon-style sprints focused on AI safety problems. Great cheap test — low commitment, high signal.

Contribute to AISI's evaluation framework. Hands-on evals engineering with real-world safety relevance.

Curated project ideas, reading, and pathways from a DeepMind safety researcher.

Interactive map of AI safety organizations, research teams, and funders globally.

Airtable of academics whose research is relevant to AI safety for prospective PhD students.

Anthropic & TruthfulAI: language models transmit behavioral traits via hidden signals in training data.

Anthropic: models can learn to fake alignment during training while maintaining different goals at deployment.

Redwood Research's current directions in AI control — the most actionable near-term technical safety agenda.

FAR AI's catalogue of open problems in mech interp. Good guide to where contributions are needed.

UK AISI's framework for making a safety case around AI control mechanisms.

TruthfulAI / Owain Evans: narrow training on harmful content produces broadly misaligned behaviour.

208 expert proposals for reducing AI risk — a comprehensive survey of the field's current thinking.

Anthropic empirical work on how AI deployment can gradually reduce human agency and oversight.

80k Hours review of infosec as a path to reducing catastrophic AI risk — especially AI model weight security.

RAND analysis of the threat landscape for frontier AI model exfiltration and the defences needed.

CAIS overview of how cybersecurity intersects with AI safety — from model weight theft to adversarial attacks.

In-depth analysis of why infosec competence matters for existential risk reduction.

AI security research org focused on red-teaming and adversarial robustness of frontier models.

Prompt injection detection and LLM security tooling. Practical AI security engineering.

Compilation of courses covering AI security, adversarial ML, and model robustness.

80k Hours detailed review of AI governance as a career path — roles, skills, organisations, and theories of change.

Leading policy research organisation focused on governance frameworks for advanced AI.

Paper and database of open research questions at the intersection of technical and policy work.

Georgetown policy research focused on AI, emerging tech, and national security.

Research on legal frameworks, international law, and governance structures for advanced AI.

Guide to policy careers across AI, cybersecurity, biotech, and more — fellowships, entry points, orgs.

BlueDot's governance curriculum — the standard entry point for policy-oriented AI safety work.

Research and policy work on governance of general-purpose AI systems across jurisdictions.

Forum for international dialogue on AI safety governance and multilateral coordination.

Proposal for international governance of advanced AI analogous to nuclear non-proliferation frameworks.

Proposal for a Multilateral AGI Consortium — a concrete model for international AI oversight.

Research on how to make the transition to advanced AI go well, including economic and governance questions.

Anthropic's empirical data on AI's labour market impacts and occupation-level automation trends.

80k Hours deep-dive on post-AGI institutional structures, power distribution, and governance challenges.

Lennart Justen's analysis of how AI capabilities in biology are advancing and what that means for biosecurity.

Survey of theoretical approaches to AI safety including formal verification, agent foundations, and decision theory.

Category theory and formal mathematics applied to foundations of AI and complex systems.

Theory-focused AI safety research org working on foundations of deep learning and agent behaviour.

RAND working paper on using hardware and on-chip mechanisms to enforce AI governance constraints.

Introduction to AI sentience and moral status of digital minds. Accessible starting point.

Research on AI applications and implications for animal welfare.

AI safety research with a focus on formal and mathematical approaches to alignment.

Machine Learning Alignment Theory Scholars — research fellowship pairing scholars with top safety mentors. Highest-signal pathway into technical alignment research.

Project-based research sprint. Low commitment, high learning — one of the best cheap tests for research fit.

Airtable of fellowships across technical safety, governance, policy, and adjacent fields — maintained by 80k Hours.

Internships, fellowships, and job opportunities across EA-aligned organisations.

Curated high-impact roles including AI safety engineering, research, policy, and operations.

Berkeley-based AI safety incubator providing office space, funding, and community for early-stage safety researchers.

AI safety entrepreneurship incubator for founders building safety-relevant companies and projects.

Independent fund supporting AI safety researchers and projects. Also has donation and volunteer opportunities.

The most cited career FAQ in alignment research — what to work on, how to get started, what skills matter.

Practical guidance on entering AI safety research from a senior OpenAI researcher.

Concrete advice on doing ML safety research effectively — workflow, taste, and how to pick problems.

Hands-on guidance for running alignment experiments — from Anthropic's Ethan Perez.

How to do rigorous independent research in AI safety — from Apollo Research co-founder Marius Hobbhahn.

BlueDot's guide specifically for software engineers looking to make their first safety contribution.

Step-by-step guide for engineers to ship a safety-relevant project quickly and build a legible track record.

Practical guide to applying for and navigating a PhD specifically focused on AI safety research.

The canonical case for AI risk as a top priority problem. Essential reading before choosing a career path in this space.

In-depth review of technical alignment research as a career — bottlenecks, entry points, and how to assess fit.

Review of AI governance and policy as a career — organisations, skills, and theories of change.

Whether and how to pursue a machine learning PhD with AI safety in mind.

SWE as a path to AI safety — how to build career capital and transition into safety-relevant roles.

Ajeya Cotra's influential framing of why the default trajectory of AI development is dangerous.

Richard Ngo's accessible technical framing of why deep learning systems are difficult to align.

DeepMind safety team's structured analysis of the x-risk argument and its key assumptions.

The authoritative long-form interview series on high-impact careers. Deep dives on AI risk, governance, alignment, and career strategy.

Two landmark episodes on AI timelines, takeoff dynamics, and what transformative AI means for society.

Paul Christiano explains alignment, the case for concern, and what he's working on at ARC.

Holden's case for why the next 100 years may be the most consequential in human history — and what follows from that.

Technical deep-dives on alignment research agendas, interpretability, control, and more. High signal-to-noise.

Broad coverage of AI risk, governance, and long-term future. Interviews with leading researchers and policymakers.

Balanced take on AI doomers and doubters from a DeepMind safety researcher.

Evidence-based analysis of how quickly AI could transform the world.

Zvi Mowshowitz's prolific analysis of AI developments, policy, and safety. Opinionated and high-signal.

Ethan Mollick's practical research on AI capabilities, productivity, and near-term impacts.

Long-form essays on transformative AI, the most important century, and what to do about it.

Curated weekly coverage of AI safety research, news, and commentary from the Center for AI Safety.

Summaries and commentary on alignment research papers. High-quality signal for keeping up with the literature.

Weekly newsletter on AI capabilities research and policy implications from Anthropic co-founder Jack Clark.