Many Models, Many Policies: Aligning AI to Custom, Written Policies

Beyond a few general-purpose models (aligned once and served to everyone) a growing share of AI is deployed as specialized models, adapted by hospitals, banks, schools, platforms, and governments, each with its own requirements. Yet alignment remains a bespoke practice, re-engineered for every setting: it rarely starts from a policy written by those who deploy the model, yielding a model that follows that policy to the letter. It starts from data instead: preference labels, curated datasets, lists of forbidden concepts. As a result, the policy a model ends up following is implicit, hard to inspect, and expensive to change. In this talk, I argue for alignment that starts from that explicit, written policy, and I organize the problem around three questions. How do we build a model aligned to a given policy? I will show how Constitutional AI turns a written constitution into a training signal, and discuss what it takes to extend it to small language models and to modalities beyond text. How do we evaluate it? When the policy changes, so does the definition of the task, and static benchmarks end up measuring someone else’s policy. When and why does it fail? I will show how alignment can quietly erode even without malicious intent, and propose ways to detect and study these failures.

Oct 22, 2026

CCSAD Cybersecurity Seminar