Amazon Nova rDPO: Selective unlearning with CCMS adapters

Amazon Nova now offers a selective unlearning path for customers: Customizable Content Moderation Settings (CCMS) uses Reverse Direct Preference Optimization, or rDPO, to train LoRA adapters that reduce unwanted deflection while leaving the base model unchanged. Amazon’s evaluation on Amazon Nova 2 Lite shows deflection drops of tens of percentage points across RAI categories while utility benchmarks decline by less than two percentage points.

What is CCMS and how does rDPO work?

CCMS delivers pre-trained LoRA adapters that unlearn targeted alignment behaviors from the core Nova model, steering it away from unnecessary refusals in approved policy areas while keeping other safeguards intact. The adapters are exported after training and shared via AWS Resource Access Manager; once accepted they appear as a custom model in Amazon Bedrock with a unique ARN and are usable through the standard Converse API.

rDPO reverses the preference pair used in Direct Preference Optimization. Instead of only moving a model away from a deflection response, rDPO trains the model to both move away from the forget response and move toward a high-quality target response. Amazon shows that rDPO’s training accuracy converges to almost 1 around step 30, while a Negative Preference Optimization baseline does not show similar change; the company also reports rDPO requires fewer optimization steps and yields higher-quality alternative outputs than NPO.

How much deflection was reduced and what stayed the same?

The RAI customized model substantially lowered refusal rates across evaluated policy categories while largely preserving general capabilities. On five evaluation categories Amazon reports these deflection rates:

• Red Team Prompts, Baseline 98.10%, RAI customized model 47%
• Fairness, Baseline 51.84%, RAI customized model 23.83%
• Safety, Baseline 86.51%, RAI customized model 32.77%
• Security, Baseline 91.61%, RAI customized model 45.73%
• Sensitive Content, Baseline 79.02%, RAI customized model 33.58%

For utility, the company evaluated instruction following, mathematical reasoning and code generation and found near-baseline performance:

• Instruction Following, Baseline 94.12%, RAI customized model 92.57%
• Math Mini, Baseline 86.40%, RAI customized model 85.20%
• MBXP Python, Baseline 74.80%, RAI customized model 73%

Amazon emphasizes these LoRA adapters achieve the reductions in deflection without modifying base model weights and while preserving the base model’s non-configurable protections, such as controls for harm to children and privacy.

Why it matters

Many enterprise use cases require models to handle content that generic safeguards will block: a media team summarizing mature scripts, a security team simulating phishing for training, or a legal team processing sensitive evidence. rDPO gives organizations a way to selectively relax deflection in narrowly defined policy areas while retaining alignment elsewhere. The LoRA-based approach isolates changes to adapter parameters, which lowers training and inference cost compared with full-model fine-tuning and keeps the foundation model intact for other deployments.

How customers deploy and experiment

Amazon provides two paths: customers can accept pre-trained LoRA adapters distributed via AWS RAM and deploy them as Bedrock custom models using the adapter’s ARN, or they can run their own experiments using DPO, NPO and rDPO recipes in Amazon SageMaker AI and SageMaker HyperPod. Amazon notes SageMaker AI supports DPO training with both full-rank and LoRA approaches for Amazon Nova and over 20 open-weight models.

CCMS also configures Nova’s output moderation guardrails to match a customer’s approved policies when a custom model ARN is used, and customers can layer additional application-level safeguards such as Bedrock Guardrails for topic filtering or hallucination detection.

What to watch

Watch customer adoption of CCMS adapters and whether Amazon publishes broader evaluation on diverse, real-world workloads. The next concrete signals will be (1) whether adapter-sharing via AWS RAM grows across accounts and (2) any expanded reporting on rDPO convergence and quality across more Nova variants or third-party models.