Rule Violation Score (RVS): Measuring logical compliance

Guillaume Olivier Delplanque and coauthors submitted "Beyond Accuracy: Measuring Logical Compliance of Predictive Models" to arXiv on 18 Jun 2026, and introduced the Rule Violation Score, or RVS, as a complementary metric that quantifies how much a predictive model respects a given set of logical rules, independently of its predictive accuracy.

What is the Rule Violation Score (RVS)?

RVS is a complementary evaluation metric that measures the extent to which a predictive model respects a specified set of logical rules, treating hard rules and soft rules differently. The paper describes RVS as applicable to any dataset and any predictive model expressed over a relational vocabulary, and notes it can be computed using SQL queries that are automatically generated for Horn rules.

RVS is positioned as orthogonal to standard predictive performance metrics such as ranking quality, prediction error, or classification accuracy. The authors argue that while those metrics measure closeness to ground truth, they do not assess whether outputs satisfy domain or logical constraints, which RVS explicitly quantifies.

How did the authors evaluate RVS?

The authors evaluated RVS on three benchmarks covering knowledge graph link prediction and relational regression, and compared rule-based, embedding-based, and neuro-symbolic predictive models. Their experiments show that two models with comparable predictive accuracy can exhibit substantially different levels of logical compliance, revealing behavioral differences that standard metrics miss.

The paper also reports that RVS can be applied beyond model evaluation: it can evaluate the logical consistency of training datasets and help identify poorly defined rules. For Horn rules the method includes automatic generation of SQL queries, allowing RVS to be computed on relational representations.

Why it matters

Logical consistency can be as critical as predictive accuracy in high-stakes domains such as healthcare, finance, and autonomous systems, the authors note. A model that attains strong accuracy but violates domain constraints may produce outputs that are unsafe, legally problematic, or simply unusable in constrained workflows. RVS offers a concrete way to surface those violations and compare models on a dimension that standard metrics ignore.

RVS also shifts some evaluation effort earlier: by flagging inconsistent training data or ill-defined rules, it helps teams refine datasets and rule sets before deployment. That makes the metric relevant both for model selection and for data curation.

What to watch

Look for follow-up work that publishes RVS implementations, benchmarks, or tooling that generates the SQL for Horn rules at scale, and for papers that report logical compliance alongside accuracy on established leaderboards. Evidence that communities adopt RVS as a standard complementary metric would confirm the paper's premise.

References and key facts drawn from the arXiv submission: paper title "Beyond Accuracy: Measuring Logical Compliance of Predictive Models", authors led by Guillaume Olivier Delplanque, submission date 18 Jun 2026, RVS computed via SQL for Horn rules, evaluation on three benchmarks spanning knowledge graph link prediction and relational regression, and comparisons across rule-based, embedding-based, and neuro-symbolic models.