Case study

DiffusionOOD: Class-Conditional Separation Loss for Out-of-Distribution Detection

Conditional diffusion models used as generative classifiers for OOD detection, with separation loss reaching 99.03% ± 0.07% AUROC on a within-CIFAR airplane-vs-rest binary split (single ID class).

Problem

Standard neural classifiers exhibit dangerous overconfidence on out-of-distribution samples. Generative classifiers offer principled reconstruction error scoring, but suffer from catastrophic variance across random seeds and ambiguous boundary decisions.

Role

Master thesis author: novel separation loss objective formulation, PyTorch Lightning & Hydra infrastructure, multi-seed statistical evaluation, and benchmark analysis.

Architecture & Pipeline Summary

PyTorchPyTorch LightningHydraDDPM / UNetW&BCIFAR-10

1. Class-Conditional Denoising

Reconstruct inputs under competing in-distribution hypothesis conditions across forward diffusion timesteps.

2. Separation Objective

Explicitly penalize similarity between competing noise predictions with calibrated weight lambda=0.02.

3. Statistical Anomaly Scoring

Compute difference-of-reconstruction error as a calibrated OOD score tested over 3 random seeds.

Approach

Train a binary conditional diffusion model and score reconstruction error under competing class conditions.
Add class-conditional separation loss to push conditional noise predictions apart and remove model ambiguity.
Evaluate with multiple random seeds and external zero-shot benchmarks instead of relying on one lucky run.

Results

99.03% ± 0.07% AUROC averaged over three seeds on a within-CIFAR airplane-vs-rest binary split (single ID class; not comparable to multi-class CIFAR-10 OOD benchmarks).
+6.5 percentage points gain over the non-separated baseline (which achieved 92.52% ± 11.07% AUROC at lambda=0).
Seed-42 generalized zero-shot to CIFAR-100 (96.97%), Places365 (96.50%), FashionMNIST (94.03%), Textures (92.84%), and SVHN (90.50%).

// INTERACTIVE THESIS LAB · JKU LINZ (GRADE 1.0)

Conditional Diffusion as a Generative Classifier: Interactive Evidence Explorer

Built directly from the raw PyTorch score tensors (seed42_*.pt) and ablation JSON artefacts on Hugging Face (ahmed-3m/DiffusionOOD). Explore how the class-conditional separation loss (Eq. 3.23), contrastive noise-error scoring (Eq. 3.3), Monte Carlo trial budget K, and industrial transfer behave.

Eq. 3.23 (Sep. Loss): L_sep = −‖ε_θ(x_t, t, c=0) − ε_θ(x_t, t, c=1)‖²

Eq. 3.3 (OOD Score): s_OOD(x) = ê₀(x) − ê₁(x) > τ ⇒ OOD

Training Objective Regime (Eq. 3.24: L = L_diffusion + λ · L_sep)

Evaluation Benchmark (Table 6.1 · Seed 42 Raw Score Tensors, K = 100)

Decision Threshold τ on s_OOD(x) = ê₀(x) − ê₁(x): τ = +0.00061

AUROC (Seed 42 · K=100)

98.98%

3-seed: 99.03% ± 0.07%

Live TPR(τ) · Sensitivity

95.0%

TP: 8,546 / 9,000 OOD

Live FPR(τ) · False Alarms

4.9%

FP: 49 / 1,000 Airplanes

Youden's J(τ) = TPR − FPR

90.1%

Balanced Acc: 95.0%

Empirical Score Distribution s_OOD(x) = ê₀(x) − ê₁(x) (48 bins from .pt tensors)

■ ID: CIFAR-10 Airplane (N=1,000)■ OOD Cohort (N=9,000)┆ Threshold τ

Empirical ROC Curve & Live Operating Point (FPR(τ), TPR(τ))

Operating Point: FPR = 4.9%, TPR = 95.0% (Benchmark FPR@95%TPR = 4.7%)

Lessons

Stability across random seeds matters almost as much as peak score in safety-critical deployments.
A scientific finding becomes significantly more useful when variance and error bars are reported honestly.
Cross-domain transfer should be tested empirically, not assumed from synthetic benchmarks.

Artifacts

Thesis PDF DiffusionOOD repository