Underwriting Study: Data Clarity, Randomness, and Crash Predictability

Do clearer data pools behave more randomly and medium pools carry stronger underwriting signal?

Published: February 2026 • Year 1 Features (2024) to Year 2 Target (2025) • 1.67M DOT+VIN power units

Executive Summary

This study tests an underwriting premise: in pools where losses are mostly random, pricing discipline matters more than risk selection; where outcomes are less random, underwriting selection adds more value.

Using repeated stratified cross-validation (3 folds x 2 repeats), the high data-clarity pool shows the lowest predictability and lowest variance. The medium pool shows stronger stable signal. The low pool has good mean discrimination but materially higher instability, consistent with a signal-plus-noise mix.

Hazmat cohorts are directionally consistent with a lower-frequency, more random-like profile in this dataset: lower crash rates, lower violation intensity, and lower AUC than non-hazmat peers.

0.5626

High Pool AUC Mean

Variance: 0.00015467

0.6256

Medium Pool AUC Mean

Variance: 0.00076580

0.6314

Low Pool AUC Mean

Variance: 0.00078219

1. Premise and Design

Premise: underwriting value comes from identifying pools where losses are less random and explainable by observable characteristics. Pools that remain random are better handled by pricing and portfolio controls.

Entity: DOT|VIN power-unit records.

Feature window: 2024 events and exposure variables. Target window: binary crash indicator in 2025.

Pool assignment: high, medium, low data-resolution from data/processed/data_resolution_pool_assignments.csv.

Validation: repeated stratified CV, 3 folds x 2 repeats, random seed 42.

2. Pool-Level Results

Pool Rows Positives Positive Rate AUC Mean AUC Std AUC Variance
high 670,842 18,477 2.754% 0.562638 0.012437 0.00015467
medium 755,762 15,598 2.064% 0.625564 0.027673 0.00076580
low 245,035 4,061 1.657% 0.631393 0.027968 0.00078219

Figure 1 — AUC Mean by Pool

Higher AUC indicates stronger crash discrimination

Figure 2 — AUC Variance by Pool

Lower variance indicates more stable fold-to-fold performance

3. Hazmat vs Non-Hazmat Within Each Pool

Hazmat grouping is defined as smsc_hm_flag >= 0.5. In all pools, hazmat groups show lower mean AUC and often higher instability.

Group Rows Positives Positive Rate AUC Mean AUC Std AUC Variance
high_hazmat 120,149 2,954 2.459% 0.531378 0.006989 0.00004885
high_non_hazmat 550,693 15,523 2.819% 0.563460 0.009381 0.00008800
medium_hazmat 51,267 672 1.311% 0.573895 0.032147 0.00103344
medium_non_hazmat 704,495 14,926 2.119% 0.630429 0.014244 0.00020288
low_hazmat 1,662 18 1.083% 0.571244 0.046406 0.00215352
low_non_hazmat 243,373 4,043 1.661% 0.625894 0.020490 0.00041984

Figure 3 — Hazmat Penalty by Pool

Each pair compares hazmat and non-hazmat AUC in the same data-resolution pool

4. Hazmat Directional Safety Signal

Using the same Year-1 to Year-2 modeling dataset, hazmat shows lower crash frequency and lower violation intensity overall. This supports a reasonable directional assumption that hazmat is safer on average in this dataset, while still being harder to separate with current features.

2.105%

Hazmat Crash Rate

2.302%

Non-Hazmat Crash Rate

0.683

Hazmat Viol/Inspection

1.139

Non-Hazmat Viol/Inspection

Group Rows Positives Crash Rate Mean Viol/Unit Mean Viol/Inspection
hazmat 173,078 3,644 2.105% 1.2515 0.6833
non_hazmat 1,498,561 34,492 2.302% 1.8801 1.1394

Directional deltas (hazmat vs non-hazmat): crash rate is about 8.5% lower, mean violations per unit is about 33.4% lower, and mean violations per inspection is about 40.0% lower.

Predictability is also lower for hazmat (weighted AUC about 0.544 vs 0.605), consistent with more random-like outcomes in the current feature set.

5. Findings Against the Premise

Statement 1: High data-clarity should behave more random-like. Supported. High pool has the weakest mean discrimination and the lowest fold variance, consistent with stable but low-explainability outcomes.

Statement 2: Medium data-clarity should carry stronger underwriting signal. Supported. Medium pool AUC is materially higher than high pool with acceptable stability.

Statement 3: Low data-clarity should show signal mixed with noise. Partially supported. Mean AUC is high, but fold variance is also high, indicating unstable signal quality and more sensitivity to noise.

Hazmat refinement: Hazmat groups are consistently harder to model than non-hazmat groups, but also show lower observed crash and violation rates. This supports your directional conclusion of a safer and more random-like hazmat profile in this dataset. Low-hazmat remains unstable due to small sample size.

6. Underwriting Implications

  1. High clarity pools: focus on pricing discipline and portfolio controls; incremental feature-based selection power is limited in this feature set.
  2. Medium clarity pools: strongest candidate for underwriting segmentation and rule-driven risk selection.
  3. Low clarity pools: apply conservative pricing and data-quality gates to mitigate noise risk.
  4. Hazmat sub-pools: treat lower baseline frequency and weaker model separation as a directional pricing input; still develop dedicated hazmat features before relying on fine-grained hazard segmentation.

Reproducibility

All tables and metrics on this page were sourced directly from generated artifacts in the repository.