Research Study

Key Findings

Predictive Validation (Year 1 → Year 2)

Methodology

We analyzed FMCSA Safety Measurement System (SMS) violation data, linking specific violation types (Group_Desc) to carrier crash records. Crash rates were calculated using Empirical Bayes adjustment to account for exposure differences between carriers.

Data Sources

• Violations: SMS_Input_Violation file (6,548,807 records)
• Crashes: SMS_Input_Crash file
• Carrier Census: FMCSA Motor Carrier Census
• Inspections: Roadside inspection records

Statistical Approach

For each violation type, we calculated:

• Relative Risk: Crash rate for carriers with the violation type divided by crash rate for carriers without
• Crash Probability: Percentage of carriers in each group that experienced at least one crash
• Dose-Response: Crash metrics stratified by violation count (0, 1, 2, 3-4, 5-9, 10+)

Relative Risk by Violation Type

The following table shows crash rate relative risk for carriers with each violation type compared to carriers without that violation type.

Dose-Response Analysis

A critical finding is the clear dose-response relationship: as the number of violations increases, crash probability increases monotonically.

Speeding Violations

Key insight: Carriers with 10+ speeding violations have a 132x higher fatality rate (0.5963 vs 0.0045) compared to carriers with zero speeding violations.

Dangerous Driving Violations

Behavioral vs Equipment Violations

A key finding is that driver behavioral violations are significantly stronger predictors of crash likelihood than equipment/vehicle violations.

Statistical Significance

All findings are statistically significant at p < 0.001 given the large sample sizes. Correlation coefficients between violation metrics and crash rates:

Implementation Results

Based on the findings above, we implemented a behavioral caps system that applies score limits to carriers with critical behavioral violations. The results validate the predictive value of violation-type analysis.

Caps Applied

The following score caps were implemented based on specific violation types:

Validation: Crash Probability by Behavioral Flag

Carriers flagged with behavioral violations show dramatically higher crash probabilities, validating the predictive power of violation-type analysis:

Correlation Improvement

The behavioral caps improve the score's ability to predict crash occurrence:

While correlation with crash rate is slightly lower (the uncapped score already performs well here), the capped score shows 2.4x better correlation with crash count—meaning it's significantly better at predicting which carriers will actually have crashes.

Grade Distribution After Caps

The behavioral caps create cleaner separation between grades, with crash probability increasing monotonically as grades worsen:

Example: High-Risk Carriers Now Properly Flagged

The behavioral caps catch carriers that would otherwise receive near-perfect scores despite serious violations:

These carriers had excellent aggregate metrics but critical behavioral violations. Without the caps, underwriters would have no visibility into these serious risk factors.

Predictive Validation: Year 1 → Year 2

To validate the predictive power of violation types, we conducted a temporal split analysis: using 2024 data to predict 2025 crashes. This approach tests whether behavioral violations truly predict future crashes, not just concurrent ones.

Key Question: Do Behavioral Violations Predict Future Crashes Better Than Past Crashes?

A concern with using past crashes as a predictor is size bias—larger carriers have more opportunity for crashes simply due to exposure. Behavioral violations may provide a more size-independent signal of underlying risk.

Relative Risk by Violation Type (Rate-Based, Size-Controlled)

By using crash rates (per 100k miles) instead of counts, we control for carrier size:

Size-Controlled Analysis: Violations Within Fleet Size Bands

When we analyze within fleet size bands, behavioral violations become dramatically more predictive. This reveals predictive power that's hidden when mixing carriers of different sizes:

Cumulative Effect: Multiple Behavioral Violation Types

Carriers with multiple types of behavioral violations show dramatically compounding risk:

Overall Model Performance

The predictive validation confirms strong model performance across multiple metrics:

Crash Capture by Risk Decile

Sorting carriers by their Year 1 risk score, we measure how many Year 2 crashes fall into each decile:

Conclusion

This study demonstrates that specific violation types, particularly behavioral violations related to driver decisions, are significantly stronger predictors of crash likelihood than aggregate violation counts or equipment violations alone.

The dose-response relationship—where more violations of high-risk types correspond to dramatically higher crash and fatality rates—provides strong empirical justification for weighting these violation types more heavily in carrier risk assessment.

Key Findings

• Carriers with reckless driving violations have a 71% crash probability
• Carriers with 10+ speeding violations have a 98% crash probability
• The capped score shows 2.4x better correlation with crash occurrence
• 6,956 carriers were identified and capped that would otherwise have received inappropriately high scores

Predictive Validation Confirms Strong Performance

Our Year 1 → Year 2 temporal validation proves the model's predictive power:

• AUC of 0.848 and Gini of 0.696—excellent model discrimination
• Top 10% of risky carriers capture 58% of next year's crashes (5.8x lift)
• Behavioral violations predict crashes independently of carrier size:
  • Small carriers with substance violations: 2.75x relative risk
  • XLarge carriers with speeding/reckless: 3x relative risk
• Carriers with 3+ behavioral violation types have 65% crash probability the following year

Integrating violation type analysis into the FRED Score improves crash prediction accuracy and provides underwriters with granular risk intelligence that was previously hidden within aggregate metrics. The temporal validation confirms that these signals are truly predictive—not just correlated with concurrent outcomes.