Guest Commentary: FMCSA Data on CSA/SMS Proves the Scores Cannot be Used by the Shipping Public

By Tom Sanderson, CEO, Transplace

The CSA/SMS methodology and measurement system is fatally flawed. Only a small fraction of carriers are measured, and of those that are, an astonishing 56 percent are deemed to warrant further intervention because of high scores. With highway accidents at an all-time low and improving steadily, how can more than half of all carriers measured require further scrutiny?

An astonishing finding in the FMCSA data is that only 5 percent of measured carriers are in the best 20 percent on the Fatigued Driving BASIC while 32 percent of all measured carriers are in the worst 20 percent. Safety should be an absolute, not a relative measure. Grading on a curve in safety is wrong, but to distort the curve in a fashion that awards 5 As for every 32 Fs is absurd. Here is a summary of the data downloaded over four months from the FMCSA website.

In each month, approximately the same number of carriers is in the dataset (765,000), approximately 12 percent of the carriers have an SMS score on at least one BASIC, and 56 percent of measured carriers have at least one BASIC over the intervention threshold – a golden triangle. The FMCSA claims to measure 200,000 carriers, but if that is the case perhaps the Administration should include those other 110,000 carriers in its publicly released data. While 12 percent of carriers have at least one score, fewer than 10 percent are measured on any particular BASIC and less than 5 percent are measured on 3 of the 5 publicly released BASICs. Only 3 BASICs measure more than 10,000 carriers.

The number of alerts or golden triangles by BASIC is relatively stable at the aggregate level over the four months we studied. We are working with a leading transportation and logistics professor to measure volatility at the carrier level, which we know anecdotally to be of great concern to small carriers.

The data on Fatigued Driving is particularly troubling. The intervention thresholds are set at 35 percent for non-hazardous and 40 percent for hazardous-materials fleets. In each of the four months we studied, 56-57 percent of all measured carriers on the Fatigued Driving BASIC have golden triangles. We dug into the February data set in more detail and found that less than 2,300 carriers, or only 4.6 percent of the 50,000 measured carriers, have scores in the best 20th percentile. By definition, there should be 10,000 carriers in the top 20 percent. More than 16,000 carriers scored in the worst 20th percentile, or 32 percent of all measured carriers. In this BASIC, the FMCSA is giving out 5 As for every 32 Fs.

In the graphs below, the straight lines show the number of carriers that should be at each percentile in an unbiased system.  The curved lines show the actual FMCSA data for the BASIC – highlighting the fact that far fewer carriers than should be are awarded good scores. The graph on the right indicates a similar, though less biased, pattern for the Vehicle Maintenance BASIC in which only 10 percent of the carriers are in the best 20 percent and 28 percent are in the worst 20 percent.

Furthermore, the SMS scores have no correlation with accidents per million miles according to the fine analysis done by Wells Fargo. The regional data anomalies are too numerous to discuss here. Yet, some shippers and brokers are using this faulty data to decide which carriers to use.

That is denying business to perfectly safe carriers and obviously does nothing to improve highway safety. To make matters worse the FMCSA wants to use a system where 5 percent of the carriers are in the best 20 percent and 32 percent of the carriers are in the worst 20 percent to mathematically calculate a Safety Fitness Determination. We can’t let that happen behind closed doors in DC. Let your congressional representatives know that CSA/SMS is fatally flawed and is causing real harm to the supply chain while doing nothing to improve highway safety.