Analysis: Idaho’s COVID-19 testing statistics are not good

Out of the many statistical metrics purporting to measure response and potential recovery during the pandemic, a handful recently caught the eye of the Idaho Business Review editorial staff, especially for how they affect Idaho. By the metrics we analyzed, it was clear that Idaho was not doing well in a mid-August snapshot of both health and societal measures.

The takeaway nugget of our analysis suggests that nationally, laxity in measures to contain the spread of COVID-19 loosely correlated with the test positivity rate in mid-August.

A COVID-19 testing positivity rate of over 5% indicates that an infectious disease is spreading out of control and that cases of the disease will rise in the near term. It can also indicate that a state’s testing capacity is inadequate to accurately measure the spread of the disease. In either case, a high positivity rate is not a good thing.

Idaho’s performance

Compared with other states in mid-August, Idaho was one of the five worst states in the nation for the correlation of the two-week average positivity rate and for its governmental measures to contain the spread of the SARS-CoV-2 novel coronavirus behind the COVID-19 disease.

As of Aug. 20, Idaho was testing for COVID-19 at a daily rate of 157 per 100,00. This was only 16% of what its testing target should be, according to the Global Health Institute at Harvard University. Compared with all other states, this was the worst testing effort in the country in terms of adequate testing volume.

Idaho’s hospitalization rate was much better. Perhaps the most telling and accurate of all COVID-19 statistics is the hospitalization rate, which correlates to the number of people infected and affects health care capacity, two things that impact public health directly.

Idaho had a rate of 11 per 100,000 on Aug. 20, according to the COVID Tracking Project. This rate, which was slightly better than the national average, was 12 hospitalizations per 100,000 — greater than the national median of eight.

Regarding the Global Health Institute testing targets, Idaho State Epidemiolgist, Christine Hahn, M.D., told the Idaho Business Review on Aug. 21: “I know we’ve been ramping up, but according to the Harvard Global Health Institute, we’re still not doing well enough. Yes, I think we agree that we need to do more testing. What I’ve really noticed is that our testing is still mostly demand-based. It goes up and down depending on what’s coming. Our numbers for testing actually have been dipping slightly, and we think that’s probably because fewer people are now getting sick and going in to ask for testing, which is a good thing. But what we need to do better is ramping it up as the surveillance kind of testing.”

Hahn added: “We just had a meeting recently talking about that, (especially) in our long-term care, in corrections and for other employers dealing with some of our enclosed populations. We’re hoping that with long-term care, with the new (testing) machines just coming in — and they started arriving last week — that we’ll start to see more routine surveillance testing and that we’ll get our (testing) numbers up … with(more surveillance testing), instead of just waiting for people to come in and get tested, that (positivity rate) is actually going to start coming down. So we need to switch from a demand-based testing to more of a routine surveillance-screening type of testing model. We are working on it.”

Our Analysis

We looked at several different data streams and selected COVID-19 test positivity rate, as well as a weighted metric of governmental restrictions on society in response to COVID-19 and population density. These three datasets appeared to illuminate how demographics and behavior have both affected the course of the pandemic.

We evaluated several other datasets but chose not to pursue them, including smart-device mobility data from multiple high-quality sources and COVID-19 death rate data, for example. The mobility data was impressive for its ability to measure how much and how far people traveled during the pandemic. Despite this, mobility data did not correlate with health statistics, including datasets from Google, Apple, the U.S. Department of Transportation, Unacast or the University of Maryland. The hospitalization and death rate data were not used for correlations with other August data because of the long lag times for those effects to be seen after the time infection.

The first dataset we selected was the daily 14-day sliding average of the COVID-19 test positivity rate, which is the percentage of daily tests that are positive. Our data source was the Aug. 16 version of the New York Times compilation of testing and hospitalization data for the nation, which is collected for every state and updated daily.

We correlated test positivity with the Aug. 11 update of an ongoing study of “states with the fewest coronavirus restrictions,” compiled by the online consumer financial services company, WalletHub. Because states with the least restrictions receive a higher score using WalletHub’s metric, we dubbed their statistic the COVID-19 preventative measures laxity score.

The laxity score is a weighted metric that seeks to evaluate the effect of measures such as mask mandates; closures of non-essential businesses, large venues, bars and dine-in restaurants; limits on the number of people in both public and private gatherings; travel restrictions; and traveler quarantines. The laxity score also includes some non-restrictions such as business liability protections from COVID-19 related torts and contact tracing programs.

The five-day data lag between the Aug. 11 laxity score and the Aug. 16 positivity rate is not inappropriate given the lag between infection and testing. Patients tend to seek or receive testing when symptoms manifest, and symptoms show up one to 14 days after infection, with an average of five to six days, according to the University of California San Diego School of Medicine Hospital.

Results and interpretation

Looking at the data, the biggest COVID-19 loser in the nation for mid-August was Mississippi with its abysmal positivity rate of 20% and high laxity score of 50.44. The runner-up positions for the worst in the nation were Florida, Texas and Nevada — all with a positivity rate of 17% on Aug. 16 — and Idaho in the fifth-worst position, at 16% positivity and a very high 67.27 laxity score.

Four of the five worst states had laxity scores greater than 50. The exception was Texas, which had few preventative measures in place between May and the end of July. Texas was one of the earliest states to reopen its economy; it then experienced a widespread and rapid resurgence of COVID-19 in June and July. The state reinstated several coronavirus-prevention measures in a staggered and disorganized fashion in late July. Texas also has unique demographics: along with its huge footprint, it is home to two of the nation’s 10-largest metropolitan areas despite its overall population density close to the national median and its predominately rural character.

Timing trends and demographics were apparent in the data. The highly-populated New England and Mid-Atlantic States, which saw the worst of the pandemic in the East Coast in March, April and May, now had the lowest laxity scores (i.e., the highest restrictions) and the lowest positivity rates. Rhode Island violated this trend when it suffered a resurgence of COVID-19 in June and July, which has since been contained. Its current daily positivity rate has fallen down to 2% from an earlier 6%.

Another national trend has been the shift of COVID-19 outbreaks from mostly-coastal metropolitan centers to the current surges in rural southern and western states. This trend also showed up in the data, especially for the South. All of the Deep South and Southern Plains states had mid-August positivity rates greater than 7%, and seven out of nine had positivity rates that were 10% or worse. All these states, excluding Texas, had laxity scores over 50.

The western U.S. did not really show any regional consistency in either positivity or laxity score in mid-August. Several western states like Idaho and Utah had poor laxity and poor positivity performance, while others, like New Mexico and Colorado, had low positivity rates of less than 5% and moderately-low laxity scores.

Population effects

The one demographic measure that did show some control on the mid-August snapshot data was population density, based on the 2020 estimates by the U.S. Census Bureau. Half the states in the country have population densities of over 100 people per square mile. Out of all of these, five — Florida, Tennessee, South Caroline, Georgia and Wisconsin — had mid-August laxity scores of greater than 50% and positivity rates of 7% or worse.

For the half of the states with population densities under 100 per square mile, only six had positivity rates under 5%: Vermont, Maine, Alaska, West Virginia, New Mexico and Colorado. All other less-populated states had positivity rates indicative of rapid spread and inadequate volumes of testing.

Laxity had an interesting tie to population density. Out of the 25 less-populated states (pop. den. < 100/sq. mi), only six of them (24%) had laxity scores under 40, meaning they had a high level of preventive measures in place. In comparison, over three-quarters of the less-populated states had fewer COVID-19 preventive measures in place. Because rural character and low population density go hand-in-hand, this shows a tendency for the more rural states to have the fewest COVID-19 restrictions.

Caveats

A few important caveats should be mentioned here. First, positivity rate is much more volatile than laxity score. Positivity rate, even as a two-week sliding average, changes daily while the laxity score tends to be updated every three to four weeks. It is important to remember that the trend presented here is a snapshot of an evolving situation. Next week’s data will certainly be different, though we do not anticipate the general correlation of laxity with high positivity to change over the next few weeks, especially given the impediments preventing effective public health measures and the exaggerated lag times between infection, symptoms and receiving test results.

Also, the New York Times compilation pulls data from the COVID Tracking Project, which updates and reports on a daily basis. According to Hahn, the COVID Tracking Project reports data that is a bit higher because the data is not vetted and verified as well as the data published by the Idaho Department of Health and Welfare, which is revised and corrected to remove duplicated data.

There is a second reason data from this source doesn’t look like the data reported by the Idaho Department of Health and Welfare: the state data is reported only once a week as a two-week average, which is different from a 14-day sliding average recalculated daily.

Data for the state of Washington was not included in this analysis. Washington stopped reporting its positivity rate on Aug. 5 because of changes to how its Department of Health reported its data. Washington announced that it planned to resume reporting test results, including positivity rate, on Aug. 25.

The data used for this analysis is included in the table below: