Skipped doses, upload errors, delay times and software problems. Vaccine rollout in California has been plagued with data issues, leaving the state unable to keep track of how many doses of the life-saving COVID-19 vaccine are available at any given time.
The implications are far-reaching: Gavin Newsom government has pushed for vaccinations to be speeded up, in part because state data seemed to show vaccine suppliers were on doses, threatening the governor to take supplies from those who didn’t go fast enough. Now district officials say they are concerned that data accuracy issues will lead to future allocations being curtailed based on flawed conclusions from erroneous numbers.
“We pointed out that their data has been bad since the end of December,” said Ernest Mendes, Fresno County supervisor.
After being forced to disclose more information about the progress the counties were making in their vaccination efforts, the state released a dashboard to “make vaccine data transparent and accessible to all Californians.”
But the dashboard was so riddled with errors – including displaying a handful of doses from counties in Arizona and other states – that Kat DeBurgh, executive director of the Health Officers Assn. from California, she said initially told the state it had to be pulled down. Officials in smaller California counties also reported that the dashboard reported drastically low doses for them.
“When it first went up, it was extremely inaccurate,” said DeBurgh, “but [the dashboard] looks much better now. “
The accuracy of the state’s vaccine data overall is improving, which has helped California increase its national rankings for doses administered, but the effort was often a manual and extremely slow process, said California Department of Public Health spokesman Darrel. Ng.
And the work is not finished yet. Aimee Sisson, Yolo County public health officer, told lawmakers in a hearing Wednesday that upload errors had caused the state database to under-count vaccines administered in that county by nearly 30%.
“Neither the number received nor the number administered reported by the state matches what we know to be true in Yolo County,” Sisson said. “Yolo County personnel have worked unsuccessfully with a state contractor to resolve the discrepancies.”
With California warning providers that not administering vaccines fast enough could cause the state to back off doses, Sisson stressed that the data the state relied on had to be accurate.
“If the state uses its current data to determine future allocations,” Sisson said, “Yolo County could be penalized for what the state considers a 51% administration rate, but is in fact a 74% administration rate.”
Due to the data troubles, it appears that counties have been slow to provide doses to the public, Sisson said, prompting the state to look for new ways to speed up administration. Newsom said the state will sign a contract with Blue Shield of California to review the way the state allocates vaccines and improve data collection. That contract is expected to be released this week.
Some district officials have expressed concern about the adjustment of the state’s vaccine delivery system, saying the problems have to do with the state’s data collection, not the way doses are distributed.
“The system isn’t broken,” Sisson said. “It seems that the doses given are not appearing.”
Part of the problem with the state’s data is the sheer number of vaccine providers entering information using different types of software. Sometimes the errors and delays are the result of the software itself, some district officials have said. Other times, the state’s immunization registry indicates that the dose information has been uploaded, but then fails to update the totals, district officials said.
“I hear from several local health departments in the greater Sacramento area that they are having the same problem with missing doses [in the state registry], ”Sisson said at the hearing. Dr. Paul Simon, chief science officer of the Los Angeles County Department of Public Health, added, “We’re seeing the same thing here in Los Angeles.”
The Urban Counties of California legislative advocacy group wrote in a letter to Newsom this month, noting that affiliated counties were “significantly challenged by the lack of accurate state-level data.”
Last month, Newsom praised the state’s progress in increasing daily vaccination rates, but said the “two-, three-day data delay … is killing us in terms of some of the national numbers.”
The significant problems with collecting vaccine data in California first came to light when Newsom told Californians to “hold me accountable” for the goal of administering 1 million doses of COVID-19 vaccine in 10 days. After that deadline passed, state officials said coding errors and data delays made it difficult to say whether Newsom had hit the mark.
The state’s pandemic response was hampered in other ways by poor or outdated data systems, including the discovery in August that a malfunction in the public health computer database made the numbers unreliable and cast doubt on measures taken to prevent the spread of the coronavirus.
Faced with criticism from district officials about the ongoing problems with vaccine data, Dr. Tomás Aragón, director of the California Department of Public Health and the state’s public health officer, spoke in Wednesday’s hearing that there is an “incredibly intense focus” on “cleaning up the data, improving quality.”
“We need to improve the data,” Aragón said.