What’s the impact of ransomware attacks on healthcare entities? Did you ask the people who really know?

Expect some buzz next week about a new report with significant findings about the impact of cyberattacks on patient care and mortality. The study was funded by Proofpoint and conducted independently by Ponemon Institute. The survey addresses important questions about the impact of various types of cyberattacks on patient safety and care. While DataBreaches appreciates the intent, the findings do not seem sufficiently supported by the methodology^Note.

Concerns

Ponemon appropriately and responsibly mentions some limitations of using a survey method on p. 26 of their report. One of those limitations is described as:

Non-response bias
The current findings are based on a sample of survey returns. We sent surveys to a representative sample of individuals, resulting in a large number of usable returned responses. Despite non-response tests, it is always possible that individuals who did not participate are substantially different in terms of underlying beliefs from those who completed the instrument.

Their survey had a 3.9% response rate, which is so low as to warrant concerns about nonresponse bias. Ponemon conducted non-response tests to help assess the risk of bias, which was helpful, although, as they note, it doesn’t eliminate the risk. But I am not as concerned about non-response bias as I am about the sample’s representativeness in other respects.

What would you consider a representative sample if you want to determine how various cyberattacks impact patient safety and care? What positions in a hospital or medical entity would have that information?

Ponemon’s report describes their sampling method:

A sampling frame of 16,451 IT and IT security practitioners in healthcare organizations who are responsible for participating in cybersecurity strategies including setting IT cybersecurity priorities, managing budgets and selecting vendors and contractors were selected as participants in this survey.

Ponemon designed their survey so that a certain percentage of those surveyed would be at the supervisory level. But does the supervisory level mean that the individual will know about the impact of ransomware attacks on patient safety and safety care?  And what do some of these organizations represented in the sample have to do with the provision of patient care or the ability to assess impact on patient care? (see Update for graph)

Healthcare entities have departments and personnel responsible for collecting and analyzing patient safety incidents and patient care concerns. Was the survey sent to them? If not, why not? Look at the job occupations of the 641 respondents to the survey below. Are many of them even in a position to know whether specific outcomes occurred?

Let’s take a closer look at the reported findings about the impact of ransomware attacks:

• 41% of the 641 respondents report they had one or more ransomware attacks. That’s approximately 263 respondents who claimed their entity had experienced at least one ransomware attack;
• 51% of those reported that their entity had paid the ransom; and
• 67% of the 263 who reported at least one ransomware attack said that patient care was disrupted.

So approximately 176 respondents reported that patient care was disrupted due to a ransomware attack. Those 176 respondents then went on to check off which impacts their entity reportedly experienced:

Suppose we convert the percentages back to raw numbers. In that case, we find that approximately 42 people endorsed a statement that there had been an increase in mortality rate due to a ransomware attack. About 113 people endorsed a statement that delays in procedures and tests resulted in poor outcomes.

But how do they know? Are their responses based on any actual data or reports from their entity, or are their responses just wild guesses? How reliable is their reporting?

Some of the claims reported do not strike DataBreaches as credible (e.g., percentages reported for specific types of Business Email Compromise (BEC) impact were almost identical to those reported for ransomware attacks).

Is it possible for cyberattacks to have serious consequences such as poorer outcomes or increased mortality rates? Certainly, but these survey responses do not provide any actual data or evidence that they do.

If systems and files are locked, and patient pharmacy records cannot be accessed, prescription refills may be delayed, or doctors may not know what prescriptions a new patient has been receiving. Those situations pose risks of poorer outcomes or patient safety concerns. But did the risks materialize into actual harm, and do the survey respondents know for a fact?

Suppose patients urgently need surgery and surgery is delayed because systems are locked, and the patient’s records and pre-op records cannot be accessed. In that case, that poses a risk of poorer outcomes or increased mortality. But did the risks materialize into actual harm, and do the survey respondents know for a fact?

Anticipating the Headlines

Will some try to trumpet the Proofpoint/Ponemon findings as headlines?  Probably.

DataBreaches will not be surprised to read headlines claiming that ransomware attacks may cause increased mortality rates by 24% or that ransomware attacks cause poorer outcomes in 64% of cases. Neither of those claims would be supported by the methodology and findings of this survey.

Will some people use these survey findings to urge ransomware groups to back off medical targets? Almost certainly.

Will efforts to “reason” with ransomware gangs based on these results make a difference? Does the phrase “snowball’s chance in Hell” still have meaning?

Are we likely to see accurate disclosures by healthcare entities about these critical questions? No. Any admissions by providers or business associates that care was impacted or deaths resulted would likely be used against them in class action lawsuits as evidence that they were negligent in not having enough preparation or resources devoted to cybersecurity.

If we want accurate estimates, we need an audit where the entities are given some immunity from regulator enforcement action, where the personnel who collect and analyze adverse events and incidents are involved in answering questions, and any results presented to the public are de-identified.

I’ll wait over here for that snowball to re-freeze.

Update: I forgot to upload an image earlier that shows the types of settings the survey respondents work in. Some of these settings are not directly involved in patient care or are even business associates related to provision of care, so why include them in a survey to ask them whether ransomware attacks impact patient mortality or poorer patient outcomes?

---

^Note In a previous part of her life, “Dissent Doe” spent about a decade teaching undergraduate and graduate students research design and statistical analysis. Even though she has conducted survey research in that previous life that was published in a peer-reviewed journal, she has never been a fan of survey methods.