Vendors will not allow their programs or data structures to be examined, even by the academic community. Their products, medical devices, are not vetted by regulatory agencies such as the FDA for quality.
Patients and clinical personnel are supposed to take the word of the seller that these systems are fit for purpose and of high quality.
A window into the internal working of EMR's was initiated Down Under just this year, in the form of a detailed internal analysis of an ED EMR product from America slated for roll-out in public hospitals in one of Australia's most populous states. Dr. Jon Patrick of the University of Sydney performed this work. He was not happy with what he found, in terms of software engineering, human computer interaction, and other issues related to fitness for purpose. See here and here.
(For more on healthcare IT difficulties, see "Contemporary Issues in Medical Informatics: Common Examples of Healthcare Information Technology Difficulties" at this link: http://www.ischool.drexel.edu/faculty/ssilverstein/cases/.)
One component of the healthcare IT "ecosystem" many have been unable to see, which is perhaps even more carefully guarded than the program code and data structures, is the metadata on EMR use.
A remarkable development has now changed that metadata secrecy - a public civil lawsuit court docket.
Metadata is "data about data." In this context, that means data about who accessed the EMR, what they did, when they did it, from where they did it, etc. Perhaps it should be deliberately misspelled as "meatdata."
It allows the "legible gibberish" (see my Feb. 2011 post "Electronic Medical Records: Two Weeks, Two Reams" for more on that term) produced by the clinical side of EMR to be better weaved into meat, that is, a medical narrative that is essential to truly understanding a case.
I noted the unfortunate case of Samuel Sweet at the University of Pittsburgh Medical Center (UMPC), as reported by Sean Hamill in the Pittsburgh Post-Gazette:
Trial to begin in wrongful death claimAs a result of this newspaper story, the defendants filed a complaint with the judge, who ordered a six month delay in further proceedings. The complaint and decision is at this link (PDF).
Monday, September 19, 2011
By Sean D. Hamill, Pittsburgh Post-Gazette
As far as Samuel Sweet was concerned, he only had a headache. It was a bad one, but nothing more, maybe something his chiropractor could fix with a little adjustment.
But three days after being admitted to UPMC Presbyterian for what his family was told was a treatable amount of bleeding on the brain, and six hours after his family saw him laughing and chatting about the Penguins' playoff chances, early in the morning of May 16, 2009, Mr. Sweet died unexpectedly.
Why he died is the subject of a civil case his family filed a few months later ... [Plaintiff's attorney] Ms. Maliver [actually Dr. Maliver, as she holds both the JD and MD degrees - ed.] said UPMC's timeline came into question in early August 2011, when UPMC finally turned over -- after two years of discovery [i.e, they turned them over at the last minute, a few weeks before trial was scheduled to begin - ed.] -- 1,200 pages of "results detail sheets" which show not only when something was written in Mr. Sweet's electronic chart, but who wrote it and how.
[The patient died when there was a delay in treatment due to marked difficulty in putting a breathing tube into his lungs (a difficult intubation) when he developed respiratory failure. The defendants possibly never expected someone to actually comb through the 1,200 pages of "results detail sheets" on such a short time frame, but, apparently, the plaintiff's attorney took on the challenge - ed.]
So, for example, Ms. Maliver alleges that the results detail sheets show that during the 21 minutes before Mr. Sweet was said to stop breathing, none of the information in his chart is straight from the computer monitors that would tell what his vital signs were during that period.
But the results detail sheets also show something more troubling, Ms. Maliver said.
She said she found evidence that three days after Mr. Sweet died, Dr. Richard Simmons, UPMC Presbyterian's head of quality assurance, tried to put a "Diff Intub" [difficult to intubate, due to anatomy - ed.] red-letter warning on Mr. Sweet's electronic medical record. That effort showed up on the results details sheets, according to Ms. Maliver.
Such a warning would tell ["alert" would be a more precise term - ed] anyone who opened the record that Mr. Sweet was difficult to intubate.
In further investigating what I considered a remarkable story, I came across a public docket Dr. Maliver filed on behalf of her clients in the case, "PLAINTIFF'S MOTION AND SUPPORTING BRIEF TO AMEND COMPLAINT TO SEEK PUNITIVE DAMAGES".
As I mentioned previously, this is a remarkable development that has opened a window on metadata secrecy.
This is freely and publicly downloadable from the Pittsburgh Prothonotary Office Civil Docket site at this link, case #GD-09-019407 , document #40 of 09/02/2011. (I have also cached the document at this link: http://www.ischool.drexel.edu/
As I mentioned in the aforementioned post, I found the hospital attorney's explanation of apparent tampering somewhat troubling:
Dr. Simmons wasn't trying to alter the record, [attorney for the defendants] Mr. Conti said; instead he was in Mr. Sweet's medical record as part of the "peer review" process, assessing what happened in Mr. Sweet's case.The public nature of the "PLAINTIFF'S MOTION AND SUPPORTING BRIEF TO AMEND COMPLAINT TO SEEK PUNITIVE DAMAGES" document makes it possible for the public to look at some of the actual metadata to see for themselves what might have occurred.
As part of that, he was in the medical record three days after Mr. Sweet died trying to figure out how one would create a warning about a patient's difficult intubation [in effect, altering the record, at the very least through creation of a confusing audit trail - ed.], Mr. Conti said.
Again, this is remarkable and is the first time I am aware of when the public (and the academic Medical Informatics community, of which I am a part) gets to "peek behind the curtain" of an EMR's metadata.
Let's take a look.
I will summarize relevant part of the Motion and intersperse testimony and metadata from the actual record of Mr. Sweet (again, from a now-public court document).
The purpose of the Motion was to seek punitive damages:
A. SEEKING PUNITIVE DAMAGES
(1) This is a civil action for medical malpractice in which the Plaintiff alleges in essence that Samuel Sweet, a relatively healthy man of only 62 years of age, needlessly suffocated to death in the intensive care unit at UPMC after undergoing a routine cerebral angiogram when, the following day, physicians recklessly and prematurely pulled out his breathing tube and then could not get it back in when it became obvious Mr. Sweet would die without it.
(2) This case is listed for trial on September 19, 2011, and discovery has just concluded.
(3) In light of the evidence adduced in discovery, the Plaintiff seeks to amend the Complaint to re-characterize Defendant Murugan's and Defendant actions and inactions as willful, wanton, reckless and not merely negligent. The Plaintiff similarly seeks to amend the ad damnum clause against these defendants to include a request for punitive damages because of their shocking and reckless disregard for Mr. Sweet's welfare and their attempts to alter the appearance of the medical records, in full awareness of Mr. Sweet's brain death, so as to obscure what really happened.
It is the last part of count A.(3) I will address here, a quite serious charge involving medical record evidence spoliation, which is specifically forbidden in Pennsylvania by the Medical Care Availability and Reduction of Error (MCARE) Act of 2002 (PDF, see Section 511, "Preservation and accuracy of medical records", pg. 18-19) and other regulations.
Of importance, the Motion continues:
... (15) UPMC has a special warning system that is intended to serve the very purpose of alerting all medical providers of a patient with a difficult airway. The way it works [as UMPC defendants testified under oath - ed.] is the attending physician, writes up the problem list for the patient and need only put difficult airway in the list, and this will trigger a special yellow banner to pop up on the patient's computer files for all to see every time the chart is opened and every time any page is viewed.
Not only that, but then the words "difficult airway"appear in red text within the yellow banner, and this puts the viewers on notice to check the anesthesia record as to how intubation was done successfully in the past, so they can replicate the successful technique.
This patient was extubated after a procedure to check for brain bleeding, but by late at night it became obvious he could not breathe adequately on his own, and needed to be re-intubated.
Let's see how a chart appears with, and without the banner about a patient being difficult to intubate (due to their mouth and air-passage anatomy):
Real Electronic Medical Record screen of Mr. Sweet lacking warning banner, above; fictitious chart showing banner, below. [Why wasn't it the fictitious patient chart that was fiddled with? - ed.] Click to enlarge.Such a warning apparently did not appear in Mr. Sweet's EMR chart, and that along with other failed communication (which can be read in the Motion) led to the re-intubation difficulties, and ultimately death. (I should note that it's possible the clinicians did flag the "difficult intubation" condition, but that the EHR lost the data through relational integrity problems, i.e., the information became "orphaned" from the patient's records, although still existing in the system somewhere.)
(27) ... not one person knew that there was a note from anesthesiology containing a road map for success, that included the use of a "Fast-Track LMA device", neither of which was ever brought in for Mr. Sweet that night when he needed the tube to be put back in.Exhibit E. Under Oath. "A designation of difficult intubation automatically appears on a patient's banner bar if it is entered as a problem on the Problem List by the user selecting it from a drop-down menu list." (I tend to have doubts the problem list is that intelligent, but this is what was testified.) Click to enlarge.
Sworn testimony indicates nobody took the appropriate actions to enable the alert to appear on the EMR screens:Admitted under oath. Nobody did what it would take to make a "difficult intubation" alert appear on Mr. Sweet's chart. Click to enlarge.
As a result of this missing data, the "syndrome of overconfidence in computers", and the toxic effect of poorly done health IT impairing clinician-clinician communications, physicians seeing this patient when he was under duress admit under oath they did not know of this intubation problem:Dr. Kaura didn't know. Click to enlarge.
Dr. Sappington didn't know. Click to enlarge.
This is bad enough. The post-mortem chart fiddling issues I am addressing here begin with this:
C. (39) In the present case, after rounds of complex discovery battles, it became apparent that UPMC altered the records ... in this case, much of the record alteration was conducted by quality assurance officer, one Dr. Richard Simmons.
(40) Once the brain death of Mr. Sweet was known, a wave of record alteration and supplementation occurred, largely though not exclusively at the behest of Dr. Simmons ...
(41) For example, after Mr. Sweet was dead, when it could obviously not benefit his medical care, Dr. Simmons entered Mr. Sweet's computerized medical record, clearly looking into whether there had been a proper yellow-banner warning created through a problem list (which, of course, there was not).
Again, these are serious charges. Why would the plaintiff attorney raise these charges? Let's look at some metadata:
Exhibit K, showing quality assurance officer Dr. Simmons fiddling with Mr. Sweet's chart on May 19th several times during the day, as well as on the 22nd. Note circled entries. Note the gap (labeled) and the time of the last entry on the 19th, 15:23. Also note all these dates are post mortem by several days; death was on the 16th. Also note the times of entry, and the part of the chart accessed in the second round of accesses on May 19th - including "problem list" at 15:02. Click to enlarge.
I note that we are left to wonder what happened between access round two by the QA Officer on May 19 beginning 15:01 PM, and round three on May 22 at 14:03 PM. (Where are data for May 19th after 15:23 and May 20th and 21st, I wonder as well? This was not a weekend but midweek ...)
The Motion continues:
(42) Thereafter, even though Mr. Sweet was now long dead and such action could do him no good, Dr. Simmons tried to create a problem list that had originally been omitted by Defendant. He listed a difficult airway as the problem (even though obviously Mr. Sweet's problem list would have included many things, such as having undergone an angiogram and a minor cerebral bleed). In other words, Dr. Simmons tried to take the action necessary to make the computer show the yellow banner warning that was missing.
Here's more metadata seemingly supporting this statement. It's clear that nobody, including especially someone in Dr. Simmons' role should have been doing ANYTHING to the data that would or could raise suspicion of alterations via the metadata, such as times. Surely a presumably competent senior physician in a peer review role would have known this.
(I also note, shouldn't the record have been electronically "locked" by several days after death, thus causing any changes after the lock to appear in the normal "give us the legal medical record" outputs, not just in metadata?):
Exhibit L. Metadata showing Dr. Simmons on May 19th in the drop-down menu to make a problem list - then retracting it ("active --> cancelled") shortly afterwards. (I note the apparent cancellation time of 3:24 PM, i.e., 15:24, is just after the final May 19th entry of 15:23 on Exhibit K, when the several-day "gap" begins. I also note the patient died on May 16th. Curious... ) Click to enlarge.But problems apparently occurred:
(43) It did not work. The yellow banner would not appear retroactively onto the otherwise completed record although he did succeed in altering the original record to look like there had always been a problem list. Further, during review of the "detail sheets," the Plaintiff discovered that Dr. Simmons, having failed to create a banner warning, tried to then delete or undo what he had attempted, but could not get it out of the computer tracking system. See Exhibit L hereof.
Let's look at the next part of Exhibit L:
Note the postmortem date of 5/19/2009 on this "problem list", and the attempted cancellation, presented as evidence. Note the time of the Problem list addition - 3:02 PM (i.e., 15:02 as in exhibit K above). Canceled as above at 3:24. Someone in Dr. Simmons' position should not have been fiddling with the problem list, knowing (or should have known, or should have made it his business to know) that footprints of fiddling in the metadata would, at the very least, raise eyebrows. Click to enlarge.This does not look like the metadata that would be generated by peer review that I myself would perform ...
In summary, this unique Motion containing actual EMR metadata is, in my personal view, suspect for 1) imprudent post-death fiddling with the EMR records at best, and/or 2) attempts to alter the EMR to unjustly protect those who caused harm to come to this patient at worst.
I do not know which occurred, but hospital attorney Conti's excuse that "Dr. Simmons was in the medical record three days after Mr. Sweet died trying to figure out how one would create a warning about a patient's difficult intubation" [that is, entering new data not present before - ed.] could be either:
- A sign of lack of computer and legal savvy of a senior clinician/UPMC Presbyterian's head of quality assurance, but whose intentions were pure, as explained above; or
- The confabulations of an attorney caught with his pants down.
My personal opinion?
Want to buy a bridge in Brooklyn, members of the jury and Your Honor?
NY bridge for sale. Cheap.The following also does not favorably influence my interpretation of the metadata:
(44) It is noteworthy that, prior to producing these results detail sheets that indicated such changes, UPMC attempted to file a motion for a protective order to bring all further discovery to a halt (a motion which was denied) even though the discovery deadline had not yet arrived.
Obviously, this is not for me to decide and will need to await court adjudication, but as a Medical Informatics specialist and someone with some investigative experience regarding computers and medicine, I am not happy with the looks of this.
I report, you decide.
There were other allegations in this plaintiff's Motion, again based on clinical IT metadata, regarding: unaccounted-for drugs of potential abuse (e.g., sedative propofol, the drug that killed Michael Jackson), the entry of fictitious data (such as vital signs, timing and medication administration by others including nurses), dictations coerced out of junior staff by senior personnel that might have been "spun" favorably to the medical center, etc., but those are the subject of later posts.
I again repeat the six-month delay in this case could potentially leave various problems with this EMR system unexposed and unremediated. This could unnecessarily expose the public to six more months of risk.
Addendum: if UPMC or the vendor take issue with posting of this material due to IP or other issues such as health IT gag clauses (which by the way are signed by hospital executives in violation of Joint Commission Safety Standards per my 2009 JAMA letter here and longer essay here), contact the Prothonotary Office in Pittsburgh, not me.
-- SS
Addendum:
What type of person was Mr. Sweet? This type:
-- SS
Samuel E. Sweet Jr.
Cheswick
Samuel E. Sweet Jr., 62, of Cheswick, died Saturday, May 16, 2009, at UPMC Presbyterian Hospital. He was born Dec. 9, 1946, in New Kensington, son of the late Samuel E. Sweet Sr. and Marjorie (Secrist) Sweet. He was a resident of Cheswick for 33 years. He worked for the Army Corps of Engineers at Lock No. 2 of the Allegheny River and served in the Marine Corps during the Vietnam Era. He was a member of Hoboken Presbyterian Church, Blawnox Sportsmen Club, Blawnox "A" Club and the Springdale VFW. He enjoyed gardening and cutting grass. Survivors include his wife, Janet A. Sweet; sons, Samuel E. "Chip" (Michele) Sweet III, of Natrona, and William E. (Tracy) Sweet, of Edgewood; granddaughters, Kristin Ruff, of South Carolina, and Virginia Sweet, of Natrona; brother, John Thomas (Debbie) Sweet, of Warren; and nieces and nephews. Friends received from 2 to 6 p.m. Monday only, at the THOMAS M. SMITH FUNERAL HOME AND CREMATORY LTD., 930 Center Ave., Blawnox, followed by a funeral service at 7 p.m. in Hoboken Presbyterian Church, 921 North Ave., Blawnox, PA 15238. In lieu of flowers, the family respectfully suggests memorial contributions to Hoboken Presbyterian Church.
Addendum:
This case was settled before going to trial for $1,375,000.
-- SS
