Perioperative services departments have been slow to adopt the black boxes used in the aviation industry, partly because it can take a large team of experts hours to analyze data collected by the box. However, the team behind a surgical black box is using artificial intelligence (AI) to significantly cut analysis time.
Teodor Grantcharov, MD, PhD, FACS
"We've had a lot of success with automating much of the analysis and processing of the data captured by the OR Black Box®," says Teodor Grantcharov, MD, PhD, FACS, a professor of surgery at the University of Toronto and holder of the Keenan Chair in Surgery at St Michael's Hospital, both in Toronto, Ontario, Canada. That data can be used to identify areas of improvement in surgical workflow and technique.
The intent behind using the OR Black Box has been to enhance safety in the OR. "It's giving a voice to the frontline staff who are working in the room every day," says Catherine Hogan, PhD, RN, program director for perioperative services and infection prevention and control at St Michael's Hospital. Hogan has collaborated with Dr Grantcharov for several years.
Currently 12 hospitals in Canada, the United States, and Europe are using the OR Black Box, but AI increases the chances of more widespread use in the ORs of many nurse leaders (sidebar below).
The other side of the OR Black Box
Hospital leaders wanting to use the OR Black Box® may contact Teodor Grantcharov, MD, PhD, FACS, at St. Michael's Hospital in Toronto, Ontario, using the contact form at Surgical Safety Technologies (SST, https://www.surgicalsafety.com). SST is an academic start-up founded by Dr Grantcharov to advance use of the OR Black Box.
"Hospitals that are ready for this are those that have the right culture and right approach to quality and safety and don't want just to use it for publicity purposes," he says.
OR Black Box data from other hospitals are transmitted to Dr Grantcharov and his team in Toronto, where it's processed and returned to the originating hospital. "They can access it [the report] online and see how they did and how they compare with other hospitals of similar size and resources," Dr Grantcharov says. Some hospitals are using the analysis to drive their quality initiatives.
If the OR Black Box is being used as part of a research project, informed consent from the patient has to be obtained, but if it's used as a standard tool for quality improvement purposes, informed consent isn't necessary, Dr Grantcharov says. "Patients have been our biggest supporters," he adds. "They start going to hospitals that have the OR Black Box and going to surgeons who work in hospitals with it."
Inside the OR Black Box
The OR Black Box is a tool that supplements ongoing efforts to analyze events in the OR. "We've been working on developing methods to evaluate performance from a human factors point of view and design educational and quality improvement interventions for close to 20 years," says Dr Grantcharov.
The main purpose of the OR Black Box is to identify the chain of events leading to intraoperative near misses, which, if not recognized, could lead to an adverse outcome. "The intraoperative adverse event is the central focus of our analysis," Dr Grantcharov says. "We want to see it. We want to study it. We want to develop processes that allow us to prevent it."
Initially, the work relied on manual observation, but in 2010, the team started developing the OR Black Box to enable better analysis. The platform now captures and synchronizes intraoperative data from several sources, including the laparoscopic camera (currently, use of the OR Black Box is limited to laparoscopic procedures), panoramic room cameras, audio capturing devices, anesthesia monitors, and other sensors. Over the years, the team has refined the box to enable it to capture large amounts of data that could be used to incorporate AI.
The AI techniques that Dr Grantcharov and his team are using include machine learning and computer vision, which enable computers to learn from images and videos that are fed into them. In the case of the OR Black Box, a large library of surgical videos was used to teach the computer, which can now analyze much of a video on its own. For example, because the computer has seen thousands of different types of bleeding, it can now determine whether bleeding is (or isn't) significant.
Now a team of nearly 20 full-time AI engineers on the OR Black Box team is driving to automate as much of the analysis as possible. "Our goal is that by the end of 2019, 50% of the processing will be machine," says Dr Grantcharov. "Some aspects of the analysis, for example, looking at distractions, disruptions, and device performance, are now fully automated."
Useful analysis depends on collecting data from ORs in a particular location. "Otherwise, we just develop generic one-size-fits-all solutions that don't always work," Dr Grantcharov says. "What works in Toronto may not work in New York, Amsterdam, or Copenhagen."
So, what have OR Black Box data revealed? A 2018 study by Jung and colleagues analyzed data from 132 elective laparoscopic procedures and found that auditory distractions occurred a median of 138 times per case, and at least one cognitive distraction occurred in nearly two-thirds (64%) of cases. The OR door opened about once every 2 minutes. Events, defined as tissue injuries caused by healthcare providers that have potential to cause patient harm, occurred at a median rate of five per hour of operating.
Success stories
St Michael's Hospital has had OR Black Boxes for the past 4 years. The hospital has six ambulatory ORs and 15 inpatient ORs, two of which have the OR Black Box platform. The plan is to implement the platform in each of the eight new ORs that will soon be added.
Hogan says the OR Black Box has reduced disruption in the OR from people coming in and out. "We gained insight into the number of times the doors were opening, and it prompted the team to think twice about why they are going into a room at that time," she says. Team members now consider other ways to communicate, including whether the information can wait until after the case. For instance, a nurse who wants to switch shifts does not need an immediate answer.
Use of the OR Black Box also has limited the number of people in the room for a case. "As a teaching hospital, we often have people who want to come and watch surgery, but we now look more closely at who is in the room and whether they really need to be there," Hogan says. "We're finding that once you get beyond eight in the theater, the dynamics in the room shift, and communication becomes a little more challenging."
Catherine Hogan, PhD, RN
Multiple conversations result in people becoming distractors instead of enablers. "We will ask people to leave the theater if they aren't necessary," says Hogan, who adds that she validates findings from the OR Black Box with staff. For instance, staff agreed that the number of people in the OR was often too high.
Hogan says an important value of the OR Black Box is that it's research based. "We have data to back up our recommendations," she says. Those recommendations can be as simple as lowering the volume of music played in the OR based on auditory analysis of data from the OR Black Box.
Dr Grantcharov adds that the OR Black Box allows perioperative teams to quantify not only safety threats but also resilience supports (factors that allow systems and teams to be successful despite conditions that can lead to failure) using the Systems Engineering Initiative for Patient Safety Model. (This model is described at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2464868/.)
The research team at St Michael's Hospital in Toronto and The Academic Medical Centre in Amsterdam use packaged video reports of the events as part of safety discussion during a debriefing of the surgical team. The debriefings have prompted positive practice changes.
Culture change
Successful implementation of the OR Black Box depends on having the right organizational culture. "With the OR Black Box, we don't only focus on things that go wrong," Dr Grantcharov says. "We also want to capture things that we do exceptionally well." He notes that clinicians often achieve good patient outcomes even when an adverse event occurs, and there are lessons to be learned from that.
"We want to change the way hospitals usually address quality improvement," Dr Grantcharov says. Too often, action is taken only after an adverse event, which, he says, creates a natural resistance. "People believe that quality and safety is a bad thing because the only time they talk to quality and safety people is when they make a mistake," he notes.
A better approach is to use OR Black Box data to reinforce positive behaviors and examine processes that near misses have indicated need to be evaluated. In addition, the entire process should be transparent.
Initial resistance to an OR Black Box is common. "The first response is, ‘Is this big brother?' ‘Are you going to use this data to blame and shame people?'" Dr Grantcharov says.
He emphasizes that the system doesn't capture any personal data. "We don't care who the doctor was, who the nurse was, or who the patient was," he says. "All we want to know is what happened; we're only interested in the aggregate data." He uses the analogy of how large airlines analyze the data collected by black boxes during routine flights for analysis by a corporate safety team. The overall issues identified are discussed, but no individual data are shared.
When the OR Black Box was first implemented at St Michael's Hospital, Hogan says she and Dr Grantcharov worked together to educate the team, explaining that the data obtained wouldn't be used for disciplinary purposes. "It's not a camera on them; it's a camera on the room," Hogan says. "We were very transparent and showed them what the tape looked like."
"The OR Black Box is used to improve our practice, our team dynamics, and our communication," Hogan adds. She notes that physicians, nurses, and anesthesia providers in the OR are collaborating to create safer, more positive outcomes for patients, rather than working as independent disciplines. For example, the team meets monthly to review findings and decide what needs to be examined more closely.
As clinicians see the benefit of the OR Black Box to the team, acceptance increases. "We now have 100% of the general surgeons and 100% of the nurses [at St Michael's Hospital] on board," Dr Grantcharov says. He notes that most people go to work wanting to do a good job, and if something goes wrong, it's usually because of a systems issue that needs to be addressed.
That's one reason why Dr Grantcharov is concerned about the recent initiative in Wisconsin to require a camera in every OR. Assembly Bill 863 would require facilities providing surgery to give patients the option to have their surgery and discharge instructions recorded and made part of the healthcare record.
"The sad thing is that it would not lead to transparency and improvement," he says. "It will alienate us from patients and from each other, while fueling the litigation industry." The bill failed to pass in March 2018.
"A fundamental principle is that this is an initiative that comes out of our profession," Dr Grantcharov says. "It's designed to make all of us better."
Into the future
"We know that AI will help us put the OR Black Box in every operating room around the world," says Dr Grantcharov. Challenges to this expansion include computational power. "A lot of these algorithms require very powerful computers that cost hundreds of thousands of dollars," he says, though he foresees prices dropping to more affordable levels worldwide within a few years.
"We feel that the dramatic impact where we will save thousands of lives is going to happen, but we won't be able to do it if we don't make it affordable," he says.
As with driverless cars, Dr Grantcharov says, it's simply a matter of time before the OR Black Box will be able to provide real-time suggestions and feedback in the OR, for example, letting the surgeon know when he or she may be about to make a mistake. "We have put ourselves on a timeline of 5 years to develop the first version of real-time feedback," he says. "We're confident we can do at least some aspects of it earlier." ✥
Cynthia Saver, MS, RN, is president of CLS Development, Inc, Columbia, Maryland, which provides editorial services to healthcare publications.
References
Bonrath E M, Gordon L E, Grantcharov T P. Characterising ‘near miss' events in complex laparoscopic surgery through video analysis. BMJ Qual Saf. 2015;24(8):516-521.
Jung J, Jüni P, Lebovic G, et al. First-year analysis of the operating room black box study. Ann Surg. 2018;e-pub ahead of print. https://elautoclave.files.wordpress.com/2018/08/jung.pdf.
Wisconsin State Legislature. Assembly Bill 863. http://docs.legis.wisconsin.gov/2017/proposals/ab863.