importance of safety case study

Case Studies in Workplace Safety: Real-Life Examples of Successful Safety Initiatives

Introduction

Workplace safety is not just about compliance; it’s about saving lives, preventing injuries, and creating a culture of well-being. To illustrate the power of successful safety initiatives, we’ll dive into real-life case studies where organizations have gone above and beyond to ensure the safety of their employees. These inspiring stories showcase how proactive safety measures can lead to reduced incidents, improved morale, and enhanced overall productivity.

1. Tesla’s “Safety First” Culture

Tesla, the electric vehicle pioneer, places a strong emphasis on workplace safety. The company has developed a comprehensive “Safety First” culture, which goes beyond just meeting regulatory requirements. Tesla’s initiative includes rigorous safety training, daily huddles to discuss safety concerns, and a proactive approach to addressing any issues.

Result: Tesla has seen a steady decrease in workplace incidents and an increase in employee engagement. Their safety culture not only protects workers but also reinforces the organization’s commitment to innovation and excellence.

2. Alcoa’s Transformation Through Safety

Alcoa, a global aluminum manufacturer, underwent a significant transformation in the 1980s under the leadership of Paul O’Neill. O’Neill made safety the company’s top priority. His philosophy was simple: “If you want to improve safety, you have to improve everything.”

Result: Alcoa’s safety initiative not only led to a remarkable reduction in workplace injuries but also significantly boosted productivity and profitability. By focusing on safety, the company transformed its entire organizational culture.

3. DuPont’s STOP Program

DuPont, a multinational chemical company, introduced the STOP (Safety Training Observation Program) initiative to empower employees at all levels to identify and report safety hazards. The program encourages workers to speak up, and it trains them to spot potential dangers in the workplace.

Result: DuPont’s STOP Program led to a dramatic decrease in accidents and injuries. It also fostered a culture of transparency and accountability, where every employee played a vital role in ensuring their safety.

4. The Massachusetts General Hospital’s “Safety Matters” Campaign

Healthcare organizations are not exempt from the importance of workplace safety. The Massachusetts General Hospital initiated the “Safety Matters” campaign to reduce medical errors and ensure the safety of patients and healthcare workers.

Result: By emphasizing the importance of safety protocols and encouraging communication among staff, the hospital saw a significant reduction in patient safety incidents and workplace injuries. The “Safety Matters” campaign not only saved lives but also improved the quality of patient care.

5. Singapore Airlines’ Safety Excellence

In the aviation industry, safety is paramount. Singapore Airlines is a shining example of a company that prioritizes safety. The airline invests heavily in safety training and has a robust reporting system that encourages employees to report safety concerns without fear of reprisals.

Result: Singapore Airlines’ relentless commitment to safety has led to an impeccable safety record and a strong reputation for being one of the world’s safest airlines. Passengers and employees alike can feel confident in their commitment to safety.

These real-life case studies demonstrate that a proactive approach to workplace safety not only saves lives but also has a positive impact on an organization’s bottom line. Successful safety initiatives are not merely a box-ticking exercise; they are a reflection of an organization’s values and a commitment to the well-being of its employees. By learning from these examples, we can inspire a culture of safety in our own workplaces and reap the benefits of reduced incidents, improved morale, and increased productivity.

• Magazine Subscription
• ENewsletters
• EHS Education
• Safety Leadership Conference
• America's Safest Companies
• Search Search
• Members Only
• Environment
• Safety Technology
• Training & Engagement

• Safety Leadership

How Did They Do That? Case Studies on EHS Excellence

While it might be an overused phrase — “no need to reinvent the wheel” — there is a lot to be learned from others who have tackled and found solutions to safety issues that many companies face. 

In this vein, the winners of the Campbell Award offer materials including business case studies, teaching notes and case study presentations. 

Frist a word about the award, which is a National Safety Council award. Its mission is to “identify and provide evidence-based findings that enable current and future business leaders to effectively advance business vitality through embracing the value of environmental, health and safety management.” The award is supported by a network of Global Partners across five continents. 

The Campbell Award recognizes organizations for commendable leadership and excellence in integrating EHS management with business operations systems. The award aims to:

• Establish a validated process by which organizations can measure the performance of their EHS operations system against well tested and internationally accepted key performance indicators.
• Capture and evaluate the successes and lessons learned through a rigorous systematic review process.
• Foster the sharing of leading-edge EHS management systems and best practices for educational purposes worldwide.
• Recognize organizations that have EHS well integrated as a key business value and in which measurable achievements in EHS performance are productive and profitable.

Here are some samples (and excerpts) of these case studies:

Johnson & Johnson -- Social Responsibility & Sustainable Competitive Advantages 

This business case study starts with the assumption that EHS is a core value — because, for them, it is. Johnson & Johnson’s leaders, however, are aware that this belief is not shared by everyone — including some of its shareholders, who may be more focused on profit margin. By examining the company's credo-based culture and a number of its key EHS initiatives, Johnson & Johnson wanted students to come to understand the myriad ways in which social responsibility offers the company a truly sustainable competitive advantage. 

Alcan- Leadership Challenges in Cross-Culture Ventures

This business case study takes a well-developed integrated EHS management system and puts it to the test in an extreme physical and cultural environment. In examining the astonishing results at the Ningxia facility, Alcan leadership wanted students to understand the difference an integrated, transferable system can make. Through this remarkable example, students come to see the value of integration and transfer as it extends not only to the business but the wider community — making the decision they’re faced with at the end of the study even more crucial.

Dow—Inseparability of Safety

“If you can’t do it better, why do it?” These famous words of Dow Chemical Company founder Herbert H. Dow begin the organization’s Campbell Award Case Study. It is this history of innovation, coupled with aspirational goal-setting and world-class expertise, which has enabled Dow to continue to reach and exceed its vision. In this study, readers gain insight into Dow’s transformational goal development process, including a look at incentives, empowerment, risk assessment, leading indicators, and more. Readers have the opportunity to put themselves in the shoes of Dow leadership and ask themselves, “What should we do next?”

More  examples can be found here .  

There is also a Campbell Institute that was launched from the Campbell Award winners as well as thought-leaders from the National Safety Council. The mission of the Institute is to help organizations achieve and sustain EHS excellence through participation, research and events.

Adrienne Selko | Senior Editor

Email [email protected]

Adrienne Selko is also the senior editor at  Material Handling and Logistics  and is a former editor of  IndustryWeek. 

Continue Reading

Johnson & Johnson Takes Worldwide Approach to Ergonomics

NSC: Award Winner Puts EHS First

Sponsored recommendations.

Free Webinar: ISO 45001 – A Commitment to Occupational Health, Safety & Personal Wellness

ISO 45003 – Psychological Health and Safety at Work

DH Pace, national door and dock provider, reduces TRIR and claims with EHS solution

Case Study: Improve TRIR from 4+ to 1 with EHS Solution and Safety Training

Voice your opinion, to join the conversation, and become an exclusive member of ehs today, create an account today.

DOL Mine Inspections Result in Significant Violations

Emergency Responder Standards, Heat Stress and Training Tactics: What You’re Reading This Month

Safety Leadership Conference 2024 to Co-locate with VPPPA’s Safety+

SLC 2024 Preview: The 4 Keys to a World-Class Safety Culture

Photo Gallery: Photos from the Deadly Yarnell Hill Fire

6 Steps to Build Leadership Buy-In for EHS Technology

6 Qualities That Make a Safety Leader

Soft Skills 101: Three Essential Personal Attributes for Safety Professionals

OSHA's most interesting cases

What happened – and lessons learned.

Every OSHA investigation offers an opportunity for using what comes to light to help prevent similar incidents.

At the 2022 NSC Safety Congress & Expo in September, OSHA staffers highlighted three investigations – and the lessons learned – during the agency’s “Most Interesting Cases” Technical Session.

• Brian Elmore , an OSHA inspector based in Omaha, NE
• Marie Lord , assistant area director of the OSHA office in Marlton, NJ
• Peter Vo , safety engineer in OSHA’s Houston South area office

Here are the cases they presented.

• Shelving collapse in a cold storage warehouse
• Lockout/tagout-related amputation
• Crane collapse  

Related Articles

More of osha’s ‘most interesting cases’, post a comment to this article.

Safety+Health welcomes comments that promote respectful dialogue. Please stay on topic. Comments that contain personal attacks, profanity or abusive language – or those aggressively promoting products or services – will be removed. We reserve the right to determine which comments violate our comment policy. (Anonymous comments are welcome; merely skip the “name” field in the comment box. An email address is required but will not be included with your comment.)

Report Abusive Comment

Why Safety Culture Improvement Matters: Case Studies

By Josh Williams, Ph.D.

There is a large body of evidence showing the benefits of safety culture advancement including improved: safety motivation and participation (Neal & Griffin, 2006), employee commitment (Clarke, 2006), perceptions of leadership buy-in (Brown & Holmes, 1986), and other organizational factors like job satisfaction, likelihood of staying with the job, and decreased stress (Morrow & Crum, 1988). I would like to share a few examples of client case studies showing why safety culture improvement matters.

Improving safety culture is also associated with fewer workplace injuries (Barling et al., 2002; Clarke, 2006; Gillen et al., 2002; Zohar, 2000, 2002). There is no single metric to describe how long it will take incidents to reduce after a change effort because every company and every culture are different. There are variables that influence changes in target outcomes including the size of the company (e.g., Scott, 1998), the industry and sector of the company (e.g., Hoonakker et al., 2005), the maturity of the culture at the onset of the change process (e.g., Kezar, 2001), the amount of time the organization spends on the development effort (e.g., Weick & Quinn, 1999), how comprehensive the change efforts are (e.g., Krause et al., 1999), and the effectiveness of safety culture improvement interventions.   

Improving safety culture takes smart planning, effort, and long-term commitment. Here are a few examples of client case studies showing how safety culture advancement helps prevent incidents.

Unpacking the Importance of Safety Culture: Insightful Case Studies

Case study 1: power generation facility.

Propulo partnered with a large coal mine and power generation facility to help improve safety culture. In the shadow of a large-scale transition away from coal, they engaged Propulo to improve safety awareness and performance. Recognizing the disruptive impact of the transition away from coal, Propulo focused on understanding the degree to which change had affected the workers’ focus on safety. Targeting points of pain between leadership and the frontline workforce, we sought to create a more cohesive workplace which could facilitate more evolved safety conversation.

After initial scoping to understand the dynamics of their safety culture, we delivered person-based (e.g., attitudes and behavior) safety training, conducted safety leadership ownership activities, and provided guidance on process improvements for safety inspections, meetings, job planning, corporate communications, and change management. This resulted in a 38% reduction in TRIR and more than 900,000 exposure hours without an injury – the longest stretch in company history.

Case Study 2: Energy Provider

Propulo partnered with a very large energy provider to develop a customized and targeted safety program to reduce injuries. This organization had a vertically integrated operation across production, trading, and retail, and also used a variety of generation technologies like natural gas, LPG, hydro, wind, and thermal. Given their broad operational variety and the challenges that accompany vertical integration , our team focused on understanding how to address each of the diverse subcultures within their organization. We provided safety leadership and field level safety training and coaching that was customized for each group along with focused programs to provide structure for supporting this training. Targeted training is considerably more effective when paired with structural changes which permit the embedding of training concepts and learning transfer. This led to an 89% reduction in total recordables, a 55% decrease in lost time incidents, and a more than 1,000% reduction in average compensations claim costs (relative to the national average).

Case Study 3: Canadian Mining Company

Propulo worked with new ownership of a leading Canadian diamond mining company to assess their safety culture maturity, increase safety engagement, and improve overall safety performance. This organization was struggling with a perceived lack of a unified safety vision, noncompliance with site policies, low levels of hazard identification and risk awareness, and subpar leadership capabilities among some team leaders. Also, there was a lack of personal accountability and belief that employees should look out for one another.

Propulo partnered with this organization to:

• Establish a steering team to guide the change effort.
• Support a communication strategy to personalize safety messaging that employees would “feel” as a value instead of just another company program.
• Identify and train process champions across departments who would serve as leaders in their respective community to support the change effort.
• Implement a safety culture leadership training strategy to all site leaders, from executives to frontline supervisors.
• Deliver safety culture and ownership training to all employees.
• Develop a sustainment strategy that included embedding training concepts in safety meetings and pre-job briefs, implementing monthly safe production communication themes, and developing a visual campaign to reinforce safe production values.
• Support executive messaging to better align corporate communications and reinforce the message that “safe mining” is what we do and not just a corporate goal.

Over this two-year period, the organization was able to reduce total recordable rates from 4.17 to 1.1 which was one of the lowest in the Canadian mining industry (average was 3.28).

Managing Expectations

It is important to understand that changes in lagging safety indicators (incidents, injuries, property damage) typically lag safety culture improvements. Although the trend is that companies will enjoy a gradual decrease in injuries across several years after implementing quality safety improvement programs, the effects of change on safety outcomes do not happen in a predictable fashion (a straight line). Across a multi-year time period, there will likely be years where there is a steeper reduction in injuries, while other years could more steadily decline or plateau. For example, Krause et al. (1999) studied the effect of behavior-based safety management changes on injury reduction over time in 73 companies. Results showed the change was weakest (on average) in the earlier time periods and were the strongest at year five (i.e., five years after the interventions were completed). So, it is reasonable to expect that culture change takes time, some of the strongest effects may be seen after several years, and the results won’t be a linear and consistent trend.

The end game for conscientious leaders is to decrease the quantity and severity of incidents and injuries. Improving safety culture has been demonstrated to reduce the likelihood of these events occurring, even if it doesn’t follow a predictable path or happen overnight. Conducting safety culture assessments with strategic planning, delivering safety leadership skills training and coaching, and providing ongoing executive coaching are just a few ways that Propulo can help leaders improve safety culture to prevent serious injuries and fatalities and other incidents.

Case Study: BBS (Behavior-Based Safety) 2.0

Improve Your Safety Systems

RELATED POSTS

Change Management for Safety Derailers: Look Out for These Red Flag Conditions

A Novel Way to Incentivize Safety: Case Study

Improving Safety Leadership: Leading with Emotional Intelligence

• Browse All Articles
• Newsletter Sign-Up

• 14 Jul 2022
• Research & Ideas

When the Rubber Meets the Road, Most Commuters Text and Email While Driving

Laws and grim warnings have done little to deter distracted driving. Commuters routinely use their time behind the wheel to catch up on emails, says research by Raffaella Sadun, Thomaz Teodorovicz, and colleagues. What will it take to make roads safer?

• 15 Mar 2022

This Workplace Certification Made Already Safe Companies Even Safer

New research by Michael Toffel and colleagues confirms what workplace safety advocates have long claimed: Adopting OHSAS 18001 reduces worker injuries and improves a brand's image. Open for comment; 0 Comments.

• 17 Aug 2021

Can Autonomous Vehicles Drive with Common Sense?

Driverless vehicles could improve global health as much as the introduction of penicillin. But consumers won't trust the cars until they behave more like humans, argues Julian De Freitas. Open for comment; 0 Comments.

• 17 Sep 2019
• Cold Call Podcast

How a New Leader Broke Through a Culture of Accuse, Blame, and Criticize

Children’s Hospital & Clinics COO Julie Morath sets out to change the culture by instituting a policy of blameless reporting, which encourages employees to report anything that goes wrong or seems substandard, without fear of reprisal. Professor Amy Edmondson discusses getting an organization into the “High Performance Zone.” Open for comment; 0 Comments.

• 11 Jun 2019
• Working Paper Summaries

Throwing the Baby Out with the Drinking Water: Unintended Consequences of Arsenic Mitigation Efforts in Bangladesh

In this study, households that were encouraged to switch water sources to avoid arsenic exposure experienced a significant rise in infant and child mortality, likely due to diarrheal disease from exposure to unsafe alternatives. Public health interventions should carefully consider access to alternatives when engaging in mass behavior change efforts.

• 31 Jan 2019

How Wegmans Became a Leader in Improving Food Safety

Ray Goldberg discusses how the CEO of the Wegmans grocery chain faced a food safety issue and then helped the industry become more proactive. Open for comment; 0 Comments.

• 09 May 2018

A Simple Way for Restaurant Inspectors to Improve Food Safety

Basic tweaks to the schedules of food safety inspectors could prevent millions of foodborne illnesses, according to new behavioral science research by Maria Ibáñez and Michael Toffel. Open for comment; 0 Comments.

• 12 Sep 2016

What Brands Can Do to Monitor Factory Conditions of Suppliers

For better or for worse, it’s fallen to multinational corporations to police the overseas factories of suppliers in their supply chains—and perhaps make them better. Michael W. Toffel examines how. Open for comment; 0 Comments.

• 17 Jun 2016

Companies Need to Start Marketing Security to Customers

The recent tragedies in Orlando underscore that businesses and their customers seem increasingly vulnerable to harm, so why don't companies do and say more about security? The ugly truth is safety doesn't sell, says John Quelch. Open for comment; 0 Comments.

• 05 Jan 2016

The Integrity of Private Third-party Compliance Monitoring

Michael Toffel and Jodi Short examine how conflict of interest and other risks lead to inaccurate monitoring of health, labor, and environmental standards.

• 21 May 2012

OSHA Inspections: Protecting Employees or Killing Jobs?

As the federal agency responsible for enforcing workplace safety, the Occupational Safety and Health Administration is often at the center of controversy. Associate Professor Michael W. Toffel and colleague David I. Levine report surprising findings about randomized government inspections. Key concepts include: In a natural field experiment, researchers found that companies subject to random OSHA inspections showed a 9.4 percent decrease in injury rates compared with uninspected firms. The researchers found no evidence of any cost to inspected companies complying with regulations. Rather, the decrease in injuries led to a 26 percent reduction in costs from medical expenses and lost wages—translating to an average of $350,000 per company. The findings strongly indicate that OSHA regulations actually save businesses money. Closed for comment; 0 Comments.

• 24 Jan 2011

Terror at the Taj

Under terrorist attack, employees of the Taj Mahal Palace and Tower bravely stayed at their posts to help guests. A look at the hotel's customer-centered culture and value system. Open for comment; 0 Comments.

• WordPress.org
• Documentation
• Learn WordPress
• Request Membership

Latest Articles

Workplace Fire Safety: Essential Strategies for a Safer Office

Comprehensive Guide to Fire Safety: Strategies for Prevention and Protection

Continual Professional Development (CPD) in Health, Safety, and Environment (HSE): A Guide for Ongoing Excellence

Preparing a Risk Assessment: A Comprehensive Guide

Creating an Effective Fire Response Plan for Your Office

Fire Safety Training: Empowering Employees for Emergency Situations

Office Fire Hazards: Identifying and Mitigating Risks

The Role of Fire Drills in Ensuring a Safe Workplace

Navigating Workplace Fire Safety: Essential Tips for Employees

Case studies in construction safety: lessons learned from real incidents.

Case studies in construction safety provide invaluable insights into real incidents, helping to understand the consequences of safety lapses and the importance of preventive measures. Here are summaries of some impactful case studies, highlighting key lessons learned:

Case Study 1: Structural Collapse Due to Inadequate Shoring

• Incident : A multi-story building under construction collapsed due to inadequate shoring and bracing during the concrete curing process.
• Consequences : Multiple injuries and fatalities occurred, along with significant project delays and financial losses.
• Lessons Learned : The importance of adhering to engineering specifications for temporary structures and the need for regular inspections by qualified personnel.

Case Study 2: Electrocution from Overhead Power Lines

• Incident : A worker operating a crane came into contact with overhead power lines, resulting in electrocution and death.
• Consequences : Loss of life, legal action against the company, and significant financial settlements.
• Lessons Learned : The critical need for hazard assessment regarding overhead power lines and the importance of safety training for equipment operators.

Case Study 3: Trench Collapse

• Incident : A trench collapse occurred due to a lack of proper shoring and trench boxes, trapping workers.
• Consequences : Fatalities and serious injuries, alongside OSHA fines and criminal charges.
• Lessons Learned : The vital role of following trench safety guidelines, including proper shoring and having a trench rescue plan in place.

Case Study 4: Fall from Height Due to Inadequate Fall Protection

• Incident : A worker fell from a significant height due to inadequate fall protection measures.
• Consequences : Fatal injury, increased insurance premiums, and significant OSHA fines.
• Lessons Learned : The necessity of fall protection systems for work at height and regular safety training on their use.

Case Study 5: Fire and Explosion Due to Chemical Mishandling

• Incident : Incorrect storage and handling of flammable chemicals led to a major fire and explosion.
• Consequences : Severe injuries, property damage, and long-term environmental impact.
• Lessons Learned : The importance of proper chemical storage, handling procedures, and emergency response planning.

Case Study 6: Scaffolding Failure Due to Overloading

• Incident : Scaffolding collapsed under the weight of too many workers and equipment.
• Consequences : Multiple injuries and a halt in construction activities.
• Lessons Learned : Adherence to load capacity guidelines for scaffolding and regular safety checks.

Case Study 7: Health Issues from Prolonged Exposure to Hazardous Materials

• Incident : Workers developed chronic respiratory issues from prolonged exposure to asbestos and silica without proper PPE.
• Consequences : Long-term health problems for workers, lawsuits, and compensation claims.
• Lessons Learned : The necessity of providing adequate PPE and training workers on handling hazardous materials.

These case studies underscore the multifaceted nature of construction safety, emphasizing the need for comprehensive safety management systems. They highlight the importance of adhering to safety protocols, continuous training, and proactive risk management to prevent accidents and protect workers. By learning from these incidents, construction managers can better prepare and mitigate similar risks in their projects.

You May Also Like

More From Author

+ there are no comments, cancel reply.

Save my name, email, and website in this browser for the next time I comment.

Notify me of follow-up comments by email.

Notify me of new posts by email.

Safety First: Implementing a Culture of Safety in Construction

Personal Protective Equipment (PPE) Essentials for Every Construction Worker

You may also like:.

EHS Daily Advisor

Practical EHS Tips, News & Advice. Updated Daily.

Injuries and Illness

Case studies in safety: a great training tool.

Updated: Nov 6, 2011

Case studies are a great safety training tool. It’s like CSI. Employees can really get involved examining the evidence and seeing why an accident occurred.

Safety case studies are fun, challenging, interactive, and a highly effective training method.

Armed with the knowledge they gain from examining the facts of real workplace accidents, workers can learn how to avoid similar incidents and injuries.

Here’s an example of such a case from BLR’s OSHA Accident Case Studies . This case is about a confined space incident.

The Incident

Two employees arrived at concrete pit at demolition site where they’d been working to salvage the bottom part of a cardboard baler imbedded in the pit. When the employees uncovered the pit, they both felt a burning sensation in their eyes.

Employee #1 climbed down into the pit to determine what might be causing their eyes to burn. He immediately climbed back out of the pit because it was hot. He decided to put a water hose into the pit to help cool it down.

The employees climbed down into the pit with the water hose. Both employees experienced chest tightness, difficulty breathing, and burning eyes. They decided to exit the pit because of the intolerable conditions.

Employee #2 climbed out first. As Employee #1 was climbing the ladder to get out, he was overcome by the fumes and fell back into the pit. He landed on his back, unconscious.

Employee #2 climbed down into the pit in an attempt to rescue employee #1, but was unable to lift him. Employee #2 exited the pit in order to get help. Unfortunately, by the time help arrived, Employee #1 had died of asphyxiation.

The accident investigation determined that employee #1 had attempted to extinguish a small cutting torch fire the day before by covering it with sand and dirt. Apparently the fire was not extinguished and smoldered overnight, which resulted in a build up of carbon monoxide inside the pit.

Try OSHA Accident Case Studies and give a boost to your safety training program with real-life case studies of actual industrial accidents from OSHA files. We have a great one on lifting. Get the details.

Discussion Questions

Once the case has been presented, some discussion questions can help kick off the analysis of the incident. For example:

• What are the potential hazards of confined spaces?
• What was the specific hazard in this case that cause a fatality?
• Were these workers properly trained and equipped to enter a confined space?
• What type of air monitoring should be done before entering a confined space?
• Was this a permit-required confined space? If so, were the workers familiar with the safety requirements of the permit?
• Was confined space rescue equipment readily accessible?
• Training? There is no indication on the accident report that the employees were trained as authorized entrants of confined spaces. If they did receive any confined space entry training, they clearly didn’t apply what they learned. Authorized entrants are trained on the hazards of confined spaces, atmosphere testing procedures, symptoms of lack of oxygen or exposure to toxic chemicals, personal protective equipment (PPE), communication equipment, rescue retrieval equipment, etc.
• Hazard warning? These employees entered the space despite experiencing "red flags," such burning eyes and unusual heat. An important part of training for confined space workers includes learning about hazards such as the symptoms of a lack of oxygen or exposure to toxic chemicals. Workers should never enter a space, and should immediately leave a space, in which they experience signs of hazardous conditions.

Even your most skeptical workers will see what can go wrong and become safety-minded employees with OSHA Accident Case Studies . They’ll learn valuable safety training lessons from real mistakes—but in classroom training meetings instead of on your shop floor. Get more info.

• Permit-required? Most confined spaces require a permit before workers can enter the space. Permit-required confined spaces have the potential for hazards such as hazardous atmospheres, engulfment, entrapment, falls, heat, combustibility, etc. By reviewing a permit, entrants know they have obtained all the necessary equipment and the atmosphere has been monitored so they know the space is safe to enter.
• Testing? This worker died of asphyxiation, or lack of oxygen. If the atmosphere in the pit had been tested prior to entry, this accident would not have occurred. Common monitoring practices require a check of the oxygen concentration, a check for flammable gases or vapors (especially important if welding is going to be done in the space), and finally, a check for any other toxic chemicals known to potentially be in the space. Monitoring is conducted before entering the space and periodically while workers are in the space.
• Rescue procedures and equipment? The worker who collapsed back into the pit while climbing out could not be rescued because he was not wearing required rescue equipment. He should have been wearing a full-body harness attached to a retrieval line that was connected to a winch-type system that could have been used to pull the unconscious worker out of the pit. Of course, the other employee would have had to have been trained in confined space rescue procedures.

Tomorrow, we’ll introduce you to another case from OSHA Accident Case Studies, this one about a materials handling accident that resulted in a serious back injury.

More Articles on Injuries and Illness

1 thought on “case studies in safety: a great training tool”.

PingBack from http://savant7.com/workaccidentreport/workplace-safety/case-studies-train-employees-to-look-for-accident-causes-and-prevention/

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

This site uses Akismet to reduce spam. Learn how your comment data is processed .

Case studies

Successful leadership.

There are many benefits to be gained from successful leadership in health and safety, as these case studies show.

Case study - North Staffordshire Combined Healthcare NHS Trust

The board found itself facing service improvement targets. Using new corporate and clinical guidance, it set about taking a 'whole systems' approach to managing corporate risk, giving one of its directors responsibility for the leadership of health and safety for the first time. Health and safety was also made a key item on the board agenda.

This has resulted in a much better integrated health and safety management system that increases the opportunity to identify and manage all corporate risks, and a much more open culture, improving reporting and monitoring. The board actively promotes a culture that gives staff the confidence to report incidents. This has resulted in:

• incidence rates reduced by 16% over two years;
• insurance premiums reduced by 10%.

Case study - British Sugar

British Sugar had an excellent safety record and was devastated in 2003 when it suffered three fatalities. Although health and safety had always been a business priority, the company recognised that a change in focus was needed to achieve behavioural change. This included:

• the CEO assigning health and safety responsibilities to all directors, and monthly reports go to the board;
• creating effective working partnerships with employees, trade unions and others;
• overseeing a behavioural change programme and audits;
• publishing annual health and safety targets, and devising initiatives to meet them.

Results include:

A two thirds reduction in both lost time and minor injury frequency rates over a 10 year period.

much greater understanding by directors of health and safety risks.

Case study – Mid and West Wales Fire and Rescue Service

To give health and safety a high priority, Mid and West Wales Fire and Rescue Service recognised that it was critical for its leadership to demonstrate to its staff that accountability for health and safety was a fundamental element in the success of its overall service delivery. The director of service policy and planning was nominated as the health and safety director for the service in order to clearly define the importance this subject held within the organisation. The director implemented a revised health and safety framework, which included a programme of fire station visits to engage the workforce, and placed a renewed emphasis on improving incident reporting, investigation and monitoring procedures. The service has reported:

• £100,000 reduction in insurance liability premiums in one year through improved corporate strategic risk management;
• 50% reduction in sickness absence through work related injury over a two year period;
• 50% reduction in injury incidence rate over a three year period.

Case study – Sainsbury's

An external health and safety audit identified a need to develop a unified approach, and also recommended more direction from the board, to develop an effective strategy.

The result was a radical revision of the company's approach, including:

• the group human resources director creating a health and safety vision, supported by a plan with targets over three years;
• training on health and safety responsibilities was introduced for all board directors.

This has resulted in:

• the board providing a role model for health and safety behaviour;
• 17% reduction in sickness absence;
• 28% reduction in reportable incidents;
• improved morale and pride in working for the company;
• raising the profile of health and safety so it is becoming embedded in the culture of the organisation.

When leadership falls short

Many high-profile safety cases over the years have been rooted in failures of leadership. When board members do not lead effectively on health and safety management, the consequences can be severe. These examples mark issues for all boards to consider.

Competent advice, training and supervision

Following the fatal injury of an employee maintaining machinery at a recycling firm employing approximately 30 people, a company director received a 12-month custodial sentence for manslaughter. The machinery was not properly isolated and started up unexpectedly.

An HSE and police investigation revealed there was no safe system of work for maintenance; instruction, training and supervision were inadequate. HSE's investigating principal inspector said: 'Evidence showed that the director chose not to follow the advice of his health and safety adviser and instead adopted a complacent attitude, allowing the standards in his business to fall.'

The managing director of a manufacturing company with around 100 workers was sentenced to 12 months' imprisonment for manslaughter following the death of an employee who became caught in unguarded machinery. The investigation revealed that, had the company adequately maintained guarding around a conveyor, the death would have been avoided.

The judge made clear that whether the managing director was aware of the situation was not the issue: he should have known as this was a long-standing problem. An area manager also received a custodial sentence. The company received a substantial fine and had to pay the prosecution's costs.

Risk assessment

A company and its officers were fined a total of £245,000 and ordered to pay costs of £75,500 at Crown Court in relation to the removal of asbestos. The company employed ten, mostly young, temporary workers; they were not trained or equipped to safely remove the asbestos, nor warned of its risk. The directors were also disqualified from holding any company directorship for two years and one year respectively.

Is this page useful?

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Published: 18 November 2019

A review and critique of academic lab safety research

• A. Dana Ménard   ORCID: orcid.org/0000-0002-3503-5559 1 &
• John F. Trant   ORCID: orcid.org/0000-0002-4780-4968 2  

Nature Chemistry volume  12 ,  pages 17–25 ( 2020 ) Cite this article

41k Accesses

117 Citations

221 Altmetric

Metrics details

• Chemical education
• Chemical safety
• Scientific community
• Social sciences

Over the past ten years, there have been several high-profile accidents in academic laboratories around the world, resulting in significant injuries and fatalities. The aftermath of these incidents is often characterized by calls for reflection and re-examination of the academic discipline’s approach to safety research and policy. However, the study of academic lab safety is still underdeveloped and necessary data about changes in safety attitudes and behaviours has not been gathered. This Review article critically examines the state of academic chemical safety research from a multifactorial stance, including research on the occurrence of lab accidents, contributors to lab accidents, the state of safety training research and the cultural barriers to conducting safety research and implementing safer lab practices. The Review concludes by delineating research questions that must be addressed to minimize future serious academic laboratory incidents as well as stressing the need for committed leadership from our research institutions.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 print issues and online access

251,40 € per year

only 20,95 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Testing the effectiveness of interactive training on sexual harassment and assault in field science

Inappropriate behaviours in a dental training environment: pilot of a UK-wide questionnaire

Biomedical doctoral students’ research practices when facing dilemmas: two vignette-based randomized control trials

Baudendistel, B. Investigation Report University of California, Los Angeles , Case No. S1110-003-09 (Department of Industrial Relations, Division of Occupational Safety and Health, Los Angeles, 2009).

Technical Bulletin AL-134: Handling air-sensitive reagents (Sigma Aldrich, 2012).

Benderley, B. L. California investigation report explains what went wrong for Sangji. Science http://blogs.sciencemag.org/sciencecareers/2012/01/yesterday-we-pu.html (2012).

Allen, K. A young lab worker, a professor and a deadly accident. Toronto Star (2014); https://www.thestar.com/news/world/2014/03/30/a_young_lab_worker_a_professor_and_a_deadly_accident.html

Benderly, B. L. Danger in school labs. Sci. Am. 303 , 18–20 (2010).

PubMed   Google Scholar  

Grabowski, L. E. & Goode, S. R. Review and analysis of safety policies of chemical journals. J. Chem. Health Saf. 23 , 30–35 (2016).

Google Scholar  

Langerman, N. Warning to all principal investigators. J. Chem. Health Saf. 19 , 42–43 (2012).

Kemsley, J. N. University of Hawaii fined $115,500 for lab explosion. Chem. Eng. News (2016); http://cen.acs.org/articles/94/web/2016/09/University-Hawaii-fined-115500-lab.html

Pinghui, Z. Three students die in blast at Beijing university laboratory. South China Morning Post (2018); http://www.scmp.com/news/china/society/article/2179543/three-students-die-blast-beijing-university-laboratory

Van Noorden, R. A death in the lab. Nature 472 , 270–271 (2011).

Texas Tech University laboratory explosion (U.S. Chemical Safety and Hazard Investigation Board, 2010).

Kemsley, J. N. 10 years after Sheri Sangji’s death, are academic labs any safer? Chem. Eng. News (2018).

Hunter, K. et al. Guidelines for chemical laboratory safety in academic institution s (American Chemical Society, 2016); https://www.acs.org/content/dam/acsorg/about/governance/committees/chemicalsafety/publications/acs-safety-guidelines-academic.pdf

Kaufman, J. A. Killed in lab accidents: Memorial Wall. Lab Safety https://www.labsafety.org/memorial-wall (2019).

Miller, A. J. M. & Tonks, I. A. Let’s talk about safety: Open communication for safer laboratories. Organometallics 37 , 3225–3227 (2018).

CAS   Google Scholar  

Young, J. A. How “safe” are the students in my lab? Do teachers really care. J. Chem. Educ. 60 , 1067–1068 (1983).

Accidents in waiting. Nature 472 , 259 (2011).

Jorgensen, E. F. Development and psychometric evaluation of the Research Laboratory Safe Behavior Survey (RLSBS). J. Chem. Health Saf. 24 , 38–43 (2017).

Peplow, M. & Marris, E. How dangerous is chemistry? Nature 441 , 560–561 (2006).

CAS   PubMed   Google Scholar  

Hellman, M. A., Savage, E. P. & Keefe, T. J. Epidemiology of accidents in academic chemistry laboratories. Part 1. Accident data survey. J. Chem. Educ. 63 , A267 (1986).

Van Noorden, R. Safety survey reveals lab risks. Nature 493 , 9–10 (2013).

Ayi, H.-R. & Hon, C.-Y. Safety culture and safety compliance in academic laboratories: A Canadian perspective. J. Chem. Health Saf. 25 , 6–12 (2018).

Simmons, H. E., Matos, B. & Simpson, S. A. Analysis of injury data to improve safety and training. J. Chem. Health Saf. 24 , 21–28 (2017).

Sieloff, A. C., Shendell, D. G., Marshall, E. G. & Ohman-Strickland, P. An examination of injuries and respiratory irritation symptoms among a sample of undergraduate chemistry students from a Public Northeastern University. J. Chem. Health Saf. 20 , 17–26 (2013).

Probst, T. M., Barbaranelli, C. & Petitta, L. The relationship between job insecurity and accident under-reporting: A test in two countries. Work Stress 27 , 383–402 (2013).

Rathman, T. L. & Schwindeman, J. A. Preparation, properties, and safe handling of commercial organolithiums: Alkyllithiums, lithium sec-organoamides, and lithium alkoxides. Org. Process Res. Dev. 18 , 1192–1210 (2014).

Mikula, H. et al. Practical and efficient large-scale preparation of dimethyldioxirane. Org. Process Res. Dev. 17 , 313–316 (2013).

Morandi, B. & Carreira, E. M. Iron-catalyzed cyclopropanation in 6 M KOH with in situ generation of diazomethane. Science 335 , 1471–1474 (2012).

Busura, S., Khan, F., Hawboldt, K. & Iliyas, A. Quantitative risk-based ranking of chemicals considering hazardous thermal reactions. J. Chem. Health Saf. 21 , 27–38 (2014).

Frontier, A. Laboratory techniques and methods to improve your experimental skills. Not Voodoo http://chem.chem.rochester.edu/~nvd/index.php (2019).

Lowe, D. How not to do it: Tertiary butyllithium. Science Mag https://blogs.sciencemag.org/pipeline/archives/2007/03/01/how_not_to_do_it_tertiary_butyllithium (2007).

Snyder, S. A. Essential Reagents for Organic Synthesis (eds Fuchs, P., Bode, J., Charette, A. & Rovis, T) (Wiley, 2019).

Urben, P. G. Bretherick’s Handbook of Reactive Chemical Hazards 7th edn (Elsevier, 2017).

Bertozzi, C. R. Ingredients for a positive safety culture. ACS Cent. Sci. 2 , 764–766 (2016).

CAS   PubMed   PubMed Central   Google Scholar  

Huising, R. & Silbey, S. S. Constructing consequences for noncompliance: The case of academic laboratories. Ann. Am. Acad. Pol. Soc. Sci. 649 , 157–177 (2013).

Hendershot, D. C. Process safety: Is safety “common sense”? J. Chem. Health Saf. 19 , 35–36 (2012).

Kemsley, J. N. Learning from UCLA. Chem. Eng. News 87 , 29–34 (2009).

Schmidt, H. Anatomy of an incident—Multiple failure of safety systems under stress. J. Chem. Health Saf. 25 , 6–11 (2018).

Cournoyer, M. E., Trujillo, S., Lawton, C. M., Land, W. M. & Schreiber, S. B. Anatomy of an incident. J. Chem. Health Saf. 23 , 40–48 (2016).

Phifer, R. Case study – Incident investigation: Laboratory explosion. J. Chem. Health Saf. 21 , 2–5 (2014).

Reason, J. The contribution of latent human failures to the breakdown of complex systems. Philos. Trans. R. Soc., B 327 , 475–484 (1990).

Young, J. A. How complacency can jeopardize safety. Chem. Health Saf. 6 , 5 (1999).

Wu, T.-C., Liu, C.-W. & Lu, M.-C. Safety climate in university and college laboratories: Impact of organizational and individual factors. J. Saf. Res. 38 , 91–102 (2007).

Steward, J. E., Wilson, V. L. & Wang, W.-H. Evaluation of safety climate at a major public university. J. Chem. Health Saf. 23 , 4–12 (2016).

Schröder, I., Huang, D. Y. Q., Ellis, O., Gibson, J. H. & Wayne, N. L. Laboratory safety attitudes and practices: A comparison of academic, government, and industry researchers. J. Chem. Health Saf. 23 , 12–23 (2016).

McEwen, L., Stuart, R., Sweet, E. & Izzo, R. Baseline survey of academic chemical safety information practices. J. Chem. Health Saf. 25 , 6–10 (2018).

King, M. F. & Bruner, G. C. Social desirability bias: A neglected aspect of validity testing. Psychol. Market. 17 , 79–103 (2000).

Edwards, A. L. The social desirability variable in personality assessment and research. (Dryden Press, 1957).

Wardlaw, M. J. Three lessons for a better cycling future. BMJ 321 , 1582–1585 (2000).

Finkelstein, E. A., Strombotne, K. L., Chan, N. L. & Krieger, J. Mandatory menu labeling in one fast-food chain in King County, Washington. Am. J. Prev. Med. 40 , 122–127 (2011).

Ménard, A. D., Houser, C., Brander, R. W., Trimble, S. & Scaman, A. The psychology of beach users: Importance of confirmation bias, action, and intention to improving rip current safety. Nat. Hazards 94 , 953–973 (2018).

Bretherick, L. Chemical laboratory safety: The academic anomaly. J. Chem. Educ. 67 , A12 (1990).

Hill, R. H. Make safety a habit! J. Chem. Health Saf. 25 , 12–17 (2018).

Darley, J. M. & Latane, B. Bystander intervention in emergencies: Diffusion of responsibility. J. Person. Soc. Psychol. 8 , 377–383 (1968).

Leggett, D. J. Identifying hazards in the chemical research laboratory. Process Saf. Prog. 31 , 393–397 (2012).

Stuart, R. Emergency response training for laboratory workers. J. Chem. Health Saf. 17 , 29–32 (2010).

Mogielnicki, R. P., Stevenson, K. A. & Willemain, T. R. Patient and bystander response to medical emergencies. Med Care 13 , 753–762 (1975).

Shotland, R. L. & Heinold, W. D. Bystander response to arterial bleeding: Helping skills, the decision-making process, and differentiating the helping response. J. Person. Soc. Psychol. 49 , 347–356 (1985).

Hill, R. H. & Finster, D. C. Academic leaders create strong safety cultures in colleges and universities. J. Chem. Health Saf. 20 , 27–34 (2013).

West, S. S., Westerlund, J. F., Stephenson, A. L., Nelson, N. C. & Nyland, C. K. Safety in science classrooms: What research and best practice say. Educ. For. 67 , 174–183 (2003).

Withers, J. H., Freeman, S. A. & Kim, E. Learning and retention of chemical safety training information: A comparison of classroom versus computer-based formats on a college campus. J. Chem. Health Saf. 19 , 47–55 (2012).

Nelson, D. A. Incorporating chemical health and safety topics into chemistry curricula: Past accomplishments and future needs. Chem. Health Saf. 6 , 43–48 (1999).

Fivizzani, K. P. Where are we with lab safety education: Who, what, when, where, and how? J. Chem. Health Saf. 23 , 18–20 (2016).

Wood-Black, F. Incorporating safety into the general chemistry curriculum. J. Chem. Health Saf. 21 , 14–21 (2014).

Crockett, J. M. Laboratory safety for undergraduates. J. Chem. Health Saf. 18 , 16–25 (2011).

Bradley, S. Integrating safety into the undergraduate chemistry curriculum. J. Chem. Health Saf. 18 , 4–10 (2011).

Burchett, S., Pfaff, A., Hayes, J. & Woelk, K. Exploding misconceptions: Developing a culture of safety through learner driven activities. J. Chem. Health Saf. 24 , 36–42 (2017).

Matson, M. L., Fitzgerald, J. P. & Lin, S. Creating customized, relevant, and engaging laboratory safety videos. J. Chem. Educ. 84 , 1727–1728 (2007).

Karapantsios, T. D., Boutskou, E. I., Touliopoulou, E. & Mavros, P. Evaluation of chemical laboratory safety based on student comprehension of chemicals labelling. Ed. Chem. Eng. 3 , e66–e73 (2008).

Reniers, G. L. L., Ponnet, K. & Kempeneers, A. Higher education chemical lab safety interventions: Efficacious or ineffective? J. Chem. Health Saf. 21 , 4–8 (2014).

Gallion, L. A., Samide, M. J. & Wilson, A. M. Demonstrating the importance of cleanliness and safety in an undergraduate teaching laboratory. J. Chem. Health Saf. 22 , 28–31 (2015).

Alaimo, P. J., Langenhan, J. M., Tanner, M. J. & Ferrenberg, S. M. Safety teams: An approach to engage students in laboratory safety. J. Chem. Educ. 87 , 856–861 (2010).

Kennedy, S. & Palmer, J. Teaching safety: 1000 students at a time. J. Chem. Health Saf. 18 , 26–31 (2011).

Makransky, G., Thisgaard, M. W. & Gadegaard, H. Virtual simulations as preparation for lab exercises: Assessing learning of key laboratory skills in microbiology and improvement of essential non-cognitive skills. PLoS ONE 11 , e0155895 (2016).

PubMed   PubMed Central   Google Scholar  

Staehle, I. O. et al. An approach to enhance the safety culture of an academic chemistry research laboratory by addressing behavioral factors. J. Chem. Educ. 93 , 217–222 (2016).

McGarry, K. A. et al. Student involvement in improving the culture of safety in academic laboratories. J. Chem. Educ. 90 , 1414–1417 (2013).

Ritch, D. & Rank, J. Laboratory safety in the biology lab. Bioscene 27 , 17–22 (2001).

Kapin, J. M. Beyond chemical safety— an integrated approach to laboratory safety management. Chem. Health Saf. 6 , 20–22 (1999).

Shariff, A. M. & Norazahar, N. At-risk behaviour analysis and improvement study in chemical engineering laboratories. Int. J. Chem. Environ. Eng. 2 , 51–55 (2011).

Wyllie, R., Lee, K., Morris-Benavides, S. & Matos, B. What to expect when you’re inspecting: A summary of academic laboratory inspection programs. J. Chem. Health Saf. 23 , 18–24 (2016).

Ferjencik, M. & Jalovy, Z. What can be learned from incidents in chemistry labs. J. Loss Prev. Process Ind. 23 , 630–636 (2010).

Young, J. A. The professional practice of chemical safety. Chem. Health Saf. 6 , 41–42 (1999).

Marendaz, J.-L., Friedrich, K. & Meyer, T. Safety management and risk assessment in chemical laboratories. CHIMIA 65 , 734–737 (2011).

Camino, F. E. Make safety awareness a priority: Use a login software in your research facility. J. Chem. Health Saf. 24 , 22–25 (2017).

Nitsche, C. I., Whittick, G. & Manfredi, M. Collecting reaction incident information: Engaging the community in sharing safety learnings. J. Chem. Health Saf. 25 , 2–5 (2018).

LaPierre, J. It’s 1:30 a.m.-Do you know who’s in your laboratories? Chem. Health Saf. 6 , 31–33 (1999).

Mulcahy, M. B. et al. College and university sector response to the U.S. Chemical Safety Board Texas Tech incident report and UCLA laboratory fatality. J. Chem. Health Saf. 20 , 6–13 (2013).

National Research Council. Prudent practices in the laboratory: Handling and disposal of chemicals (The National Academies Press, 1995).

Bayer, R. Lab safety as a collateral duty in small colleges. J. Chem. Educ. 61 , A259 (1984).

Kaufman, J. A. Safety in the academic laboratory. J. Chem. Educ. 55 , A337 (1978).

Scherz, P. Risk, prudence and moral formation in the laboratory. J. Moral Educ. 47 , 304–315 (2018).

Ashbrook, P. Laboratory safety in academia. J. Chem. Health Saf. 20 , 62 (2013).

Ashbrook, P. C. Hazard assessment. J. Chem. Health Saf. 21 , 35 (2014).

Ashbrook, P. C. Accountability. J. Chem. Health Saf. 20 , 48 (2013).

Czornyj, E., Newcomer, D., Schroeder, I., Wayne, N. L. & Merlic, C. A. Proceedings of the 2016 Workshop Safety By Design – Improving safety in research laboratories. J. Chem. Health Saf. 25 , 36–49 (2018).

Backus, B. D. et al. Laboratory safety culture: Summary of the chemical education research and practice – Safety in chemistry education panel discussion at the 46th Midwest and 39th Great Lakes Joint Regional American Chemical Society Meeting, St. Louis, Missouri, on October 21, 2011. J. Chem. Health Saf. 19 , 20–24 (2012).

Langerman, N. Laboratory safety? J. Chem. Health Saf. 16 , 49–50 (2009).

Langerman, N. Reactive chemistry incidents in laboratories. J. Chem. Health Saf. 16 , 23–26 (2009).

McCroskey, J. C. in Teaching communication: Theory, research, and methods (eds Daly, J.A., Friedrich, G.W. & Vangelisti, A.L.) 471–479 (Erlbaum Associates, 1990).

One injured in lab explosion at UCLA. Los Angeles Daily News https://www.dailynews.com/2019/02/05/one-injured-in-lab-explosion-at-ucla/ (2019).

Download references

Acknowledgements

ADM and JFT would like to thank the University of Windsor for salary support for the preparation of this work. We would also like to thank C. Houser, K. Soucie, M. Bondy, J. Hayward and D. Cavallo-Medved for their comments on earlier drafts of this paper.

Author information

Authors and affiliations.

Faculty of Science, University of Windsor, Windsor, ON, Canada

A. Dana Ménard

Department of Chemistry and Biochemistry, University of Windsor, Windsor, ON, Canada

John F. Trant

You can also search for this author in PubMed   Google Scholar

Contributions

A.D.M. wrote the draft of the paper; both A.D.M. and J.F.T. conducted the literature search and analysis; both A.D.M. and J.F.T. revised the paper.

Corresponding authors

Correspondence to A. Dana Ménard or John F. Trant .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

The investigation report prepared by California’s Division of Occupational Safety & Health relating to the laboratory accident at UCLA in 2008 that led to the death of Sheharbano Sangji.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Ménard, A.D., Trant, J.F. A review and critique of academic lab safety research. Nat. Chem. 12 , 17–25 (2020). https://doi.org/10.1038/s41557-019-0375-x

Download citation

Received : 18 January 2019

Accepted : 11 October 2019

Published : 18 November 2019

Issue Date : January 2020

DOI : https://doi.org/10.1038/s41557-019-0375-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Laboratory safety regulations and training must emphasize the underpinning research ethics perspectives.

• Bor Luen Tang

Journal of Academic Ethics (2024)

Developing a gamified artificial intelligence educational robot to promote learning effectiveness and behavior in laboratory safety courses for undergraduate students

• Qi-Fan Yang
• Li-Wen Lian
• Jia-Hua Zhao

International Journal of Educational Technology in Higher Education (2023)

Insights from a laboratory fire

• Mitchell P. Jones
• Kathrin Weiland
• Alexander Bismarck

Nature Chemistry (2023)

Large language models for chemistry robotics

• Naruki Yoshikawa
• Marta Skreta
• Animesh Garg

Autonomous Robots (2023)

Rank Has Its Privileges: Explaining Why Laboratory Safety Is a Persistent Challenge

• Gokce Basbug
• Ayn Cavicchi
• Susan S. Silbey

Journal of Business Ethics (2023)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Research article
• Open access
• Published: 31 August 2020

Exploring psychological safety in healthcare teams to inform the development of interventions: combining observational, survey and interview data

• Róisín O’Donovan   ORCID: orcid.org/0000-0003-1993-5319 1 &
• Eilish McAuliffe 1  

BMC Health Services Research volume  20 , Article number:  810 ( 2020 ) Cite this article

28k Accesses

64 Citations

40 Altmetric

Metrics details

Psychological safety allows healthcare professionals to take the interpersonal risks needed to engage in effective teamwork and to maintain patient safety. In order to improve psychological safety in healthcare teams, an in-depth understanding of the complex and nuanced nature of psychological safety is needed. Psychological safety concepts, including voice, silence, learning behaviour, support and familiarity, informed the current study’s investigation of psychological safety. This study aims to use a mixed-methods approach to develop an in-depth understanding of psychological safety within healthcare teams and to build on this understanding to inform the development of future interventions to improve it.

Survey, observational and interview data are triangulated in order to develop an in- depth understanding of psychological safety within four healthcare teams, working within one case study hospital. The teams taking part included one multidisciplinary and three unidisciplinary teams. Observational and survey data were collected during and immediately following team meetings. Individual interviews were conducted with 31 individuals across the four teams. Thematic analysis was used to analyse these interviews.

Survey results indicated a high level of psychological safety. However, observations and interviews captured examples of silence and situations where participants felt less psychologically safe. Findings from across all three data sources are discussed in relation to voice and silence, learning, familiarity and support.

The results of this study provide a detailed description and in-depth understanding of psychological safety within four healthcare teams. Based on this, recommendations are made for future research and the development of interventions to improve psychological safety.

Peer Review reports

Psychological safety is a multi-dimensional, dynamic phenomenon that concerns team members’ perception of whether it is safe to take interpersonal risks at work [ 1 ]. It is particularly important within healthcare teams who need to work interdependently to co-ordinate safe patient care within a highly complex, dynamic and high stakes work environment [ 2 ]. However, a culture of fear and low psychological safety still exists within healthcare organisations [ 3 , 4 , 5 , 6 , 7 ]. There is a need to develop and implement interventions to improve psychological safety within these teams [ 8 ]. The ongoing Covid-19 pandemic has highlighted the importance of psychological safety within healthcare teams. Cultivating psychological safety is necessary in order to enable healthcare teams to collectively redesign processes and services to cope with new challenges, learn from mistakes and implement changes accordingly [ 9 ]. In order to improve psychological safety, we must first understand the complexity and nuance of psychological safety within healthcare teams [ 8 , 10 ].

To date, there has been a paucity of cross-level and multilevel research on psychological safety [ 10 ]. This has limited our understanding of psychological safety, the concepts related to it and whether it varies across teams within the same organisation [ 10 ]. Within organisational research, collecting different kinds of data on the same phenomenon and triangulating this data can help researchers assess complex phenomena, such as psychological safety, more accurately [ 8 , 11 , 12 , 13 ]. In the current study we use survey, observational and interview data to develop an in-depth understanding of psychological safety within healthcare teams. Building on this understanding, we aim to inform the development of an intervention to improve psychological safety.

The constructs which informed our study design our outlined below. These include constructs which have been linked to either low or high psychological safety (voice and learning behaviour) and which have been found to support psychological safety (positive interpersonal relationships). These constructs play a particularly important role within the context of healthcare teams.

Psychological safety promotes voice and learning behaviour. Speaking up and voice behaviour are interpersonally risky behaviours which play an important role in healthcare teams [ 8 ]. Feeling psychologically safe can enable team members to engage in speaking up behaviour, such as asking questions, pointing out a mistake or near miss and making suggestions for improvement [ 2 , 13 , 14 , 15 , 16 ]. Psychological safety also enables learning behaviours, such as seeking help or feedback [ 1 , 17 , 18 ]. Learning behaviours are integral to healthcare teams’ ability to manage demanding conditions, with rapidly evolving knowledge and practice as well as their ability to learn from failure [ 17 , 18 ]. When healthcare professionals prioritise patient safety by engaging in speaking up and learning behaviours, it is indicative of their levels of psychological safety [ 19 ].

On the other hand, lack of psychological safety inhibits team members from speaking up and causes them to opt for avoidance behaviours, such as silence [ 20 ]. It is important to acknowledge that although employees may frequently engage in voice behaviour, they could also be withholding other ideas, suggestions or concerns [ 20 ]. This highlights the need to go beyond observable behaviours to explore the nuance and complexities of individuals’ experience of psychological safety. In order to do so, this study uses a combination of survey, observation and interview data to gain a full understanding of psychological safety.

Positive interpersonal relationships drive psychological safety [ 21 , 22 ]. Within healthcare teams, having positive relationships, effective role models and better teamwork climates encourages healthcare professionals to speak up for safety [ 19 ]. Team members relationship with the team leader has been found to influence their sense of psychological safety. For example, when leaders engage in supportive behaviour, such as inclusiveness and openness, they foster psychological safety for other team members [ 23 , 24 , 25 ]. Peer support and trust among team members also improves psychological safety within teams [ 10 ]. Psychological safety can build across time, increasing as team members become more familiar with one another and have positive experiences of engaging in interpersonally risky behaviour [ 10 , 13 , 26 , 27 ]. While positive, supportive and trusting interpersonal relationships can foster psychological safety, it is important to note that psychological safety does not imply a team without any conflict or problems [ 5 ]. Psychological safety is needed in order for productive conflict, such as task conflict, to occur [ 28 ]. Task conflict concerns disagreements related to differences in viewpoints, ideas and opinions about the task being performed and can result in learning and improved performance [ 13 , 28 , 29 ].

In this study, focusing on a single hospital as a case study, we use a mixed methods approach to further our understanding of psychological safety within four healthcare teams. We combine data collected through surveys, observations and individual interviews to gain a holistic understanding of psychological safety in these teams. Building on this understanding, we aim to inform the development of future interventions to improve psychological safety in healthcare teams.

Research setting and participants

This study was undertaken by the authors as part of a wider body of research aiming to develop an intervention to improve psychological safety in healthcare teams. This research was conducted with healthcare professionals working on one of four healthcare teams from within the same acute, suburban hospital. Three of the teams were unidisciplinary – physiotherapists, nurses and speech and language therapists – and one was multidisciplinary. The authors collaborated closely with hospital management in order to recruit healthcare teams from within the hospital. These teams were selected in collaboration with hospital management using purposive sampling in order to identify different team types as well as teams that held meetings amenable to observation. The lead researcher contacted the leader of each team to ask if their team would be interested in taking part in the team observation and survey. Before beginning observations, team members from within each team were asked to consent to the meeting being observed and to completing the survey following the meeting. For interviews, a combination of purposive sampling and snowball sampling were employed to recruit team members from within each team. After the observation and surveys were complete, the researcher informed the team that anyone who was willing to take part in an interview could contact them (the researcher) or their team leader who would then refer them to the lead researcher. Team members were recruited from across all staff grades and included team leaders as well as senior and junior team members [ 30 ]. The total number of participants who took part in each phase of data collection can be found in Table  1 .

The team as a whole were observed during regularly scheduled team meetings. Of those who attended the meeting, there was 100% participation rate in the survey within teams A and B. There was one team members in team C and another in team D who was part of the observations but did not complete a survey. Since surveys were kept completely anonymous, it was not possible to assess whether the same participants who completed the survey also took part in an interview. In teams A ( n  = 2), B ( n  = 1) and D ( n  = 1), participants who were not present during observations and who did not complete a survey were recruited by the team leader to take part in an interview.

Data collection

Within case study designs, the use of multiple sources of evidence in recommended in order to capture a holistic understanding of the phenomena being studied [ 31 ]. The current study triangulates multiple sources of data in order to gain an in-depth understanding of psychological safety in healthcare teams. Rather than adopting the commonly used approach to triangulation to gain more credible or valid results, we draw on a post-modern paradigm which views reality as having multiple, fractured dimensions and being socially constructed. This approach requires acknowledgment that all research findings are shaped by the approach used to collect them and that different methods of data collection will offer different results [ 32 ]. Therefore, we use multiple methods to deepen our understanding of psychological safety by encouraging re-interpretation of findings as data sources reveal new insights and, thus, facilitating a more complex and in-depth exploration of healthcare professionals experiences of psychological safety. This approach to triangulation of data has been termed crystallization [ 33 ].

Composite measure of psychological safety

Survey and observational data were collected using the composite measure developed and presented in a recent paper focused on measuring psychological safety in healthcare teams [ 34 ]. For each team, one weekly meeting was observed using the observation measure, which captured voice, silence, supportive or unsupportive and familiarity behaviours. The observational data were collected as part of a pilot test of the measure and, as a result, behaviours were amended following each observation, however the overall categories remained the same. Observations were completed by one researcher who sat at the table or in the room where each meeting was conducted. The observer tracked the behaviours displayed by the team leader and team members by making a mark in the “behaviour count” box for the relevant behaviour. Observations were made by only one researcher in order to reduce the inhibiting effect the presence of two researchers might have on the team’s behaviour. The observed team meetings varied in length. The meeting held by Team A lasted 30 min, Team B’s meeting lasted 90 min, Team C’s meeting lasted 70 min and Team D had a meeting which lasted 120 min. All meeting were the teams regularly scheduled team meetings. Each discussed clinical and/or management issues that were relevant to their team.

Following the meeting, team members were asked to complete the survey component of the composite measure. There were three sections in the survey which assessed participants’ psychological safety in relation to: the team leader, other team members and the team as a whole. As part of a pilot test of the survey, a 7 point Likert scale was used for teams A-C. For team D, a 10 point Likert scale was tested. This was done in order to check if using a 10 point Likert scale would give participants an even wider spectrum of response options and, thus, allow the survey to capture more variability in participants’ responses [ 35 ]. Surveys did not ask for any identifiable information and were kept completely anonymous. Participants completed their survey in the same room as one another. There was plenty of space for them to move freely in order to complete their survey in private and surveys were handed directly to the researcher once completed.

Semi-structured interviews were conducted with 31 participants from across the four teams. The full interview schedule is presented as a supplementary file . This interview data was collected in order to gain an in-depth understanding of individuals’ experience of psychological safety and explore whether there were any emerging differences compared to the team level observations or survey responses. Interviews were conducted in a private room located within the case study hospital and lasted an average of 28 min. A full description of the process used to collect and analyse the interview data can be found in O'Donovan, De Brún & McAuliffe (in preparation). Hybrid inductive-deductive thematic analysis was used to identify themes which correspond to the concepts covered in the observations and survey data. Descriptive, open codes were assigned to each interview. These codes were then reviewed and refined, with reference to the psychological safety literature, in order to identify overarching themes. Analysis also compared findings from individuals in the same team to explore the consistencies and inconsistencies across cases. Thematic analysis was chosen because it is a theoretically flexible approach to qualitative analysis which allows the combination of inductive and deductive methods [ 36 , 37 ]. As highlighted by Braun and Clarke [ 36 ], thematic analysis is a useful method for working within a participatory research paradigm, with participants as collaborators, and for producing qualitative analysis which can inform policy development. Since the overarching aim of this study was to inform the development of an intervention to improve psychological safety which is grounded in the experiences of healthcare professionals, thematic analysis was particularly suitable. Our analysis focused on themes which captured participants’ experiences of speaking up or remaining silent, engaging in learning behaviour and their experience of support, or lack of support and familiarity within the team.

Ethical approval was obtained for this study from the Human Research Ethics Committee in University College Dublin (Reference number: LS-17-67). Written informed consent was obtained from all participants prior to each stage of data collection. In order to maintain anonymity, no identifiable information was collected during observations or surveys. Interviews were assigned a code made up of P (participant), interview number (e.g. the first interview conducted within each team was given the number 1) and team letter (A, B, C or D) and any identifiable characteristics were removed from the interview transcripts.

Survey results

All survey responses are displayed in Table  2 . They indicated that team members felt psychologically safe. In team A, a mean response of 6.700 was given for section 1, 6.597 for section 2 and 6.212 for section 3.

Observations

A positive, constructive atmosphere was observed during the team meeting. While the team leader spoke the most, team members were given opportunities to speak up. However, five to six team members dominated the discussion and not all team members spoke. These observations indicated that both team members and team leaders engaged in voice, learning, supportive and familiarity behaviours. There were no counts of defensive voice, silence or unsupportive behaviour. The specific behaviours displayed can be seen in Table  3 and observer ratings can be seen in Table  4 .

Voice and silence

Team members described an open team atmosphere where they felt listened to, respected and psychologically safe. They felt comfortable speaking up about work issues or things “they felt very strongly” about and would go to their team leader if they were “frustrated” or “struggling” . Interviews highlighted that there were opportunities to speak up during meetings. However, team members remained silent about certain issues. Some team members believed that discussing conflict, personal or confidential issues within a team setting may not be “appropriate” . They would discuss these issues outside the team setting instead.

“but I would say maybe it’s when the group disassembles that some of those opinions come out, you know, it mightn’t always be that effective.”

Conflict was “pushed under the carpet” because team members wanted to avoid insulting or questioning others or didn’t want to “rock the boat or cause issues” . Conflict avoidance was linked to small team size which meant that “everybody knows each other” and it would “make it harder for yourself” to speak up about conflict. One junior team member felt uncomfortable speaking up about confrontational issues with more experienced team members due to a fear that they would be dismissive and think “sure what does {team member} know” . However, junior team members all felt comfortable or “confident” asking for help.

A senior member of team A suggested that explicitly asking for input from junior team members could improve psychological safety and speaking up.

“looking for people’s opinions rather than waiting for somebody to offer, like asking, maybe some of the younger members, because I actually do think their opinion is really valuable.”

A relaxed atmosphere was deliberately cultivated to encourage learning behaviour.

“we’re very aware of trying to create an environment, em, like relaxed environment because you know when they’re relaxed, they’re going to learn more, they’ll ask more questions”

Team members considered patient care to be their “focus” and felt they could speak up about patient safety issues. They recognised that speaking up about errors was important for learning and improvement within the team.

“there’s lots of different failures in the system that probably will lead to that happening again but it is important that we try, I suppose, to remedy them.”

Members of team A talked about their team having a solution-focused approach to problem solving and highlighted that the team’s supervision model created a psychologically safe space for learning. The team leader advocated for the importance of trying new things and learning from them. However, three team members said there was not enough time for learning and there was a need to schedule time for supervision, where concerns and ideas could be voiced.

“it’s just something that kind of does get put on the longer finger a little bit, so that, it probably would be good to actually have time like an actual scheduled time to do that.”

Members of team A talked about feeling supported by their team leader and their peers. This encouraged them to speak up.

“you can go to the group and they’ll have your back, in terms of, yeah, your kind of professional role.”

The team leader provided support in relation to career development as well as personal and work needs. Team members were confident they would be supported by their leader if/when they asked for it.

“whatever small little rubbish is going on in your life, she will take that, you know, really into consideration and she is really so much about the staff member.”

Familiarity

Familiarity between team members facilitated psychological safety. Team members found it easier to speak openly as they got to know one another better and worked together for longer.

“Yeah she’s very easy to talk to… because I’ve worked with her for about X years.”.

Lack of familiarity had a negative impact on team members’ feelings of psychological safety. One team member identified themselves as being new to the team. In addition, team members whose roles were more separate from the rest of the team felt less comfortable.

“I probably wouldn’t feel as comfortable getting involved in some of their discussions.”

Team members highlighted the need to build relationships with those who work in these separate roles.

Survey results indicated that team members felt psychologically safe. Participants gave a mean response of 6.750 for section 1, 6.405 for section 2 and 5.667 for section 3.

At first, the team leader gave feedback, then each team member had an opportunity to contribute. At the end of the meeting, team members were given an opportunity to raise any concerns. There were some tense moments where there may have been an undertone of confrontation. Jokes were used to defuse these moments. During these instances the observer felt that people could be holding back. All participants engaged in voice, supportive, learning and familiarity behaviours but also displayed unsupportive behaviours. Team members displayed defensive voice and silence behaviours.

According to interviews, the leader of team B created an open, inclusive team atmosphere which made team members feel psychologically safe.

“it’s an open forum and I’ve never really felt that I couldn’t say anything.”

Team members prioritised patients and would speak up about patient safety issues.

“because I would kind of be out for the patient, you know, so I would have enough {confidence}, to say that’s not acceptable, or it’s not acceptable behaviour.”

However, according to the team leader, meetings could be “more participative” without certain team members. This suggests that the presence of these team members reduces psychological safety for others. Participants described negative reactions to people speaking up during meetings, such as “tut tutting” , “rolling eyes” or “sighing” . The team leader highlighted the need for improving people’s behaviour during team meetings.

“I suppose people are less likely to contribute if they feel like that’s a risky response or there is the risk of that being a response.”

According to the team leader, improving psychological safety would involve making it clear to all team members that they play a valuable role in the team.

“it’s to convince, like everybody at that table has a critical role to play.”

Similar to Team A, conflict, personal or confidential issues were not deemed “appropriate” for the group setting and were discussed outside of team meetings. While conflict existed within the team it was not “open” and was “sometimes ignored” . Team members would withhold their “true feelings” to try to “keep the peace” , “incubate the mess in front of everybody” and to avoid making other team members feel attacked. When asked why conflict isn’t addressed, the team leader said, “we’re not there yet” .

Team members felt comfortable admitting mistakes and considered it to be the “whole point of the forum {team meeting}” and necessary for learning.

“you can’t fix them if you don’t highlight them.”

However, they also referred to individuals who have not admitted when they haven’t done something and have covered it up because “they don’t like to show up they’re not doing {something}.”

Interviews indicated that the team was going through an “evolution” and trying to become more focused on learning. This involved having dedicated time within and outside regular team meeting to discuss errors and concerns. According to the team leader, this improved speaking up and psychological safety in the team.

“We have had people say ‘I completely messed up’.”

Team members said that their leader is “100% behind you” and that leadership support played an important role in creating a psychologically safe environment.

“I think too it’s down to having the confidence in our leaders, in our leader as well. That you know that it’s kind of a safe space to talk.”

There was one reference to lack of peer support on the team. According to one participant, another team member has complained about a lack of support within the team, but this individual has not given support to others. This presented peer support as a reciprocal relationship between members.

“he wants support, he’s not giving support on the other side of it, he’s not giving support to us.”

One team member said they have received support in the form of other team members’ expertise.

“I’m not the expert in that field, I would be kind of guided by our {lists specific roles}.”

Most team members have worked in the hospital for long enough to be familiar with their colleagues. This made it easier for them to speak up.

“maybe because I’m here so long that maybe it’s a thing with age (laughs). You know, I don’t have a problem really in that kind of a setting speaking.”

There were three team members who identified themselves as being new to the team. A new team member didn’t feel the same level of comfort as others because she felt the team didn’t know her well enough. This team member highlighted the need for time with the team in order for her to become more comfortable.

“I’m still not 100% comfortable, I don’t think they know me yet.”

Survey results indicated that team members felt psychologically safe. Participants gave a mean response of 6.611 for section 1, 6.064 for section 2 and 5.308 for section 3.

There was a collaborative, inclusive and constructive atmosphere during the team meeting. While there were opportunities for participants to speak up, certain individuals dominated the discussions. All participants displayed voice, supportive, learning and familiarity behaviours. Team members displayed one count of unsupportive behaviour. There were no defensive voice or silence behaviours recorded.

Interviews referenced a historical culture of fear which lead to a lack of honesty and low psychological safety. One team member said that it has been difficult to change this culture completely, since the same people are still working there. As a result, some of this culture remained and team members reported silence and a lack of encouragement to speaking up.

“I think overall, the consensus was not to speak, there was never encouragement to speak so I can’t think of any occasion where I actually felt comfortable, there may have been occasions where I became so frustrated, that then I would have, you know, given my opinions, but that would not have been done in a comfortable environment.”

Position in the hierarchy influenced speaking up behaviour. The team leader was aware that team members may remain silent because of her role as leader. According to one team member, there was a reluctance to speak up when the team leader was present. This team member thought that there would be more open discussions and more things would “come out” if they could run their own meetings and meet their leader less regularly.

“they don’t want to say with the management there, because it will show them to be not coping as well.”

However, Team C was going through a “transition” and was changing from the historically “negative” culture. The team leader aimed to make the team more inclusive and let ideas “come from them {team members} up” . Team members felt psychologically safe with the leader because she is “reasonable” and they felt valued by her.

“I actually would be 100% confident, that if I did have to challenge any of her, em, any concerns that there wouldn’t be a problem that she’s extremely reasonable and yeah.”

Team members discussed patient safety and actively raised issues in order to have an open discussion and avoid creating tension or misunderstandings.

“I can voice it and everyone knows what my opinion is and I know what other people’s opinion is rather than them go to a one to one, and say, I don’t agree with x, y and z, because I think that’s kind of going behind people’s backs, in a certain way.”

However, like the other teams, they would discuss personal or confidential issues outside of the team setting.

Learning behaviour

The team leader said that by speaking up about mistakes, she could foster trust and encourage other team members to do the same.

“so I think the more they see me owning up to mistakes, the more they’re going to trust me {…} and the more they’ll own up to mistakes.”

One team member referred to the team as being very “pro-learning” and that team members were encouraged to speak up so they could learn from every “meeting or conflict” .

“they’re very much encouraging you to say it, it’s a safe space we’re not going to go back and, and talk about it, and we can learn, because I might be struggling with something, that someone else mentions, and I go, ‘oh god, I’m actually struggling with that area too, how are you going about it?’”

Team members referred to the leader being supportive, inclusive and “open” . One team member has learned from experience that she was more likely to get support if she approached team members one to one, rather than in a group setting

“I realised you’re better off actually going from one to one to one before than bringing it up {in a group setting}, and then you might get some support behind you.”

Familiarity between team members and the fact that they were all part of the same discipline, encouraged psychological safety.

“most of us have worked together for a while so we know what’s going on.”

One team member identified themselves as being new to the team. There was a lack of familiarity between the team leader and the other team members and there was still a need for trust to build between them.

“going to take a while though, {…} for me to be able to say to you, you know, tell you what, they do trust me and we’re open.”

One team member said that if the team had been given more opportunities to become familiar with one another when they first joined the team, they would feel more comfortable within team meetings.

“to get more familiar with each other and not to be as worried about somebody else being in the room.”

Survey results indicated that team members felt psychologically safe. Based on pilot testing, survey responses were altered to provide participants with a wider spectrum of response options (O’Donovan et al., in press). Responses could be between 1 and 10, 1 being “strongly disagree” and 10 being “strongly agree”. Participants gave a mean response of 7.704 for section 1, 8.071 for section 2 and 7.333 for section 3.

There was a positive, friendly and constructive atmosphere during the team meeting. While some tension was noted between the senior team members, there were opportunities for speaking up and most team members communicated openly. Decisions were made together, and team members seemed not be holding anything back. All participants displayed voice, silence, supportive, learning and familiarity behaviours. There was one count of unsupportive behaviour for the team leader.

All team members felt there were opportunities to speak up within team meetings and gave examples of speaking up about work related issues, including patient safety issues.

“the staff meeting, the biggest decisions are made at those and I think everyone gets an opportunity to weigh in.”

However, team members also noted that meetings mostly focused on operational issues. As a result, the issues that the team members wanted to raise were not given time.

“the bits that maybe us minions (laughs) want to talk about is back loaded.”

Participants suggested building time into meetings to discuss team members’ concerns and for them to connect in “a meaningful way” and understand where one another’s “emotions are at on a daily basis” in order to provide support and reassurance.

There was more pressure when raising issues in the group and so team members would speak to the leader about personal issues or would discuss conflict with one other team member.

“I would be more likely to bring things to my {team leader}, or to talk to people at an individual level em, if it was something sort of, I don’t know, beyond those boundaries.”

The majority of team members’ silence occurred when they gathered together as a group. Conflict or disagreements happened “under the covers” or got “brushed away” and were not discussed openly. Team members said they remained silent in team meetings to be polite and respectful to one another. They position this as a functional way to maintain good working relationships.

“so yeah, I think politeness really underpins a lot of our interactions.”

The influence of hierarchy and experience was also noted. Senior members were aware that junior team members were less likely “to rock the boat or make suggestions” . While junior team members were more nervous about speaking up, they became more psychologically safe as they gained experience.

“knowing what’s too much for one person really helped me figure out when to kind of just get on with it or when to kind of ask around.”

There were team members who remained silent about their ideas for change rather than risk sharing them with the team.

“don’t know if I’d feel comfortable suggesting too much change, just because I’d feel like that’s not really my remit or you know, everything, there’s a lot of well-established routines and I wouldn’t really be one to rock the boat too much.”

However, participants commented that the team has improved and become more open to learning and change. One member said it was easier for them to speak up and share their opinions during the meeting which was observed as part of this study.

“people were speaking up and we were able to push what was going to work best for us across.”

Examples were given of peer support. One participant referred to being supported during a difficult experience, explained that the team is very supportive, and that providing support is part of their jobs.

“I think this is one of the most supportive teams that I have ever experienced, em like I said there is nobody on the team that I haven’t spoken to about one thing or another.”

The leader was described as approachable, inclusive and supportive. Receiving support from the team leader made participants “confident in their choice” to speak up and share their opinion.

Familiarity between team members influenced psychological safety. They were more comfortable speaking to the team members who they worked closely with or considered good friends. There was one team member who identified themselves as being new to the team. One participant said that she would feel more “conscious” of what she was saying to team members who she didn’t work closely with, suggesting lower psychological safety. Others said their psychological safety improved as they got to know their colleagues better.

“I’ve become much more comfortable as I’ve gotten to know people to ask for help.”

This study provides an in-depth understanding of psychological safety within four healthcare teams working in the same case study hospital. It addresses recent calls for the use of multiple methods to capture a more accurate and nuanced understanding of psychological safety in healthcare teams [ 8 , 10 ]. While survey results from each team reflected high psychological safety, observations and interviews captured exceptional cases and more subtle dynamics within teams. Observations captured behaviours displayed during team meetings while interviews offered an insight into psychological safety both within and outside meetings, as well as within a historical context. An overview of the results can be seen in Fig.  1 . In this figure, we aim to highlight the indicators of psychological safety identified at both team and individual levels. Figure  1 also illustrates the added degree of variance in psychological safety captured by observations and interviews, in comparison to survey results. While survey results indicated medium to high levels of psychological safety within all teams, both observations and interviews captured examples of silence and an absence of learning behaviour. In addition, interviews highlighted examples of low levels of support from other team members and a lack of familiarity between certain team members. Survey and observation results captured team level dynamics which gave context to our understanding of individual level perceptions of psychological safety. This was particularly important, given that the overarching aim of this research is to inform the development of a team-level intervention to improve psychological safety.

Indicators of Psychological Safety Identified at the Team and Individual Levels. This figure illustrated the key findings coming from each data collection source. Positive and negative indicators of psychological safety are indicated in relation to voice, learning, support and familiarity

According to interviews, team members felt more psychologically safe speaking up about certain topics. All teams prioritised patient safety and felt comfortable speaking up about concerns related to patient safety, indicating a sense of psychological safety [ 19 ]. However, silence was observed in teams A, B and C, there were lower survey scores for questions referring to speaking up about personal issues or disagreements, and interviews indicated that team members found speaking up about conflict or personal issues difficult. While junior team members could ask senior members for help and advice, they found it difficult to raise issues that could be deemed as challenging or confrontational. The influence of hierarchy was seen in all teams. This corresponds with research illustrating that those with higher status reported higher levels of psychological safety [ 26 , 27 , 38 ]. While team members found it difficult to engage with conflict or disagreements during team meetings, some found it easier during one-to-one discussions with either the team leader or another team member involved in the issue. Learning behaviours were displayed throughout team observations, and team members’ survey responses indicated that they could ask questions and share ideas. Interviews highlight that teams B, C and D are going through transitions, engaging in more learning initiatives, and becoming more psychologically safe. The leaders of teams B and C are both actively creating an open and inclusive team environment. According to interviews, the leader of team C engages in inclusive leadership by explicitly asking team members for their input and sharing her mistakes with the team in order to role model speaking up behaviour. Research has shown that doing this encourages psychological safety [ 18 , 39 , 40 , 41 ]. In team B, the leader introduced protected time for developing teamwork and discussing concerns or mistakes made. While interviews indicated that team D has become more open, team members highlighted the need for protected time during team meetings for people to raise issues that were important to them, to connect to one another and to discuss their experiences that week.

All leaders displayed supportive behaviour during observations. The leader of team B showed the most supportive behaviour, using inclusive language throughout the meeting. Survey and interview data indicated that team members felt supported by their team leaders. However, interviews highlighted that there were still examples of team members not feeling comfortable taking interpersonal risks. While past research has highlighted the role of supportive leaders play in promoting psychological safety [ 23 , 24 , 25 ], supportive leadership alone is not enough. As discussed above, there were still issues that team members didn’t feel comfortable discussing and the impact of hierarchy and historical cultures of fear could still be seen.

Interpersonal dynamics also influenced psychological safety. As outlined in previous literature, peer support improves psychological safety within teams [ 10 ]. During observations of team B, team members showed lower supportive behaviours than their team leader and, during interviews, only one team member mentioned receiving peer support. Lower levels of peer support were found in the multidisciplinary team, where team members worked in different departments. When team members worked within the same discipline, it was easier for them to support one another in their roles and this facilitated psychological safety.

Since members of team B had been working in the hospital for a long time, they were familiar with one another. However, on teams A and D, some team members worked separately from the rest of the team. As a result, they were less familiar with and comfortable around other team members. A member of team C suggested that if team members were given time to get to know one another without the presence of their leader, they would become more comfortable and issues could be discussed more openly. Having close and connected working roles facilitates familiarity and, as a result, psychological safety. This highlights the need for teams whose roles are more separate from one another to make deliberate efforts to cultivate familiarity. This corresponds with the need to develop interventions which are suited for use across and between multiple disciplines [ 8 ]. This is particularly important in a healthcare context, where psychological safety is needed for “teaming”, an active process which allows multidisciplinary healthcare teams to work together to deliver increasingly complex patient care [ 10 , 42 ].

Implications for practice and future research

In addition to gaining an understanding of psychological safety within healthcare teams, this study aimed to inform the development of interventions to improve it. A recent systematic review of such interventions has highlighted that in order to improve their effectiveness, interventions targeting psychological safety need to be grounded in the experiences of the target audience [ 8 ]. This study contributes to a more in-depth understanding of psychological safety within healthcare teams which is needed to develop future interventions. Compared to observation and survey data, the data collected through interviews with team members provided the most valuable insights into the specific areas which interventions can target. This is because participants were given the opportunity to discuss areas in which they thought psychological safety in their team could be improved. Firstly, the issue of time was raised in each team. This mainly involved giving time during meetings for discussing more personal issues or experiences rather than only focusing on operational issues. This had already been done within team B and the team leader had noticed some improvements in trust and openness as a result. Time was also needed for prioritising learning and becoming more familiar with one another. Secondly, since team members felt more psychologically safe talking about difficult subjects during one-to-one interactions, these opportunities for interactions outside of a team setting should be encouraged. Interviews also highlighted the need to build relationships and foster familiarity with new team members and team members who work separately from the rest of the team. Lastly, interventions should encourage an awareness that all team members play a valuable role and explicitly ask for input from team members who are more vulnerable to low psychological safety, such as junior team members.

For future research, it is important to note the impact different data collection methods has on our understanding of psychological safety. As can be seen from Fig. 1 , surveys provided an overview of the levels of psychological safety within the teams, however, observations and interviews provide more detailed and nuanced understanding. Observations provided a more objective view of behaviours relating to psychological safety [ 34 ] and interviews offered insight into team members’ past and present experiences both within and outside team meetings. Researchers should be aware of the different levels of understanding gained from the use of these different methods of data collection and use this to ensure the type of data they collect is suited to their research question.

Strengths and limitations

This study combined survey, observation and interview data to gain an in-depth understanding of psychological safety within four healthcare teams. The deliberate inclusion of common components across the survey, observations, and interview analysis facilitated the triangulation of data. This provided a more detailed and holistic understanding of psychological safety.

However, some limitations must also be noted. Data was collected within one case study hospital, restricting the generalisability of findings. To address this, we have presented detailed analysis and given as much contextual information as possible for each team, without compromising the anonymity of participants. This should allow readers to determine whether they are applicable in other settings [ 32 , 43 ].

Through examining the different outputs using multiple methods of data collection, this study provides a more in-depth understanding of psychological safety in healthcare teams. Within observations, it was difficult to accurately observe silence, count individual episodes of silence and determine the motivation behind silence. However, interviews were a valuable source for exploring team members’ reasons for remaining silent. While both interviews and surveys are vulnerable to self-report bias [ 44 ], the observation measure offers a more objective measure of psychological safety. The observation measure also provides information on team-level dynamics related to psychological safety. This team-level understanding aided our analysis of interviews in order to capture individual team members’ perceptions of psychological safety. This understanding is important since the overarching aim of the programme of research (of which this study is one component) is to inform the design of an intervention to improve psychological safety at the team level. In order to calculate inter-rater reliability for the observation measure without inhibiting teams by having multiple coders present, future research could use a video camera to record meetings [ 45 , 46 ].

This study examines psychological safety within four healthcare teams. Results from surveys, observations and interviews are considered together in order to gain an in-depth understanding of psychological safety within these teams. Based on our findings, recommendations are made for future research and the development of interventions to improve psychological safety.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Edmondson A. Psychological safety and learning behavior in work teams. Adm Sci Q. 1999;44(2):350–83.

Google Scholar  

Nembhard IM, Edmondson AC. Making it safe: the effects of leader inclusiveness and professional status on psychological safety and improvement efforts in health care teams. J Organ Behav. 2006;27(7):941–66.

Moore L, McAuliffe E. To report or not to report? Why some nurses are reluctant to whistleblow. Clin Gov Int J. 2012;17(4):332–42.

Moore L, McAuliffe E. Is inadequate response to whistleblowing perpetuating a culture of silence in hospitals? Clin Gov Int J. 2010;15(3):166–78.

Edmondson AC. Speaking up in the operating room: how team leaders promote learning in interdisciplinary action teams. J Manag Stud. 2003;40(6):1419–52.

Unal A, Seren S. Medical error reporting attitudes of healthcare personnel, barriers and solutions: a literature review. J Nurs Care. 2016;5(6):377.

Swendiman RA, Edmondson AC, Mahmoud NN. Burnout in surgery viewed through the lens of psychological safety. Ann Surg. 2019;269(2):234–5.

PubMed   Google Scholar  

O’Donovan R, McAuliffe E. A systematic review exploring the content and outcomes of interventions to improve psychological safety, speaking up and voice behaviour. BMC Health Serv Res. 2020;20(1):1–11.

Stoller JK. Reflections on leadership in the time of COVID-19. BMJ Lead. 2020;4(2):77–9.

Newman A, Donohue R, Eva N. Psychological safety: a systematic review of the literature. Hum Resour Manag Rev. 2017;27(3):521–35.

Molina-Azorin JF, Bergh DD, Corley KG, Ketchen DJ Jr. Mixed methods in the organizational sciences: taking stock and moving forward. Organ Res Methods. 2017;20(2):179–92.

Edmondson AC, McManus SE. Methodological fit in management field research. Acad Manag Rev. 2007;32(4):1246–64.

Edmondson AC, Lei Z. Psychological safety: the history, renaissance, and future of an interpersonal construct. Annu Rev Organ Psychol Organ Behav. 2014;1(1):23–43.

Pearsall MJ, Ellis AP. Thick as thieves: the effects of ethical orientation and psychological safety on unethical team behavior. J Appl Psychol. 2011;96(2):401.

Bienefeld N, Grote G. Speaking up in ad hoc multiteam systems: individual-level effects of psychological safety, status, and leadership within and across teams. Eur J Work Organ Psychol. 2014;23(6):930–45.

Leroy H, Dierynck B, Anseel F, Simons T, Halbesleben JR, McCaughey D, et al. Behavioral integrity for safety, priority of safety, psychological safety, and patient safety: a team-level study. J Appl Psychol. 2012;97(6):1273–81.

Carmeli A, Sheaffer Z. How learning leadership and organizational learning from failures enhance perceived organizational capacity to adapt to the task environment. J Appl Behav Sci. 2008;44(4):468–89.

Hirak R, Peng AC, Carmeli A, Schaubroeck JM. Linking leader inclusiveness to work unit performance: the importance of psychological safety and learning from failures. Leadersh Q. 2012;23(1):107–17.

O’Donovan R, Mcauliffe E. A systematic review of factors that enable psychological safety in healthcare teams. Int J Qual Health Care. 2020;32(4):240–50.

Sherf EN, Parke MR, Isaakyan S. Distinguishing voice and silence at work: unique relationships with perceived impact, psychological safety, and burnout. Acad Manag J. 2020. https://doi.org/10.5465/amj.2018.1428 .

Carmeli A, Gittell JH. High-quality relationships, psychological safety, and learning from failures in work organizations. J Organ Behav Int J Ind Occup Organ Psychol Behav. 2009;30(6):709–29.

May DR, Gilson RL, Harter LM. The psychological conditions of meaningfulness, safety and availability and the engagement of the human spirit at work. J Occup Organ Psychol. 2004;77(1):11–37.

Appelbaum NP, Dow A, Mazmanian PE, Jundt DK, Appelbaum EN. The effects of power, leadership and psychological safety on resident event reporting. Med Educ. 2016;50(3):343–50.

Aranzamendez G, James D, Toms R. Finding antecedents of psychological safety: a step toward quality improvement. Nurs Forum. 2015;50(3):171–8.

Edmondson AC. Managing the risk of learning: Psychological safety in work teams. In: International Handbook of Organizational Teamwork. Chichester: Wiley; 2008. p. 255.

Jain AK, Fennell ML, Chagpar AB, Connolly HK, Nembhard IM. Moving toward improved teamwork in cancer care: the role of psychological safety in team communication. J Oncol Pract. 2016;12(11):1000–11.

Reese J, Simmons R, Barnard J. Assertion practices and beliefs among nurses and physicians on an inpatient pediatric medical unit. Hosp Pediatr. 2016;6(5):275–81.

Hoenderdos JW. Towards an observational measure for team psychological safety [dissertation]. The Netherlands: University of Twente; 2013.

Bradley BH, Postlethwaite BE, Klotz AC, Hamdani MR, Brown KG. Reaping the benefits of task conflict in teams: the critical role of team psychological safety climate. J Appl Psychol. 2012;97(1):151.

Devers K, Frankel R. Study design in qualitative research—2: sampling and data collection strategies. Educ Health. 2000;13(2):263–71.

CAS   Google Scholar  

Yin RK. Case study research and applications: design and methods. Thousand Oaks: Sage publications; 2017.

Tracy SJ. Qualitative quality: eight “big-tent” criteria for excellent qualitative research. Qual Inq. 2010;16(10):837–51.

Ellingson LL. Engaging crystallization in qualitative research: an introduction. Thousand Oaks: Sage; 2009.

O’Donovan R, Van Dun D, McAuliffe E. Measuring psychological safety in healthcare teams: developing a more accurate and nuanced mixed-methods approach. BMC Med Res Methodol. 2020;20(1):1–17.

Darbyshire P, McDonald H. Choosing response scale labels and length: Guidance for researchers and clients. Australas J Market Res. 2004;12(2):17–26.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

Boyatzis RE. Transforming qualitative information: Thematic analysis and code development. Thousand Oaks: Sage; 1998.

Schwappach DL, Gehring K. Trade-offs between voice and silence: a qualitative exploration of oncology staff’s decisions to speak up about safety concerns. BMC Health Serv Res. 2014;14(1):303.

PubMed   PubMed Central   Google Scholar  

Halbesleben JR, Rathert C. The role of continuous quality improvement and psychological safety in predicting work-arounds. Health Care Manag Rev. 2008;33(2):134–44.

Raes E, Decuyper S, Lismont B, Van den Bossche P, Kyndt E, Demeyere S, et al. Facilitating team learning through transformational leadership. Instr Sci. 2013;41(2):287–305.

Alingh CW, van Wijngaarden JD, van de Voorde K, Paauwe J, Huijsman R. Speaking up about patient safety concerns: the influence of safety management approaches and climate on nurses’ willingness to speak up. BMJ Qual Saf. 2019;28(1):39–48.

Edmondson AC. Teaming: how organizations learn, innovate, and compete in the knowledge economy. San Francisco: Wiley; 2012.

Gomm R, Hammersley M, Foster P. Case study method: key issues, key texts. Thousand Oaks: Sage; 2000.

Donaldson SI, Grant-Vallone EJ. Understanding self-report bias in organizational behavior research. J Bus Psychol. 2002;17(2):245–60.

Hoogeboom MAMG, Wilderom CPM. A complex adaptive systems approach to real-life team interaction patterns, task context, information sharing, and effectiveness. Group Organ Manag. 2020;45(1):3–42.

Waller MJ, Kaplan SA. Systematic Behavioral observation for emergent team phenomena: key considerations for quantitative video-based approaches. Organ Res Methods. 2018;21(2):500–15.

Download references

Acknowledgements

Not applicable.

This research is funded by the Irish Research Council and the Ireland East Hospital Group under the Employment Based Partnership Scholarship. The funders did not play a role in the design of the study, data collection, analysis and interpretation or in writing the manuscript.

Author information

Authors and affiliations.

Centre for Interdisciplinary Research, Education, and Innovation in Health Systems (IRIS), School of Nursing, Midwifery & Health Systems, Health Sciences Centre, University College Dublin, Dublin 4, Ireland

Róisín O’Donovan & Eilish McAuliffe

You can also search for this author in PubMed   Google Scholar

Contributions

Both authors were involved in the design and planning of this study. ROD completed data collection and analysis. EMcA analysed 10% of interviews completed. ROD drafted the manuscript and EMcA contributed to writing and revising the paper. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Róisín O’Donovan .

Ethics declarations

Ethics approval and consent to participate.

Ethical approval was obtained for this study from the Human Research Ethics Committee in University College Dublin (Reference number: LS-17-67). Written informed consent was obtained from all participants prior to each stage of data collection.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

O’Donovan, R., McAuliffe, E. Exploring psychological safety in healthcare teams to inform the development of interventions: combining observational, survey and interview data. BMC Health Serv Res 20 , 810 (2020). https://doi.org/10.1186/s12913-020-05646-z

Download citation

Received : 14 April 2020

Accepted : 12 August 2020

Published : 31 August 2020

DOI : https://doi.org/10.1186/s12913-020-05646-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Psychological safety
• Mixed methods
• Healthcare teams

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

To read this content please select one of the options below:

Please note you do not have access to teaching notes, relative importance of safety against natural disasters for residential selection: a case study at osaka prefecture, japan.

International Journal of Disaster Resilience in the Built Environment

ISSN : 1759-5908

Article publication date: 5 January 2023

Issue publication date: 13 February 2024

This study aims to clarify the following research questions: to what extent do people consider natural disaster risks as important for residential selection? what personal demographics and attitudes toward natural disaster risks are associated with the relative importance of natural disasters for residential selection? and to what extent do the associated personal attributes influence the relative importance of natural disasters for residential selection?

Design/methodology/approach

An internet-based survey was performed to collect 2,000 responses from residents of Osaka Prefecture, Japan, to gauge people’s relative importance of safety against natural disasters regarding residential preference. The obtained results were analysed using two types of statistical analysis, specifically chi-square test and multivariable logistic regression analyses.

It was found that 37.3% of the respondents in Osaka Prefecture, Japan, considered the “safety against natural disasters” relatively important when selecting a residential location. The statistical analysis also demonstrated that those having a relatively higher level of disaster awareness and preparedness were 1.41 times more likely to prefer to live in a place that is safer from natural disasters. Thus, it was suggested that disaster education aimed at raising the level of people’s disaster awareness could be effective to increase the number of people who choose to live in a safer place from natural disasters.

Originality/value

Living in an area that is safer from natural disasters can effectively minimize human and property damage. Recently, several measures have been taken in Japan to guide people to live in a safer place. The clarification of the extent to which people consider natural disaster risks as important for residential selection and the understanding of the categories of the people who are likely to do so is important to develop more effective natural disaster measures; however, there has been less attention on such investigation. Therefore, this study conducted an internet-based survey and examined it.

• Natural disasters
• Residential preference
• Residential choice
• Disaster management
• City planning
• Disaster prevention

Takabatake, T. , Hasegawa, N. and Nishigaki, S. (2024), "Relative importance of safety against natural disasters for residential selection: a case study at Osaka prefecture, Japan", International Journal of Disaster Resilience in the Built Environment , Vol. 15 No. 2, pp. 303-319. https://doi.org/10.1108/IJDRBE-08-2022-0082

Emerald Publishing Limited

Copyright © 2022, Emerald Publishing Limited

Related articles

All feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

• Business Essentials
• Leadership & Management
• Credential of Leadership, Impact, and Management in Business (CLIMB)
• Entrepreneurship & Innovation
• Digital Transformation
• Finance & Accounting
• Business in Society
• For Organizations
• Support Portal
• Media Coverage
• Founding Donors
• Leadership Team

• Harvard Business School →
• HBS Online →
• Business Insights →

Business Insights

Harvard Business School Online's Business Insights Blog provides the career insights you need to achieve your goals and gain confidence in your business skills.

• Career Development
• Communication
• Decision-Making
• Earning Your MBA
• Negotiation
• News & Events
• Productivity
• Staff Spotlight
• Student Profiles
• Work-Life Balance
• AI Essentials for Business
• Alternative Investments
• Business Analytics
• Business Strategy
• Business and Climate Change
• Design Thinking and Innovation
• Digital Marketing Strategy
• Disruptive Strategy
• Economics for Managers
• Entrepreneurship Essentials
• Financial Accounting
• Global Business
• Launching Tech Ventures
• Leadership Principles
• Leadership, Ethics, and Corporate Accountability
• Leading Change and Organizational Renewal
• Leading with Finance
• Management Essentials
• Negotiation Mastery
• Organizational Leadership
• Power and Influence for Positive Impact
• Strategy Execution
• Sustainable Business Strategy
• Sustainable Investing
• Winning with Digital Platforms

5 Benefits of Learning Through the Case Study Method

• 28 Nov 2023

While several factors make HBS Online unique —including a global Community and real-world outcomes —active learning through the case study method rises to the top.

In a 2023 City Square Associates survey, 74 percent of HBS Online learners who also took a course from another provider said HBS Online’s case method and real-world examples were better by comparison.

Here’s a primer on the case method, five benefits you could gain, and how to experience it for yourself.

Access your free e-book today.

What Is the Harvard Business School Case Study Method?

The case study method , or case method , is a learning technique in which you’re presented with a real-world business challenge and asked how you’d solve it. After working through it yourself and with peers, you’re told how the scenario played out.

HBS pioneered the case method in 1922. Shortly before, in 1921, the first case was written.

“How do you go into an ambiguous situation and get to the bottom of it?” says HBS Professor Jan Rivkin, former senior associate dean and chair of HBS's master of business administration (MBA) program, in a video about the case method . “That skill—the skill of figuring out a course of inquiry to choose a course of action—that skill is as relevant today as it was in 1921.”

Originally developed for the in-person MBA classroom, HBS Online adapted the case method into an engaging, interactive online learning experience in 2014.

In HBS Online courses , you learn about each case from the business professional who experienced it. After reviewing their videos, you’re prompted to take their perspective and explain how you’d handle their situation.

You then get to read peers’ responses, “star” them, and comment to further the discussion. Afterward, you learn how the professional handled it and their key takeaways.

HBS Online’s adaptation of the case method incorporates the famed HBS “cold call,” in which you’re called on at random to make a decision without time to prepare.

“Learning came to life!” said Sheneka Balogun , chief administration officer and chief of staff at LeMoyne-Owen College, of her experience taking the Credential of Readiness (CORe) program . “The videos from the professors, the interactive cold calls where you were randomly selected to participate, and the case studies that enhanced and often captured the essence of objectives and learning goals were all embedded in each module. This made learning fun, engaging, and student-friendly.”

If you’re considering taking a course that leverages the case study method, here are five benefits you could experience.

5 Benefits of Learning Through Case Studies

1. take new perspectives.

The case method prompts you to consider a scenario from another person’s perspective. To work through the situation and come up with a solution, you must consider their circumstances, limitations, risk tolerance, stakeholders, resources, and potential consequences to assess how to respond.

Taking on new perspectives not only can help you navigate your own challenges but also others’. Putting yourself in someone else’s situation to understand their motivations and needs can go a long way when collaborating with stakeholders.

2. Hone Your Decision-Making Skills

Another skill you can build is the ability to make decisions effectively . The case study method forces you to use limited information to decide how to handle a problem—just like in the real world.

Throughout your career, you’ll need to make difficult decisions with incomplete or imperfect information—and sometimes, you won’t feel qualified to do so. Learning through the case method allows you to practice this skill in a low-stakes environment. When facing a real challenge, you’ll be better prepared to think quickly, collaborate with others, and present and defend your solution.

3. Become More Open-Minded

As you collaborate with peers on responses, it becomes clear that not everyone solves problems the same way. Exposing yourself to various approaches and perspectives can help you become a more open-minded professional.

When you’re part of a diverse group of learners from around the world, your experiences, cultures, and backgrounds contribute to a range of opinions on each case.

On the HBS Online course platform, you’re prompted to view and comment on others’ responses, and discussion is encouraged. This practice of considering others’ perspectives can make you more receptive in your career.

“You’d be surprised at how much you can learn from your peers,” said Ratnaditya Jonnalagadda , a software engineer who took CORe.

In addition to interacting with peers in the course platform, Jonnalagadda was part of the HBS Online Community , where he networked with other professionals and continued discussions sparked by course content.

“You get to understand your peers better, and students share examples of businesses implementing a concept from a module you just learned,” Jonnalagadda said. “It’s a very good way to cement the concepts in one's mind.”

4. Enhance Your Curiosity

One byproduct of taking on different perspectives is that it enables you to picture yourself in various roles, industries, and business functions.

“Each case offers an opportunity for students to see what resonates with them, what excites them, what bores them, which role they could imagine inhabiting in their careers,” says former HBS Dean Nitin Nohria in the Harvard Business Review . “Cases stimulate curiosity about the range of opportunities in the world and the many ways that students can make a difference as leaders.”

Through the case method, you can “try on” roles you may not have considered and feel more prepared to change or advance your career .

5. Build Your Self-Confidence

Finally, learning through the case study method can build your confidence. Each time you assume a business leader’s perspective, aim to solve a new challenge, and express and defend your opinions and decisions to peers, you prepare to do the same in your career.

According to a 2022 City Square Associates survey , 84 percent of HBS Online learners report feeling more confident making business decisions after taking a course.

“Self-confidence is difficult to teach or coach, but the case study method seems to instill it in people,” Nohria says in the Harvard Business Review . “There may well be other ways of learning these meta-skills, such as the repeated experience gained through practice or guidance from a gifted coach. However, under the direction of a masterful teacher, the case method can engage students and help them develop powerful meta-skills like no other form of teaching.”

How to Experience the Case Study Method

If the case method seems like a good fit for your learning style, experience it for yourself by taking an HBS Online course. Offerings span seven subject areas, including:

• Business essentials
• Leadership and management
• Entrepreneurship and innovation
• Finance and accounting
• Business in society

No matter which course or credential program you choose, you’ll examine case studies from real business professionals, work through their challenges alongside peers, and gain valuable insights to apply to your career.

Are you interested in discovering how HBS Online can help advance your career? Explore our course catalog and download our free guide —complete with interactive workbook sections—to determine if online learning is right for you and which course to take.

About the Author

An official website of the Department of Health & Human Services

• Search All AHRQ Sites
• Email Updates

1. Use quotes to search for an exact match of a phrase.

2. Put a minus sign just before words you don't want.

3. Enter any important keywords in any order to find entries where all these terms appear.

• The PSNet Collection
• All Content
• Perspectives
• Current Weekly Issue
• Past Weekly Issues
• Curated Libraries
• Clinical Areas
• Patient Safety 101
• The Fundamentals
• Training and Education
• Continuing Education
• WebM&M: Case Studies
• Training Catalog
• Submit a Case
• Improvement Resources
• Innovations
• Submit an Innovation
• About PSNet
• Editorial Team
• Technical Expert Panel

What is diagnostic safety? A review of safety science paradigms and rethinking paths to improving diagnosis.

Choi JJ. What is diagnostic safety? A review of safety science paradigms and rethinking paths to improving diagnosis. Diagnosis (Berl). 2024;Epub May 27. doi:10.1515/dx-2024-0008.

Improving diagnostic safety requires a variety of approaches. This review presents Safety-I, Safety-II , and Safety-III approaches do not only identify and reduce diagnostic errors, but also to improve diagnostic performance.

Prevalence and causes of diagnostic errors in hospitalized patients under investigation for COVID-19. April 12, 2023

Context matters: toward a multilevel perspective on context in clinical reasoning and error. June 21, 2023

Readiness of US general surgery residents for independent practice. October 4, 2017

Towards diagnostic excellence on academic ward teams: building a conceptual model of team dynamics in the diagnostic process. September 20, 2023

Optimizing measurement of misdiagnosis-related harms using symptom-disease pair analysis of diagnostic error (SPADE): comparison groups to maximize SPADE validity. May 3, 2023

How strong is the evidence for the use of perioperative beta blockers in non-cardiac surgery? Systematic review and meta-analysis of randomised controlled trials. September 7, 2005

Ethical considerations and patient safety concerns for cancelling non-urgent surgeries during the COVID-19 pandemic: a review. May 12, 2021

End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. June 5, 2019

Primary care pediatricians' interest in diagnostic error reduction. July 20, 2016

Organizational response to known medical errors: does peer review protection impede improvement? May 30, 2018

Diagnostic errors in primary care pediatrics: Project RedDE. November 29, 2017

Implementation of the I-PASS handoff program in diverse clinical environments: a multicenter prospective effectiveness implementation study. November 16, 2022

Acceptability and feasibility of the Leapfrog computerized physician order entry evaluation tool for hospitals outside the United States. July 22, 2015

Optimizing Pediatric Patient Safety in the Emergency Care Setting. October 19, 2022

Physician and nurse well-being and preferred interventions to address burnout in hospital practice: factors associated with turnover, outcomes, and patient safety. July 19, 2023

Differential safety between top-ranked cancer hospitals and their affiliates for complex cancer surgery. April 24, 2019

Composite measures for profiling hospitals on bariatric surgery performance. October 30, 2013

Adverse events among emergency department patients with cardiovascular conditions: a multicenter study. March 10, 2021

Perceived disability-based discrimination in health care for children with medical complexity. July 19, 2023

Training safer surgeons: how do patients view the role of simulation in orthopaedic training? May 13, 2015

Detecting adverse drug events through data mining. March 24, 2010

The influence of resident involvement on surgical outcomes. January 30, 2005

Surgical skill and complication rates after bariatric surgery. October 23, 2013

What every graduating resident needs to know about quality improvement and patient safety: a content analysis of 26 sets of ACGME milestones. July 18, 2018

Safety considerations in learning new procedures: a survey of surgeons. January 31, 2018

Hospital-Acquired Condition Reduction Program is not associated with additional patient safety improvement. December 4, 2019

Surgeon age and operative mortality in the United States. October 25, 2006

Reducing nosocomial transmission of COVID-19: implementation of a COVID-19 triage system. October 28, 2020

Inviting patients to identify diagnostic concerns through structured evaluation of their online visit notes. April 13, 2022

Do patient engagement IT functionalities influence patient safety outcomes? A study of US hospitals. August 10, 2022

Understanding the nature of medication errors in an ICU with a computerized physician order entry system. February 11, 2015

Optimizing medication safety in the home. July 15, 2015

A novel approach to increase residents' involvement in reporting adverse events. May 18, 2011

Two-year longitudinal assessment of physicians' perceptions after replacement of a longstanding homegrown electronic health record: does a J-curve of satisfaction really exist? August 3, 2016

Safety culture in the operating room: variability among perioperative healthcare workers. September 13, 2017

Parental misinterpretations of over-the-counter pediatric cough and cold medication labels. July 1, 2009

Deprescribing as a clinical improvement focus. November 20, 2019

Preparedness for COVID-19: in situ simulation to enhance infection control systems in the intensive care unit. May 13, 2020

Digital health intervention on patient safety for children and parents: a scoping review. November 29, 2023

Role of knowledge and reasoning processes as predictors of resident physicians' susceptibility to anchoring bias in diagnostic reasoning: a randomised controlled experiment. May 22, 2024

Effects of interorganisational information technology networks on patient safety: a realist synthesis. December 9, 2020

Bar-code verification: reducing but not eliminating medication errors. December 12, 2012

TRIAD XII: are patients aware of and agree with DNR or POLST orders in their medical records. September 11, 2019

More than an apple a day: factors associated with avoidance of doctor visits among transgender, gender nonconforming, and nonbinary people in the USA. September 23, 2020

Good and bad reasons: the Swiss cheese model and its critics. August 12, 2020

What does safety in mental healthcare transitions mean for service users and other stakeholder groups: an open-ended questionnaire study. March 3, 2021

The RCA ReCAst: a root cause analysis simulation for the interprofessional clinical learning environment. July 14, 2021

Patient Safety Innovations

Handshake antimicrobial stewardship as a model to recognize and prevent diagnostic errors

Changes in hospital-acquired conditions and mortality associated with the hospital-acquired condition reduction program. September 29, 2021

Decreased incidence of cesarean surgical site infection rate with hospital-wide perioperative bundle. September 29, 2021

How structural racism works - racist policies as a root cause of U.S. racial health inequities. December 17, 2020

Handshake antimicrobial stewardship as a model to recognize and prevent diagnostic errors. September 8, 2021

Show me the money, I'll show you my complications: impacts of incentivized incident self-reporting among surgeons. March 2, 2022

Improving responses to safety incidents: we need to talk about justice. February 23, 2022

Drivers of unprofessional behaviour between staff in acute care hospitals: a realist review. December 20, 2023

Addressing veteran health-related social needs: how Joint Commission standards accelerated integration and expansion of tools and services in the Veterans Health Administration. December 6, 2023

Prevented harm and cost avoidance with pharmacist intervention while utilizing a discharge medication reconciliation tool. November 29, 2023

Failure of crisis leadership in a global pandemic: some reflections on COVID-19 and future recommendations. November 16, 2022

Understanding factors that could influence patient acceptability of the use of the PINCER intervention in primary care: a qualitative exploration using the Theoretical Framework of Acceptability. November 9, 2022

A novel study of situational awareness among out-of-hospital providers during an online clinical simulation. October 5, 2022

Improving emergency medicine clinician awareness of prehospital-administered medications. August 9, 2023

Patient safety for people experiencing advanced dementia in hospital: a video reflexive ethnography. July 19, 2023

Evaluation of detected medication errors within the operating room at an academic medical center. June 21, 2023

What can safety cases offer for patient safety? A multisite case study. October 11, 2023

Clinical pathway adherence and missed diagnostic opportunities among children with musculoskeletal infections. September 6, 2023

The Ohio Maternal Safety Quality Improvement Project: initial results of a statewide perinatal hypertension quality improvement initiative implemented during the COVID-19 pandemic. June 7, 2023

Using sociotechnical theory to understand medication safety work in primary care and prescribers' use of clinical decision support: a qualitative study. May 24, 2023

Proactive patient safety: focusing on what goes right in the perioperative environment. May 24, 2023

ASHP Guidelines on the Safe Use of Automated Compounding Devices for the Preparation of Parenteral Nutrition Admixtures. June 15, 2022

Assessment of perioperative outcomes among surgeons who operated the night before. June 8, 2022

Can a targeted educational approach improve situational awareness in paramedicine during 911 emergency calls? June 8, 2022

Clinicians' perspectives on proactive patient safety behaviors in the perioperative environment. April 26, 2023

Impact of sleep deficiency on surgical performance: a prospective assessment. April 19, 2023

Predictors of serious opioid-related adverse drug events in hospitalized patients. July 8, 2020

Implementing a survey for patients to provide safety experience feedback following a care transition: a feasibility study October 9, 2019

The development and implementation of cognitive aids for critical events in pediatric anesthesia: the Society for Pediatric Anesthesia Critical Events Checklists. March 15, 2017

PReSaFe: A model of barriers and facilitators to patients providing feedback on experiences of safety. December 7, 2016

Failure to debrief after critical events in anesthesia is associated with failures in communication during the event. March 20, 2019

A checklist-based intervention to improve surgical outcomes in Michigan: evaluation of the Keystone Surgery program. January 28, 2015

Evaluation of the effectiveness of a surgical checklist in Medicare patients. January 7, 2015

Hospital credentialing and privileging of surgeons: a potential safety blind spot. April 15, 2015

Association of hospital participation in a quality reporting program with surgical outcomes and expenditures for Medicare beneficiaries. February 18, 2015

Enhancing surgical safety using digital multimedia technology. November 25, 2015

Triggers, bundles, protocols, and checklists—what every maternal care provider needs to know. April 13, 2016

Emergency hospitalizations for unsupervised prescription medication ingestions by young children. October 8, 2014

Early impact of the 2011 ACGME duty hour regulations on surgical outcomes. January 13, 2016

Copy, paste, and cloned notes in electronic health records: prevalence, benefits, risks, and best practice recommendations. May 7, 2014

The impact of internal service quality on preventable adverse events in hospitals. March 27, 2019

Scaffolding our systems? Patients and families 'reaching in' as a source of healthcare resilience. January 16, 2019

Patients' conceptualizations of responsibility for healthcare: a typology for understanding differing attributions in the context of patient safety. June 5, 2019

Is it time for safeguards in the adoption of robotic surgery? May 15, 2019

Medication reconciliation during internal hospital transfer and impact of computerized prescriber order entry. December 22, 2010

Hospital process compliance and surgical outcomes in Medicare beneficiaries. October 27, 2010

The effect of resident duty hour restriction on trauma center outcomes in teaching hospitals in the state of Pennsylvania. September 29, 2010

Assessment of a safety enhancement to the hospital medication reconciliation process for elderly patients. May 19, 2010

Medication errors involving oral chemotherapy. March 24, 2010

Variation in hospital mortality associated with inpatient surgery. October 14, 2009

Clinicians' assessments of electronic medication safety alerts in ambulatory care. October 7, 2009

"Water cooler" learning: knowledge sharing at the clinical "backstage" and its contribution to patient safety. September 1, 2010

Policy and practice in the use of root cause analysis to investigate clinical adverse events: mind the gap. July 20, 2011

Cognitive Load Theory and its Impact on Diagnostic Accuracy. May 22, 2024

Addressing electronic health record contributions to diagnostic error. May 15, 2024

Comparative evaluation of LLMs in clinical oncology. May 8, 2024

The role for policy in AI-assisted medical diagnosis. May 8, 2024

Diagnostic error in mental health: a review. April 24, 2024

Inattentional blindness in medicine. April 17, 2024

Assessing diagnostic performance. February 14, 2024

Factors influencing diagnostic accuracy among intensive care unit clinicians - an observational study. January 24, 2024

Diagnostic Quality Improvement Repository. January 10, 2024

Error codes at autopsy to study potential biases in diagnostic error. January 10, 2024

Patient reasoning: patients' and care partners' perceptions of diagnostic accuracy in emergency care. January 10, 2024

"ChatGPT, can you help me save my child's life?" - Diagnostic accuracy and supportive capabilities to lay rescuers by ChatGPT in prehospital basic life support and paediatric advanced life support cases - an in-silico analysis. December 6, 2023

The PRIDx framework to engage payers in reducing diagnostic errors in healthcare. November 1, 2023

Pediatric Diagnostic Safety: State of the Science and Future Directions. September 13, 2023

Assessing the utility of ChatGPT throughout the entire clinical workflow: development and usability study. September 13, 2023

Diagnostic errors in uncommon conditions: a systematic review of case reports of diagnostic errors. September 13, 2023

Experimental evidence for structured information-sharing networks reducing medical errors. August 9, 2023

Opportunities for diagnostic improvement among pediatric hospital readmissions. June 28, 2023

The AHRQ Report on Diagnostic Errors in the Emergency Department: the wrong answer to the wrong question. June 28, 2023

Prevalence and characteristics of diagnostic error in pediatric critical care: a multicenter study. June 14, 2023

Types of diagnostic errors reported by paediatric emergency providers in a global paediatric emergency care research network. May 10, 2023

A critical appraisal of AHRQ's "Diagnostic Errors" report. May 10, 2023

Identifying electronic health record contributions to diagnostic error in ambulatory settings through legal claims analysis. May 3, 2023

Annual Perspective

Anesthesiology patient handoff education interventions: a systematic review. March 29, 2023

Managing interruptions to improve diagnostic decision-making: strategies and recommended research agenda. March 29, 2023

Frequency of medication administration timing error in hospitals: a systematic review. March 29, 2023

Diagnostic Centers of Excellence (X01 Clinical Trial Not Allowed). March 22, 2023

Connect With Us

Sign up for Email Updates

To sign up for updates or to access your subscriber preferences, please enter your email address below.

Agency for Healthcare Research and Quality

5600 Fishers Lane Rockville, MD 20857 Telephone: (301) 427-1364

• Accessibility
• Disclaimers
• Electronic Policies
• HHS Digital Strategy
• HHS Nondiscrimination Notice
• Inspector General
• Plain Writing Act
• Privacy Policy
• Viewers & Players
• U.S. Department of Health & Human Services
• The White House
• Don't have an account? Sign up to PSNet

Submit Your Innovations

Please select your preferred way to submit an innovation.

Continue as a Guest

Track and save your innovation

in My Innovations

Edit your innovation as a draft

Continue Logged In

Please select your preferred way to submit an innovation. Note that even if you have an account, you can still choose to submit an innovation as a guest.

Continue logged in

New users to the psnet site.

Access to quizzes and start earning

CME, CEU, or Trainee Certification.

Get email alerts when new content

matching your topics of interest

in My Innovations.

Supreme Court Imperils an Array of Federal Rules

A foundational 1984 decision required courts to defer to agencies’ reasonable interpretations of ambiguous statutes, underpinning regulations on health care, safety and the environment.

• Share full article

Adam Liptak

Reporting on the Supreme Court since 2008

The decision is the latest upending longstanding precedents.

The Supreme Court swept aside a longstanding legal precedent on Friday, reducing the power of executive agencies and endangering countless regulations by transferring power from the executive branch to Congress and the courts. Chief Justice John G. Roberts Jr., writing for the majority, said that “agencies have no special competence” and that judges should determine the meaning of federal laws.

The precedent, Chevron v. Natural Resources Defense Council , is one of the most cited in American law, underpinning 70 Supreme Court decisions and roughly 17,000 in the lower courts. Critics of regulatory authority immediately hailed the decision, suggesting it could open new avenues to challenge federal rules in areas ranging from abortion pills to the environment.

The court has now overturned major precedents in each of the last three terms: on abortion in 2022, on affirmative action in 2023 and now on the power of administrative agencies. In a dissenting opinion, Justice Elena Kagan said the ruling amounted to the Supreme Court’s latest judicial power grab. “A rule of judicial humility,” she wrote, “gives way to a rule of judicial hubris.”

Here’s what else to know:

What is Chevron deference? It is the principle from the Supreme Court’s 1984 ruling that gave regulatory agencies leeway to interpret laws that Congress had left vague. When Congress passes a law, it cannot anticipate all the ways that the economy, the nation and the world will change. If regulators had only the powers that Congress explicitly gave them, many regulations would be vulnerable to legal challenges. The ruling could have broad implications for the regulation of food and drugs , the banking and financial sector , taxation , as well as conservative activists’ targeting of medication abortion and rights for transgender people .

A major goal of the conservative legal movement: Friday’s ruling undoes a precedent that empowered executive branch agencies, which many conservatives have come to believe are dominated by liberals under both parties’ administrations — a critique often described as “the deep state.” Elizabeth Murrill, the Republican attorney general of Louisiana who has taken a leading role in lawsuits against the Biden administration’s environmental regulations, said Chevron deference had been “wildly abused by this administration more than any other.” Read about conservatives’ view of the precedent.

The White House reaction: Karine Jean-Pierre, the White House press secretary, said Friday’s decision was the latest example of the Supreme Court siding with Republican-backed special interests to block “common-sense rules that keep us safe, protect our health and environment, safeguard our financial system, and support American consumers and workers.”

Supporters of regulatory oversight criticized the decision: Critics of the decision said it would empower the courts, not Congress, to dictate policy. “Getting rid of Chevron deference says, you know what? The courts will be the decider of how to interpret these laws instead of experts who are knowledgeable in the field,” said Senator Jeff Merkley, Democrat of Oregon, who sits on the Senate Environment and Public Works Committee.

The case started with fishermen: The court heard two almost identical cases, Loper Bright Enterprises v. Raimondo, and Relentless v. Department of Commerce. Both cases involved a 1976 federal law that requires herring boats to carry federal observers to collect data used to prevent overfishing.

Under a 2020 regulation interpreting the law, owners of the boats were required not only to transport the observers but also to pay $700 a day for their oversight. Fishermen in New Jersey and Rhode Island — backed by two conservative organizations that decry the “administrative state” — sued, saying the 1976 law did not authorize the relevant agency, the National Marine Fisheries Service, to impose the fee.

Charlie Savage

The decision is the latest blow to regulatory agencies.

Overturning the Chevron deference precedent is just the latest in a series of ringing blows the Supreme Court’s Republican-appointed conservative bloc has delivered to the ability of regulatory agencies to impose rules on powerful business interests, advancing a longstanding goal of the conservative legal movement and the donors who have funded its rise .

Just yesterday, the majority struck down the ability of agencies to enforce their rules via in-house tribunals before technical-expert administrative judges. Instead, it ruled, agencies must sue accused malefactors in federal court before juries.

In recent years, the Republican majority has also made it easier to sue agencies and get their rules struck down, including by advancing the so-called major questions doctrine. Under that idea, courts should nullify economically significant regulations if judges decided Congress was not clear enough in authorizing them.

Advancing and entrenching that idea, the court has struck down an E.P.A. rule aimed at limiting carbon pollution from power plants , and barred the Occupational Safety and Health Administration from telling large employers they must either have their workers vaccinated against the Covid-19 virus or have them undergo frequent testing.

And in a 2020 ruling , the five Republican appointees then on the Supreme Court struck down a provision of the law Congress enacted to create the Consumer Financial Protection Bureau that had protected its head from being fired by a president without a good cause, like misconduct.

Karine Jean-Pierre, the White House press secretary, said Friday’s decision was the latest example of the Supreme Court blocking “common-sense rules that keep us safe, protect our health and environment, safeguard our financial system, and support American consumers and workers.”

The court has not always gone as far as libertarians wanted, however. Earlier this term, the court rejected a challenge to the way the Consumer Financial Protection Bureau is funded. Striking it down would have opened the door to lawsuits to nullify every regulation and enforcement action it has taken in its 13 years of existence, including ones concerning mortgages, credit cards, consumer loans and banking.

While overturning Chevron is now the capstone victory for the conservative legal movement’s assault on the administrative state, it may not be the end of the story. More extreme opponents of regulation hope the court will someday embrace a sweeping version of the so-called nondelegation doctrine.

Under that vision, the Constitution does not allow Congress to delegate any of its legislative authority to executive branch agencies. If so, all regulations should be struck down because the only way society can impose a legally binding rule on business interests is if Congress manages to specifically enact one via statute.

Read the Court’s Decision to Overrule the Chevron Doctrine

The ruling sweeps aside a legal precedent that required courts to defer to the expertise of federal administrators in carrying out laws passed by Congress.

Advertisement

Deborah B. Solomon

Karine Jean-Pierre, the White House press secretary, called the Supreme Court’s ruling “another deeply troubling decision that takes our country backwards” and the latest decision by the court to side with Republican-backed special interests and block commonsense rules on health, the enviroment and worker protection.

She said in a statement that President Biden had directed his legal team to work with the Justice Department and other lawyers “to review today’s decision carefully and ensure that our administration is doing everything we can to continue to deploy the extraordinary expertise of the federal workforce to keep Americans safe and ensure communities thrive and prosper.”

Coral Davenport

The Chevron decision is the latest major blow in a yearslong coordinated strategy to weaken the authority of what conservative activists call the “administrative state.” One big step came two years ago, with the Supreme Court decision in West Virginia vs. E.P.A., sharply curtailing the agency’s authority to regulate climate-warming pollution from power plants. That ruling essentially told regulators to stay in their lane and not attempt broader interpretations of the law.

The Chevron decision advances that precedent, essentially applying it to all regulations, large and small. Together, experts say that the two rulings could mean that more government regulations are struck down or scaled back by the courts, and that government agencies could be more timid or restrained in writing new rules.

Christina Jewett

Utah geared up for a fight in anticipation of the court’s ruling.

The possibility of new limits on the regulatory power of the federal government had already spurred one state to identify regulations ripe for a challenge.

A law passed in Utah directs agriculture and environmental agencies to identify federal rules carried out in the state that might be vulnerable under a ruling that limits the Chevron precedent. The analyses are due at the start of 2025 and the law says the state attorney general will issue his own report by midyear to declare his plans for possible lawsuits.

Sean D. Reyes, the Utah attorney general, has made well known his distaste for the Chevron precedent, which gives federal agencies leeway to interpret laws that Congress left vague. In a news release, he called the standard “one of the greatest threats to individual liberty.”

“For far too long, it has been wielded by big government proponents, unaccountable federal bureaucrats, and activist courts to destroy the freedoms of hard-working Americans and rob local control from our states,” he said in a statement in August.

Mr. Reyes signed on with about two dozen other Republican attorneys general to a friend-of-the-court brief decrying the onus on small businesses, the vast costs and the volume of regulations, which they said vastly outpace the number of laws passed by Congress, though that has been on a downward trend for decades.

Utah is not entirely alone in its war room crouch, said Gary Feldon, an attorney with Hollingsworth who noted Utah’s work in a recent article anticipating the ruling.

“I don’t know that anybody is doing it quite as systemically as the state of Utah seems to be, but industry and businesses are certainly aware that we are on the edge of a major shift,” Mr. Feldon said. “And the savvy among them are making sure that they’re in position to take advantage of it now.”

In addition to cutting back on the power of executive agencies, the Supreme Court on Friday issued decisions in two other closely watched cases: upholding a city’s laws aimed at banning homeless residents from sleeping outdoors and ruling that federal prosecutors had overstepped in using an obstruction law to prosecute a Jan. 6 rioter .

Teddy Rosenbluth and Roni Caryn Rabin

The ruling is likely to stymie public health initiatives, experts said.

The Supreme Court decision overturning the so-called Chevron doctrine is likely to hamstring the federal government’s public health efforts and invite waves of litigation from parties opposed to regulations aimed at safeguarding Americans, scientific and legal experts said.

By gutting federal agencies’ power to interpret ambiguous laws and fill in gaps in statutes, forcing them instead to defer to protracted judicial or legislative processes, the ruling also could prevent regulators from acting quickly and creatively in the face of a catastrophic emergency, such as climate change or another deadly pandemic.

“We anticipate that today’s ruling will cause significant disruption to publicly funded health insurance programs, to the stability of this country’s health care and food and drug review systems, and to the health and well-being of the patients and consumers we serve,” several of the nation’s largest health organizations, including the American Public Health Association and the American Cancer Society, said in a joint statement on Friday.

Federal officials will feel a “chilling effect” that will slow regulations in areas in which they do not have explicit authority, said Dr. Reshma Ramachandran, co-director of the Yale Collaboration for Regulatory Rigor, Integrity and Transparency, an initiative that studies medical product evaluations and coverage in order to improve patient outcomes.

Instead of hiring more scientific and technological experts, federal agencies will have to arm themselves with lawyers, she predicted.

Zachary L. Baron, director of the Health Policy and the Law Initiative at the O’Neill Institute at Georgetown Law in Washington, said one result of the ruling “is likely to be an increase in litigation and an increase in uncertainty.”

“It seems like, as Justice Kagan wrote in her dissent, the court is flipping the script today, giving more authority to courts and judges and less authority to federal agencies and the expertise that they have,” Mr. Baron said.

Today’s ruling is one in a string of court decisions in recent years in which the court has given itself “more and more power over every significant policy dispute, and closing the door on agency experts that have been working on these issues for years,” he said.

Indeed, Justice Elena Kagan’s dissent offered an example of the type of detailed scientific question judges may now face in court: When does an alpha amino acid polymer qualify as a protein?

“I don’t know many judges who would feel confident resolving that issue,” she wrote. “(First question: What even is an alpha amino acid polymer?)”

The Food and Drug Administration, she added, has scores of experts who could “collaborate with each other on its finer points, and arrive at a sensible answer.”

The Chevron doctrine has its roots in public health: a 1984 Supreme Court case involving air pollution. At issue was the Environmental Protection Agency’s interpretation of an ambiguous provision in the Clean Air Act that affected companies subject to pollution controls.

The court determined that federal agencies should receive “deference” for reasonable interpretations of gaps or ambiguities in the statutes that Congress could not have anticipated when it crafted the laws.

The court gave federal agencies leeway because of their subject matter and scientific expertise, experience and political accountability.

Now that this authority has been curtailed, public health agencies simply may regulate less, a goal long sought by proponents of a smaller federal government and companies eager to pursue unfettered growth.

“If agencies know that everything they do that is not perfectly aligned with a statute will be scrutinized by the court, they will be less likely to promulgate expansive rules or swift rules,” said Selina Coleman, a health care partner at Reed Smith, a large law firm.

Other experts also predicted an explosion in litigation and uncertainty. The ruling will “signal to industry and aggressive state attorneys general to open the floodgates to more litigation to block federal regulatory efforts,” Mr. Baron said.

Moving public health decisions from federal agencies to Congress and the courts will lead to “incoherence, chaos and endless litigation,” said Paul Billings, national senior vice president of public policy at the American Lung Association.

The Supreme Court and lower courts have already chipped away at the authority of regulatory bodies to make public health decisions. Many such rulings were handed down during the coronavirus pandemic.

In November 2021, the Supreme Court upheld an injunction that barred the Centers for Disease Control and Prevention from enforcing a national moratorium on evictions from rental housing, despite fears that a wave of such displacements would exacerbate the spread of Covid-19.

In January 2022, the Supreme Court ruled that the Occupational Safety and Health Administration could not require large businesses to vaccinate their employees against Covid. In April 2022, a federal judge in Florida struck down a C.D.C. mandate that required passengers to wear masks on public buses, trains and planes.

Today’s Supreme Court decision will task Congress with spelling out exactly what agencies like the C.D.C. can and cannot do, several observers said. “Nobody has any confidence that Congress can get its act together to do that,” said Dr. Georges C. Benjamin, executive director of the American Public Health Association.

“I think the decision as written solidifies employment for lawyers and judges, and undermines the authority of experts,” he added.

Other scientists also expressed doubt that Congress or the judiciary could remain abreast of constantly evolving scientific evidence. “To keep up with that pace of change, even for a medical or scientific professional, is very challenging,” said Karen Knudsen, chief executive of the American Cancer Society.

Consumer advocates are calling the decision a travesty that could upend the rules and regulations Americans depend upon for their safety.

“It’s going to affect everything from airbags in peoples’ cars to the quality of the food they feed their families and the water they drink,” said Stephen Hall, legal director of Better Markets, which pushes for tougher regulation. “This decision threatens to return the United States to the 1910s when the government had very limited ability to protect the health, safety, and welfare of America.”

Alan Rappeport

The ruling could undermine the Treasury Department and the I.R.S.

The Supreme Court’s knockdown of Chevron deference could complicate the ability of the Treasury Department and the Internal Revenue Service to craft federal regulations that are central to President Biden’s economic agenda.

The Treasury Department is responsible for implementing major pieces of legislation such as the Inflation Reduction Act, including determining who qualifies for billions of dollars worth of tax credits. At the same time, the I.R.S. has vast leeway to administer the tax code. The agency has faced criticism recently for its decision to halt some pandemic relief tax credits to businesses because of concerns about fraud and delaying collection of new taxes on digital wallet transactions.

“Taxpayers are likely to challenge the validity of dozens of tax regulations and those challenges are much more likely to prevail,” said Robert J. Kovacev, a lawyer at the firm Miller & Chevalier who specializes in tax litigation and represents businesses engaged in disputes with the tax agency. “For years the I.R.S. has issued regulations expanding its power and restricting tax benefits that Congress intended taxpayers to receive.”

The ruling will also present new challenges as the Biden administration rolls out its alternative energy credit regulations, Mr. Kovacev said, because the I.R.S. will not be able to take for granted that courts will defer to its regulations.

The Tax Policy Center said in an analysis last fall that such a Supreme Court decision would make it harder for an agency such as the I.R.S. to write rules to address industries that are quickly evolving, such as cryptocurrencies, and that it would be more difficult to fill in the gaps for Congress when lawmakers rush to write tax legislation.

Critics of the tax agency said on Friday expressed optimism the ruling would limit its powers.

“Today’s decision will level the playing field for taxpayers and government agencies,” said Joe Bishop-Henchman, executive vice president at the National Taxpayers Union Foundation. “Unreasonable I.R.S. interpretations will no longer automatically win in court, which is as it should be, and reasonable interpretations will still have the force of law.”

Treasury Department and the I.R.S. did not immediately have a comment.

Former top Trump officials are gloating about the overturning of the Chevron doctrine. Mandy Gunasekara, who served as chief of staff at the E.P.A. during the Trump administration and has helped write Project 2025 , a policy blueprint for a next Republican administration, wrote on the social media site X, that the era of “trust the experts” had ended. She called it a “great day for our constitutional integrity and the American people.”

Chris Cameron

House Republican leadership praised the Supreme Court ending the Chevron doctrine. “House Republican committees will be conducting oversight to ensure agencies follow the Court’s ruling,” Speaker Mike Johnson wrote in a joint statement with Representatives Steve Scalise and Tom Emmer.

Elizabeth Dias

Conservative Christian activists see Chevron as major win to push their causes.

Conservative Christian activists and lawyers are celebrating the Chevron decision as a significant win for their ambitions to target medication abortion and rights for transgender people.

Anti-abortion activists see the ruling as a critical tool to fight the Food and Drug Administration, especially after the court rejected their bid to undo the F.D.A.’s approval of a medication abortion drug earlier in June. “Getting rid of Chevron is the first domino to fall,” Kristi Hamrick, a strategist for Students for Life, said in a statement.

They see the decision as a new precedent as they seek to bring a future case against the F.D.A. to the Supreme Court. Ms. Hamrick said such a case was likely to get a better reception “when the F.D.A. is no longer given the benefit of the doubt.”

The Alliance Defending Freedom, the conservative Christian legal advocacy group that argued against the F.D.A.’s approval of the abortion pill and lost, also praised the ruling.

Federal agencies “frequently disrespect Americans’ most cherished principles — including religious freedom and the sanctity of life,” said Julie Marie Blake, senior counsel at A.D.F. “Now, the court has wiped away a major roadblock that prevented Americans from holding government officials accountable.”

A.D.F. had filed an amicus brief in the case on behalf of Christian Employers Alliance, a group that defends freedoms for Christian businesses. The brief criticized a range of federal agencies, including the Department of Education and Health and Human Services, for what it said was the agencies’ efforts on “ending women’s sports” to imposing “radical gender ideology” to “forcing employers to pay for puberty blockers, cross-sex hormones and amputating healthy organs.”

Now, the brief’s argument looks like a road map for what lawyers may want to pursue with Chevron gone.

Business groups are cheering the Chevron decision. The National Federation of Independent Business, which represents small businesses, said the Supreme Court’s ruling will allow companies “to breathe a sigh of relief.”

“For 40 years, Chevron deference has allowed administrative agencies to enact regulations with little accountability,” Beth Milito, Executive Director of NFIB’s Small Business Legal Center, said in a statement. “Abandoning Chevron will hold agencies accountable and level the playing field in court cases between small businesses and administrative agencies.”

Democrats, anticipating Chevron’s demise, gave E.P.A. more power in recent climate law.

The Biden administration has been preparing for the overturn of Chevron, knowing that conservative activists have pushed cases like this, and that the majority of justices on the Supreme Court were expected to look favorably on it.

That’s why two years ago the White House worked with congressional Democrats to squeeze through legislation that could help protect the Environmental Protection Agency’s authority to craft climate change regulations, even if the Chevron doctrine was struck down.

Climate change rules could be particularly vulnerable to legal attack in a post- Chevron world. That’s because the E.P.A. wrote them under the authority of the 1970 Clean Air Act, a sweeping law that directs the agency to regulate all pollutants that endanger human health.

But the legislators of 1970 did not specify anywhere in the law that carbon dioxide emissions, the chief cause of climate change, should be regulated. It doesn’t even mention climate change.

Democrats changed that in the 2022 Inflation Reduction Act, a law chiefly focused on spending billions of dollars on clean energy technology to fight climate change. But the law amends the Clean Air Act to define the carbon dioxide produced by the burning of fossil fuels as an “air pollutant.”

That language, according to legal experts as well as the Democrats who worked it into the legislation, explicitly gives the E.P.A. the authority to regulate greenhouse gases and to use its power to push the adoption of wind, solar and other renewable energy sources.

The specificity of that legal language should protect E.P.A.’s authority to regulate carbon dioxide pollution by limiting their emission from tailpipes and smokestacks.

However, opponents of the rule — chiefly, the fossil fuel industry — are still expected to use the demise of the Chevron doctrine to attempt to weaken the specifics of those rules.

Overturning Chevron is just the latest in a series of ringing blows the Supreme Court’s Republican-appointed conservative bloc has delivered to the ability of regulatory agencies to impose rules on powerful business interests, advancing a long-standing goal of the conservative legal movement and the donors who have funded its rise . Here are some previous steps:

In recent years, the Republican majority has also made it easier to sue agencies and get their rules struck down, including by advancing the so-called major questions doctrine. Under that idea, courts should nullify economically significant regulations if judges decided Congress was not clear enough in authorizing them. Advancing and entrenching that idea, the court has struck down an E.P.A. rule aimed at limiting carbon pollution from power plants , and barred the Occupational Safety and Health Administration from telling large employers they must either have their workers vaccinated against the Covid-19 virus or have them undergo frequent testing.

Overturning the Chevron decision has been a major goal of the conservative legal movement.

After taking aim at abortion and affirmative action, the conservative legal movement set its sights on a third precedent: Chevron v. Natural Resources Defense Council .

The 1984 decision, one of the most cited in American law but largely unknown to the public, bolstered the power of executive agencies that regulate the environment, the marketplace, the work force, the airwaves and countless other aspects of modern life. Overturning it was a key goal of the right and is part of a project to demolish the “administrative state.”

The decision rejecting Chevron threatens regulations covering — just for starters — health care, consumer safety, government benefit programs and climate change.

Chevron — and bear with me here, this will hurt only for a minute — established the principle that courts must defer to agencies’ reasonable interpretations of ambiguous statutes. The theory is that agencies have more expertise than judges, are more accountable to voters and are better able to establish uniform national policies.

“Judges are not experts in the field, and are not part of either political branch of the government,” Justice John Paul Stevens wrote in 1984 for a unanimous court (though three of its justices recused for reasons of health or financial conflict). Justice Stevens later said of the opinion , which was easily his most influential, that it was “simply a restatement of existing law.”

The decision was not much noted when it was issued. “If Chevron amounted to a revolution, it seems almost everyone missed it,” Justice Neil Gorsuch, the harshest critic of the doctrine on the current court, wrote in 2022 , saying that courts had read it too broadly.

At first, conservatives believed that empowering agencies would constrain liberal judges. So the Reagan administration, which had interpreted the Clean Air Act to allow looser regulations of emissions, celebrated the decision.

Justice Stevens, rejecting a challenge from environmental groups, wrote that the Environmental Protection Agency’s reading of the statute was “a reasonable construction” that was “entitled to deference.”

The head of the E.P.A. when the regulation was issued? Anne Gorsuch, Justice Gorsuch’s mother.

Most surprisingly, given its current bad odor with the right, Chevron was at least initially championed, celebrated and elevated by Justice Antonin Scalia, a revered conservative figure who died in 2016 . “In the long run Chevron will endure and be given its full scope,” he wrote in a law review article in 1989, adding that this was so “because it more accurately reflects the reality of government.”

What, then, accounted for the decision’s place on the conservative hit list? After all, as the case itself demonstrates, it requires deference to agency interpretations under both Republican and Democratic administrations.

The answers are practical, cultural and philosophical. Business groups on the whole remain hostile to regulation. Many conservatives have come to believe that executive agencies are dominated by liberals under both parties’ administrations — the shorthand for this critique is “the deep state.” And some on the right have become hostile to the very idea of expertise.

The majority opinion by Justice Roberts notes: “Chevron’s presumption is misguided because agencies have no special competence in resolving statutory ambiguities. Courts do.” Justice Elana Kagan, in a fiery dissent, disagreed and predicted “large-scale disruption,” as judges are called upon to answer questions that expert agencies have been entrusted to handle.

Ken Bensinger

Conservative pundits, already celebrating last night’s debate, are now in a mood of downright jubilation after the Supreme Court’s rulings today rolling back the power of regulatory agencies and overturning the Justice Department’s use of an obstruction statute in the January 6 criminal cases. “Huge, huge 24 hours for Donald Trump/GOP WOW,” wrote Megyn Kelly, the right-wing podcaster and former Fox anchor.

Stacy Cowley and Emily Flitter

The ruling will embolden challenges against financial regulators.

The end of Chevron deference is a boon for banking lobbyists, who have in recent years intensified their pushback against the agencies that oversee them — especially the Consumer Financial Protection Bureau, one of the industry’s most aggressive regulators.

The consumer bureau’s interpretations “may now be subject to heightened attack and may require far more justification than formerly was the case,” said Joseph Lynyak, a partner at Dorsey & Whitney who specializes in financial regulation.

While the decision will complicate regulators’ jobs, its effects will likely seem familiar to them. Losing the Chevron deference will amplify a shift already underway in the lower courts, which have in recent years been receptive to lawsuits challenging financial regulators’ actions. The U.S. Court of Appeals for the Fifth Circuit, in particular — and the federal courts under its purview — has been a major roadblock, preventing the bureau from imposing credit card late fee limits and expanding its interpretation of anti- discrimination laws .

One recent action that may now be ripe for a challenge is the bureau’s decision that Buy Now, Pay Later lenders are credit card providers, giving buyers a right to dispute charges and demand refunds.

“Because this interpretive rule pushes the envelope past existing law into pure agency interpretation, it will be an attractive target for industry challenge,” said Erin Bryan, another partner at Dorsey & Whitney.

In addition to the C.F.P.B., trade groups representing banks have sued other federal bank regulators, including the Office of the Comptroller of the Currency and the Federal Reserve. They have challenged those regulators over a host of rules, from a sweeping anti-redlining regulation to one requiring banks to disclose detailed data about their small business loans.

Outside advocacy groups have also gotten into the habit of suing the regulators, though the bulk of their activity took place during the Trump administration, when proponents of stricter financial regulation felt that government officials were unlawfully loosening rules on banks and other firms. Their preferred appeals circuit was the Ninth; they often filed federal court cases in the Northern District of California, where they expected judges to treat their arguments favorably.

Both sides won rulings by judges who declined to defer to the regulators.

“A court can always avoid getting to the Chevron deference in the first place by saying that a statute is not ambiguous, and that’s what happens the vast majority of the time,” said Randy Benjenk, a partner at Covington & Burling who focuses on financial regulation.

“In practice it’s been rare for a judge to conclude that a statute is ambiguous and defer to an agency’s interpretation of law. Judges routinely reach their own interpretations that contradict the agencies. That’s true in courts nationwide, whether in Texas, California or anywhere else.”

The oversight of food, drugs and tobacco is expected to be a target.

The Food and Drug Administration, which oversees a vast swath of items people use every day, is expected to see an increase — perhaps an onslaught — of lawsuits following the Supreme Court’s decision on Friday.

“This is disastrous for public health. This is disastrous for the critical role of science-based regulatory agencies,” said Mitch Zeller, a former F.D.A. associate commissioner and tobacco division director. “Chevron has worked well for half a century and makes a lot of sense.”

Challenges could range from whether tainted spinach can be traced back to a farm to the very core of the F.D.A.’s decisions on whether drugs are safe and effective enough to be sold in the United States.

“F.D.A. has always been called the gold standard for product approval throughout the world,” said Perham Gorji, a partner at the law firm DLA Piper and former deputy chief counsel at the F.D.A. “Less deference to F.D.A. is going to obviously change what’s available in terms of products that are available here in the United States.”

The agency employs about 18,000 people, many of whom are doctors or have advanced degrees in biostatistics, chemistry and toxicology. Given the complexity of some scientific decisions the agency makes, attorneys who focus on the F.D.A. said initial challenges might focus on areas in which the F.D.A. exerts policy clout, including some that touch on drug pricing.

Chad Landmon, an attorney with Axinn who leads the F.D.A. practice group, predicted that early lawsuits could stem from a mix of problems companies face.

“I think companies are going to be much more aggressive and generally are going to be looking for opportunities to challenge the F.D.A.,” Mr. Landmon said.

Others expect a broad onslaught from tobacco companies regulated by the agency. “I would expect the tobacco industry to target every aspect of the F.D.A.’s regulatory infrastructure,” said Desmond Jensen of the Public Health Law Center. The agency decides which e-cigarettes are authorized for sale and can reject new cigarettes that could attract new smokers.

Limits on Chevron are widely thought to favor industry, but the reality could be more complex if advocacy groups gear up, said Nick Shipley, a former lobbyist for BIO and PhRMA and the founder of Cronus Consulting. He cited the group that challenged the F.D.A.’s approval of abortion medications .

“Industry,” he said, “could be caught in the crossfire.”

While the Chevron decision could imperil the standing of hundreds of recent and future regulations, Chief Justice Roberts was careful to write in his opinion that the decision is not retroactive.

Justice Roberts wrote that it does “not call into question prior cases that relied on the Chevron framework. The holdings of those cases that specific agency actions are lawful — including the Clean Air Act holding of Chevron itself — are still subject to statutory stare decisis despite our change in interpretive methodology.”

Coral Davenport ,  Christina Jewett ,  Alan Rappeport ,  Margot Sanger-Katz ,  Noam Scheiber and Noah Weiland

Here’s what the Chevron ruling could mean in everyday terms.

The Supreme Court’s decision on Friday to limit the broad regulatory authority of federal agencies could lead to the elimination or weakening of thousands of rules on the environment, health care, worker protection, food and drug safety, telecommunications, the financial sector and more.

The decision is a major victory in a decades-long campaign by conservative activists to shrink the power of the federal government, limiting the reach and authority of what those activists call “the administrative state.”

The court’s opinion could make it easier for opponents of federal regulations to challenge them in court, prompting a rush of new litigation, while also injecting uncertainty into businesses and industries.

“If Americans are worried about their drinking water, their health, their retirement account, discrimination on the job, if they fly on a plane, drive a car, if they go outside and breathe the air — all of these day-to-day activities are run through a massive universe of federal agency regulations,” said Lisa Heinzerling, an expert in administrative law at Georgetown University. “And this decision now means that more of those regulations could be struck down by the courts.”

The decision effectively ends a legal precedent known as “Chevron deference,” after a 1984 Supreme Court ruling. That decision held that when Congress passes a law that lacks specificity, courts must give wide leeway to decisions made by the federal agencies charged with implementing that law. The theory was that scientists, economists and other specialists at the agencies have more expertise than judges in determining regulations and that the executive branch is also more accountable to voters.

Since then, thousands of legal decisions have relied on the Chevron doctrine when challenges have been made to regulations stemming from laws like the 1938 Fair Labor Standards Act, the 1970 Clean Air Act , the 2010 Affordable Care Act and others.

In writing laws, Congress has frequently used open-ended directives, such as “ensuring the rule is in the public interest,” leaving it to agency experts to write rules to limit toxic smog, ensure that health plans cover basic medical services, ensure the safety of drugs and cosmetics and protect consumers from risky corporate financial behavior.

But that gave too much power to unelected government officials, according to conservatives, who ran a coordinated, multiyear campaign to end the Chevron doctrine. They believe the courts, not administrative agencies, should have the power to interpret statutes. The effort was led by Republican attorneys general, conservative legal activists and their funders, several with ties to large corporations, and supporters of former President Donald J. Trump.

“Overturning Chevron was a shared goal of the conservative movement and the Trump administration. It was expressed constantly,” said Mandy Gunasekara, who served as chief of staff at the E.P.A. under President Trump and has helped write Project 2025 , a policy blueprint for a next Republican administration. “It creates a massive opportunity for these regulations to be challenged. And it could galvanize additional momentum toward reining in the administrative state writ large if the administration changes in November.”

Still, Jonathan Berry, who served as a senior Labor Department official under Mr. Trump, noted that overturning the Chevron doctrine itself “doesn’t immediately blow anything up.”

Rather, Mr. Berry said, the fate of the regulations will be determined by what happens when they start moving through the courts without the protection of Chevron. “The mystery is exactly how much of this stuff goes down,” Mr. Berry said.

Here is a look at how the decision might affect various government agencies.

The Environmental Protection Agency

Environmentalists fear that the end of the Chevron doctrine will mean the elimination of hundreds of E.P.A. rules aimed at limiting air and water pollution, protecting people from toxic chemicals and, especially, tackling climate change.

Over the past six months, the Biden administration has issued the most ambitious rules in the country’s history aimed at cutting climate-warming pollution from cars , trucks , power plants and oil and gas wells . Without those rules, it would very likely be impossible for President Biden to achieve his goal of cutting greenhouse gas emissions in half by the end of the decade, which analysts say all major economies must do to avoid the most deadly and catastrophic impacts of global warming.

All of the Biden climate rules have already been the target of lawsuits that are winding their way through the courts.

Legal experts say that the reversal of Chevron will not remove E.P.A.’s foundational legal obligation to regulate climate-warming pollution: that was explicitly detailed in a 2007 Supreme Court decision and in 2022 legislation passed by Democrats in anticipation of challenges to that authority.

But the specific regulations — such those designed to cut car and truck pollution by accelerating the transition to electric vehicles, or to slash power plant pollution with the use of costly carbon capture and sequestration technology — could now be more legally vulnerable.

The result would quite likely be that stringent climate rules designed to sharply reduce emissions could be replaced by much looser rules that cut far less pollution. Experts say that could also be the fate of existing rules on smog, clean water and hazardous chemicals.

Labor Agencies

The elimination of the Chevron deference could affect workers in a variety of ways, making it harder for the government to enact workplace safety regulations and enforce minimum wage and overtime rules.

One recent example was in April, when the Biden administration raised the salary level below which salaried workers automatically become eligible for time-and-a-half overtime pay, to nearly $59,000 per year from about $35,000, beginning on Jan. 1. Business groups have challenged the Labor Department’s authority to set a so-called salary threshold and such challenges will have far better odds of success without the Chevron precedent, experts said.

The shift could also rein in protections for workers who publicly challenge the policies of their employers, according to Charlotte Garden, a professor of labor law at the University of Minnesota. The National Labor Relations Board often concludes that a single worker has the right to protest low pay or harassment or attendance policies without being disciplined or fired. But the relevant law refers to “concerted activities,” meaning the protection may now apply only to groups of employees who stage such protests, not individuals, Professor Garden said.

Food and Drug Administration

The Food and Drug Administration flexes significant power when it sets the standards for how new drugs must be studied and whether they are safe and effective before they are approved for use. Attorneys who worked at the agency said that companies chafing at that high bar for approvals might now challenge those regulations. Others said legal challenges could ultimately affect drug prices.

Challenges are also expected in the agency’s tobacco division, which authorizes the sale of new cigarettes and e-cigarettes with the intent to protect public health. “I would expect the industry to attack the F.D.A.’s authority to do premarket review at all,” said Desmond Jenson, deputy director of the commercial tobacco control program at the Public Health Law Center.

Others noted the Chevron decision could have a chilling effect, compelling the F.D.A. to proceed quite carefully, given the potential for litigation, if it moves forward with proposals to ban menthol cigarettes or make them less addictive by slashing nicotine levels.

Abortion opponents say the ruling could work in their favor as they seek to bring another case against the Food and Drug Administration’s approval of an abortion medication to the Supreme Court, which rejected their effort to undo the agency’s approval of the drug this month.

Kristi Hamrick, a strategist for Students for Life of America, an anti-abortion organization, said in a statement that such a case was likely to get a better reception “when the F.D.A. is no longer given the benefit of the doubt.”

Health Care

The court’s ruling could affect how Medicare, Medicaid and Affordable Care Act insurance plans are administered, health law experts said, as opponents gain an opportunity to challenge how these huge programs operate.

The health care system is governed by elaborate regulations covering how hospitals operate, what providers are paid for medical services and how insurance companies are monitored by the government. Much of that regulation is grounded in interpretation of laws that date back decades. Major industries could be affected if rules are changed.

“There’s an awful lot of regulation that flies under the radar that’s just about making sure the trains run on time,” said Nicholas Bagley, a law professor at the University of Michigan.

Rachel Sachs, a health law expert at the Washington University School of Law in St. Louis, said that the complex set of rules devised and governed by the Department of Health and Human Services and the Centers for Medicare and Medicaid Services could be challenged in new ways.

“There’s a lot of work to do in that process,” she said. “And therefore there are a lot of opportunities for challengers to pick at specific choices that C.M.S. and H.H.S. are making in the interpretation of these rules.”

The Supreme Court decision will require Congress to specify exactly what agencies like the C.D.C. can and cannot do, several analysts said. “Nobody has any confidence that Congress can get its act together to do that,” said Dr. Georges C. Benjamin, executive director of the American Public Health Association.

The Biden administration has written health regulations anticipating a world without the Chevron deference, said Abbe R. Gluck, a health law expert at Yale Law School who served in the White House at the beginning of Mr. Biden’s term. For that reason, she thinks litigation over the most recent rules may be less influenced by this change than challenges concerning some older regulations.

“The Supreme Court has not relied on Chevron in quite a few years,” she said. “So the federal government, including H.H.S., has become accustomed to drafting regulations and making its interpretation arguments as if Chevron did not exist.”

“They’ve already adjusted,” Ms. Gluck said.

Treasury and the Internal Revenue Service

The Treasury Department and the Internal Revenue Service both have broad mandates to interpret legislation when they write rules and regulations and enforce the tax code.

Since the Inflation Reduction Act passed in 2022, the Treasury Department has been racing to roll out regulations related to billions of dollars of clean energy tax credits that provide huge incentives for things such as the manufacturing of batteries or the purchase of electric vehicles. The Treasury Department has received pushback from some lawmakers who contend that it has not followed the intent of the law.

Although Congress creates the tax code through legislation, the I.R.S. has wide latitude in how the tax laws are administered. Accounting experts have suggested that the court’s ruling could complicate the agency’s ability to administer the tax code without specific direction from Congress.

A recent example is how the agency last year delayed enforcement of a contentious tax policy that would require users of digital wallets and e-commerce platforms to report small transactions. The new provision was introduced in the tax code in 2021 but was strongly opposed by lobbyists and small businesses.

The I.R.S. received criticism from some lawmakers for delaying the policy, but the agency defended its decision by arguing that taxpayers needed a longer transition period before the measure should be enforced to avoid a chaotic tax season.

Elizabeth Dias , Teddy Rosenbluth and Roni Rabin contributed reporting.

UCL Mathematical & Physical Sciences

• News and events
• Departments
• About the Faculty
• Innovation & Enterprise
• Equity (EDI)

Exploring the importance of International Asteroid Day

Why should we care about asteroids? Our student Mayukh answers this question and talks about space research in the blog marking International Asteroid Day.

30 June 2024

Amidst black holes, quasars, and the Big Bang, something equally intriguing caught my attention as I studied the module, “Modern Physics, Astronomy and Cosmology” at UCL: asteroids. Yep, those seemingly harmless space rocks have a lot more going on than meets the eye.  Enter International Asteroid Day , a global initiative that aims to raise public awareness about the potential hazards of asteroid impacts and inform us about what to do in case a near-Earth Object (NEO) decides to pay us an uninvited visit. Why should we care about asteroids? Asteroids might sound like the stuff of sci-fi movies, but they’re very real and can pose serious risks to our planet. These space rocks range from tiny pebbles to giant boulders stretching several kilometres.  While most hang out in the asteroid belt between Mars and Jupiter, some take a detour and head our way due to gravitational interactions with planets or other asteroids, altering their orbits. And when they do, they can cause quite a ruckus. Remember the dinosaurs? Their untimely demise 66 million years ago was partly thanks to a massive asteroid impact. More recently, in 2013, a small asteroid exploded over Chelyabinsk, Russia, injuring over 1,000 people. So, understanding these threats isn’t just for scientists — it’s for all of us. What’s International Asteroid Day all about? International Asteroid Day,  marked each year on 30 June, is like humanity’s annual wake-up call about asteroids. This date marks the anniversary of the Tunguska event in 1908, when an asteroid flattened about 80 million trees in Siberia. That’s a pretty big hint that we should take these things seriously. This day brings together experts from around the globe — astronomers, physicists, disaster management pros — to share their knowledge and strategies for spotting and dealing with potential threats. It’s a chance for all of us to get clued up on what’s out there and what we can do about it. Teamwork makes the dream work One of the coolest things I’ve learned in my studies with UCL Physics and Astronomy is how international cooperation is at the heart of space research. Agencies like NASA and the European Space Agency (ESA) aren’t just staring at the stars — they’re working tirelessly to keep us safe. Using advanced telescopes, satellites, and radar systems, they track asteroids and predict their paths. If a threat is detected, these organisations have detailed plans to inform the public and coordinate a response. Think of it as a global neighbourhood watch, but for space rocks. They’ve even got plans for deflecting or destroying dangerous asteroids, just in case. The future of space defence Despite all the progress, we can’t afford to sit back and relax. Continuous research and investment in space technology are essential. Take NASA’s DART (Double Asteroid Redirection Test) mission, for instance. Launched in November 2021, DART successfully demonstrated the ability to change the orbit of an asteroid in September 2022 by crashing into the moonlet Dimorphos of the asteroid Didymos. This historic achievement proved that with the right technology, we can alter the path of an asteroid and potentially prevent a catastrophic impact on Earth. Pretty cool, right? My studies have shown me how interconnected and collaborative space research is. It’s a field that needs global cooperation, constant learning, and public support. Through initiatives like International Asteroid Day, we can help ensure these efforts get the attention and funding they deserve. Whether you’re a student like me, a space enthusiast, or just curious about the cosmos, there’s always more to discover and contribute. International Asteroid Day isn’t just about acknowledging a potential threat — it’s about celebrating our collective achievements in protecting our planet and inspiring future generations. Let’s stay curious, stay informed, and stay prepared. After all, the universe is a big place, and who knows what’s out there waiting to be discovered? Mayukh Das MSci Chemistry student (Department of Chemistry)

Keep up to date:

Funnelback feed: https://cms-feed.ucl.ac.uk/s/search.json?collection=drupal-mathematical-... Double click the feed URL above to edit

R news and tutorials contributed by hundreds of R bloggers

The hidden challenges of working with cloud large language models: a text2graph case study.

Posted on July 1, 2024 by Gift Kenneth in R bloggers | 0 Comments

Staying ahead of the curve in machine learning often means adapting to unexpected changes. Recently, our team at Appsilon encountered a situation that highlights the importance of constant monitoring and flexible solutions when working with cloud-based Large Language Models (LLMs).

Interested in a demo of our Text2Graph application? Reach out to experts to set up a call today.

Today, we’d like to share our experience with GPT-4 and how it impacted Text2Graph (our R/Shiny application designed to transform your data into insights).

The Scenario: Using LLM APIs In Text2Graph

Our Text2Graph platform relies on GPT-4 to generate code based on specific prompts. The core of our solution involved sending a carefully crafted prompt to GPT-4 and expecting the generated code to be neatly wrapped in triple backticks (“`).

The Unexpected Change: Challenges With LLM API Updates

After a recent update to GPT-4, we noticed something peculiar. In about 50% of cases, the model started including the language type within the code block, resulting in outputs like ```r instead of just ``` . This subtle change, while seemingly minor, had a significant impact on our application’s ability to process the generated code correctly. Let’s make it clear. The whole prompt was exactly the same, but the output has changed in a systematic way!

Our Solution: Adjusting Prompts For LLM APIs

Interestingly, our approach to solving this issue wasn’t to modify our application to handle both “` and “`r. Instead, we found that adjusting our prompt was the most effective solution. This experience underscores the importance of prompt engineering and the delicate balance between the prompt, the model, and the application processing the output.

The Takeaway

This incident brings us to a crucial point that all developers and companies working with cloud LLMs should keep in mind:

With cloud LLMs, there’s no guarantee that a solution working today will continue to work tomorrow.

Unlike traditional software where you have control over the version and behavior of your tools, cloud-based AI models can be updated at any time, potentially altering their output in ways that might affect your applications.

Interested in learning how we apply machine learning to drug discovery? Check out Crystal Clear Vision, our model for protein crystal detection.

The Importance Of Monitoring

This experience showed us the importance of monitoring production applications that rely on LLMs.

Here are a few key reasons why:

• Regular monitoring can help you quickly identify when model outputs start deviating from expected patterns.
• By catching issues early, you can adjust your prompts or application logic to maintain consistent performance.
• Continuous monitoring helps ensure that your AI-powered solutions remain reliable and trustworthy for your users.
• With proper monitoring in place, you can adapt to changes in model behavior swiftly, minimizing downtime or degradation in service quality.

Additional Challenges Of Using LLM APIs

While building with LLMs, here are some other challenges that should be considered:

• Context Window Limitations: LLMs have input size limits , necessitating creative solutions like chunking data or using embeddings.
• Latency and Performance Issues: LLMs can be slow, and chaining calls makes latency problems worse. Sometimes you may need to opt-in for a more performant model of a lesser quality.
• Prompt Engineering Complexity: Crafting effective prompts requires continuous experimentation. It’s worth pointing out that an optimal prompt for OpenAI’s GPT-4o will be different from Anthropic’s Claude 3.5. It will be even different from the GPT-4 prompt!
• Prompt Injection Risks: Guardrails are needed to mitigate security risks.
• Product Development Realities: LLMs are tools for features, not complete products, requiring standard design and validation processes.
• Legal and Compliance Concerns: Ensuring data privacy and regulatory compliance is essential.

Summing Up Challenges Of Working With Cloud LLMs

As we continue to push the boundaries of what’s possible with AI and LLMs, we should remember that these powerful tools come with their own set of challenges. By being ready to adapt, we can utilize these tools to their full potential. Sometimes it’s worth using additional layers between you and the LLMs with tools like LangChain .

At Appsilon, we’re committed to sharing our experiences and insights as we navigate this exciting and rapidly changing landscape. We hope that by sharing this, we can help other teams better prepare for the unique challenges of working with cloud-based LLMs.

Interested in more insights and best practices in R/Shiny and Machine Learning? Subscribe to our newsletter to stay up to date.

Note : Thank you Pasza Storożenko for providing guidance in writing this article.

The post appeared first on appsilon.com/blog/.

Copyright © 2024 | MH Corporate basic by MH Themes

Never miss an update! Subscribe to R-bloggers to receive e-mails with the latest R posts. (You will not see this message again.)

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

Research: Using AI at Work Makes Us Lonelier and Less Healthy

• David De Cremer
• Joel Koopman

Employees who use AI as a core part of their jobs report feeling more isolated, drinking more, and sleeping less than employees who don’t.

The promise of AI is alluring — optimized productivity, lightning-fast data analysis, and freedom from mundane tasks — and both companies and workers alike are fascinated (and more than a little dumbfounded) by how these tools allow them to do more and better work faster than ever before. Yet in fervor to keep pace with competitors and reap the efficiency gains associated with deploying AI, many organizations have lost sight of their most important asset: the humans whose jobs are being fragmented into tasks that are increasingly becoming automated. Across four studies, employees who use it as a core part of their jobs reported feeling lonelier, drinking more, and suffering from insomnia more than employees who don’t.

Imagine this: Jia, a marketing analyst, arrives at work, logs into her computer, and is greeted by an AI assistant that has already sorted through her emails, prioritized her tasks for the day, and generated first drafts of reports that used to take hours to write. Jia (like everyone who has spent time working with these tools) marvels at how much time she can save by using AI. Inspired by the efficiency-enhancing effects of AI, Jia feels that she can be so much more productive than before. As a result, she gets focused on completing as many tasks as possible in conjunction with her AI assistant.

• David De Cremer is a professor of management and technology at Northeastern University and the Dunton Family Dean of its D’Amore-McKim School of Business. His website is daviddecremer.com .
• JK Joel Koopman is the TJ Barlow Professor of Business Administration at the Mays Business School of Texas A&M University. His research interests include prosocial behavior, organizational justice, motivational processes, and research methodology. He has won multiple awards from Academy of Management’s HR Division (Early Career Achievement Award and David P. Lepak Service Award) along with the 2022 SIOP Distinguished Early Career Contributions award, and currently serves on the Leadership Committee for the HR Division of the Academy of Management .

Partner Center

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• BMC Health Serv Res

Experience of learning from everyday work in daily safety huddles—a multi-method study

Karina wahl.

1 Department of Paediatrics, Region Jönköping County, 55185 Jönköping, SE Sweden

2 Department of Clinical and Experimental Medicine, Linköping University, Linköping, Sweden

Margaretha Stenmarker

3 Futurum/Department of Paediatrics, Region Jönköping County, Jönköping, Sweden

4 Department of Paediatrics, Institute of Clinical Sciences, The Sahlgrenska Academy at the University of Gothenburg, Gothenburg, Sweden

5 Jönköping Academy for Improvement of Health and Welfare, Jönköping University and Futurum, Jönköping Region Jönköping County, Jönköping, Sweden

Associated Data

The datasets generated and analysed during the current study are not publicly available due to the ethical approval but are available from the corresponding author on reasonable request.

To reduce patient harm, healthcare has focused on improvement based on learning from errors and adverse events (Safety-I). Daily huddles with staff are used to support incident reporting and learning in healthcare. It is proposed that learning for improvement should also be based on situations where work goes well (Safety-II); daily safety huddles should also reflect this approach. A Safety-II-inspired model for safety huddles was developed and implemented at the Neonatal Care Unit at a regional hospital in Sweden. This study followed the implementation with the research questions:

Do patient safety huddles with a focus on Safety-II affect the results of measurements of the patient safety culture? What are the experiences of these huddles amongst staff? What experiences of everyday work arise in the patient safety huddles?

A multi-method approach was used. The quantitative part consisted of a questionnaire (151 items), submitted on four different occasions, and analysed using Mann Whitney U-test and Kruskal Wallis ANOVA-test. The qualitative data were analysed using thematic content analyses of interviews with staff ( n  = 14), as well as answers to open questions in the questionnaires.

There were 151 individual responses to the questionnaires. The response rates were 44% to 59%. For most comparisons, there were no differences. There were minor changes in patient safety culture measurements. A lower rating was found in December 2020, compared to October 2019 ( p  < 0.05), regarding whether the employees pointed out when something was about to go wrong. The interviews revealed that, even though most respondents were generally positive towards the huddles (supporting factors), there were problems (hindering factors) in introducing Safety-II concepts in daily safety huddles. There was a challenge to understanding and describing things that go well.

Conclusions

For patient safety huddles aimed at exploring everyday work to be experienced as a base for learning, including both negative and positive events (Safety-II); there is a need for an open and permissive climate, that all professions participate and stable conditions in management. Support from managers and knowledge of the underpinning Safety-II theories of those who lead the huddles, may also be of importance.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08462-9.

To improve patient safety and reduce patient harm, healthcare providers have focused on routines for risk management and systems to identify and remedy deficiencies. Despite all efforts made in patient safety work, the number of healthcare injuries is still high [ 1 ]. In the work on systems for incident reporting, too much focus has been on collecting reports, with less on learning, improvement, and the social processes around incidents [ 2 ]. More focus on these latter factors may help in reducing the rate of incidents, as the actions and efforts tried previously have not brought about the desired improvements in healthcare.

It has been suggested that patient safety work is facing a paradigm shift [ 3 ]. Instead of only looking at incidents and accidents (termed a Safety-I perspective), it is also important to focus on learning from what employees do well, and how they adapt to the varying and difficult conditions in which they work (Safety-II) [ 4 ]. In a Safety-II perspective, safety work should be based on understanding how individuals act and perform their work in everyday life, so that things most often go right, and then based on that understanding, try to support work so that safety is improved [ 4 ].

A system is said to be resilient when it can achieve what it is intended to accomplish, both under expected and unexpected conditions [ 5 ]. Resilience engineering (RE) is a field that studies system resilience, and entails the conscious design of a system to make it resilient [ 6 ]. In resilience, four potentials are suggested as important in a system, namely: to be able to respond—to know what to do in different situations; to monitor—to know what to look for and measure it;—to learn from all experiences; and to anticipate—to know what to expect in the future [ 7 ]. The Safety-II perspective is necessary in order to describe how these potentials are expressed in a system (for example a workplace in healthcare), and to develop an understanding of how work in ordinary life is accomplished, taking into account all variability that there is [ 4 ].

The literature on resilience and Safety-II and their application in healthcare is expanding, but mainly based on case studies and from theoretical and methodological perspectives. It has been argued that there is a need for more empirical research in the field [ 8 ]; and to design interventions and operationalise changes based on RE principles and to measure their effectiveness [ 9 ]. A review of RE literature highlighted the need for studies on hindering factors for implementation of Safety-II and RE principles in practice, as well as the need to provide practical guidance to managers on how to design and operate resilient organisations [ 10 ].

It has been argued that a Safety-II and resilience approach to learning from everyday work stimulates staff participation in the learning process, which supports patient safety improvement [ 11 ]. There are existing models for reflection in daily huddles [ 12 – 14 ]. The Green Cross model is commonly used in healthcare in Sweden with the purpose of learning from adverse events [ 14 ]. In the Green Cross model, staff meet in a daily huddle to discuss and reflect on patient safety incidents and risks for patient harm that have occurred during the day in order to support incident reporting and learning – a Safety-I approach. The Resilient Performance Enhancement Toolkit (RPET), is suggested as a tool that supports daily conversations at a workplace to promote learning and improvement based on everything that happens in ordinary work – a Safety-II approach. Within an improvement work, a daily safety huddle based on a Safety-II approach, called the Green Line, was developed and introduced at the neonatal intensive care unit (NICU) at the Ryhov county hospital in Jönköping, Sweden in October 2018. The Green Line was developed in close cooperation with Erik Hollnagel while he was writing up his RPET whitepaper [ 15 ]. The overall aim of the improvement work in the NICU is to improve patient safety through introducing a Safety-II approach to learning from everyday work.

The Green Line is a tool to support daily conversations and to promote learning and improvement based on everything that happens in ordinary work in a patient safety huddle. There are emerging descriptions on the use of different methods to use a Safety-II approach to support learning in healthcare settings based on written accounts of experiences of everyday work [ 11 , 16 ]. Patient safety huddles offer a simpler and more direct way of communication than written accounts, and might thus be a more convenient way to promote learning and improvement from ordinary work.

The aim of this study was to describe experiences of changing the focus of patient safety work from only learning from deviations (Safety-I) to also learning from when things go well (Safety-II) during patient safety huddles at a hospital unit, using the research questions:

• Do reflections with a focus on Safety-II in patient safety huddles affect the results of measurements of the patient safety culture conducted using questionnaires?
• What experiences of the Green Line with a Safety-II approach have staff had?
• What experiences of everyday work arise in the patient safety huddles that can be classified according to the potentials defined in resilience engineering; respond, monitor, learn, anticipate?

This study follows the implementation of the Green Line method at an NICU with a multi-method approach, using separate quantitative and qualitative analyses. The study was originally planned as an improvement work and the instruments used for the study were chosen based on the methods used in the improvement work in combination with data from it. A description of how the study relates to changes in the improvement work is described in Fig.  1 .

Study and improvement work. Description of changes in the local context and the design of the study and the improvement work. Description of when the survey and interviews in the study were conducted linked to the situation at the unit. J1 = January, F = February, M = March, A = April, M = May, J2 = June, J3 = July, A = August, S = September, O = October, N = November, D = December

Context/settings

The NICU studied is part of the department of paediatrics at a regional hospital in the southern part of Sweden. There are 16 patient beds, and approximately 40 nurses, 20 assistant nurses and seven doctors are employed. The unit cares for newborn infants and premature infants born from gestational week 27, in need of intensive care. Two nurse managers and a consultant physician are responsible for management of the unit. In each work shift, the management has appointed a bed and staff coordinator. A quality and patient safety developer at the department of paediatrics supports improvement works at the NICU.

The Green Line improvement work at the NICU

The NICU management team decided to introduce the Green Line method to improve and support patient safety work at the unit. An inter-professional improvement group with nurses and assistant nurses introduced and led the implementation, starting in October 2018. All employees were introduced to the Green Line model and trained in how to use it, including the theoretical concepts of resilience and Safety-I and -II. To facilitate the implementation of the Green Line, the inter-professional improvement group had support from the quality and patient safety developer and the unit management team. Regular meetings with the improvement group were planned to follow up on and change the improvement work when needed. Improvements were tested and changed according to the PDSA model for improvement work: Plan, Do, Study, and Act [ 17 ] (Additional file 1 ). Professor Erik Hollnagel was present at a huddle at the beginning and participated in a meeting with the improvement group to provide input on the improvement work. In discussions within the improvement group together with the quality and patient safety developer the general questions to support Safety-II-inspired safety huddles suggested in RPET [ 15 ] were adapted to suite the workplace and the intentions of the improvement work. The questions were open with the intention to use follow-up questions: “How did you manage that?”, “Can you describe more?” Based on the experiences of the patient safety huddles, questions were changed over time so that the reflections in the huddle were perceived as valuable for learning. Some examples are: “How have we succeeded today?”, “What have we done to ensure that all children receive food?” To sharpen the patient safety huddles focus topics for reflection were introduced, for example encounters with parents and staff, and breaks to provide relief from work.

The Green Line reflections was planned to be 5–10 min long in the afternoon and all staff that could attend were invited to take part. During the meeting, short notes were taken by one of the participants in the patient safety huddle, to insure that ideas for improvement could be collected. The bed and staff coordinator led and facilitated the reflection. Sometimes this task was performed by members of the improvement group, when they were on duty. At the end of the reflection, the meeting was classified according to its main content: discussion and learning based on a Safety-II perspective, discussion and learning based on a negative event (Safety-I), only reporting of a negative event, or no reflection at all. A classification, based on a colour scheme, was used as is described in Erik Hollnagels RPET whitepaper [ 15 ]. To support the learning process and ensure deeper reflection, the quality and patient safety developer participated once or twice a month in the safety huddles. This person also wrote a summary every month of the number of participants, profession, colour classification and what potentials the conversations could be classified within. The summary was shown to the employees to further strengthen their awareness of the role of resilience and the Safety-II approach. During the improvement work, the questionnaire described in the quantitative method section was used to follow up on the patient safety culture at the unit. One statement was added so that the questionnaire had items to cover all four potentials described in resilience. The question added was formulated “At my workplace, we have routines and working methods that allow us to be prepared for challenges that may arise”, and is not reported in this study.

Quantitative method

Data collection and analysis.

The questionnaires used in the improvement work to investigate the patient safety culture formed the basis for the quantitative analyses of this study. The questionnaire contained 11 items phrased as statements and was distributed and repeated four times during the study period. This questionnaire has been developed by the Swedish association of local authorities and regions, together with researchers and experts in patient safety, as a simplified tool for patient safety culture assessment [ 18 , 19 ]. The answer alternatives in the questionnaire use a five-point Likert scale, 1–5 (1 = I totally disagree, 5 = I fully agree) (Additional file 2 ). One open-ended question to capture the views of the individual was added in the last three surveys, namely “Comments on the Green Line reflections?” The questionnaires were sent out to all employees by work e-mail on four occasions, October 2018, March 2019, October 2019, and December 2020, along with a request for voluntary participation (Additional file 3 ). The answers were collected using the web survey system esMaker [ 20 ], which enables anonymous participation.

Statistical analysis

Results are presented as numbers, frequencies, medians and 25th and 75th percentiles when suitable. A Mann Whitney U-test was used to analyse differences between two groups. A Kruskal Wallis ANOVA-test was used when there were comparisons between more than two groups. The adjusted P -value is shown. The following were compared: occasion for the surveys, profession and years of employment at the unit (≤ 10 year, > 10 year), The P -value is reported as * < 0.05, ** < 0.01, *** < 0.001. The data analysis was generated using SPSS version 25 (IBM Corp., Armonk, NY, USA).

Qualitative method

Two sources were used for the qualitative analysis: the answers to the open question in the questionnaire “Comments on the Green Line reflections? Enter positive as well as negative views”, and semi-structured individual interviews with employees ( N  = 14). These interviews were performed from December 2020 to January 2021.

In the interviews, employees were asked about participation by a strategic selection with maximum variability sampling to achieve a wide distribution by profession and by years of employment, and by convenience (Fig.  2 ). Information was provided in writing (Additional file 4 ) and orally. The interviews were conducted by KW, an experienced quality and patient safety developer, under the guidance of AR and MS who are experienced researchers. The interviews were recorded, a question guide was used (Additional file 5 ) and notes were taken. At the end of the interview a summary was made by KW and additional comments and changes could be made. The interviews lasted 5—24 min (mean 14 min) and were conducted during working hours. The interviews were transcribed verbatim prior to the analysis and were anonymised. The results were analysed on a group level, and are presented with de-identified illustrative quotes.

Interviewees – demographic information. Description of demographic information regarding the interview participants, their profession, number of times they participated in reflections and number of years at the unit

Three different qualitative analyses were performed.

• I). A deductive analysis of the interviews to provide an answer to the question of what experiences of everyday work are highlighted in the reflections. The four potentials described in resilience—respond, monitor, learn and anticipate—were used as a framework in the analysis [ 7 ].
• II and III). Two different inductive thematic content analyses were performed separately on the interviews and the open answers in the questionnaire to the question about experiences of the Green Line approach. The content analysis according to Malterud is based on themes, codes and meaning units and is an analysis of different types of data, such as interviews and written text [ 21 ].

The first analysis was made by KW, and an initial version of themes and codes was developed. The analysis was then adjusted and verified in discussions with AR and discussed and confirmed with MS. The results were validated through feedback from the interviewees.

The results section contains a description of changes during the improvement work as well as quantitative and qualitative results.

Changes and results in the improvement work

During the study period the improvement work changed over time. The changes were made by the improvement group, based on experiences of the implementation and according to the PDSA model. These changes and some contextual factors are displayed in Fig.  1 . The frequency of safety huddles changed during the studied period; the initial intention was to have them daily, but that was intentionally changed to twice a week. But even so, not all planned safety huddles were performed as planned. Changes were made to the questions used to support the reflections and also to how often the patient and safety developer participated in the safety huddles. There were also changes in management; in one period there was one manager instead of two. On average, eight to ten professionals participated in each patient safety huddle. During the Covid-19 pandemic no intentional changes were made in the safety huddles, except that they were moved from a small room to a larger one. Examples of the monthly summaries intended to further strengthen awareness of resilience and safety-II is displayed in Additional file 6 , with the number of participants, profession, colour classification and what potentials of resilience the conversations could be classified within.

Examples of practical improvements at NICU as a result of the safety huddles are provided in Additional file 7 .

Quantitative result / results of survey

There were 151 individual responses to the questionnaires, submitted on four different occasions. The response rates were 44% to 59% (Tables  1 , ​ ,2 2 ,  3 ). The results are presented in Tables ​ Tables4, 4 , ​ ,5, 5 , ​ ,6 6 ,  7 . For most comparisons, there were no differences. For some there were significant differences, but with small differences in specific numbers. The statement “I point out when I think something is about to go wrong” was valued significantly lower in December 2020 compared to October 2019 (Table ​ (Table4). 4 ). The following statements: “In my workplace, we always act on the risks we see”, “I dare to talk about my mistakes” and “I would feel safe if a close relative was cared for at my workplace” were rated significantly lower by those who had worked > 10 years compared to those who had worked ≤ 10 years (Table ​ (Table5). 5 ). Physicians indicated to a higher degree their ability to adapt and collaborate than other professions did (Table ​ (Table6). 6 ). There was no difference in comparisons of the statement “The Green Line reflections lead to learning” within the entire staff group i.e. not between professions, or according to years employed in the last three surveys (Table ​ (Table7 7 ).

Survey responses, by four occasions for questionnaires

Survey responses on four occasions. Questionnaires, number of responses, number of questionnaires and response rate in %

Survey responses by profession

Survey responses by professions number and %

Survey responses by years of employment at the unit

Survey responses by year at the unit, number and %. (One person had not specified year at the unit, 150 answers to this question)

Survey responses to questionnaires on four occasions

The answer alternatives in the questionnaire were a five-point Likert scale, 1–5 (1 = I totally disagree, 5 = I fully agree) (Additional file 1 ). N is number of answers. Median, percentile 25th, percentile 75th, * =  P  < 0.05, ** =  P  < 0.01, *** =  P  < 0.001

* is December 2020 in comparison with October 2019

Survey responses to questionnaires by years of employment at the unit

Survey responses to questionnaires by profession

The answer alternatives in the questionnaire were a five-point Likert scale, 1–5 (1 = I totally disagree, 5 = I fully agree) (Additional file 1 ). N is number of answers. Median, percentile 25th, percentile 75th, * =  P  < 0.05, ** =  P  < 0.01, *** =  P  < 0.001. * 1 and ** 1 is doctor in comparison with assistant nurses. ** 2 is doctor in comparison with nurse

Due to the small number of respondents in the group, other occupational group, 25th and 75th is not reported in question 4, 8, and 13

Survey responses to the question, “The green line reflections lead to a learning”

The answer alternatives in the questionnaire were a five-point Likert scale, 1–5 (1 = I totally disagree, 5 = I fully agree) (Additional file 1 ). The comparison is made by three occasions, by profession and by year at the unit respectively

N is number of answers. Median, percentile 25th, percentile 75th, * =  P  < 0.05, ** =  P  < 0.01, *** =  P  < 0.001

Qualitative results

Demographic data for each one of the interviews, are presented in Table ​ Table8 8 .

Demographic information of interviewed persons, their profession, years of employment at the unit and number of occasions they participated in the safety huddles

a In the article cited interviewed persons

Experiences of everyday work

The experiences reported were categorised according to the potentials described in resilience, i.e. respond, monitor, learn, and anticipate. There were many examples from learn and respond, fewer from anticipate and only one example from monitor.

The respondents gave examples of how they adapted to the conditions of work, for example by distributing patients and staff more evenly over the ward. NICU staff, both new and experienced ones, said that training was necessary to be able to respond properly to unusual situations.

‘When a very sick child arrives, suddenly large parts of the staff disappear to care for that child. How did we then manage to take care of all the other patients? A parent could take care of their child in a way that we had not planned. Someone from the general paediatric unit came to help us…” (P3).

“Everyone knows this except me…..then you realise that almost everyone wanted training…. some things are unusual…..”(P6).

This staff member indicated that she thought she was the only one interested in these issues, but the safety huddles indicated that her concerns were shared by her colleagues. The respondents believe that it is important to highlight examples in the safety huddles on how things work out, and how they have managed and responded to different situations.

Only one experience emerged that can be traced to the monitor potential. The respondent describes the importance of staff being with the patients and observing changes in the patient´s status so they do not miss anything. “You should not miss anyone…you have to observe, it is how I think…we observe them” (P12). This is an experience that is at the individual patient level; there were no experiences reported of the potential monitor at the system level.

Experiences were described of learning from activities in daily work, from mistakes and learning from colleagues but it was described as difficult to learn from situations that had been resolved.

“Narcanti, we have two different dilutions. First a mistake was discussed in the Green Line and then it was close to a mistake again ….. it was a nurse who reacted before the drug was given, we had talked about that kind of problem before (in the safety huddles)” (P7).

“….we highlight the good examples and learn from them …….what have we done well today?—Well, we have substituted for each other in coffee breaks, that was good …..There were no real learning opportunities” (P5).

The learning in the safety huddles were mainly from negative events, very few from things that had gone well, when problems were resolved. It seems difficult to get an in-depth reflection on why situations were resolved in a good way.

The respondents experienced that everyday work was becoming more and more unpredictable and complex, and some examples came up in the safety huddles of how they anticipated problems to be better prepared to deal with them.

“….plan your day with the person you work with…lunch and everything…who should go first…otherwise…no one has a break” (P1).

“Say you need help, instead of saying today I did not get a break…don’t think that you should sort it out yourself” (P1).

Work is often unpredictable, but it is important to plan the day when it is busy, for example, who should go on a break first and when, and ask for help instead of complaining afterwards. So even if it was not intentional in the format of the safety huddles, anticipation and preparedness for difficulties in the coming work shifts were subjects raised according to the respondents.

Reflections on the “Green Line method”

In both the analysis of interviews and the open answers from the questionnaire, two themes emerged, "Supporting factors" and "Hindering factors”. There was also an overlap between the codes; hence the results are reported together since both sources were reflections on the same phenomenon. Three and two codes respectively were found for each theme (Table ​ (Table9, 9 , Additional file 8 ).

Themes and codes from the analysis of the interviews and from the questionnaire

Supporting factors

The theme “Supporting factors” describes the codes "Seeing benefits with reflection", "Learning from what happens" and "Finding improvements for a rewarding reflection".

Seeing benefits with reflection

All respondents mentioned that it was valuable to have reflections in general. The safety huddles offer an opportunity for those who do not speak out in any other context in daily work or staff meetings at the unit. There is an opportunity to get confirmation that they have done the right thing and they can get input from others´ solutions.

On the basis of good examples from the reflections in the huddles it can be easier to address negative things. The safety huddles also support creating common values and cohesion in a unit where the employees are working in different areas.

“I really think it's great …. It’s not only mistakes that should be noticed, you can learn a lot from each other, everyone has different experiences….” (P7). "It's such a scattered department you may not even see others throughout the shift… I think there will be a little more cohesion in the group because of reflections…" (P11).

It is said that it is valuable to reflect on what happened during a day, and the safety huddles improve the cohesion in the working group. Without reflections in the huddles it is difficult to get the opportunity to share experiences because the unit is divided into different care rooms.

Learning from what happens

It was experienced as difficult to talk about and learn from things that went well; these positive experiences are taken for granted, and it was easier to talk about something negative. But the view was expressed that it was good to highlight and concretise things that went well so others could learn from them. There were also notes that negative comments were not taken seriously. Sometimes it was perceived as taboo to talk about when something went wrong.

“It is difficult because…for everything in life really, if you do not hear anything, then it is probably often good… you are only told the bad things” (P1).

“No one dared to say anything that was negative” (free comment survey 2020).

This may depend on expectations that only positive events should be addressed in the huddle, or on the working climate and the role of openness. Hence, it is difficult to focus on what goes well when nothing negative happens and the challenge of talking about both the positive and negative things instead of just the negative things.

Finding improvements for a rewarding reflection

The respondents pointed out suggestions for improvements to make reflections in the patient safety huddles more useful and valuable. The role of the safety huddle leader was important; he/she needed to be interested, direct the conversations and believe that the reflection was important. It was good that the quality and patient safety developer sometimes facilitated the huddles. It was important to develop the method without changing to new methods or giving new names to the method; the method was just the tool. The huddles needed to be varied and inspiring, not static with too limited conversation. It was good to vary the questions for example with different focus topics.

“… Someone who is clear about the purpose and who agrees with the purpose, I think so, not just someone who is set to lead that reflection” (P9).“…they became more inspired… the (reflection) leader must have the ability to angle the questions” (P2). “It became more lively when we started using focus topics” (P11).

The safety huddles should be regular, short, objective and with the right focus. There were different opinions about the frequency: every day, twice a week or on demand. The safety huddles needed to be planned in the schedule, so that doctors could also participate, since they were better if all professions participated. It was also regarded as important that the managers were involved and supportive.

“…that the managers try to participate and are interested and also think it is important” (P3).

It seems necessary to clarify the purpose of the safety huddles and to find ways to spread lessons from them. The leadership role of the safety huddle is important, as well as the ability of the leader of the huddle to get in-depth reflections. It is important to involve all, to schedule all professions so that they can participate in safety huddles.

Hindering factors

The theme “Hindering factors” describes the codes: "Seeing difficulties with reflection" and "The impact of the work climate”.

Seeing difficulties with reflection

Some respondents said that safety huddles got stuck in the format, so that the format was more important than what was reflected on, which was perceived as inhibiting, and there was a need to clarify the purpose of the huddles. There were no learning opportunities and it was hard to keep them serious and focused. There was more focus on staff working hours and breaks than on the actual task of creating good care. When the reflection was based on what went well, it was often the same things that came up, which was not useful. It was difficult to find times that suited all employees to attend the huddles. Things that needed to come up were not discussed, and things that came up were not carried forward, since those who could answer were not present, which was frustrating.

“..There was often a lot of repetition, it was the same thing. And everything that becomes the same thing becomes very boring” (P2). “Feedback is given but stays there …it may need to reach other people … or make improvements… often it stays in the small group… and the challenge may continue to bother you” (P6).

Difficulties with the safety huddles were described; it was hard to keep the reflections serious and focused, there were no learning opportunities and the purpose was not clear. It was difficult to make improvements when not all professions participated.

The impact of the work climate

The respondents felt that how easy it was to dare to reflect openly varied, depending on the situation and the constellation of participants. Comments emerged that sometimes the atmosphere during the safety huddles was not inviting and the conversations were superficial. There was a desire for an open and permissive climate but the experience was that this was not always the case. The work in the unit meant that the employees were scattered inside different care rooms and did not see each other during the working day if there is no opportunity to gather, for example during a safety huddle. There was a great work experience and skill in the group; many had worked for a long time and it was difficult as a new person to dare to talk, and it was difficult to join the working group. The managers were important for creating the climate and supporting the Green Line reflections.

“We should have a slightly more open climate in our department….the attitude of some in the staff group may be….judges a little too easily sometimes” (P6). “…when you are new, you are invisible” (P5). “Managers must be involved in the Green Line project and support an open climate…..” (P3).

The impact of the work climate and the difficulties that exist when employees were scattered in individual care rooms throughout the day were described. It was difficult to join the working group when there was great work experience and skill in the group. A safety huddle could lead to improved cohesion and community in the group but needed support from the managers.

This study evaluates the introduction of Safety-II inspired reflections in patient safety huddles for staff at a hospital ward. Thus, it is an attempt to draw empirical knowledge from interventions designed to operationalise changes based on Safety-II and resilience engineering principles [ 8 , 9 ]. Most respondents were positive towards safety huddles generally, but it was found that to really lead to learning and improvement, the format and support for a Safety-II inspired reflection needs to be developed and the purpose needs to be clarified further. There were different opinions about what was easy or difficult when performing the safety huddles. Our findings suggest these matters depended on the situation, who took part in the safety huddle and who led it. There were minor changes in some aspects of patient safety culture measurements over time during the study period. In the experiences discussed in the safety huddles, there were examples of the system potentials of resilience: learn, respond and anticipate, but only one of the potential monitor.

It was perceived difficult to reflect on and learn from what was going well, the Safety-II perspective. The literature on learning in a Safety-II perspective is still sparse. In one study on an intervention based on written reports on things that had gone well, the number of reports was smaller than expected, problems getting staff engaged on a wide scale were discussed, and it was concluded that learning from how things go well is a simple yet compelling concept [ 16 ]. Our study supports this reflection; Safety-I learning took precedence in relation to Safety-II in the safety huddles at the NICU, even though both can co-exist. We normally “see” when an adverse event takes place, but we do not “see” when an adverse event does not take place, when things go well [ 15 ]. Healthcare professionals are trained to see and report adverse events [ 2 ]. If they do not see what is going well, it is difficult for them to understand and describe it. In a study of nurses' experiences of the incident reporting culture after implementation of the Green Cross method, it was found that it was not good to focus only on things that went wrong, and it was suggested that health care would benefit from learning both from successes and errors [ 22 ]. In the present study it was found necessary that staff understood that shifting focus from Safety-I to II should include learning from both Safety-I and II perspectives [ 23 ].

One goal of the Green Line reflections was to support learning. Adults learn what they experience as meaningful, they take as much responsibility as they are interested in, and they do not get involved if they do not see any meaning to what they are learning [ 24 ]. Leadership is important to create good conditions and a permissive climate for learning [ 25 ]. Our study supports the view that the role of managers is important; reflections in patient safety huddles need support from clear leadership by the managers at the unit, and the purpose of the safety huddles needs to be constantly clarified [ 14 ]. Managers at the clinical level are central to the system’s capacity for expressing resilience but they need more models and training in how to approach their work [ 25 ]. Managers need to continuously follow up an intervention to reinforce commitment for a change to be fully accepted and established in the workplace [ 26 ]. A development-oriented leadership where managers support employees' learning as part of development can be successful. The manager's role is to clarify expectations, prioritise development issues, create resources, and to follow up [ 27 ]. In this study, there were shortcomings in how the improvement work was followed up in the long term.

The impact of the work environment is central. Psychological safety, a belief that one will not be punished or humiliated for speaking up with ideas, questions, concerns or mistakes, is important in a workplace to support tolerance and openness [ 28 ]. Tolerance and openness in a workplace helps patient safety huddles to be perceived as rewarding so that they support learning based on reflections on both negative and positive events. The desire is to improve the work environment based on experiences expressed in the reflections; but there also needs to be a good work environment to encourage people to risk sharing experiences. There was a significant change from October 2019 and December 2020 regarding whether the employees pointed out when something was about to go wrong (with a lower rating in December 2020). Apart from aspects of psychological safety that were expressed in the interviews, this may also be explained by the management changes, and changes in format of and support for the safety huddles. In an NICU, collective learning based on safety reporting and accumulated knowledge in prioritizing of and performing the work may be difficult, since the work is performed inside individual care rooms, as has been pointed out by Hybinett et al. [ 25 ]. To see each other and share experiences may contribute to psychological safety. Safety huddles for all staff can offer an opportunity to share experiences and increase knowledge in a unit where work is dispersed, as in a NICU.

In this study, the Safety-II inspired safety huddles were found sometimes to be worthwhile and sometimes not. They turned out differently depending on who led the reflections, their experience in doing so, their knowledge of the theoretical background, and their ability to get an in-depth reflection. How open and tolerant the participants and the work climate were, was also of importance. Furthermore, it was appreciated if someone else from outside the unit, for example the quality and patient safety developer, sometimes led the reflection to ensure it was deeper. In a study of the Green Cross method in healthcare (i. e. Safety-I), Schwarz et al. also found that the leadership role in the meeting is important [ 14 ]. It helps to have supporting questions and open questions, and the questions need to be varied. In addition, it is good if all professions participate; in our study it was reported that physicians did not attend the safety huddles as regularly as other professions. In another study on the effect of the Green Cross method on incident reporting the participation of physicians was also highlighted [ 29 ]. For learning according to Safety-II to happen, the importance of reflecting on everyday practice, and ensuring that such reflection is routinely carried out in practice, is important [ 30 ]. Our findings support this; schedule planning is needed and the safety huddles have to occur regularly. The safety huddles need to be tailored to the staff's needs and have an actual impact on improving their work to be experienced as rewarding and valuable. There is a need to develop methods to spread lessons and support improvements based on positive events in the same manner as from negative events, i. e. to explore everyday work [ 31 ].

The respondents experienced that everyday life was becoming increasingly complex and there was a need to adapt to different situations. In complex enterprises such as modern health care it is necessary to make pragmatic adaptations to changing contexts, including in the introduction of improvement interventions [ 32 ]. To support the development and testing of improvements in complex healthcare systems, PDSA cycles are well established. In these, ideas are transformed into action, the actions are tested and studied to learn and to improve them, and this is continued in a cycle, for continuous improvement [ 17 , 33 ]. PDSA is an established improvement tool in the NICU, and it was used initially to support the Green Line work, but not thoroughly over time. It may have been valuable to continue the PDSA cycles until the Green Line reflections had been satisfactory established.

All four potentials that have been suggested to describe a resilient system [ 7 ], were exemplified in the reflections, but to varying degrees. Communication, for example the safety huddles in this study, can contribute to the four potentials, but do not directly contribute to resilient performance [ 34 ]. There were more examples reported in the interviews from the potentials learn and respond, while there were fewer from anticipate and only one from monitor. There is possibly a greater propensity for healthcare professionals to act, than to be actively aware of what they can expect from the future and from measurements. This can be exemplified by the work situation at an NICU, as has been described by Hybinette et al. The focus in an NICU is on unpredictable factors such as acute admissions, where one has to quickly readjust plans and actions and where the inflow of emergency patients may have the highest priority [ 25 ]. One possible way forward to better highlight and describe examples of expression of resilient capacity might be to use a number of pre-defined issues for the reflections which can reflect and draw attention to all four potentials.

Method discussion and limitation

Different methods were used with the aim of capturing aspects of the experience and impact of the Green Line method. This project was originally designed as an improvement project, not a research study. Had it been so, another approach to evaluating possible effects on patient safety culture would have been chosen. Measuring the patient safety culture using questionnaires can be useful. Hospitals that have good results in patient safety measurements also have lower numbers of adverse safety events; but further research is needed to investigate the relationship between the measured safety culture and the improvement in clinical safety [ 35 ]. The instrument chosen for evaluation of patient safety culture is widely known and used in Swedish healthcare in different contexts [ 18 , 19 ], and was therefore chosen by the improvement group. The introduction of the Green Line reflections was less likely to improve patient safety culture. However, the hope for that was one of the reasons behind the project for the management of the ward, and the improvement group. Therefore it was found most true and honest to the project and the workplace to include this in the study.

When the interviews were conducted, there had been no safety huddles for a while, which may have affected the answers. It is difficult to draw conclusions from comparisons (surveys) over time when the conditions are constantly changing; there were changes in the local context and in the design of the improvement work, which may have affected the results. The response rate was quite low in the survey responses; hence, conclusions have to be drawn carefully. However, the survey was supplemented with interviews and the results were largely consistent across both methods.

The first author's pre-understanding from being a quality and patient safety developer at the department of paediatrics and from taking an active part in the improvement work may have affected the results. Pre-understanding can also be important in the analysis and interpretation of data and can contribute to in-depth knowledge and understanding [ 36 ].

Based on this study´s results, it may be difficult to introduce reflections based on learning from everything that happens, including when things go well (Safety-II) into patient safety huddles. Careful planning is important for such interventions to be able to succeed. To make the reflections better, it is important to have support from managers, and for those who lead the safety huddles to have knowledge of the theories underpinning the Safety-II approach. For the participants, there needs to be an open and permissive climate, a plan to ensure that all professions can participate, and stable conditions in management and support of the safety huddles for them to be experienced as valuable for learning. Further studies are needed to understand how Safety-II-inspired safety huddles are best implemented and to determine whether increased understanding amongst employees of the purpose of the huddles may contribute to better patient safety and an improved patient safety culture.

Acknowledgements

Erik Hollnagel, for support when designing the improvement work.

Bo Rolander, statistician at Futurum, for help with the statistics.

Maria Olsson, librarian at Futurum library, for help with the reference list.

Authors’ contributions

All authors contributed to the design of the study. KW made the quantitative analysis, along with a statistician. All authors contributed to the qualitative analysis as described in the methods section. KW made the first draft of the manuscript which was then finalised by all authors, who then read and approved the final manuscript.

Research funding was obtained from Futurum – the academy of health and care, Region Jönköping County.

Availability of data and materials

Declarations.

Ethical approval for the study was obtained from the Swedish Ethical Review Authority (Registration number 2020–04448). Participation was voluntary and oral consent was collected from each participant. Additionally, oral informed consents have been approved by the Swedish Ethical Review Authority. The material was saved without linking it to individual participants. The study was in all aspects performed in accordance with the Declaration of Helsinki guidelines.

Not applicable.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Karina Wahl, Email: [email protected] .

Margaretha Stenmarker, Email: [email protected] .

Axel Ros, Email: [email protected] .

Opinion: As conservatives target same-sex marriage, its power is only getting clearer

• Copy Link URL Copied!

It’s been two years since the U.S. Supreme Court ruling in the Dobbs case that overturned the federal right to an abortion, and the troubling concurring opinion by Justice Clarence Thomas in which he expressed a desire to “revisit” other landmark precedents, including the freedom to marry for same-sex couples, codified nationally by the Obergefell Supreme Court decision, nine years ago Wednesday

Since that ruling, the LGBTQ+ and allied community has done much to protect the fundamental freedom to marry — passing the Respect for Marriage Act in Congress in 2022; sharing their stories this year to mark the 20th anniversary of the first state legalization of same-sex marriages, in Massachusetts; and in California , Hawaii and Colorado launching ballot campaigns to repeal dormant but still-on-the-books anti-marriage constitutional amendments.

California Democratic Party endorses ballot measures on same-sex marriage, taxes, rent control

The party’s executive board voted Sunday on which measures they would endorse.

May 19, 2024

This winter, I worked with a team at the Williams Institute at UCLA School of Law to survey nearly 500 married LGBTQ+ people about their relationships. Respondents included couples from every state in the country; on average they had been together for more than 16 years and married for more than nine years. Sixty-two percent married after the court’s 2015 Obergefell marriage decision, although their relationships started before before that. More than 30% of the couples had children and another 25% wanted children in the future.

One finding that jumped out of the data: Almost 80% of married same-sex couples surveyed said they were “very” or “somewhat” concerned about the Obergefell decision being overturned. Around a quarter of them said they’d taken action to shore up their family’s legal protections — pursuing a second-parent adoption, having children earlier than originally planned or marrying on a faster-than-expected timeline — because of concerns about marriage equality being challenged. One respondent said, “We got engaged the day that the Supreme Court ruled on the Dobbs decision and got married one week after.”

World & Nation

Same-sex marriage ruling creates new constitutional liberty

The Supreme Court’s historic ruling Friday granting gays and lesbians an equal right to marry nationwide puts an exclamation point on a profound shift in law and public attitudes, and creates the most significant and controversial new constitutional liberty in more than a generation.

June 26, 2015

As we examined the survey results, it became clearer than ever why LGBTQ+ families and same-sex couples are fighting so hard to protect marriage access — and the answer is really quite simple: The freedom to marry has been transformative for them. It has not only granted them hundreds of additional rights and responsibilities, but it has also strengthened their bonds in very real ways.

Nearly every person surveyed (93%) said they married for love; three-quarters added that they married for companionship or legal protections. When asked how marriage changed their lives, 83% reported positive changes in their sense of safety and security, and 75% reported positive changes in terms of life satisfaction. “I feel secure in our relationship in a way I never thought would be possible,” one participant told us. “I love being married.”

The evolution of same-sex marriage

I’ve been studying LGBTQ+ people and families for my entire career — and even still, many of the findings of the survey touched and inspired me.

Individual respondents talked about the ways that marriage expanded their personal family networks, granting them (for better and worse!) an additional set of parents, siblings and loved ones. More than 40% relied on each other’s families of origin in times of financial or healthcare crisis, or to help out with childcare. Some told of in-laws who provided financial assistance to buy a house, or cared for them while they were undergoing chemotherapy for cancer.

Analysis:: Antonin Scalia’s dissent in same-sex marriage ruling even more scornful than usual

The legal world may have become inured to wildly rhetorical opinions by Justice Antonin Scalia, but his dissent in the Supreme Court’s same-sex marriage decision Friday reaches new heights for its expression of utter contempt for the majority of his colleagues.

And then there was the effect on children. Many respondents explained that their marriage has provided security for their children, and dignity and respect for the family unit. Marriage enabled parents to share child-rearing responsibilities — to take turns being the primary earner (and carrying the health insurance), and spending more time at home with the kids.

The big takeaway from this study is that same-sex couples have a lot on the line when it comes to the freedom to marry — and they’re going to do everything possible to ensure that future political shifts don’t interfere with their lives. As couples across the country continue to speak out, share their stories — and in California, head to the ballot box in November to protect their hard-earned freedoms — it’s clear to me that it’s because they believe wholeheartedly, and with good reason, that their lives depend on it.

Abbie E. Goldberg is an affiliated scholar at the Williams Institute at UCLA School of Law and a psychology professor at Clark University, where she directs the women’s and gender studies.

More to Read

Opinion: Interracial marriage went from criminal to commonplace. Could it go back?

June 9, 2024

Newsom urges California voters to protect same-sex marriage amid Supreme Court distrust

June 7, 2024

The U.S. has caught up to California on views of LGBTQ+ rights, poll shows

June 6, 2024

A cure for the common opinion

Get thought-provoking perspectives with our weekly newsletter.

You may occasionally receive promotional content from the Los Angeles Times.

More From the Los Angeles Times

Opinion: We should all dissent from the Supreme Court’s immunity decision, and not respectfully

July 1, 2024

Opinion: If you were relieved by the Supreme Court’s abortion rulings this term, think again

Calmes: The Biden debate and the Supreme Court — the full catastrophe

June 30, 2024

Litman: A Supreme Court ruling may help Jan. 6 rioters. Here’s why it’s less likely to help Trump

June 28, 2024

Calculation of the minimum clinically important difference (MCID) using different methodologies: case study and practical guide

• Original Article
• Open access
• Published: 28 June 2024

Cite this article

You have full access to this open access article

• Anita M. Klukowska 1 , 2 ,
• W. Peter Vandertop 1 ,
• Marc L. Schröder 3 &
• Victor E. Staartjes   ORCID: orcid.org/0000-0003-1039-2098 4  

82 Accesses

Explore all metrics

Introduction

Establishing thresholds of change that are actually meaningful for the patient in an outcome measurement instrument is paramount. This concept is called the minimum clinically important difference (MCID). We summarize available MCID calculation methods relevant to spine surgery, and outline key considerations, followed by a step-by-step working example of how MCID can be calculated, using publicly available data, to enable the readers to follow the calculations themselves.

Thirteen MCID calculations methods were summarized, including anchor-based methods, distribution-based methods, Reliable Change Index, 30% Reduction from Baseline, Social Comparison Approach and the Delphi method. All methods, except the latter two, were used to calculate MCID for improvement of Zurich Claudication Questionnaire (ZCQ) Symptom Severity of patients with lumbar spinal stenosis. Numeric Rating Scale for Leg Pain and Japanese Orthopaedic Association Back Pain Evaluation Questionnaire Walking Ability domain were used as anchors.

The MCID for improvement of ZCQ Symptom Severity ranged from 0.8 to 5.1. On average, distribution-based methods yielded lower MCID values, than anchor-based methods. The percentage of patients who achieved the calculated MCID threshold ranged from 9.5% to 61.9%.

Conclusions

MCID calculations are encouraged in spinal research to evaluate treatment success. Anchor-based methods, relying on scales assessing patient preferences, continue to be the “gold-standard” with receiver operating characteristic curve approach being optimal. In their absence, the minimum detectable change approach is acceptable. The provided explanation and step-by-step example of MCID calculations with statistical code and publicly available data can act as guidance in planning future MCID calculation studies.

Similar content being viewed by others

Determining the clinical importance of treatment benefits for interventions for painful orthopedic conditions.

The anchor design of anchor-based method to determine the minimal clinically important difference: a systematic review

Values derived from patient reported outcomes in spine surgery: a systematic review of the minimal clinically important difference, substantial clinical benefit, and patient acceptable symptom state

Avoid common mistakes on your manuscript.

The notion of minimum clinically important difference (MCID) was introduced to establish thresholds of change in an outcome measurement instrument that are actually meaningful for the patient. Jaeschke et al . originally defined it “as the smallest difference in score in the domain of interest which the patient perceives as beneficial and which would mandate, in the absence of troublesome side-effects and excessive cost, a change in the patient’s management” [ 1 ].

In many clinical trials statistical analyses only focuses on intergroup comparisons of raw outcome scores using parametric/non-parametric tests and deriving conclusions based on the p -value. Using the classical threshold of p- value < 0.05 only suggests that the observed effect is unlikely to have occurred by chance, but it does not equate to a change that is clinically meaningful for the patient [ 2 ]. Calculating MCID scores, and using them as thresholds for “treatment success”, ensures that patients’ needs and preferences are considered and allows for comparison of proportion of patients experiencing a clinically relevant improvement among different groups [ 3 ]. Through MCID, clinicians can better understand the impact of an intervention on their patients’ lives, sample size calculations can become more robust and health policy makers may decide which treatments deserve reimbursement [ 4 , 5 , 6 ].

The MCID can be determined from the patient’s perspective, where it is the patient who decides whether a change in their health was meaningful [ 4 , 7 , 8 , 9 ]. This is the most common “gold-standard” approach and one that we will focus on. Occasionally, the clinician’s perspective can also be used to determine MCID. However, MCID for a clinician may not necessarily mean an increase in a patient’s functionality, but rather a change in disease survival or treatment planning [ 10 ]. MCID can also be defined at a societal level, as e.g. improvement in a patient’s functionality significant enough to aid their return to work [ 11 ].

MCID thresholds are intended to assess an individual’s clinical improvement and ought not to be applied to mean scores of entire groups post-intervention, as doing so may falsely over-estimate treatment effectiveness. It is also noteworthy to mention that obtained MCID values are not treatment-specific but broadly disease category-specific. They rely on a patient’s perception of clinical benefit, which is influenced by their diagnosis and subsequent symptoms, not just treatment modality.

In this study, we summarize available MCID calculation methods and outline key considerations when designing a MCID study, followed by a step-by-step working example of how MCID can be calculated.

Navigating the case study

To illustrate the MCID methods and to enable the reader to follow the practical calculation guide of different MCID values, based on the described methods along the way, a previously published data set of 84 patients, as described in Minetama et al ., was used based on CC0.10 license [ 12 ]. Data can be downloaded at https://data.mendeley.com/datasets/vm8rg6rvsw/1 . The statistical R code can be found in Supplementry content  1 including instructions on formatting the data set for MCID calculations The title of different MCID methods in the paper (listed below) and their number correspond to the same title and respective number in the R code. All analyses in this case study were carried out using R version 2023.12 + 402 (The R Foundation for Statistical Computing, Vienna Austria) [ 13 ].

The aim of Minetama et al . was to assess the effectiveness of supervised physical therapy (PT) with unsupervised at-home-exercises (HE) in patients with lumbar spinal stenosis (LSS). The main inclusion criteria were presence of neurogenic intermittent claudication and pain/or numbness in the lower extremities with or without back pain and > 50 years of age; diagnosis of LSS confirmed on MRI and a history of ineffective response to therapy for ≥ 3 months. Patients were then randomized into a 6-week PT or HE programme [ 12 ]. All data was pooled, as a clinically significant benefit for patients is independent of group allocation and because MCID is disease-specific. Therefore, the derived MCID will be applicable to most patients with lumbar spinal stenosis, irrespective of treatment modality. Change scores were calculated by subtracting baseline scores from follow-up scores.

MCID calculation methods

There are multiple approaches to calculate MCID, mainly divided into anchor-based and distribution-based methods (Fig.  1 ) [ 4 , 10 , 14 , 15 , 16 , 17 ]. Before deciding on the method, it needs to be defined whether the calculated MCID will be for improvement or deterioration [ 18 ]. Most commonly, MCID is used to measure improvement (as per Jaeschke et al . definition) [ 1 , 4 , 7 , 14 , 15 , 16 , 19 , 20 ]. The value of MCID for improvement should not be directly applied in reverse to determine whether a decrease in patients' scores signifies a clinically meaningful deterioration – those are two separate concepts [ 18 ]. In addition, the actual MCID value ought to be applied to post-intervention score of an individual patient (not the overall score for the whole group), to determine whether, at follow-up, he or she experienced a change equating to MCID or more, compared to their baseline score. Such patient is then classified as “responders”.

Flow diagram presenting range of Minimum clinically important difference calculation methods stratified into anchor, distribution-based and “other” described in the study. MCID, Minimum Clinically Important Difference; MIC, Minimal Important Change

According to the Consensus-based Standards for the selection of health measurement instruments (COSMIN) guidelines, the “anchor-based” approach is regarded as the “gold-standard” [ 21 , 22 , 23 ]. In this approach, we determine the MCID of a chosen outcome measurement, based on whether a pre-defined MCID (usually derived from another published study) was achieved by an external criterion, known as the anchor, usually another patient-reported outcome measure (PROM) or an objective test of functionality [ 4 , 7 , 8 , 15 , 16 , 17 , 18 , 20 ]. It is best to use scales which allow the patient to rate the specific aspect of their health related to the disease of interest post-intervention compared to baseline on a Likert-type scale. This scale may range, for example, from “much worse”, “somewhat worse”, “about the same”, “somewhat better”, to “much better”, such as the established Global Assessment Rating tool [ 7 , 8 , 24 , 25 ]. Depending on the scale, some studies determine MCID by calculating change scores for patients who only ranked themselves as “somewhat better”, and some only consider patients who ranked themselves as “much better” [ 7 , 25 , 26 , 27 , 28 , 29 ]. This discrepancy is likely an explanation for a range of MCID for a single outcome measure dependent on the methodology. There appears to be no singular “correct” approach. One of the alternatives to the Global assessment rating is the use of the health transition item (HTI) from the SF-36 questionnaire, where patients are asked about their overall health compared to one year ago [ 7 , 30 , 31 ]. Although quick and easy to conduct, the patient’s response may be influenced by comorbid health issues other than those targeted by intervention. Nevertheless, any anchor where the patient is the one to decide what change is clinically meaningful, captures the true essence of the MCID. One should however, be mindful of the not easily addressed recall bias with such anchors – patients at times do not reliably remember their baseline health status [ 32 ]. Moreover, what the above anchors do not consider is, whether the patient would still choose the intervention for the same condition despite experiencing side-effects or cost. That can be addressed through implementing anchors such as the Satisfaction with Results scale described in Copay et al ., who found that MCID values based on the Satisfaction with Results scale were slightly higher than those derived from HTI-SF-36 [ 7 , 33 ].

Other commonly used outcome scales, such as Oswestry Disability Index (ODI), Roland–Morris Disability Questionnaire (RMDQ), Visual Analogue Scale (VAS), or EQ5D-3L Health-Related Quality of Life, can also act as anchors [ 7 , 14 , 16 , 34 , 35 ]. In such instances, patients complete the “anchor” questionnaire at baseline and post-intervention and the MCID of that anchor is derived from a previous publication [ 12 , 16 , 35 ]. Before deciding on the MCID, full understanding of how it was derived in that previous publication is crucial. Ideally, this should be done for a population similar to our study cohort, with comparable follow-up periods [ 18 , 20 ]. Correlations between the anchor instrument and the investigated outcome measurement instrument must be recorded, and ought to be at least moderate (> 0.05), as that is the best indicator of construct validity (whether both the anchor instrument and outcome instrument represent a similar construct of patient health) [ 18 , 36 ]. If such correlation is not available, the anchor-based MCID credibility instrument is available to aid in assessing construct proximity between the two [ 36 , 37 ].

Once the process for selecting an anchor and classifying “responders” and “non-responders” is established, the MCID can be calculated. The outcome instrument of interest will be defined as an outcome for which we want to calculate the MCID. The first anchor-based method (within-patient change) focuses on the average improvement seen among clear responders in the anchor. The between-patient change anchor-based method additionally subtracts the average improvement seen among non-responders (unchanged and/or worsened) and consequently ends up with a smaller MCID value. Finally, an anchor-based method based on Receiver Operating Characteristic (ROC) curve analysis–that can be considered the current “gold standard”- also exists, which effectively looks at the MCID calculation as a sort of diagnostic instrument and aims to improve the discriminatory performance of our MCID threshold. In the following paragraphs, the three anchor-based methods are described in more detail. The R code (Supplementry Content  1 ) enables the reader to follow the text and to calculate MCID for the Zurich Claudication Questionnaire (ZCQ) Symptom Severity domain, based on a publicly available dataset [ 12 ].

Choice of outcome measurement instruments for MCID calculation case study

The chosen outcome measurement instrument in this case study for which MCID for improvement will be calculated is ZCQ Symptom Severity domain [ 12 ]. The ZCQ is composed of three subscales: symptom severity (7 questions, score per question ranging from 1 to 5 points); physical function (5 questions, score per question ranging from 1 to 4 points) and patient satisfaction with treatment scale (6 questions, score per question ranging from to 4 points). Higher scores indicate greater disability/worse satisfaction [ 38 ]. To visualize different MCID values, Numeric Rating Scale (NRS) for Leg Pain (score from 0 “no pain” to 10 “worse possible pain) and Japanese Orthopaedic Association Back Pain Evaluation Questionnaire (JOABPEQ) Walking Ability domain are chosen, as they showed high responsiveness in patients with LSS post-operatively [ 39 ].Through 25 questions, the JOABPEQ assesses five distinctive domains: pain-related symptoms, lumbar spine dysfunction, walking ability, impairment in social functioning and psychological disturbances. The score for each domain ranges from 0 to 100 points (higher score indicating better health status) [ 40 ]. The correlation of ZCQ symptom severity with NRS Leg Pain and JOABPEQ Walking Ability domain, is 0.56 and − 0.51, respectively [ 39 ]. For a patient to be classified as a “responder”, using the NRS for Leg pain or JOABPEQ walking ability, the score at 6-week follow-up must have improved by 1.6 points or 20 points, respectively [ 7 , 40 , 41 ].

This publicly available dataset does not report patient satisfaction or any kind of global assessment rating.

To enable calculation of global assessment rating-based MCID methods for educational purposes, despite very limited availability of studies providing MCID for deterioration of JOABPEQ, we decided to stratify patients in this dataset into the three following groups, based on the JOABPEQ Walking Ability as an anchor: likely improved (change score above 20 points according to Kasai et al . ), no significant change (− 20– + 20 points change score), and likely deteriorated (lower than − 20 points change score) [ 41 ]. As obtained MCID values were expected to be negative, all values, for clarity of presentation, were multiplied by − 1, except in Method (IX), where graphical data distribution was shown.

The different methods in detail

Method (i) calculating mcid using “within-patient” score change.

The first method focuses on calculating the change between baseline and post-intervention score of our outcome instrument, for each patient classified as a “responder”. A “responder” is a patient who, at follow-up, has achieved the pre-defined MCID of the anchor (or ranks themselves high enough on Global assessment rating type scale based on our methodology). The MCID is then defined as the mean change in the outcome instrument of interest of those classified as “responders” [ 4 , 7 , 16 , 31 ].

The corresponding R-Code formula is described in Step 5a of Supplementry Content  1 . Calculated within-patient MCID of ZCQ Symptom Severity based on NRS Leg Pain and JOABPEQ Walking Ability domain was 4.4 and 4.2, respectively.

Method (II) calculating MCID using “between-patient” score change

In this approach, the mean change in our outcome instrument is calculated for not only “responders” but also for “non-responders”. “Non-responders” are patients who did not achieve the pre-defined MCID of our anchor or who did not rank themselves high enough (unchanged, or sometimes: unchanged + worsened) on Global Assessment Rating type scale according to our methodology. The minimum clinically important difference of our outcome instrument is then defined as the difference between the mean change scores of “responders” and “non-responders” [ 4 , 7 , 16 , 19 ].

The corresponding R-Code formula is described in Step 5b of Supplementry content  1 . Calculated between-patient MCID of ZCQ Symptom Severity based on NRS Leg Pain and JOABPEQ Walking Ability domain was 3.5 and 2.8, respectively.

Method (III) calculating MCID using the ROC analysis

Here the MCID is derived through ROC analysis to identify the “threshold” score of our outcome instrument that best discriminates between “responders” and “non-responders” of the anchor [ 4 , 7 , 16 , 19 , 27 ]. To understand ROC, one must familiarize oneself with the concept of sensitivity and specificity. In ROC analysis, sensitivity is defined as the ability of the test to correctly detect “true positives”, which in this context refers to patients who have achieved a clinically meaningful change.

“False negative” would be a patient, who was classified as “non-responder” but is really a “responder”. Specificity is defined as the ability of a test to correctly detect a “true negative” result- a patient who did not achieve a clinically meaningful change – a “non-responder” [ 25 ].

A “false positive” would be a patient, who was classified as a “responder” but who was a “non-responder”. Values for sensitivity and specificity range from 0 to 1. Sensitivity of 1 means that the test can detect 100% of “true positives”’ (“responders”), while specificity of 1 reflects the ability to detect 100% of “true negatives” (“non-responders”). It is unclear what the minimum sensitivity and specificity should be for a “gold-standard” MCID, which is why the most established approach is to opt for a MCID threshold that maximizes both sensitivity and specificity at the same time, which can be done using ROC analysis [ 4 , 7 , 25 , 31 , 42 ]. During ROC analysis, the “closest-to-(0,1)-criterion” (the top left most point of the curve) or the Youden index are the two methods to automatically determine the optimal threshold point [ 43 ].

When conducting the ROC analysis, the Area under the curve (AUC) is also determined–a measure of how well the MCID threshold discriminates responders and non-responders in general. Values in AUC can range 0–1. An AUC of 0.5 signifies that the score discriminates no better than random chance, whereas a value of 1 means that the score perfectly discriminates between responders and non-responders. In the literature, an AUC of 0.7 and 0.8 is deemed fair (acceptable), while ≥ 0.8 to < 0.9 is considered good and values ≥ 0.9 are considered excellent [ 44 ]. Calculating the AUC provides a rough estimate of how well the chosen MCID threshold performs. The corresponding R-Code formula is described in Step 5c of Supplementry content  1 . Statistical package pROC was used. The calculated MCID of ZCQ symptom severity based on NRS Leg Pain and JOABPEQ Walking Ability domain was for both 1.5.

Calculation of MCID through distribution-based methods

Calculation of MCID using the distribution-based approach focuses on statistical properties of the dataset [ 7 , 14 , 16 , 27 , 45 ]. Those methods are objective, easy to calculate, and in some cases, yield values close to anchor-based MCID. The advantage of this approach is that it does not rely on any external criterion or require additional studies on previously established MCIDs or other validated “gold standard” questionnaires for the specific disease in each clinical setting. However, it fails to include the patient’s perspective of a clinically meaningful change, which will be discussed later in this study. In this sense, distribution-based methods focus on finding MCID thresholds that enable mathematical distinction of what is considered a changed vs. unchanged score, whereas anchor-based methods focus on finding MCID thresholds which represent a patient-centered, meaningful improvement.

Method (IV) calculating MCID through Standard Error of Measurement (SEM)

The standard error of measurement conceptualizes the reliability of the outcome measure, by determining how repeated measurements of an outcome may differ from the “true score”. Greater SEM equates to lower reliability, which is suggestive of meaningful inconsistencies in the values produced by the outcome instrument despite similar measuring conditions. Hence, it has been theorized that 1 SEM is equal to MCID, because a change score ≥ 1 SEM, is unlikely to be due to measurement error and therefore is also more likely to be clinically meaningful [ 46 , 47 ]. The following formula is used: [ 1 , 7 , 35 , 46 , 48 ].

The ICC, also called reliability coefficient, signifies level of agreement or consistency between measurements taken on different occasions or by different raters [ 49 ]. There are various ways of calculating the ICC depending on the used model with values < 0.5, 0.5– 0.75, 0.75–0.9 and > 0.90 indicating poor, moderate, good and excellent reliability, respectively [ 49 ]. While a value of 1 × SEM is probably the most established way to calculate MCID, in the literature, a range of multiplication factors for SEM-based MCID have been used, including 1.96 SEM or even 2.77 SEM to identify a more specific threshold for improvement [ 48 , 50 ]. The corresponding R-Code formula is described in Step 6a of Supplementry Content  1 . The chosen ZCQ Symptom Severity ICC was 0.81 [ 51 ]. The SEM-based MCID was 1.9.

Method (V) calculating MCID through Effect Size (ES)

Effect size (ES) is a standardized measure of the strength of the relationship or difference between two variables [ 52 ]. It is described by Cohen et al . as “degree to which the null hypothesis (there is no difference between the two groups) is false”. It allows for direct comparison of different instruments with different units between studies. There are multiple forms to calculate ES, but for the purpose of MCID calculations, the ES represents the number of SDs by which the post-intervention score has changed from baseline score. It is calculated based on the following formula incorporating the average change score divided by the SD of the baseline score: [ 52 ].

According to Cohen et al . 0.2 is considered small ES, 0.5 is moderate ES and 0.8 or more is large ES [ 53 ]. Most commonly, a change score with an ES of 0.2 is considered equivalent to MCID [ 7 , 16 , 31 , 54 , 55 , 56 ]. Using this method, we are basically identifying the mean change score (in this case reflecting the MCID) that equates to an ES of 0.2: [ 7 , 55 ].

Practically, if a patient experienced small improvement in an outcome measure post intervention, the ES will be smaller than for a patient who experienced a large improvement in outcomes measure. The corresponding R-Code formula is described in Step 6b of Supplementry Content  1 . The ES-based MCID was 0.9.

Method (VI) calculating MCID through Standardized Response Mean (SRM)

The Standardized Response Mean (SRM) aims to gauge the responsiveness of an outcome similarly to ES. Initially described by Cohen et al . as a derivative of ES assessing differences of paired observations in a single sample, later renamed as SRM, it is also considered an “index of responsiveness” [ 38 , 53 ]. However, the denominator is SD of the change scores–not the SD of the baseline scores–while the numerator remains the average change score from baseline to follow-up: [ 10 , 45 , 57 , 58 , 59 ].

Similarly, to Cohen’s rule of interpreting ES, it has been theorized that responsiveness can be considered low if SRM is 0.2–0.5, moderate if > 0.5–0.8 and large if > 0.8 [ 58 , 59 , 60 ]. Again, a change score equating to SRM of 0.2 (although SRM of 1/3 or 0.5 were also proposed) can be considered MCID, although studies have used the overall SRM as MCID as well [ 45 , 54 , 56 , 61 ]. However, since SRM is a standardized index, similarly to ES, the aim of the SRM-based method ought to be to identify a change score that indicates responsiveness of 0.2: [ 61 ].

Similar to the ES-based method, the SRM-based approach for calculating the MCID is not commonly used in in spine surgery studies [ 14 ]. It is a measure of responsiveness, which is the ability to detect change over time in a construct to be measured by the instrument, and ought to be therefore calculated for the study-specific change score rather than extrapolated as a “universal” MCID threshold to other studies. The corresponding R-Code formula is described in Step 6c of Supplementry Content  1 . The SRM-based MCID was 0.8.

The limitation of using Method (V) and (VI) in MCID calculations will be later described in Discussion.

Method (VII) calculating MCID through SD

Standard Deviation represents the average spread of individual data points around the mean value of the outcome measure. Norman et al . found in their review of studies using MCID in health-related quality of life instruments that most studies had an average ES of 0.5, which equated to clinically meaningful change score of 0.5 × SD of baseline score [ 7 , 16 ,  30 ].

The corresponding R-Code formula is described in Step 6d of Supplementry content  1 . The SD-based MCID was 2.1.

Method (VIII) calculating MCID through 95% Minimum Detectable Change (MDC)

The MDC is defined as the minimal change below which there is a 95% chance that it is due to measurement error of the outcome measurement instrument: [ 7 , 61 ].

Usually, value corresponding to z is the desired level of confidence, which for 95% confidence level is 1.96. Although MDC–like all distribution-based methods–does not consider whether a change is clinically meaningful, the calculated MCID should be at least the same or greater than MDC to enable distinguishing true mathematical change from measurement noise. The 95% MDC calculation, is the most common distribution-based approach in spinal surgery, and it appears to most closely resemble anchor-derived MCID values, as demonstrated by Copay et al . [ 7 , 14 , 62 ]. The corresponding R-Code formula is described in Step 6e of Supplementry Content  1 . The 95% MDC was 5.1.

Method (IX) calculating MCID through Reliable Change Index

Another less frequently applied method through which “responders and “non-responders” can be classified but which does not rely on an external criterion is the Reliable Change Index (RCI), also called the Jacobson–Truax index [ 63 , 64 ]. It indicates whether an individual change score is statistically significantly greater than a change in score that could have occurred due to random measurement error alone [ 63 ].

In theory, a patient can be considered to experience a statistically reliably identifiable improvement ( p  < 0.05), if the individual RCI is > 1.96. Again, it does not reflect whether the change is clinically meaningful for the patient but rather that the change should not be attributed to measurement error alone and likely has a component of true score change. Therefore, this method is discouraged in MCID calculations as it relies on statistical properties of the sample and not patient preferences–as all distribution-based methods do [ 65 ]. In the example of Bolton et al . who focused on the Bournemouth Questionnaire in patients with neck pain, RCI was subsequently used to discriminate between “responders” and “non-responders”. The ROC analysis approach was then used to determine the MCID [ 64 ]. The corresponding R-Code formula is described in Step 6f of Supplementry Content  1 . Again, pROC package was used. The ROC-derived MCID was 2.5.

Other methods

Method (x) calculating mcid through anchor-based minimal important change (mic) distribution model.

In theory, combining anchor- and distribution-based methods could yield superior results. Some suggestions include averaging the values of various methods, simply combining two different methods (i.e. both an anchor-based criterion such as ROC-based MCID from patient satisfaction and 95% MDC-based MCID have to both be met to consider a patient as having achieved MCID) [ 25 ]. In 2007, de Vet et al . introduced a new visual method of MCID calculations that does not only combine but also integrates both anchor- and distribution-based calculations [ 25 ]. In addition, their method allows the calculation of both MCID for improvement and for deterioration, as these can differ.

In short form, using an anchor, patients were divided into three “importantly improved”, “not importantly changed” and “importantly deteriorated” groups (Fig.  2 ) . Then distribution expressed in percentiles of patients who “importantly improved”, “importantly deteriorated” and “not importantly changed” were plotted on a graph. This is the anchor-based part of the approach, ensuring that MCID thresholds chosen have clinical value.

Distribution of the Zurich Claudication Questionnaire Symptom Severity change scores for patients categorized as experiencing “important improvement”, “no important change” or “important deterioration” in JOABPEQ walking ability as an anchor (Method (X)). For ZCQ Symptom Severity score to improve, the actual value must decrease explaining the negative values in the model. ROC , Receiver Operating Characteristic; ZCQ , Zurich Claudication Questionnaire; JOABPEQ , Japanese Orthopaedic Association Back Pain Evaluation Questionnaire

The second part of the approach is then entirely focused on the group of patients determined by the anchor to be “unchanged”, and can be either distribution- or anchor-based:

In the first and more anchor-based method, the ROC-based method described in Method (III) is applied to find the threshold for improvement (by finding the ROC-based threshold point that optimizes sensitivity and specificity of identifying improved vs unchanged patients) or for deterioration (by finding the ROC-based threshold point that optimizes sensitivity and specificity of identifying deteriorated vs unchanged patients). For example, the threshold for improvement is found by combining the improved and unchanged groups, and then testing out different thresholds for discriminating those two groups from each other. The optimal point on the resulting ROC curve based on the closest-to-(0,1)-criterion is then found.

In the second method, which is distribution-based, the upper 95% (for improvement) and lower 95% (for deterioration) limits are found based solely on the group of patients determined to be unchanged. The following formula is used (instead, subtracting instead of adding the 1.645 × SD for deterioration or improvement, respectively): [ 25 ]

The corresponding R-Code formula can be found under Step 7a in Supplementry Content  1 . The model is presented in Fig.  2 . The 95% upper limit and 95% lower limit was 4.1 and − 7.2 respectively. The ROC-derived MCID using RCI was − 2.5 (important improvement vs unchanged) and − 0.5 (important deterioration vs unchanged). For the purpose of the model, MCID values were not multiplied by − 1 but remained in original form.

Method (XI) calculating MCID as 30% Reduction from Baseline

In recent years, a simple 30% reduction from baseline values has been introduced as an alternative to MCID calculations [ 66 ]. It has been speculated that absolute-point changes are difficult to interpret and have limited value in context of “ceiling” and “floor” effects (i.e. values that are on the extreme spectra of the measurement scale) [ 4 ]. To overcome this, Khan et al . found that 30% reduction in PROMs has similar effectiveness as traditional anchored or distribution-based methods in detecting patients with clinically meaningful differences post lumbar spine surgery [ 15 ]. The corresponding R-Code formula can be found under Step 7b in Supplementry Content  1 .

Method (XII) Calculating MCID through Delphi method

The Delphi Method is a systemic approach using the collective opinion of experts to establish a consensus regarding a medical issue [ 67 ]. It has mostly been used to develop best practice guidelines [ 68 ]. However, it can also be used to aid MCID determination [ 69 ]. The method focuses on distributing questionnaires or surveys to panel of members. The anonymized answers are grouped together and shared again with the expert panel in subsequent rounds. This allows the experts to reflect on their opinions and consider strengths and weaknesses of the others response. The process is repeated until consensus is reached. Ensuring anonymity, this prevents any potential bias linked to a specific participant’s concern about their own opinion being viewed or influenced by other personal factors [ 67 ].

Method (XIII) calculating MCID through Social Comparison Approach

The final approach is asking patients to compare themselves to other patients, which requires time and resources [ 70 ]. In a study by Redelmeier et al . patients with chronic obstructive pulmonary disease in a rehabilitation program were organized into small groups and observed each other at multiple occasions [ 70 ]. Additionally, each patient was paired with another participant and had a one-to-one interview with them discussing different aspects of their health. Finally, each patient anonymously rated themselves against their partner on a scale “much better”, “somewhat better”, “a little bit better”, “about the same”, “a little bit worse” “somewhat worse” and “much worse”. MCID was then calculated based on the mean change score of patients who graded themselves as “a little bit better” (MCID for improvement) or a “little bit worse” (MCID for deterioration), like in the within-patient change and between-patient change method described in Method (I) and (II) [ 70 ].

Substantial Clinical Benefit

Over the years, it has been noted that MCID calculations based either purely on distribution-based method or only group of patients rating themselves as “somewhat better” or “slightly better” does not necessarily constitute a change that patients would consider beneficial enough “to mandate, in the absence of troublesome side effects and excessive cost, to undergo the treatment again” [ 3 , 24 ]. Therefore, the concept of substantial clinical benefit (SCB) has been introduced as a way of identifying a threshold of clinical success of intervention rather than a “floor” value for improvement- that is MCID [ 24 ]. For example, in Carreon et al ., ROC derived SCB “thresholds” were defined as a change score with equal sensitivity and specificity to distinguish “much better” from “somewhat better” patients post cervical spinal fusion [ 71 ]. Glassman et al . on the other hand used ROC derived SCB thresholds to discriminate between “much better” and “about the same” patients following lumbar spinal fusion. The authors stress that SCB and MCID are indeed separate entities, and one should not be used to derive the other [ 24 ]. Thus, while the methods to derive SCB and MCID thresholds can be carried out similarly based on anchors, the ultimate goal of applying SCB versus MCID is different.

Using the various methods explained above, overall, MCID for improvement for ZCQ Symptoms Severity domain ranged from 0.8 to 5.1 (Table  1 ). Here, the readers obtained results can be checked for correctness. On average distribution-based MCID values were lower than anchor-based MCID values. Within distribution-based approach, method (VIII) “Minimum detectable change” resulted in MCID of 5.1, which exceeded the MCID’s derived using the “gold-standard” anchor-based approaches. The average MCID based on anchor of NRS Leg pain and JOABPEQ walking ability was 3.1 and 2.8, respectively. Dependent on methods used, percentage of responders to HE and PT intervention fell within range of 9.5% for “30% Reduction from Baseline” method to 61.9% using ES- and SRM-based method (Table  2 ). Method (X) is graphically presented in Fig.  2 .

As demonstrated above, the MCID is dependent upon the methodology and the chosen anchor, highlighting the necessity for careful preparation in MCID calculations. The lowest MCID of 0.8 was calculated for Method (VI) being SRM. Logically, if a patient on average had a baseline ZCQ Symptom Severity score of 23.2, an improvement of 0.8 is unlikely to be clinically meaningful, even if rounded up. It rather informs on the measurement error property of our instrument as explained by COSMIN. Additionally, the distribution-based methods rely on statistical properties of the sample, which varies from cohort to cohort making it only generalizable to patient groups with similar SD but not applicable to others with a different spread of data [ 52 ]. Not surprisingly, anchor-based methods considering patient preferences yielded on average higher MCID values than distribution-based methods, which again varied from anchor to anchor. The mean MCID for improvement calculated for NPRS Leg Pain was 3.1, while for JOABPEQ Walking Ability it was 2.8—such similar values prove the importance of selecting responsive anchors with at least moderate correlations. Despite assessing different aspects of LSS disease, the MCID remained comparable in this specific case.

Interestingly, Method (VIII) MDC yielded the highest value of 5.1, exceeding the “gold-standard” ROC-derived MCID. This suggests that, in this example, using this ROC-derived MCID in clinical practice would be illogical, as the value falls within the measurement error determined by MDC. Here it would be appropriate to choose MDC approach as the MCID. Interestingly, ROC-derived MCID values based on Global Assessment Rating like stratification of patients based on their JOABPEQ Walking Ability (Method X) yielded higher MCID, than in Method (III). This may be attributed to a more a balanced distribution of “responders” and “non-responders” (only unchanged patients) in Method (X), unlike in the latter (Method III) where patients were strictly categorized into “responders” and “non-responders” (including both deteriorated and unchanged). This further highlights the importance of using global assessment rating type scales in determining the extent of clinical benefit.

Although ES-based (Method (V)) and SRM-based (Method (VI)) MCID calculations have been described in the literature, ES and SRM were originally created to quantify the strength of relationship between scores of two samples (in case of ES) and change score of paired observations in one sample (in case of SRM) [ 53 , 58 , 59 ]. They do offer an alternative to MCID calculations. However, verification with other MCID calculation methods, ideally anchor-based, is strongly recommended. As seen in this case study and other MCID’s derived similarly, they often result small estimates [ 7 , 55 ]. There is also no consensus regarding the choice of SD of Change Score vs. SD of Baseline Score as denominator. Additionally, whether the calculated MCID (mean change score) should represent value, such as the ES is 0.2 indicating small effect, or value should be 0.5 suggesting moderate effect is currently arbitrary and often relies on the researcher’s preference [ 53 , 55 , 59 ]. Both ES and SRM can be used to assess whether the overall change score observed in single study is suggestive of a clinically meaningful benefit in that specific cohort or in case of SRM, whether the outcome measure is responsive. However, it is our perspective that extending such value as “MCID” from one study to another is not recommended.

One can argue whether there is even a place for distribution-based methods in MCID calculations. They ultimately fail to provide an MCID value that meets the original definition of Jaeschke et al . “of smallest change in the outcome that the patient would identify as important”. At no point are patients asked about what constitutes a meaningful change for them, and the value is derived from statistical properties of the sample solely [ 1 ]. Nevertheless, conduction of studies on MCID implementing scales such as Global Assessment Rating is time-consuming and performing studies for each patient outcome and each disease is likely not feasible. Distribution-based methods still have some merit in that they–like the 95% MDC method—can help distinguish measurement noise and inaccuracy from true change. Even if anchor-based methods should probably be used to define MCID thresholds, they ought to be supported by a calculation of MDC so that it can be decided whether the chosen threshold makes sense mathematically (i.e., can reliably be distinguished from measurement inaccuracies) as seen in our case study.

Calculating MCID for different diagnoses

Previously, MCID thresholds for outcome measurement instruments were calculated for generic populations, such as patients suffering from low back pain. More recently, MCID values for commonly used PROMs in spine surgery, such as ODI, RMDQ or NRS have been calculated for more narrowly defined diagnoses, such as lumbar disc herniation (LDH) or LSS. The question arises as to whether a separate MCID is needed for all the different spinal conditions. In general, establishing an MCID specific to these patient groups is only recommended if these patient’s perception of meaningful change is different from that of low back pain in general. Importantly, again, the MCID should not be treatment-specific, but rather broadly disease specific. Therefore, it is advisable to use MCID based on patients who had the most similar disease characteristics to our cohort. For example, an MCID for NRS Back Pain based on study group composed of different types of lumbar degenerative disease, may in some cases, be applied to study cohort composed solely of patients with LDH. However, no such extrapolation should be performed for populations with back pain secondary to malignancy, due to a totally different pathogenesis and associated symptoms that may influence the ability to detect a clinically meaningful change in the above NRS Back Pain such as fatigue or anorexia.

Study cohort characteristics that influence MCID

Regardless of robust methodology, it can be expected that it is impossible to obtain the same MCID on different occasions even in the same population due to the inherent subjectivity of what is perceived as “clinically beneficial” and day-to-day symptom fluctuation. However, it was found that patients who have worse baseline scores, reflecting e.g., more advanced disease, require greater overall change at follow-up to report it as clinically meaningful [ 72 ]. One should also be mindful of “regression to the mean” where extremely high or low-scoring patients then subsequently score closer to baseline at second measurement [ 73 ]. Therefore, adequate cohort characteristics need to be presented, for the readers to judge how generalizable the MCID may be to their study cohort. If a patient pre-operatively experiences NRS Leg Pain of 1, and the MCID is 1.6, they cannot achieve MCID at all, as the maximum possible change score is smaller than the MCID threshold (“floor effect”). A similar situation can occur with patients closer to the higher end of the scale (“ceiling effect”). The general rule is, that if at least 15% of the study cohort has the highest or lowest possible score for a given outcome instrument, one can expect significant “ceiling/floor effects” [ 50 ]. One way to overcome this, is through transferring absolute MCID scores to percentage change scores [ 4 , 45 ]. However, percentage change scores only account for high baseline scores, if high baseline scores indicate larger disability (as seen with ODI) and have a possibility of larger change. If a high score in an instruments reflects better health status (as seen in in SF-36), than percentage change scores will increase the association with baseline score [ 4 ]. In general, it is important to consider which patient to exclude from certain analyses when applying MCID: For example, patients without relevant disease preoperatively (for example, those exhibiting so-called “patient-accepted symptom states”, PASS) should probably be excluded altogether when reporting the percentage of patients achieving MCID [ 74 ].

Establishing reliable thresholds for MCID is key in clinical research and forms the basis of patient-centered treatment evaluations when using patient-reported outcome measures or objective functional tests. Calculation of MCID thresholds can be achieved using a variety of different methods, each yielding completely different results, as is demonstrated in this practical guide. Generally, anchor-based methods relying on scales assessing patient preferences/satisfaction or global assessment ratings continue to be the “gold-standard” approach- the most common being ROC analysis. In the absence of appropriate anchors, the distribution-based MCID based on the 95% MDC approach is acceptable, as it appears to yield the most similar results compared to anchor-based approaches. Moreover, we recommend using it as a supplement to any anchor-based MCID thresholds to check if they can reliably distinguish true change from measurement inaccuracies. The explanation provided in this practical guide with step-by-step examples along with public data and statistical code can add as guidance for future studies calculating MCID thresholds.

Jaeschke R, Singer J, Guyatt GH (1989) Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials 10:407–415. https://doi.org/10.1016/0197-2456(89)90005-6

Article   CAS   PubMed   Google Scholar  

Concato J, Hartigan JA (2016) P values: from suggestion to superstition. J Investig Med 64:1166. https://doi.org/10.1136/jim-2016-000206

Article   PubMed   PubMed Central   Google Scholar  

Zannikos S, Lee L, Smith HE (2014) Minimum clinically important difference and substantial clinical benefit: Does one size fit all diagnoses and patients? Semin Spine Surg 26:8–11. https://doi.org/10.1053/j.semss.2013.07.004

Article   Google Scholar  

Copay AG, Subach BR, Glassman SD et al (2007) Understanding the minimum clinically important difference: a review of concepts and methods. Spine J 7:541–546. https://doi.org/10.1016/j.spinee.2007.01.008

Article   PubMed   Google Scholar  

Lanario J, Hyland M, Menzies-Gow A et al (2020) Is the minimally clinically important difference (MCID) fit for purpose? a planned study using the SAQ. Euro Respirat J. https://doi.org/10.1183/13993003.congress-2020.2241

Neely JG, Karni RJ, Engel SH, Fraley PL, Nussenbaum B, Paniello RC (2007) Practical guides to understanding sample size and minimal clinically important difference (MCID). Otolaryngol Head Neck Surg 136(1):14–18. https://doi.org/10.1016/j.otohns.2006.11.001

Copay AG, Glassman SD, Subach BR et al (2008) Minimum clinically important difference in lumbar spine surgery patients: a choice of methods using the Oswestry disability index, medical outcomes study questionnaire short form 36, and pain scales. Spine J 8:968–974. https://doi.org/10.1016/j.spinee.2007.11.006

Andersson EI, Lin CC, Smeets RJ (2010) Performance tests in people with chronic low back pain: responsiveness and minimal clinically important change. Spine 35(26):E1559-1563. https://doi.org/10.1097/BRS.0b013e3181cea12e

Mannion AF, Porchet F, Kleinstück FS, Lattig F, Jeszenszky D, Bartanusz V, Dvorak J, Grob D (2009) The quality of spine surgery from the patient’s perspective. Part 1: the core outcome measures index in clinical practice. Euro Spine J 18:367–373. https://doi.org/10.1007/s00586-009-0942-8

Crosby RD, Kolotkin RL, Williams GR (2003) Defining clinically meaningful change in health-related quality of life. J Clin Epidemiol 56:395–407. https://doi.org/10.1016/S0895-4356(03)00044-1

Gatchel RJ, Mayer TG (2010) Testing minimal clinically important difference: consensus or conundrum? Spine J 10:321–327. https://doi.org/10.1016/j.spinee.2009.10.015

Minetama M, Kawakami M, Teraguchi M et al (2019) Supervised physical therapy vs. home exercise for patients with lumbar spinal stenosis: a randomized controlled trial. Spine J 19:1310–1318. https://doi.org/10.1016/j.spinee.2019.04.009

R Core Team (2021) R A Language and Environment for Statistical Computing

Chung AS, Copay AG, Olmscheid N, Campbell D, Walker JB, Chutkan N (2017) Minimum clinically important difference: current trends in the spine literature. Spine 42(14):1096–1105. https://doi.org/10.1097/BRS.0000000000001990

Khan I, Pennings JS, Devin CJ, Asher AM, Oleisky ER, Bydon M, Asher AL, Archer KR (2021) Clinically meaningful improvement following cervical spine surgery: 30% reduction versus absolute point-change MCID values. Spine 46(11):717–725. https://doi.org/10.1097/BRS.0000000000003887

Gautschi OP, Stienen MN, Corniola MV et al (2016) Assessment of the minimum clinically important difference in the timed up and go test after surgery for lumbar degenerative disc disease. Neurosurgery. https://doi.org/10.1227/NEU.0000000000001320

Kulkarni AV (2006) Distribution-based and anchor-based approaches provided different interpretability estimates for the hydrocephalus outcome questionnaire. J Clin Epidemiol 59:176–184. https://doi.org/10.1016/j.jclinepi.2005.07.011

Wang Y, Devji T, Qasim A et al (2022) A systematic survey identified methodological issues in studies estimating anchor-based minimal important differences in patient-reported outcomes. J Clin Epidemiol 142:144–151. https://doi.org/10.1016/j.jclinepi.2021.10.028

Parker SL, Godil SS, Shau DN et al (2013) Assessment of the minimum clinically important difference in pain, disability, and quality of life after anterior cervical discectomy and fusion: clinical article. J Neurosurg Spine 18:154–160. https://doi.org/10.3171/2012.10.SPINE12312

Carrasco-Labra A, Devji T, Qasim A et al (2021) Minimal important difference estimates for patient-reported outcomes: a systematic survey. J Clin Epidemiol 133:61–71. https://doi.org/10.1016/j.jclinepi.2020.11.024

Prinsen CAC, Mokkink LB, Bouter LM et al (2018) COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res 27:1147–1157. https://doi.org/10.1007/s11136-018-1798-3

Article   CAS   PubMed   PubMed Central   Google Scholar  

Mokkink LB, de Vet HCW, Prinsen CAC et al (2018) COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures. Qual Life Res 27:1171–1179. https://doi.org/10.1007/s11136-017-1765-4

Terwee CB, Prinsen CAC, Chiarotto A et al (2018) COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res 27:1159–1170. https://doi.org/10.1007/s11136-018-1829-0

Glassman SD, Copay AG, Berven SH et al (2008) Defining substantial clinical benefit following lumbar spine arthrodesis. J Bone Joint Surg Am 90:1839–1847. https://doi.org/10.2106/JBJS.G.01095

de Vet HCW, Ostelo RWJG, Terwee CB et al (2007) Minimally important change determined by a visual method integrating an anchor-based and a distribution-based approach. Qual Life Res 16:131–142. https://doi.org/10.1007/s11136-006-9109-9

Solberg T, Johnsen LG, Nygaard ØP, Grotle M (2013) Can we define success criteria for lumbar disc surgery? Acta Orthop 84:196–201. https://doi.org/10.3109/17453674.2013.786634

Power JD, Perruccio AV, Canizares M et al (2023) Determining minimal clinically important difference estimates following surgery for degenerative conditions of the lumbar spine: analysis of the Canadian spine outcomes and research network (CSORN) registry. The Spine Journal 23:1323–1333. https://doi.org/10.1016/j.spinee.2023.05.001

Asher AL, Kerezoudis P, Mummaneni PV et al (2018) Defining the minimum clinically important difference for grade I degenerative lumbar spondylolisthesis: insights from the quality outcomes database. Neurosurg Focus 44:E2. https://doi.org/10.3171/2017.10.FOCUS17554

Cleland JA, Whitman JM, Houser JL et al (2012) Psychometric properties of selected tests in patients with lumbar spinal stenosis. Spine J 12:921–931. https://doi.org/10.1016/j.spinee.2012.05.004

Norman GR, Sloan JA, Wyrwich KW (2003) Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care 41:582–592. https://doi.org/10.1097/01.MLR.0000062554.74615.4C

Parker SL, Mendenhall SK, Shau DN et al (2012) Minimum clinically important difference in pain, disability, and quality of life after neural decompression and fusion for same-level recurrent lumbar stenosis: understanding clinical versus statistical significance. J Neurosurg Spine 16:471–478. https://doi.org/10.3171/2012.1.SPINE11842

Gatchel RJ, Mayer TG, Chou R (2012) What does/should the minimum clinically important difference measure?: a reconsideration of its clinical value in evaluating efficacy of lumbar fusion surgery. Clin J Pain 28:387. https://doi.org/10.1097/AJP.0b013e3182327f20

Lloyd H, Jenkinson C, Hadi M et al (2014) Patient reports of the outcomes of treatment: a structured review of approaches. Health Qual Life Outcomes 12:5. https://doi.org/10.1186/1477-7525-12-5

Beighley A, Zhang A, Huang B et al (2022) Patient-reported outcome measures in spine surgery: a systematic review. J Craniovertebr Junction Spine 13:378–389. https://doi.org/10.4103/jcvjs.jcvjs_101_22

Ogura Y, Ogura K, Kobayashi Y et al (2020) Minimum clinically important difference of major patient-reported outcome measures in patients undergoing decompression surgery for lumbar spinal stenosis. Clin Neurol Neurosurg 196:105966. https://doi.org/10.1016/j.clineuro.2020.105966

Wang Y, Devji T, Carrasco-Labra A et al (2023) An extension minimal important difference credibility item addressing construct proximity is a reliable alternative to the correlation item. J Clin Epidemiol 157:46–52. https://doi.org/10.1016/j.jclinepi.2023.03.001

Devji T, Carrasco-Labra A, Qasim A et al (2020) Evaluating the credibility of anchor based estimates of minimal important differences for patient reported outcomes: instrument development and reliability study. BMJ 369:m1714. https://doi.org/10.1136/bmj.m1714

Stucki G, Daltroy L, Liang MH et al (1996) Measurement properties of a self-administered outcome measure in lumbar spinal stenosis. Spine 21:796

Fujimori T, Ikegami D, Sugiura T, Sakaura H (2022) Responsiveness of the Zurich claudication questionnaire, the Oswestry disability index, the Japanese orthopaedic association back pain evaluation questionnaire, the 8-item short form health survey, and the Euroqol 5 dimensions 5 level in the assessment of patients with lumbar spinal stenosis. Eur Spine J 31:1399–1412. https://doi.org/10.1007/s00586-022-07236-5

Fukui M, Chiba K, Kawakami M et al (2009) JOA back pain evaluation questionnaire (JOABPEQ)/ JOA cervical myelopathy evaluation questionnaire (JOACMEQ) the report on the development of revised versions April 16, 2007: the subcommittee of the clinical outcome committee of the Japanese orthopaedic association on low back pain and cervical myelopathy evaluation. J Orthop Sci 14:348–365. https://doi.org/10.1007/s00776-009-1337-8

Kasai Y, Fukui M, Takahashi K et al (2017) Verification of the sensitivity of functional scores for treatment results–substantial clinical benefit thresholds for the Japanese orthopaedic association back pain evaluation questionnaire (JOABPEQ). J Orthop Sci 22:665–669. https://doi.org/10.1016/j.jos.2017.02.012

Glassman SD, Carreon LY, Anderson PA, Resnick DK (2011) A diagnostic classification for lumbar spine registry development. Spine J 11:1108–1116. https://doi.org/10.1016/j.spinee.2011.11.016

Perkins NJ, Schisterman EF (2006) The inconsistency of “optimal” cutpoints obtained using two criteria based on the receiver operating characteristic curve. Am J Epidemiol 163:670–675. https://doi.org/10.1093/aje/kwj063

Nahm FS (2022) Receiver operating characteristic curve: overview and practical use for clinicians. Korean J Anesthesiol 75:25–36. https://doi.org/10.4097/kja.21209

Angst F, Aeschlimann A, Angst J (2017) The minimal clinically important difference raised the significance of outcome effects above the statistical level, with methodological implications for future studies. J Clin Epidemiol 82:128–136. https://doi.org/10.1016/j.jclinepi.2016.11.016

Wyrwich KW, Tierney WM, Wolinsky FD (1999) Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life. J Clin Epidemiol 52:861–873. https://doi.org/10.1016/s0895-4356(99)00071-2

Wolinsky FD, Wan GJ, Tierney WM (1998) Changes in the SF-36 in 12 months in a clinical sample of disadvantaged older adults. Med Care 36:1589–1598

Wyrwich KW, Nienaber NA, Tierney WM, Wolinsky FD (1999) Linking clinical relevance and statistical significance in evaluating intra-individual changes in health-related quality of life. Med Care 37:469–478. https://doi.org/10.1097/00005650-199905000-00006

Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15:155–163. https://doi.org/10.1016/j.jcm.2016.02.012

McHorney CA, Tarlov AR (1995) Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res 4:293–307

Hara N, Matsudaira K, Masuda K et al (2016) Psychometric assessment of the Japanese version of the Zurich claudication questionnaire (ZCQ): reliability and validity. PLoS ONE 11:e0160183. https://doi.org/10.1371/journal.pone.0160183

Kazis LE, Anderson JJ, Meenan RF (1989) Effect sizes for interpreting changes in health status. Med Care 27:S178–S189. https://doi.org/10.1097/00005650-198903001-00015

Cohen J (1988) Statistical power analysis for the behavioral sciences. L Erlbaum Associates, Hillsdale, NJ

Franceschini M, Boffa A, Pignotti E et al (2023) The minimal clinically important difference changes greatly based on the different calculation methods. Am J Sports Med 51:1067–1073. https://doi.org/10.1177/03635465231152484

Samsa G, Edelman D, Rothman ML et al (1999) Determining clinically important differences in health status measures: a general approach with illustration to the health utilities index mark II. Pharmacoeconomics 15:141–155. https://doi.org/10.2165/00019053-199915020-00003

Wright A, Hannon J, Hegedus EJ, Kavchak AE (2012) Clinimetrics corner: a closer look at the minimal clinically important difference (MCID). J Man Manip Ther 20:160–166. https://doi.org/10.1179/2042618612Y.0000000001

Stucki G, Liang MH, Fossel AH, Katz JN (1995) Relative responsiveness of condition-specific and generic health status measures in degenerative lumbar spinal stenosis. J Clin Epidemiol 48:1369–1378. https://doi.org/10.1016/0895-4356(95)00054-2

Liang MH, Fossel AH, Larson MGS (1990) Comparisons of five health status instruments for orthopedic evaluation. Med Care 28:632–642

Middel B, Van Sonderen E (2002) Statistical significant change versus relevant or important change in (quasi) experimental design: some conceptual and methodological problems in estimating magnitude of intervention-related change in health services research. Int J Integr care. https://doi.org/10.5334/ijic.65

Revicki D, Hays RD, Cella D, Sloan J (2008) Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 61:102–109. https://doi.org/10.1016/j.jclinepi.2007.03.012

Woaye-Hune P, Hardouin J-B, Lehur P-A et al (2020) Practical issues encountered while determining minimal clinically important difference in patient-reported outcomes. Health Qual Life Outcomes 18:156. https://doi.org/10.1186/s12955-020-01398-w

Parker SL, Mendenhall SK, Shau D et al (2012) Determination of minimum clinically important difference in pain, disability, and quality of life after extension of fusion for adjacent-segment disease. J Neurosurg Spine 16:61–67. https://doi.org/10.3171/2011.8.SPINE1194

Jacobson NS, Truax P (1991) Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 59:12–19

Bolton JE (2004) Sensitivity and specificity of outcome measures in patients with neck pain: detecting clinically significant improvement. Spine 29(21):2410–2417. https://doi.org/10.1097/01.brs.0000143080.74061.25

Blampied NM (2022) Reliable change and the reliable change index: Still useful after all these years? Cogn Behav Ther 15:e50. https://doi.org/10.1017/S1754470X22000484

Asher AM, Oleisky ER, Pennings JS et al (2020) Measuring clinically relevant improvement after lumbar spine surgery: Is it time for something new? Spine J 20:847–856. https://doi.org/10.1016/j.spinee.2020.01.010

Barrett D, Heale R (2020) What are Delphi studies? Evid Based Nurs 23:68–69. https://doi.org/10.1136/ebnurs-2020-103303

Droeghaag R, Schuermans VNE, Hermans SMM et al (2021) Evidence-based recommendations for economic evaluations in spine surgery: study protocol for a Delphi consensus. BMJ Open 11:e052988. https://doi.org/10.1136/bmjopen-2021-052988

Henderson EJ, Morgan GS, Amin J et al (2019) The minimum clinically important difference (MCID) for a falls intervention in Parkinson’s: a delphi study. Parkinsonism Relat Disord 61:106–110. https://doi.org/10.1016/j.parkreldis.2018.11.008

Redelmeier DA, Guyatt GH, Goldstein RS (1996) Assessing the minimal important difference in symptoms: a comparison of two techniques. J Clin Epidemiol 49:1215–1219. https://doi.org/10.1016/s0895-4356(96)00206-5

Carreon LY, Glassman SD, Campbell MJ, Anderson PA (2010) Neck disability index, short form-36 physical component summary, and pain scales for neck and arm pain: the minimum clinically important difference and substantial clinical benefit after cervical spine fusion. Spine J 10:469–474. https://doi.org/10.1016/j.spinee.2010.02.007

Wang Y-C, Hart DL, Stratford PW, Mioduski JE (2011) Baseline dependency of minimal clinically important improvement. Phys Ther 91:675–688. https://doi.org/10.2522/ptj.20100229

Tenan MS, Simon JE, Robins RJ et al (2021) Anchored minimal clinically important difference metrics: considerations for bias and regression to the mean. J Athl Train 56:1042–1049. https://doi.org/10.4085/1062-6050-0368.20

Staartjes VE, Stumpo V, Ricciardi L et al (2022) FUSE-ML: development and external validation of a clinical prediction model for mid-term outcomes after lumbar spinal fusion for degenerative disease. Eur Spine J 31:2629–2638. https://doi.org/10.1007/s00586-022-07135-9

Download references

Open access funding provided by University of Zurich. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and affiliations.

Department of Neurosurgery, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam Movement Sciences, Amsterdam, The Netherlands

Anita M. Klukowska & W. Peter Vandertop

Department of Neurosurgery, University Clinical Hospital of Bialystok, Bialystok, Poland

Anita M. Klukowska

Department of Neurosurgery, Park Medical Center, Rotterdam, The Netherlands

Marc L. Schröder

Machine Intelligence in Clinical Neuroscience and Microsurgical Neuroanatomy (MICN) Laboratory, Department of Neurosurgery, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland

Victor E. Staartjes

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Victor E. Staartjes .

Ethics declarations

Conflict of interest.

The authors declare that the article and its content were composed in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (TXT 6 KB)

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Klukowska, A.M., Vandertop, W.P., Schröder, M.L. et al. Calculation of the minimum clinically important difference (MCID) using different methodologies: case study and practical guide. Eur Spine J (2024). https://doi.org/10.1007/s00586-024-08369-5

Download citation

Received : 03 May 2024

Revised : 17 May 2024

Accepted : 10 June 2024

Published : 28 June 2024

DOI : https://doi.org/10.1007/s00586-024-08369-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Minimum clinically important difference
• Anchor-based methods
• Distribution-based methods
• Change scores
• Clinical outcomes
• Spine surgery
• Find a journal
• Publish with us
• Track your research