[MUSIC] Hello, and thank you for watching this video on measuring teamwork for improvement. In this video, we're going to introduce the options for measuring teamwork in healthcare and talk about why this is a good idea in general. Of course, if we're going to improve something we need to know where we stand and we need to assess over time to make sure that what we're doing is being beneficial and positive. But that can be challenging for healthcare, even though there are many, many good options for developing measurement systems that can be helpful in this area. So we're going to talk about those options, and what we can do to be effective and data driven in our approach to managing teamwork. To start off, to kind of frame the conversation, if you are developing a measurement system for understanding team performance or how well we're doing with teamwork, there's a couple of key questions you're going to have to answer. And how you answer those is going to determine what your measurement strategy should be. So first of all, of course, kind of what will be measured? And we can, of course, measure teamwork skills of individuals. We can measure performance of a team as a whole. If you think back to the model we introduced of the inputs, the mediators and the outputs, we can measure things about how the team is put together, its composition. We can measure things about the organizational context it's in, and we can also measure things about the outcomes of individual team members, of patients, of the team as a whole. Good reason to measure all of those things. Difficult to measure all of them, of course, at once so making tough decisions based on why. Why are we trying to measure team performance? In any given situation will help us focus on what we should be measuring. In general, there's a couple of large scale reasons. In other industries, less so in health care, they measure it for certifications or selection of people. In healthcare, we typically measure it for training evaluation, or we measure it to improve performance over time. We want information about how we're doing, either as a team positively or how we're doing in terms of minimizing communication failure risks, so that we can prioritize our efforts for improvement. So that's where we'll focus our time moving forward is training evaluation and measuring for improvement. So the how will be measured. In general, they're too big with methods for measuring teamwork or team performance. And that survey based measures either from the team members themselves, from leaders in organizations, or from patients and family members, asking them, were they receiving consistent information? Did it appear that their team was functioning cohesively or not? Those can be really strong predictors of many outcomes that the organization cares about. So survey based measures, asking lots of different people. Or there's observational measures too. We can go and look and see how well teams are doing in different situations. Pros and cons to both of those, we'll talk a little bit about those moving forward. And then who will be assessing performance? We kind of talked about this in the survey portion of this. Teamwork may look different from very different perspectives, is it the nurses perception, the therapists, the physicians, the patients and family members? From the observational phase, who is actually going to go look and observe our performance? Who are our raters? So those we need to figure out if we're designing a measurement system. And then where are we going to assess this performance? Is it on the job? Is it in simulations? Those things make a difference too. All good options with pros and cons. And then when? For a training evaluation, typically, we may ask people's perceptions as they're leaving the training. But we know what we really care about is do they transfer the job and do any changed behaviors persist over time. So we need to think about how are we checking in with people over time about the skills that they're using and developing. So that first piece that we'll talk through, overall, this is similar to the typical training evaluation that you'd use for any kind of program. There's levels or steps you'd like to see to ensure that you're doing a good job. And the first level are just people's reactions. Did they like the program? Do they intend to use it? And do they find value in it? And those are the two biggest things that we care about. People's judgements of utility, do the teamwork skills, will they help me? And then our intent to transfer, our intent to change our behavior and do something differently? So Level II is about learning. It's not just my emotional response to it, or my judgements about it, but am I actually learning the behaviors and the knowledge that's a part of the program? We can assess that with knowledge tests or simulations, to some extent. But really, are we changing what people know what they're capable of doing in their attitudes? And again, we can do that with surveys, knowledge tests. We can do that with observations. The third level is behavior change. So am I actually doing something different on the job that I wasn't doing before? Am I doing it better than I was before due to training? And here we can use survey methods too. We can ask people their perceptions of how effective they are as a team on the job before and after training. Or we can go and look, and we'll talk about some very sophisticated observational tools out there. We'll talk about a very simple one for a very specific purpose coming up. But we can observe what people are doing, or we can ask people their perceptions of what's going on. Again, people in teams, patients and family members, leaders and other members of the organization. Level IV, the kind of top of this structure, is results. If we're changing behavior, are we actually changing something that matters outside of that? Are we connected to improving better safety? Are we actually improving quality and outcomes in the organization or not? And this is where we recommend connecting some of these interventions to clinical issues as well, to safety and quality issues but another higher level outcome or workforce issues. So what happens to burnout and fatigue? What happens to staff retention? What happens to the safety culture or workplace engagement? All of those things can be impacted by teamwork training interventions, and those things are valuable in their own right, as well as organizational results, things that matter to the organization. So one example of a relatively simple measurement tool, observational measurement tool, for teamwork, again, focuses on briefings and debriefings that we've discussed in a previous video. This one is a simple audit tool. Team performance observation systems can get really challenging. There's some phenomenal ones, but they can get challenging to implement reliably. Audit tools, however, seek to focus on very specific process changes, and they seek to capture very specific teamwork behaviors. So for us, in the OR, what we did was create an observational tool that focused on some very specific things about where briefings were happening, where they are happening, when they were happening, some basics from the literature about what we know. Good briefings should cover topics we covered in a previous video, including name introductions, role introductions, setting expectations for open communication and being proactive about identifying risks. There's, of course, specific content in the OR that people should address, including things like antibiotics and antibiotic redosing and our perceptions on whether patients are going to have a difficult airway, things like that. And then participation. You can also very easily and reliably capture who is there, who is pausing the other tasks that the're doing and who's contributing anything to the conversation. And that can give you really good information about how mindful people are engaging in this process. If they're not stopping what they're doing, or they're not contributing anything, chances are you have a situation where people are going through an administrative task, not really participating in a value-added process. So that's a really specific way to look at a specific piece of teamwork and then use that information to drive improvement overtime too. One resource you should be definitely be aware of is the Agency for Healthcare Research and Quality Care Coordination Measure Atlas. This is an ongoing document freely available online that seeks to capture all the available different types of measures for understanding coordination. And it's well organized. What they mean by coordination is they divide this into some coordination activities. And this is a lot like the behaviors we've been talking about through the series of videos, the leadership, the communication, those types of things. And some larger strategies around how you organize healthcare system to improve coordination. All of these things, you can also look at what are the outputs or the products of expected impacts of coordination, and you can measure those things as well. And you can measure them from different perspectives, again, as we've been talking about. The patient perspective, the staff perspective, the health system and leadership perspective as well. So this is a resource of most of the available measures for assessing coordination. That can be really helpful if you're doing research or if you're planning an improvement initiative around teamwork and communication or coordination, really valuable resource. Another very practical tool, again, for gathering data that will help us prioritize where we should focus our time, our effort or our energy for improvement. This comes from the Comprehensive Unit-Based Safety Program, where there's a thing called a staff safety assessment, where you ask people, how's the next patient going to be harmed? And what can we do to prevent that? When you get those results, frequently, you will see communication as one of the top themes. So you can ask people more specifically around, what are the team worker communication problems on the unit that will lead to the next patient being harmed? And you can elicit information that way. You ask people to describe those problems, and how they can be addressed, so you get some solutions as well. And then what you do is you administer that, you categorize people's responses into themes, and you can think about is this a interpersonal issue? Is this a information technology issue? Is this a work process issue? What are people having challenges with around communication and teamwork on the unit? Debrief with a staff, plan and implement changes. So we take back your own perspective. Here's where we're struggling with teamwork. What are some things that we can do? And then we can use some of the team training competencies, some of the work process changes we've discussed in these videos to implement changes to address people's' concerns about teamwork and communication on their unit, and how to improve and then you repeat. And this is a great way to turn teamwork improvement from being perceived as a project or training intervention to being an ongoing, integrated piece of the organization's approach to safety and quality improvement in management. We have a way to on an ongoing basis elicit for people are feeling or the biggest challenges they have and using that to drive improvement. And coupling that to our team training programs to some of our work process changes, like structured communications or briefings and debriefings, and integrating that into an ongoing, continuously learning process where we're improving over time. So in this module, we talked a bit about team performance measurement. It's a very large topic, but typically, in healthcare currently, it's used for training evaluation. In this for eliciting information about how we can improve and get better over time. In the next video, we'll talk about the idea of cross unit collaboration and what a multi-team system is.
