Improving Tactical Decision Making - Dr. Gary Klein

Jon Becker: My guest today is a true legend in the field of decision-making, Dr. Gary Klein.

Gary's the author of the book “Source of Power” how people make decisions and five other books.

His work includes groundbreaking research into “The Cognitive Models” we use for decision-making including the recognition Prime Decision Model, the Data Frame Model of Sense Making, the Management by Discovery Model of Planning in Complex Situations, and the Triple Path Model of Insight.

He's developed methods including: the Premortem Method of Risk Assessment, Techniques for Cognitive Task Analysis, the Shadowbox Training approach and has also helped pioneer the Naturalistic Decision Making movement in 1989.

Dr. Gary Klein has experience in dozens of work domains including: military, health care and emergency response.

I'm excited to have Gary on the show because his work with military and emergency response workers, provide a framework for us to understand not only our decision making but also how we can improve it through training and simulation.

Hope you enjoy my conversation with Dr. Gary Klein.

My name is Jon Becker.

For the past four decades, I've dedicated my life to protecting tactical operators. During this time, I've worked with many of the world's top law enforcement and military units. As a result, I've had the privilege of working with the amazing leaders who take teams into the world's most dangerous situations.

The goal of this podcast is to share their stories in hopes of making us all better leaders, better thinkers, and better people.

Welcome to The Debrief!

Jon Becker: Gary, thank you so much for being here with me today.

Gary Klein: Thank you very much for inviting me. I really appreciate it.

Jon Becker: Why don't we start for the you know unaware. Let's start with kind of a brief bio background of you and kind of your interest in decision- making because it spans almost my entire lifetime.

Gary Klein: Right. It all started in the mid 1980s. The army put out an announcement that they wanted some research done to understand how people make life and death decisions under extreme time pressure and uncertainty.

The background I found out later is that the Army had put a fair amount of funding into conventional laboratory based research, and it didn't get them very much and they were very disappointed. So they were looking for something new. So they put out this announcement.

I had started my company about seven years earlier, and I thought “I'll take a shot at it”. I didn't know what I had to offer or whether I had a chance. I think most of the people who responded that it was a competitive announcement said, “We'll take a laboratory task that we can control, and vary the time pressure and vary the uncertainty and see what we can find out”. I didn't want to do that. I was really curious, how people can make good decisions under time, pressure and uncertainty especially with high stakes? I said, “Let's let's talk to some people who do it”. And we chose not police officers but firefighters because they have to do it on a regular basis, and because we thought that they had a lot of time between fires when we could work with them sit with them, ride with them, it would just be an easier population and we had some connection.

So we did a study. It was a six-month effort to see how they proceeded. And oddly enough we found out that the conventional models weren't even close. And we came up with a different account of how they did it. That's really was what started my career and we had to develop methods to interview them to do deep what we call “Cognitive Interviewing” to get inside their heads. And we did a large number of interviews about tough cases, critical incidents, and came up with a different model of how they performed and that's changed the whole trajectory of my career.

Jon Becker: You wrote kind of the seminal book on decision making. I think that ‘Source of Power’ everybody looks back to is kind of the magna carda or the you know Rosetta Stone of this kind of research and looking at decision making and then obviously subsequently have written a lot of books. Over the past 40 years, I think if you go back and look at your body of work it's pretty broad, and you've kind of helped to guide this entire science.

Gary Klein: Thank you. I think we've we've had a much stronger impact than we expected. We didn't have any plan that we were going to go in this direction. We just followed whatever interested us that seemed like it would be useful and valuable, and we were just following our own intuition about what to do, and how to proceed.

The book ‘Sources of Power’ has it just was reissued a few years ago and its to sell very briskly but that book was an accident. So much of what I've done has been an accident.

We've been doing research for about five or seven years and I thought, “I'd like to publish what we have found but I also want to publish the stories we've heard because just giving the findings without the stories, doesn't really convey the flavor of it”. But in a scientific publication they don't really have much space for stories. There's always a pressure to try to keep the work as truncated and dense as possible but I wanted to tell the stories. I thought I have to write a book to do that. But I was never going to write a book because that takes a lot of work.

Then one day it was in 1984 I think-- no it was later than that. It was probably ‘93, now that I think of it '92, I was jogging. And there was no jogging path where we were staying. It was a rocky path and I said, “It's only a matter of time before I twist an ankle”. 10 minutes later I came down wrong, twisted an ankle, broke a bone in my ankle and managed to hobble home. I was going to be laid up. We were on a sabbatical my wife had. Was going to be laid up for the next month or two while my ankle healed, and I said, “I guess this is my time to write a book”. So that's how the book Sources of Power got written.

Jon Becker: So your broken ankle is all of our academic gain.

Gary Klein: Yes it is. It's the luckiest broken ankle that I ever had.

Jon Becker: It's interesting because I've read a lot on decision making and it all kind of does reference back. Part of the reason that I wanted to talk to you was a lot of the you know let's call them ‘modern decision-making’ books reference back to Sources of Power.

I think the story aspect is one of the things that makes Sources of Power such an interesting book because it's dense as far as the amount of knowledge that you're passing along. What's nice is it's punctuated with so many stories. The stories are so memorable that even when you don't remember the academic concept, you walk away with remembering you know the story of somebody hanging on a bridge, and how they're going to tinker down and like this series of firefighters. It is in that sense really effective at its purpose because it forces you to kind of look at the way you make decisions.

Gary Klein: Right. In many ways, I think the stories are the backbone of the book. There is research in there. There are models and theories about decision making. So that's in there but the stories are really the foreground. That's what I really wanted because the stories were exciting. I mean they were incidents that we collected when we did our interviews, and we learned so much from the stories then we would go back to our office and try to make sense of it.

Jon Becker: I think that probably the thing that struck me most with Sources of Power is that we have a concept of how we make decisions. We all think we understand how we make decisions, and what we understand about our own decision making is almost certainly wrong.

I think that that's probably as I walked away it's like the way I thought I made decisions is not the way I make decisions. Let's talk about that for a minute.

I know that we think when we make decisions, we look at all the possible outcomes and weigh them against each other and we run this big complicated model. That's not true is it?

Gary Klein: Right. The research that had been done up to when we studied the firefighters mostly talked about that, and said if you want to be a rational decision maker that's exactly what you need to do. You need to lay out the options. You need to lay out the evaluation dimensions. You need to evaluate each option on each dimension. See which one comes out ahead, and if you don't do that you're not a rational decision maker.

So that's what gets taught in schools of business. What used to get taught in schools of business and military academy and schools of engineering, that's the way to make decisions.

We were a little dubious about that because when people have to make time-pressure decisions, they don't have time to do it.

A friend of mine Danny Zakai did a study, showing that it takes at least a half hour to set up that kind of a matrix.

In law enforcement you don't have a half hour to set that up, so we thought “Okay you're now looking at a full range of options”. Maybe you're just looking at two. One that you know you think you want the other one to compare it to. So we thought that's what people would do and we were wrong. They didn't even look at two options, and that frustrated us.

It's not as if we collected data to show that our model was right. The interviews we did showed that our original model was wrong, and we didn't have any other idea, so we went back to the interviews and said, “What are these people telling us?” They're telling us that because of the experience they have they can look at a situation and quickly using the patterns they built up over five years, 10 years, 20 years. All those patterns immediately tell them here's what's going on, or suggest here's what's going on in the situation.

And that means what should they pay attention to, and what can they ignore. It means what can they expect, so they're ready for the situation to evolve. But if their expectancy is violated, that tells them maybe something else is going on here and I have to rethink it. It tells them what goals they should try to achieve. And it tells them here's the courses, here are the courses of action that are likely to be successful and it starts with the most likely one.

So within seconds people who have that kind of experience, immediately know what is likely to work in this situation. But their intuitions match, this pattern matching-- when I talk about intuitions, I don't mean anything occult or mysterious. I mean the experience that you have the patterns you built up could sometimes mislead you. That was a confusing point for us-- how can you evaluate one course of action except by comparing it to others. We looked in the interviews to see what they were telling us. What they're telling us was you don't compare one option to another. Instead what you do is you imagine the option-- How would work in this situation if I carried it out? Let me run it through in my mind to see if I think it'll succeed or if there's some problems that might arise. If you don't see any problems then you're ready to go and that's how people can make rapid decisions under time pressure and uncertainty.

If you see problems then you can change the course of action, you can modify, you can improve it for this situation so that it'll get the job done. If you can;t figure out a way to improve it to overcome the problem and you say, “Okay forget that one. What else do I have in my repertoire?” You go through the repertoire until you find one that seems to be workable and that's the one that you choose. We call this a “Recognition Primed Decision Model”. It's primed by your ability to recognize situations but it's not just recognizing the intuition part, the pattern matching part. But then this other part of imagining it we call a mental simulation you're running through in your head that's a conscious deliberate analytical part. Now people don't know that because the pattern matching is subconscious. You don't experience yourself matching a pattern. Something just pops in your head and you say, “I know what's happening here”. So people haven't appreciated that and we didn't appreciate it until we did our interviews. Plus people have the wrong idea about what a decision is.

I remember early on in the project I interviewed this one firefighter and I said, “I'm here to study how you make decisions”. He told me about one incident that stuck in his mind where there was an apartment building on fire, lots of gas, lots of smoke and he opened up one of the doors in one of the apartments and there was a family and they were all sitting in the living room and they all had died. It just hit him with a shock to see that. He remembered that the husband I think there were two kids, the husband and the wife. The husband was wearing a sleeveless undershirt and he said from that point on he could never wear a sleeveless undershirt again, it had that emotional shock on him. But there was no decision making there. It was just a vivid incident that he recalled but he wasn't making decisions. Then we realize that the word “decision” is confusing people because they think a decision means something that it isn't. Then many other people were telling us that they never made decisions that was the usual response.

I remember the first guy I interviewed, he'd been a fire ground Commander for a dozen years, He'd been in the fire department for 16 years. I said, “I'm here to study how you make decisions”. He said, “In all that time I've never made a single decision. So you're wasting your time with me.” I said “Really?” He was actually one of the first ones I interviewed. We've just gotten this Army contract and now we find out that they're saying they don't make decisions. I said, “So what do you do? How do you know what to do?” And he said, “It's just procedures”, he followed the procedures. So I thought, “Wow this project is off so it is going to fail”. I asked him, “Can I see the procedure manual? Maybe I'd find something in the procedure manual that would help me.” And he said, “Oh it's not written down you just know.” In fact what seemed like procedures to him were really his expertise coming out and that's how he got to be a commander. That's how he got to be so good. When he was thinking of decisions he thought I meant a decision was comparing one option to another and he never did that and we heard that again and again that they never compared options one to another. In fact we've been doing a number of studies after that we've done, other people have done to replicate our work and find the same thing. At least 90 to 95% of the cases people are not comparing options. Occasionally sometimes you have to but it's very rare.

So people have the wrong idea when they hear the word “decision” so we stopped asking about decisions and we asked them about tough cases. Cases where they weren't sure where somebody might have done something differently where there was some uncertainty, we asked about that because the word “decision” was sending them off in the wrong direction.

Jon Becker: As you're talking it strikes me that one of the things that makes human beings evolve, one of the things that separates us from all other species is the ability to share those patterns-- is the ability through all higher animals teach each other patterns by the mother bear takes the young bear to the river and shows them the salmon and then the young bear knows that's where food comes from. But as human beings because we have written language we can write that down and somebody across the world can read Gary Klein's book and learn from it and that's part of what makes us evolve. We undervalue those recognitions.

One of the examples I use when I talk about why we do debriefs is a rattlesnake the first guy that ever found a rattlesnake got bit by the rattlesnake and if he didn't tell anybody about it that happened ad nauseum until somebody said, “Hey, there's this long skinny thing, it's got a rattle on its tail and fangs in the front don't touch it”. Then as long as he shared that knowledge nobody else got bit. It strikes me as you're talking that that Recognition Primed Decision making is the ability to look at that long cylindrical object with the rattle on its tail and the fangs at the front and quickly say that's a rattlesnake.

Gary Klein: Right.

Jon Becker: You don't look at that and go, “Is that a rattlesnake or a goat?” That math doesn't happen in your brain, you don't look at it and think “I wonder if that's a whale”. Your brain just in a very subconscious level draws quickly to oh that is this like there may be a logic tree involved in that where your brain looks at it and says, “Well it's on land so it's not a whale”. And then you know that might happen a thousand times subconsciously but at some point your brain goes “That's a rattlesnake”.

Gary Klein: I don't think our brain is saying “Is that a whale?”

Jon Becker: I mean mine might would but the ability to differentiate those two is happening in a subconscious isn't it?

Gary Klein: And that's one reason that people don't share what they know because it's happening at a subconscious level because they're picking up cues that aren't all that obvious. You can see the long narrow tube but rattlesnakes tend to be coiled and they tend to be hiding under rocks or or behind bushes ready to to strike. So they don't announce to themselves I'm a rattlesnake. So people seeing rattlesnakes and watching them and knowing the danger that they pose can watch them and see how they move, how they slither, where they like to hang out. The fact that during the daylight they like to be out in the open warming up because they're coldblooded, and at night they tend to to preserve heat to borrow underneath. So just watching and observing you builds a mental model about rattlesnakes not just what they immediately look like but how they act and how they function and that's hard to communicate. And that's one of my big complaints about the police community and firefighting community and many others that they don't do the kind of debriefing I think could be the most powerful. They don't do the kind of on-the-job training that I think would be the most powerful.

If you go out on a call and something happens and you have an interesting incident you can come back and talk about what happened and then people can say, “Oh you did this. I would have done that.” And you can talk about the merits of one versus the other and people can all learn from that. It's hard for you to describe the subtle things you were noticing. What were the things that were making you uneasy? What was raising suspicion in your mind? Those are harder to describe and people usually do not even try to describe them when they tell the story.

The debriefing in the military after action review is usually about the action you took and doesn't also get into the depth of what was running through your mind, what were you noticing. I wouldn't ask you why did you do it but I would ask you: What were you seeing? What were you hearing? What were you not hearing? Is there anything that got you suspicious because it wasn't happening? That's all part of your expertise and that part of the incident people often don't talk about because it's not as obvious as the actions you took.

Jon Becker: And yet that's really where the magic is.

Gary Klein: That's where the magic is. Let me tell you about a flight that I took a number of years ago. We were studying wildland firefighting and we wanted to visit a station in Northern California. I flew into  San Francisco airport and then we got a small puddle jump up to it's about a 45 minute flight up to I think it was Redding where I was going to be based. It was 10 o'clock at night. It was totally dark and we're taking off from San Francisco Airport and then all of a sudden the plane starts to yaw back and forth and all the other passengers are looking around like “Are we going to crash?” Because the plane is wiggling back and forth. I wasn't too worried because I know that the planes have a stabilizer so there's a limit to how much it's going to wiggle and then sure enough we lift it off and we were fine and we all forgot about it. Then when we landed at the end of the landing the plane did some more wiggling. It was a 45 minute flight. Get into the van taking me to my hotel and as luck would have it the co-pilot he and the pilot were in the same hotel, he's sitting next to me and we're waiting. I asked him what we were waiting for and he said, “We're waiting for the pilot. He's filling out some forms.” I'm a nosy guy as that's part of my profession.

Jon Becker: Curious.

Gary Klein: Okay thank you that's a better spin. So I said, “Remember that wiggling that we did, that yawing that the plane did, what was that all about?” He said, “I don't know. I have no idea why we're waiting for the pilot, that's what he's writing up so that the maintenance crew can look it over and diagnose the problem and repair it so we can fly out tomorrow morning.” I said, “You have no idea why the plane was--?” He said, “None whatsoever”. Then the pilot shows up after a while he's sitting on my other side. I couldn't resist. I think I've told you I'm a curious guy. So I said, “That yawing that we did, what was that about?” He said, “Well as soon as the planes started yawing I figured it feels like a problem with the nose wheel and we were going so fast I didn't want to abort take off because I thought that would be dangerous. And if there's a problem with the nose wheel I just have to lift the nose up and we'd be fine and sure enough we were. I made a note to myself when we landed to keep the nose off the ground until I bled off a lot of speed and then touchdown at the end.” He remembered we did some more yawing. I said, “Yes”, and that's when I got scared because I thought these two guys (the pilot and the co-pilot) were sitting next to each other right next to each other in a small cockpit for 45 minutes and they never talked about this.

How many opportunities are missed because those conversations don't happen? I don't know why this didn't happen. Maybe the co-pilot felt it wasn't his place to ask. He was intimidated by the difference in rank; maybe the pilot didn't want to voice information or didn't even think about sharing it. But how many opportunities are there where something interesting like this happens that can help build expertise and we squander it? I see this in police departments with people riding next to each other and not sharing information, losing the opportunities.

Jon Becker: It's interesting because the way you describe it we recognize patterns where so much of this is recognition primed. And so much of it is what we would regard as intuitive right right and and I think that there is kind of this innate feeling that intuition is something we either have or we don't have. Oh well you know he's just got really good business sense or he's got good sixth sense or whatever. What I'm hearing you say is that's not true.

Gary Klein: No. This kind of intuition, this kind of pattern matching is the essence of expertise and as people develop expertise they have more experiences and they build a bigger pattern base. Now that I've said that let me walk it back because there's a belief that to build expertise you need more experience-- and that's partially true but it's partially untrue because you also hear lots of stories of somebody with 10 years of experience but it really is just one year of experience repeated 10 times because they haven't learned very much. So just having experience is not enough-- it's got to be transformed into expertise. That means when you have an experience you need to reflect on it and not just what you did but why you did it. You have to think about it-- and especially now I'm thinking the medical community where somebody wants to go into emergency medicine they want to go into a a hospital in an inner city where there's lots of accidents and lots of action because it's not just having experiences, it's having tough cases. So you want to expose yourself to tough case. Tough cases are ones where you sometimes fail. We can learn a lot from our failures because our failures expose the limits of our understanding and give us a chance to reflect and to revise some of the beliefs that may not be perfectly accurate that may be limited.

So you want these kinds of tough cases, these kinds of difficult challenging experiences but you also want to learn from them. So you want to get feedback. Simply have an experience isn't enough, you want to get feedback on what happened. You don't just want the feedback, you want to make sense to the feedback. What is the feedback telling us? This journey to expertise is not a trivial journey that's why some people move up the ranks and learn a lot and become much more sophisticated, and other people are just stuck in a rut and not not learning all that they could and not developing and not expanding their understanding.

Jon Becker: It's not just a frequency of experience question it's a breath of experience question.

Gary Klein: Right.

Jon Becker: As you're talking in thinking about failure-- I do a class called “Lessons Learned From the Debrief”, and one of the things I teach there is you have to embrace failure. Failure is where you learn the limits of your expertise. And it's also I guess as you're as you're describing it if I have a pattern that I applying to a situation that pattern failing is where I understand the limits of that pattern.

Gary Klein: Exactly that allows you to move to a higher level of expertise is when your current understanding turns out to be inadequate in some ways, and only then are you going to be open because otherwise we don't want to change our thinking because that takes work. When we failed that creates an openness in us to do this kind of rethinking.

I think you're exactly right, failures are extremely important but I'm not sure about embracing failure because failures hurt and people struggle with them. Now I know some people who say, “I want to put that behind me. I don't want to dwell on it.” I think that that's a mistake. I think what's great about failures is at least for me I'll tell you about me. When I fail if I put on a a workshop and it doesn't go the way it was supposed to I'm really irritated I'm discouraged I'm depressed I walk around for the next couple of days saying, “I don't know if I want to put on any more workshops I think I'm done”. Then eventually I realized, “Here's what I did wrong. Here's what I should have. How I should have framed it. Here's what exercise I should have introduced. Here's something that I should have waited till later and then when I rethink it that way then I can't wait to do another workshop and I'm eager to do it again.”

The importance for me of failure is it unsettles my thinking but it failures also burn at me because I can't get them out of my head. If I said, “Okay I failed. I'm just going to move on, I wouldn't learn as much that's why one of the ways where when people ask me how do I know who's an expert? There's no perfect diagnostic but one of the ways I know somebody is an expert is I'll say, “Tell me about your last failure and I find that the people who are the experts can immediately tell me what it was because it's been bothering them ever since. Whereas I encounter a lot of people who give me a blank look and say, “I can't think of any”. I sort of cross them off the list of experts because they're oblivious to failures-- they're not rethinking and so they're not growing-- they're not building expertise.

Jon Becker: So that's exactly what I mean by Embrace failure that's exactly what I mean I mean is it is the teacher. And it's funny because I use the exact same diagnostic question when I'm determining whether I'm going to interview someone for the show. So if a team had an operation that went well, didn't go well one of the first questions I asked them is “What went wrong on your operation?” If their answer is “Nothing. It was really smooth.” Then there's nothing for you to learn here-- if the pattern that you're applying to the situation if you go back to Maslo you have a hammer you tend to view all your problems as nails. If the hammer has worked every time and it's and every time is a nail then you really only understand the nail. I need you to understand nail and screw and staple and duct tape and all of these other things and you only realize that by getting bit by the rattlesnake.

Gary Klein: Right.

Jon Becker: Now am I getting bit by the rattlesnake can teach you that getting bit by the rattlesnake is what the rattlesnake is. So you might not also have to get bit by the rattlesnake but one of us is getting bit in order to tell the story. That's what I was about to say.

Gary Klein: So you don't have to have these firsthand experiences we can have vicarious experiences we can learn from others especially from the failures of others and see what they did wrong and not just what they did wrong but if we have a chance if we can interview them before they die of the rattlesnake bite ask them “How could you not notice a rattlesnake?” They might tell us that it had blended in with the vegetation”. We probably wouldn't do that kind of an interview as they were dying…

Jon Becker: Or they thought it was a whale.

Gary Klein: Or maybe they thought it was a whale. When people come back from an operation that didn't go well then we can ask them: In hindsight what should you have been looking at? In hindsight how could you have changed the way you identified the threat? In hindsight what were the tells? Now that you've been through this, what have you learned? We call these “Cognitive Debriefs”. They talk about what the person was experiencing and thinking and noticing and recognizing, and not just what did the person do. Too many debriefs are just narrowly focused on the actions people took and not on how they were sizing up the situation.

Jon Becker: It's interesting because one of the things that's changed in the time that I've been in this industry is debriefs have gone from brutal after actions to hero stories and the number of mistakes that people are admitting to and owning. When an operation goes well then yeah there's going to be a debrief and we're going to talk about it but when an operation doesn't go well, there is this resistance to debriefing it to talking about it. The thing that strikes me about that is that's exactly when we need to debrief. If it went well we can't learn as much from it went well as we can from it didn't go well.

Gary Klein: Exactly right. Now there's a trap in there about the learning from the debrief which is let's get a debrief and incident that didn't go well, and let's write it down so we have a record for other people. When I've seen that done too many times, if it's written down and made official it gets sanitized.

Jon Becker: Yep.

Gary Klein: I remember once a friend of good friend of mine told me about this firefighting incident where so many things went wrong. I said, “I want to learn more about it”. He said, “There's a report”. He showed me the report and none of those things were in the report because you don't want to single out that person failed to do this or this person was not anticipating. Those things never get included in the report because you don't want to damage people's careers. So the writing down hinders the debrief. I don't know any easy way around that but it's just something I've noticed.

Jon Becker: Well I'll give you another corollary to that too which is when something really big happens where there is a catastrophic failure and it's a very public failure and we know that that things went wrong. Rather than trying to look at the ire process and trying to look at all of the nuances to it we try to just blame it on somebody.

Gary Klein: Exactly.

Jon Becker: Look at Uvaldi as the most recent tragedy that everybody is kind of talking about. I recently interviewed the team for DOJ that did the evaluation of the Uvaldi situation and I said told them, “One of the things that drives me crazy with Uvaldi is everybody wants to look and go, “Oh well it's all that chief of police's fault” or “Those guys just didn't have any courage”. There were 370 people. Somebody had courage. We have an underlying flaw in the way that we are training but because we just try to blame it on somebody, “Oh we should charge that guy with a crime” or “Oh we should do that” we lose all of the lessons that are associated with an event like that that really could have been beneficial.

Gary Klein: Totally agree with that. As I listen to you I'm thinking when you do the investigation, when you do the after action review the debriefing what's your stance? Is your stance to try to find who to blame or can you adopt a more curious stance or empathetic stance because you're dealing with people who are reasonably caring, reasonably experienced who want to do a good job and somehow they failed, what happened? Why did it happen? There might be good reasons or useful reasons to learn from them if you take a more sympathetic or empathetic stance when you do the debrief rather than a prosecution stance to try to show that they messed up big time.

Jon Becker: I think it's easier as somebody who debriefs a lot of them and sits through in my career I sat through close to a thousand law enforcement and military debriefs. I think it is easier like The Lazy Man's way is to say, “Well Gary screwed this up” because then I don't have to look at the fact that I I could have very easily been Gary and I could have made that exact same decision and that our expertise is lacking or our method isn't working. So it's much easier for us to point the finger at somebody and blame it on them than it is to really do a true postmortem on our decision-making process and look at like why did we fail.

Gary Klein: Right. Let me tell you about a guy named Ike Bracken who's an amazing trainer in the petrochemical industry. I remember he was a petrochemical plant operator. He was really highly skilled then he became a trainer and was a trainer for several decades. He told me early in his career he trained people the way he had been trained. He waited for people to make a mistake and then he slammed him. That was the training regimen. That he was used to that's what he put into practice and he was like a rattlesnake.

Jon Becker: Yeah waits till they get bit and then go see. Ready to bite people.

Gary Klein: Then he noticed it wasn't really working as effectively as he wanted and then on his own he just started to become curious. If somebody did something that was wrong or was it a mistake instead of saying, “Ah you shouldn't have done that” he'd wonder why they did it-- what was going on with them. Because it was that kind of a sympathetic dialogue people were not shutting down. They knew that this was not a punitive interview. This was not a punitive training session. It was to be a productive training session and so it became a joint venture for them to try to understand how this could have happened and with that kind of a stance it became very productive. So for most of his career he was just seen as a great trainer but he changed his mindset-- from a mindset that was critical to a mindset that was curious and it made all the difference.

Jon Becker: It's funny because one of the biggest differences between me when I started my business and me as as now “middle-aged leader”, is I used to care who was at fault. I only care now about fixing the problem. When I teach this I tell people like “You can try with your wife”. You walk in and go, “Hey, you did this| and point your finger at your wife and let me know how that goes for you. I don't care what the problem is if I come at you with like “Hey, Gary you know I think you screwed this up”, you are instantly defensive whereas if I put the problem on the table and I come to the same side of the table as you are and I go, “Hey, how do you think this happened?”

Gary Klein: Yeah exactly that question is a question born of curiosity.

Jon Becker: Yeah.

Gary Klein: This happened.

Jon Becker: When we're debriefing we need to be focused on that. We need to be focused on why we end up where we ended up.

Gary Klein: Yep how did this happen and I might say, “I don't know, let's talk about it and see what we can figure out”.

Jon Becker: Yeah it's very interesting because often the best leaders, the best decision makers-- I look at the people in my life that are the best leaders and they are the most curious people I know. They learn from everybody they meet.

Gary Klein: Right. But for other people being curious is maybe a little vulnerable that it suggests you don't know the answer, that it may threaten your authority to be asking-- to be wondering-- to not know things. I think I once made a list of all the ways that organizations throttle curiosity. If I'm a trainer my job is to get through all of my PowerPoint slides, and if people are starting to ask me questions in the class I want to get through that. I want to shut them down so I can get through my slides whereas those questions that's where the learning is going to happen.

Jon Becker: So if our goal is to develop expertise then how do we organizationally encourage that and train it? If you were responsible for training a business leader for a SWAT team (which you are, this is what you've done for many years) what can we do culturally to create expertise in our organization?

Gary Klein: I think about the organization wanting to improve performance. To improve performance we can think of it in two ways: There's one way which is I try to reduce errors and I might try to make sure we have lots and lots of procedures so people know exactly what they're supposed to do which is impossible in a field like law enforcement because no two situations are alike. I have this procedural mentality so that if people don't follow the procedures I can criticize them. For many organizations it's all about reducing errors, and certainly we need to reduce errors. I'm not saying that errors are good. We can learn from failures that doesn't mean we want failures. We have to harvest the failures when they do occur.

Jon Becker: Yeah.

Gary Klein: The other thing is to increase expertise and to increase insights and most organizations don't pay any attention to that. So that's what you're asking me now, what about that side of the coin about building expertise. I've told you one way I think the most effective way is by this on-the-job training-- by the mentoring that goes on. The mentoring involves you know like you have a field training officer riding with a recruit just riding with the recruit you can talk about. Here's how we do this, here's how we do that but it takes more work to describe all of the subtle things that you're noticing. If there's an incident afterwards when you debrief it, you can talk about what you were spotting that made you suspicious of this person whereas the recruit might not have noticed that. So the effect of mentoring means going beyond just the actions taken and the procedures followed. It means describing how you made sense of the situation: What were the tells? What were the things you learned and what were the things you were able to spot that weren't obvious?

So that kind of on-the-job learning and mentoring I think is the most effective thing that the organization can do. Most organizations do not put much energy into this at all because they put energy into classroom training. People say most of what they learned they learned outside the classroom-- well why not boost those experiences by helping the mentors capture and describe and and make it part of their routine to describe what they were seeing? I don't want to just put it on the mentors, on the field training officers. This also can be part of the recruits expectation that if you see something that you don't understand ask about it. The co-pilot in the incident that I told you before, the co-pilot could have asked the pilot, “What happened as we were taking off? Why were we yawing?” So that could be just part of the relationship involving mutual curiosity. I don't think people are prepared for that. That would be my suggestion number one.

My second suggestion is to do more with vicarious experiences and help people just have more incidents that they're exposed to. The Marines have a technique called “Tactical Decision Games” that people go through as a way of giving them tough decision challenges so that they can think about it and they can be stretched. For a long time they had tactical decision games once a month in the Marine Corps Gazette and the Marines who got the Gazette would read it. It was just a one page with a diagram, here's your forces, here's what you know about the enemy, here's your mission, this just happened, what are you going to do now and they would sort of write down or capture or think about what I would do now. And then the next month you would get some solutions. People didn't always agree but they had interesting ideas. So this was a way of helping people get vicarious experiences.

Now the person who popularized tactical decision games in Marines is an amazing individual and a very good friend of mine John Schmidt who had been in the Marines for 12 years and left the Marine as a major because he was interested in doing other things. The main thing he's doing now is he's the CEO of my company. I was able to scarf him up and so he now runs my company. He's my boss. I just talked to him a few hours ago. What we've done is moved the next level or we've changed or modified or expanded on the idea of tactical decision games and what we have is a technique we call “Shadow Box” which is a way of giving people tough cases and helping them to learn from it. It's a great technique and I wish I could take credit for it but I can't. I learned about it from a firefighter, another friend of mine Neil Heins who had been a New York fire department Battalion Chief. He developed this technique where I would give you a scenario, a tough case. Then we walk through the scenario and at a certain point it would stop. This could be done. I could just talk to you through it or I could have it like with a diagram in a class but we mostly do it online these days. So I stopped the action and I say, “All right at this point maybe there's four courses of action. Rank order the courses of action in terms of what you are most eager to do second, third and fourth and write down your reasons.” Then we continue the scenario then we stop the action again call it a second decision point and we say, “Here's five pieces of information which would you say is the most important for you to gather as second, third, fourth and fifth rank order them and write down your reasons why you thought that was the most important?” Then we might continue the scenario, stop it again and say “At this point what's your major goal, what do you want to achieve at this point? Here's three options and you’d rank all of them and write down your reasons.

Now we've given the same scenario to a couple of what we call subject matter experts, maybe two or three subject matter experts. They've gone through the same scenario you did. They did their ranking. They wrote down their reasons. So when you say here's what my ranking is we can immediately tell you here's how these experts ranked it, and that's the game like quality because you want your ranking to match the experts. The most important part of it is here's what the rationale statement of the experts was compared to what you wrote. You get to see what implications they noticed, what were they thinking about, what were they worrying about.

So this is a way for you to see the world through the eyes of experts but the experts don't have to be there because it's difficult to arrange and too expensive. That's why we can do this online. I can give you this information about how the experts were thinking and now you are seeing the world release this incident through the eyes of the experts.

Now you might say, “I don't agree with the experts”. And that's okay too. Sometimes the experts don't agree with each other and we'll say, “Here's one view. Here's another view just to give the contrast. All we say is look at the reasons that the experts gave because they're experts for some reason. So see what can be learned from it. Even if you don't agree with what they're saying, see what you can learn from the rationale statements. Since we have like two or three of them we have to synthesize those and we combine them. But you can learn from experts even when you're holding on to your own views.” Shadow Box.

Jon Becker: This season we started running a series of decision making exercises that run with each of our significant debriefs and they're built and developed by the California Association of Tactical Officers. One of the guys that helps develop him is a Vanderbilt educational doctor who's a big Gary Klein fan and cites a lot of your work. Now I know that he wasn't as smart as I thought he was, he just read a bunch of your stuff but these decision-making exercises are exactly what you describe. It's putting the person in the shoes of the decision maker. Here's a set of facts - what information do you want - how do you see the problem - how do you frame this - here's another set of facts. It's running through that three times and then getting to a conclusion. It's funny because we were debating. Usually the way that these are run is live with a monitor and somebody who's there to kind of guide the decision-making and argument. As we started to plan through these of just putting them out there and launching the internet just turning it loose and letting you run it by self, one of the conversations that we got into was “What if the conversation doesn't go the way we want it to?” We all kind of came to the same conclusion is it really doesn't matter which way the conversation goes. It's really the process and not the destination that matters here right.

Gary Klein: Exactly. Having people go through this is a way for them to get vicarious experiences and to get feedback and to get an opportunity to reflect on their own thinking. When we construct these Shadow Box exercises we'll have like some of the options will be likely mistakes that people might make and we use those as foils to trap people. Especially less experienced people will jump for that and that's a way for them to expose their own thinking and then to see why that wouldn't work and was a bad idea. So that's the way these kinds of exercises work.

We find that in the data we've collected in about a half a day people are matching up. Their rank orderings are closer to that of the expert increases by about 25%.

Jon Becker: That’s huge.

Gary Klein: Is a huge jump in expertise in just a short period of time. John and his colleagues and our company have built about I think area of law enforcement. We're in the process of trying to get them posted to make them available for the law enforcement community.

Jon Becker: Once they're posted how would people engage? When that goes live how can people find it?

Gary Klein: I will let you know. I think they're going we're hoping to get them posted on I Addlist so that they can be more widely distributed. So we're having discussions with I Addlist in order to make these very broadly available for the law enforcement community.

I should also mention that I have a video course (I almost forgot to mention that to you) called a “Master Class in Practical Decision Making” and it's 15 episodes. Each episode is only five minutes and each one has a practical lesson that people can take away with them and put into practice. I did not properly prepare to tell you what the link is.

Jon Becker: You can send it to me and I can put it in the show notes.

Gary Klein: I will send it to you. It's on our website Shadowboxtraining.com, it's listed there but I'll send you the link so you can make people aware of it.

Jon Becker: Excellent. Gary, this has been ridiculously informative. I appreciate you taking the time to sit down with me and make me smarter in decision-making.

Gary Klein: I've enjoyed this conversation. It feels like a conversation with a friend even though I've just met you and how well it went.

Jon Becker: I'm sure we will talk again. So thank you so much for doing this.

Gary Klein: Thank you very much, Jon.