Why We Fail: Solving Life-Saving’s Most Demanding Challenges
By Matthew Mitchell, Founder and CEO, International Association of Search and Rescue Coordinators (IASARC)
Jacob survived against impossible odds, but not because we saved him. As it would turn out, he swam for more than 19 hours overnight, enduring some of the most mentally challenging circumstances one can imagine. When he finally made it to shore the following day, naked and dehydrated, he made a phone call that would change my life and set me on the path to finding solutions to the most demanding challenges in my field.
The keys to his small center console were in his pocket when he went overboard. His girlfriend was still on the small vessel but had no way to start the engine and attempt a recovery. Jacob was young and athletic, a nurse, and could swim with the best. Still, the sheer force of the currents involved could not be overcome even by an Olympic swimmer. As we later analyzed the drift scenario, it became evident that the boat, with its significant sail area upon which the forces of the wind acted, and Jacob, a person in the water with only a tiny fraction of his mass above the waves, were being pushed in near opposite directions. Jacob drifted further and further from his vessel as his girlfriend watched helplessly.
After losing sight of Jacob, she called the local authorities, who alerted the Coast Guard and initiated a rescue response. When the first Coast Guard small boat arrived, Jacob had been out of sight for nearly an hour. They had been more than seven nautical miles from shore, the winds were moderate, and they still had several hours before sunset. All in all, the search conditions were not bad–but that did not make the search easy. Locating a person in the water, especially one without the benefit of a lifejacket, is stunningly difficult. Fewer than 19% of persons in the water searches yield a life saved. This number plummets when the distressed individual has no floatation or a signaling device to aid searchers.
Why is this number so low? Why is it so hard to locate a distressed person in the water? The answer is simple yet painful: humans are just not effective searchers. But how bad can they be? The global search and rescue (SAR) enterprise locates and saves people daily. We have accounts of persons in the water being spotted by passing ships in the middle of the ocean. Don’t these anecdotes counter the above statement that humans are ineffective? Well, anecdotes of lucky rescues and news accounts may suggest otherwise, but empirical research tells a different story. It turns out humans are not just ineffective but critically so. A 2006 Coast Guard Research and Development Center (RDC) report concluded that the average probability of detection (POD) during a visual search from a Coast Guard aircraft was only 18% from 0.1 nautical mile. That means if a Coast Guard aircraft flew within two football fields of you while conducting an unaided visual search, they would have only an 18% chance of seeing you.
Back to Jacob. He had drifted nearly four nautical miles in the hour it took to get rescue resources on the scene. Given the inaccuracies of the winds and currents and the uncertainty of the exact location where he entered the water, even the initial search area was roughly 16 square nautical miles. Covering an area that large with a single small boat with parallel tracks of .1 nautical mile would require nearly 12 hours! More and more resources were brought, and search planners at the responsible Rescue Coordination Center (RCC) began developing optimal search patterns based on all the known information. They calculated the drift vectors, predicted Jacob’s location, and designed searches attempting to maximize the effectiveness of each.
We threw three boats, two helicopters, and one jet at the case… all with nothing found. We continued some searches into the night. If nothing else, we thought, Jacob may see the navigation lights of the searching craft, and his will to live would be strengthened knowing we were still looking; we hadn’t given up. Morning came, and more searches were planned, but faith in a positive outcome began to wane. I picked up the phone to call off the search when another line rang….
“Coast Guard search and rescue, state the nature of your emergency,” I repeated this greeting a hundred times daily. The other end of the line didn’t skip a beat, “Yeah, this is Mike down at the beach. Some guy just swam out of the Gulf, completely naked, asked for a bottle of water, and said you might be looking for him.” The floor seemed to open up and swallow me whole. We missed him…we ultimately failed. But how?
A few days later, I met Jacob at his home. There were no smiles exchanged. After letting me in, he settled on his couch and pointed to his chest; he was wearing a strobe light. He said, “No one will ever fail to see me again.” By this point in my career, I had conducted several survivor debriefs, but nothing prepared me for what I was about to hear. From a purely planning perspective, it turned out that we didn’t fail. We put aircraft and boats directly on Jacob’s position time and time again. He recounted aircraft flying directly over him, and not just once. One flew so close that he could look up and see that the pilot had his visor down! It was the same story for the surface craft. He noted that at one point, the boats drove so close that they made a nearly complete circle around him, which allowed him to identify the characteristics of the crew on deck. Even the jet went right over the top of Jacob. Yet, despite being well within the expected detection range of the human searcher, everyone missed Jacob.
The recounting of that night was gut-wrenching. Jacob experienced despair like few of us do. More than once, he considered dying, allowing himself to sink below the waves, only to change his mind at the last second and fight for a few more minutes. The night was an eternity, and he wondered if he had died and this was hell. How could they not see him? The surface craft sent to serve as an inspiration looped around his position, close enough to make him think he would be run over, yet time and time again, he was not detected. Suffice it to say that humans are not effective searchers.
But why is this? There are two fundamental reasons. First, the human eye is not designed to scan a relatively uniform area. We humans have exceptional resolution, the equivalent of a 576-megapixel camera, but that resolution drops to a tenth of that, only 20 degrees off the center of our vision. While our total field of view is an impressive 180 degrees, we can only see fine detail in the center few degrees. Unless you’re staring right at it, you’ll miss it.
Further, adjusting our near-to-far vision focus can take up to two seconds. Consider that pilot with his visor down that had flown right over Jacob and missed him. If he had turned his eyes down to view his instrumentation and then refocused on the surface of the water, resuming the search, there would have been up to two seconds where his vision was refocused and thus unable to see the fine detail.
Consider that a person in the water is generally only visible by their head, unless they are fortunate enough to be wearing a survival suite. From a distance of .1 nautical mile, that head has a relative size of only 50 thousandths of an inch…thinner than a dime. That is a tiny object to detect while your vision refocuses on distance.
And if that weren’t bad enough, one must also combat empty field myopia. This temporary condition occurs when we view a relatively empty field, like the open ocean. It causes our vision to blur or refocus automatically on an area only a few meters before us. Searchers are unaware of this, failing to see specific details within their field of view.
Second, while the human eye is ill-suited to the job of searching, its most significant detraction is that it is attached to the human brain. That grey matter between your ears is infinitely variable, making consistent and repeatable results nearly impossible. Volume Three of the International Aeronautical and Maritime Search and Rescue (IAMSAR) manual refers to this as the vagaries of the mind. “We can “see” and identify only what our mind permits us to see.”
The factors that negatively impact a human’s ability to detect a search object are seemingly endless: vibration of the search vehicle, atmospheric and environmental variations, glare, fatigue during extended searches, and even a searcher’s overestimation of their ability are only some of the more common impediments. I’ve had dozens of coxswains and pilots tell me, “If they were out there, we would have found them,” only to get a report later that a survivor had been found and told of being overflown. If this is in doubt, one only has to read the harrowing and tragic tale of Nick Schuyler and his friends, who capsized and were adrift in what was then the Gulf of Mexico. Nick recounts, “A giant Coast Guard plane flew directly over us…. Maybe the pilots were distracted at that particular moment they flew over us, I thought grimly. Maybe one of the Coast Guards was going to the bathroom or not looking down.” Nick was ultimately the sole survivor. At least one of Nick’s friends was still alive when they were overflown. Nick would watch him die knowing they could have been saved if only they had been seen by that Coast Guard plane.
Case closed: humans are not effective searchers.
So, what is the solution? Readers have undoubtedly been screaming, “Artificial intelligence!” Well slow down AI aficionados. Yes, the incredible power of modern computing offers some solutions, but only a piece. Let’s back up for just a moment and define the challenge. The goal of planning a search is to maximize the probability of locating survivors in the minimum amount of time, given finite resources. Regardless of how good or bad a sensor is, we must always endeavor to make maximally efficient use of it.
Every second we waste by not employing our finite resources efficiently is another second that Jacob is treading water or Nick is sitting on his capsized vessel getting pounded by waves. Thus, the challenge is twofold: 1) Field the best sensor and detection systems practicable by current technology, and 2) employ those sensors and detection systems to optimal effect. Simply deploying better sensors is not enough, not by a long shot.
So, what does it take to plan an optimal search? This is the domain of the niche science known as Search Theory. In World War II, the Allied Forces faced the problem of detecting and destroying enemy submarines, which wreaked havoc on the ocean shipping lanes. After the war was won, their research was published in a classified report, A Theoretical Basis for Methods of Search and Screening, 1946. Long since declassified, the report is now considered the founding document in modern Search Theory and is taught in operations research to this day, albeit in an increasingly dwindling number of venues. Beginning in the 1970s, a small group of lifesaving mathematicians and other scientists have been the scarce few torchbearers of the science. They are primarily responsible for all modern search planning methodologies worldwide.
To simplify the relevant component of Search Theory for the matter at hand, to plan an optimal search (one that is maximally efficient), you absolutely must have detailed quantitative measures of a sensor’s performance over its entire effective range for every possible combination of sensor, environmental condition, search object, and search characteristic. That is a vast number of scenarios. For example, as many as 25 different environmental factors influence detection, from humidity, sea state, moon illumination, etc. Concerning the search object, a person in the water without a lifejacket is much less detectable than someone in a bright orange survival suit. In the business of saving lives, the details matter.
Unfortunately, gathering such massive data requires substantial empirical experiments. The first such experiments conducted in the 1970s collected hundreds of detection opportunities to extrapolate and interpolate visual performance parameters that are still in use today. The most recent experiments, conducted in 2012, gathered over 3,500 data points to evaluate the performance of then-modern electro-optical sensor systems. Regrettably, the sensor tested has already been decommissioned, rendering those results useless. Herein lies the crux: empirically testing new sensors as they develop is prohibitively costly and time-consuming. This is particularly disheartening because, within the U.S. Coast Guard’s arsenal of 20 sensors available for search and rescue, they only have sufficient data to plan optimal searches for three. Imagine one of the world’s leading search and rescue organizations unable to plan efficient searches in most circumstances. This is not a uniquely U.S. challenge; no search and rescue authority has resolved this problem.
Enter the Private Sector.
AI is not a magic pill, although it most certainly opens the door to a solution set. First, AI, such as an autonomous detection and recognition systems, must be trained with good data. No data, no AI. The better the data, the better the AI. While the world was in the throes of COVID, a small Scottish technologies company, Zelim Ltd., was collecting what has become the largest archive of detection data of common maritime objects, to include persons-in-the-water, in the world, more than 6 million labeled frames, from all angles, in all conditions, and from various heights and speeds. Contrasting this data with the most recent traditional experiment noted above, that’s an increase of 140,000% more data!
Second, there must be a solid methodology to evaluate the performance of an AI system. Recall from above that if you don’t know the performance of a sensor or detection system, you cannot, even in theory, deploy it optimally. Therefore, if you do not have a validated and repeatable methodology to assess the performance of an autonomous detection system, it will always be used in a suboptimal way, ultimately resulting in fewer lives saved.
Zelim Ltd. entered into a collaborative research and development agreement (CRADA) with the U.S. Coast Guard’s Research and Development Center (RDC) specifically to address this challenge. The objective of the agreement was to “collaboratively develop methods to evaluate the effectiveness of … autonomous detection and tracking systems to allow for the integration with accepted search planning systems and methodologies.” This objective required the Coast Guard and Zelim to collaboratively “evaluate scientifically a method for objectively determining Effective Sweep Width (ESW) to improve the Probability of Detection (POD) of target objects in the marine environment when using Artificial Intelligence models (AI).”
The impetus for this agreement was the realization that all SAR planning techniques employed worldwide to date are reliant on human-in-the-loop detection of search objects, whether unassisted through the naked eye or interfacing with the output of a sensor. Autonomous detection systems present a categorical shift in search planning and execution, removing the human observer and the innate variability thereof from, at least, the initial detection and identification of a potential target. The CRADA sought to understand better the dynamics of planning an effective and efficient search whereby the primary observer was an autonomous detection system, such as a deep learning algorithm, versus a human being.
This agreement marks a pivotal shift in search and rescue–one that could redefine how we save lives, not only at sea, but on land and one day in space as well. Results thus far bode exceptionally well for the integration of autonomous detection systems with the life-saving missions. A search planning methodology has emerged that is currently undergoing rigorous peer review and validation, and, once validated, will arguably mark a significant step forward in the continuing progression of Search Theory.
The question now is not whether we will integrate artificial intelligence into search planning, but how quickly we can refine our methodologies and integrate this new technology to ensure no one else has to endure what Jacob did. Had this technology existed then, Jacob’s story might have been very different. The next Jacob doesn’t have to be a near-miss, we have the tools to change that.
For more information on this scientific advancement contact the Founder and CEO of the International Association of Search and Rescue Coordinators (IASARC), Matthew Mitchell, matthew.mitchell@iasarc.org or the CTO of Zelim Ltd., Doug Lothian, Doug.Lothian@zelim.com.
Notable Links
To support IASARC’s lifesaving mission now | Donate
Follow IASARC | LinkedIn
