Tuesday, January 24, 2017

Tangled Web

"Oh! What a tangled web we weave, when first we practice to deceive."

We have all heard it before. The above quote is often attributed to "the Bard", but it actually first appeared in a love triangle play, "Marmion", penned by Sir Walter Scott.

In the case of MH370, and the search for answers, it has never been more appropriate. In this case the deceived parties are the ATSB and other organizations involved in the search for the aircraft. They have deceived themselves.

In the last several days we have seen the search for MH370 abandoned (I use the terms abandoned and suspended interchangeably) by the tripartite group of nations - Malaysia, China, and Australia. What we have not heard is the honest reasons (the expense burden, and the realization that the logic and analytics supporting undertaking the underwater search were flawed) behind this decision. Quote below from Darren Chester, Australian Transport Minister.

But Mr Chester said the decision to suspend the search was not just about money.  “This is not primarily about cost at all, it’s about a whole range of factors — we certainly don’t want to raise false hope amongst the next of kin.”
Simply not a credible statement. It needs to be examined in the context of the search events to date.

The ATSB made a decision to commence an underwater search they knew full well would be costly based on the analytics of a search strategy working group, SSWG. This group consists of a bunch of very talented engineers and scientists from industries coupled to the missing aircraft and its electronic suite - Boeing, Inmarsat, Thales,... In parallel with the SSWG independent analysts tossed their hat in the ring. Perhaps the best known group of analysts calls themselves the IG (Independent Group). The SSWG and the IG (and others) all using similar methodologies concluded the aircraft terminated in the vicinity of 38S on the last AES ping ring. I could say a lot about how this conclusion was derived, but suffice to say it included notions such as "Occam's Razor" and "that is the way pilots like to fly airplanes".  In addition to questionable heuristics, the underlying physics of the hardware producing the ping rings and Doppler data, were not sufficiently well understood.

Despite the talents of the SSWG and the IG, the decision of the ATSB was poor both at the time  the decision was made and certainly in hindsight. It was like assembling a group of carpenters, plumbers, and electricians (all talented craftsmen in their own domains) to design and build a nuclear reactor. There was no overlap between the talents of the SSWG and IG and the problem statement. No one in the SSWG or IG had ever participated in an undertaking of the type they were attempting to evaluate - determine the location of a missing aircraft using electronic handshake data and other highly nuanced information.

So as a highly compensated manager of the ATSB you "circle the wagons" (assemble your trusted advisors), and make a decision relative to how to proceed. Mind you, at this time there was no aircraft debris to indicate that the plane had even terminated in the ocean much less at a latitude of 38S. As we all know, a decision was made to commence an underwater search that had gone on for more than two years without a trace of the aircraft being found.

So, now we have the tangled web. If the decision to start the search was justifiably correct, then the decision to abandon the search is logically incorrect. In addition to the data used to justify the original search we now have - debris findings and drift analytics, knowledge of a large area where the plane is not likely to have terminated, and additional information from ground truth derived by the Malay government (radar data, phone registration, pilot simulator data,...). In short, we have more and better information than was available at the time the original decision to start the search was made. There is no logical way the decision to start the search and the decision to abandon the search can both be correct. Mr. Chester would have us believe that both decisions were indeed correct, and that other "stuff" such as the NOK suffering enough and "a whole range of factors" which he neglected to enumerate led to the abandon decision. The reality is that someone needs to be tossed under the bus. 

What should have been done? You do not generally rise to a position of responsibility, and the associated spending authority, by stepping in the piles of crap inevitably situated along everyone's career path. You develop or should have developed a keen sense of self-preservation by balancing decisions with solid analytics and information. In this case there was a clear precedent, AF447. It is well known that this aircraft was found using the consulting services of Metron, an organization with a pedigree in solving problems of this sort. At the very least, spending a few dollars (maybe a $1M or two, I don't really know Metron's fee structure) to get a second opinion would have been prudent and well-advised. It actually does not matter whether Metron's involvement would have materially changed the decision to start the search or where to start the search. What would be changed is the ATSB would not be walking around with shit on their shoes today requiring the transport minister to make lame excuses. My best guess, however, is that Metron would have realized that the problem was under-constrained, and that an underwater search anywhere would entail significant risk.

A prudent manager would fast forward to the possible outcomes, one of which was that the aircraft would not be found in the search area specified. One could readily anticipate the difficulties associated with such a negative outcome, and take early steps to mitigate it. One such step (and retrospectively the correct step) would be not to commence an expensive underwater search based on the information available at the time.