Weapons of Math Destruction

The 30 November 2015 book draft, "Bayesian Methods in the Search for MH370", authored by members of the Australian DSTG (Defense Science and Technology Group) is an interesting document. Hereinafter I will refer to it as the BM. It is especially interesting to me. This short document addresses two issues in the BM that have received commentary, without clarification, by a number of independent investigators. I say without clarification since I am not aware of any published attempts to clarify them. Of course, I may have missed them, and apologies in advance if I did miss them.

The first issue has to do with measured BFO errors. There has been quite a lot of discussion relative to the accuracy that might be expected from the BFO data contained in the Inmarsat logs for the last flight of MH370. These discussions have been primarily between myself and Mike Exner of the Independent Group, IG. Until now there has been no way to quantify these errors directly since no data was available. The BM changes that, and Figure 5.4 from the BM is reproduced (without permission) below.











The BM comments on Figure 5.4 as follows:



The BM offers no additional data to support the claim of "geographic dependency" such as repeated flights exhibiting a pattern correlated with the data in Figure 5.4.

When I first read the BM, the data in Figure 5.4 did not really surprise me at all, and had questions not been raised by others, I would have probably not have given it further consideration. Without supporting data I mentally discarded the "geographic dependency" explanation, and applied my own explanation which is simply drift in the AES -> 3F1 -> Perth oscillator chain. I have long estimated this drift to be on the order of +/- 5Hz over the duration of the MH370 flight. Cut and paste below from a much earlier post in this blog, "BFO Errors (again)".

















The errors shown in Figure 5.4 of the BM are certainly this order of magnitude. My estimate was based on a few decades of experience with oscillators of the type used in the Inmarsat chain. It was not based on a knowledge of specific part numbers and manufacturer's specifications of the oscillators used in the hardware associated with the MH370 BFO data. There is not much doubt in my mind relative to the source of the errors in Figure 5.4, but I stand ready to retreat from my explanation should the DSTG present data supporting their "geographic dependency" claims.

As a parting comment relative to this first issue it important to realize that BFO errors on the order of 1 Hz produce position errors along a range ring on the order of a degree of latitude per Hz (some 60 nautical miles of error).

The second issue has to do with the "anomalous" paths show in Figure 9.2 of the BM, and reproduced below.



































The commentary offered in the BM relative to these anomalous paths is pasted below.



Basically if one draws a radial from the sub-satellite point to the aircraft flight path, the BFO data becomes ambiguous relative to a continuation along the flight path and a continuation at an angle equal to and on the "other side" of the radial to the sub-satellite point. While this statement is not quite true for satellite positions far from the extreme ends of the satellite "dither" (where the Doppler associated with satellite movement is significant), it is exactly true for times such as 19:40 data point in the MH370 log when the satellite was virtually standing still at the extreme end of its wobble. As some readers might recall, I pointed this phenomenon out on the Wise blog some time ago. It was met with the howls of protest normally associated with my blogs there. No matter. It is what it is (or is not).

Note: 12 February 2016

The text above was "struck" because it is incorrect. The bifurcations in the path for BTO are, as stated, about a radial to the sub-satellite point. The bifurcations in the paths for the BFO values are not about a radial to the sub-satellite point. They are about a N-S line, a meridian. My apologies to anyone who was misled or wasted time due to this error on my part. 

The explanation from the BM above could have been worded better, but suffice to say that the three examples of flight path deviation are simply illustrative samples from a continuous "fan" of possible paths - "the true underlying pdf has support all the way around the arc. Without dynamic constraints the location of the peak of the pdf is simply a function of the measurement noise".  What the quoted snippet in italics means is that without the assumption that the aircraft is flying in a straight line ("dynamic constraints"), the plane could be just about anywhere on the last arc while satisfying BTO and BFO.

Summary

The Inmarsat system was not designed to be a navigation system. It was designed to be a communication system. While I acknowledge the huge amount of effort put forth by various parties (including myself), that effort does not change the physics of the system. The simple truth is that this huge effort is akin to turning a sow's ear into a silk purse. It is certainly a very worthwhile effort, but it would be a mistake to think that the Inmarsat data can, in and of itself, be used to accurately estimate the terminus of MH370.

The DSTG BM adds little new to boost confidence in a terminal location. It is very "IGish" - assume a flight dynamic,  find a path for your assumed dynamic that satisfies the ISAT data, and then conclude you have high confidence in the terminal location.














Duncan Steel