Shooting Log(Fish) in a Barrel

Burkard Polster and Marty Ross

The Age, 2 December 2013

Our regular readers will be aware that Maths Master columns vary considerably in style and content: on occasion we work hard to try to explain some pretty mathematics; at other times we're happy to sit back and take easy potshots at mathematical nonsense. Both types of column are fun and rewarding, at least for us (and we hope for others).

One potshot column that we particularly enjoyed putting together concerned the SAFE index, a naive approach to measuring the vulnerability of endangered species. Though undoubtedly well-intentioned, the SAFE index is an archetypal example of mathematical fancy dress camouflaging the thinness of the underlying ideas.

An unexpected benefit of writing the SAFE index column was our meeting Associate Professor Michael McCarthy. Michael is an ecologist at The University of Melbourne, working on conservation biology and ecological modelling, and he clearly grows impatient with some of the fluff that appears in his field. Michael and his colleagues (and others) had submitted their own, detailed critique of the SAFE index. Michael has also performed yeoman's work analysing the Victorian Coalition's groundbreaking study into the effect of cows trampling on fragile alpine vegetation.

Recently, Michael alerted us to a new venture, the Ocean Health Index (OHI). True to its name, the OHI is intended to be a measure of the health of the Earth's ocean ecosystem. It was launched in 2012 with an article in the prestigious journal Nature and is supported by a large international team of respected marine scientists.

So, The OHI has to be a good thing, right? Perhaps. However Michael and a number of his colleagues have some pretty large nits to pick. (Nature declined to publish their letter in response to the original article, though an independent critique and authors' reply have just recently appeared in the journal.)

The OHI attempts to measure the status of ten public goals for a healthy ocean. Included are specifically physical and biological goals, such as water cleanliness and biodiversity, as well as human-centred goals such as tourism. Each of the ten goals are scored out of 100 and then the scores are averaged to give the overall Ocean Health Index. The Index is measured for each country as well as for the World overall. For example, Australia's OHI in 2013 is 70 (43rd in the World), compared to the Global Index Score of 65.

What's wrong with that? Plenty, some of it obvious. Michael is diplomatic, suggesting that the meaning of ocean health is "unclear". Your less diplomatic Maths Masters will be more blunt: the Ocean Health Index is pointless.

To begin, one might conceivably attempt to distill the state of "water cleanliness" or "tourism" to a single number, but then what? If water cleanliness receives a score of 90 and tourism a 10, what can the average of 50 possibly indicate? What if the numbers were reversed?

There is clearly little meaning in the average of ten such numbers. Moreover, the extensive interrelatedness of the goals being scored muddies the little meaning that might exist. It follows that the Ocean Health Index is not a true measure of anything; it is just faking simplicity.

Furthermore, as Michael and others have noted, the ten individual scores are in themselves problematic, and intrinsically so. The Ocean Health Index has the laudable purpose of measuring ocean health with respect to human interaction. However, the computational effect is that each score is an intermingling of physical and biological measures of the current ocean state with human pressures upon that state. The result is that even the individual scores are scores of nothing in particular, neither fish nor fowl.

It gets sillier. Michael has alerted us to some truly bizarre arithmetic lurking in the Index.

One of the ten goals in the OHI is food provision, being the "the amount of seafood captured or raised in a sustainable way". This is calculated as the weighted average of x_FIS, a score of "fisheries" health, and x_MAR, a "mariculture" (ocean fishing) score.

The Nature article does not indicate how x_FIS and x_MAR are calculated, however some details are provided in the 2012 supplementary materials (page 99). We're simplifying, but x_FIS is essentially a ratio, a comparison of the number of fish harvested in a year to the maximum sustainable harvest. To calculate x_MAR, one first calculates the number of fish caught divided by the area of the region being fished. (The detailed computation involves the consideration of different fish species and the incorporation of estimates of sustainability.) Then, one takes a logarithm, giving in effect the formula

Why the +1? God only knows. (In 2013 the +1 was eliminated from the formula.) Why the logarithm? Again, God only knows. And then, what to make of the overall food provision score: how does one average x_FIS , a unitless ratio, with x_MAR the logarithm of (1 more than) a fish density? We doubt that even God knows the answer to that one.

Mathematical modelling is difficult and subtle; one should avoid declaring hard and fast rules, and one should be wary of being too or too quickly critical. However, it is a fundamental rule to not sum quantities of different physical type. For example, if a car travels 200 meters in 7 seconds we don't then create a Frankenstein quantity of 207 met-secs. Furthermore, though logarithmic scales can be enlightening, we have never seen logarithms employed in the manner attempted in the Ocean Health Index.

We are sure, despite its glaring flaws, that the Ocean Health Index is well-intentioned and incorporates good and important research. It may be salvageable. However, we cannot understand why the creators of the Ocean Health Index chose to include such clunky mathematics in their model. Nor can we understand why Nature chose to accept and publish the article in that form.

Whatever the explanation, for us the expression "appeared in Nature" no longer has such an authoritative ring.

Burkard Polster teaches mathematics at Monash and is the university's resident mathemagician, mathematical juggler, origami expert, bubble-master, shoelace charmer, and Count von Count impersonator.

Marty Ross is a mathematical nomad. His hobby is helping Barbie smash calculators and iPads with a hammer.