The best laid NAPLAN

by Burkard Polster and Marty Ross

The Age, 9 May 2011

This week’s going to be a blast! Of course, every school week is chockablock with fun, but this week is special: it’s NAPLAN week.

Students all over the country in years 3, 5, 7 and 9 will take the NAPLAN tests. They’re not exactly fun. Tests tend to be scary and the Australian Curriculum, Assessment and Reporting Authority (ACARA), which is responsible for NAPLAN, assures us that “NAPLAN tests are not tests students can ‘prepare’ for”.

This makes the weeks of preparation just undertaken in thousands of schools seem a little silly. It also makes the suggestion that preparing for the tests is somehow cheating seem very silly.

Except, ACARA is clearly fibbing. Any test, unless it is unpredictable to the point of blatant unfairness, can be prepared for. The NAPLAN tests are no different.

In fact, reviewing the tests from previous years, most test questions seem entirely predictable. True, this year ACARA may bowl a bouncer, setting questions of an entirely different style. If so, we very much look forward to the reaction, and ACARA’s subsequent justification.

But who cares anyway? Do the tests actually matter? It’s hard to see how a parent, simply concerned for the progress of their child, will learn much of value. Brief numerical results from standardised tests cannot compare to the continuous, close and caring observation of parents and teachers.

No matter how it may sometimes be sold to parents, NAPLAN is not about evaluating students: it’s about evaluating teachers and schools. It’s carrot and stick, though the stick is big and ill-directed, and the carrot is small and distasteful.

Schools are labelled by their NAPLAN results on the misguided My School website. It is planned to award bonuses to teachers on the basis of NAPLAN tests. There are even reports that education ministers have concocted the featherbrained scheme to use NAPLAN results in determining university entrance. Given the current and proposed applications of NAPLAN results, it is very odd to suggest that preparing students for the NAPLAN tests is cheating, rather than just prudent.

Nonetheless, the NAPLAN tests will be well worthwhile if it encourages students to learn. That is, the NAPLAN tests are only really worthwhile if they can be prepared for, and if the preparation makes good academic sense. So, it is critical to look at what NAPLAN is actually testing. Uh-oh.

To begin, the tests exhibit a fetish for jargon. Luckily, your Maths Masters were exempt from the year 5 tests in 2009, where one question required knowledge of a “reflex angle”: before looking it up, neither Maths Master had a clue. We were also momentarily thrown by other geometry questions, because words such as “cylinder” have a different, more general, meaning for a mathematician.

Similarly, but much more annoying, consider the following question from the 2008 year 9 tests:

We are tired of people employing the weasel word “equivalent”. We wish these testing folk would simply learn to talk of quantities being equal.

However, the above examples are largely mathematicians’ nitpicks. There are others, but not enough for us to complain too loudly. Unfortunately, there’s much more than nits to pick.

The big neon sign of trouble is that NAPLAN claims to test “numeracy”, not mathematics. As to what “numeracy” actually means, we’re back in the land of jargon: it doesn’t mean much of anything. ACARA states that “the main reference for numeracy as well as mathematical knowledge, skills and understanding is the Statements of Learning for Mathematics. The document does not actually contain the word “numeracy”. The document is also a clearer demonstration of mathematical ignorance than of mathematical knowledge.

The “numeracy” game is a cup and balls swindle. While it is commonly claimed that numeracy means something like a basic understanding of mathematics, it is seldom applied that way. In practice, “numeracy” is used to refer to what would be better labelled “functional numeracy”, the bare arithmetic and such needed to get by in life.

There is obviously nothing wrong and much right with teaching functional numeracy. We do not object to most of the NAPLAN questions, and some we quite like. We agree that every student should be able to cope easily with the NAPLAN material (except for the stuff on reflex angles).

However, teaching functional numeracy is not the same as teaching mathematics, and it is not a sufficient basis for teaching mathematics. It has little to do with the learning of abstract reasoning, which is the real value in teaching the majority of school mathematics.

True to its title, the NAPLAN numeracy test is not a mathematics test. There is too much concern for placing questions in real life context and not enough testing of basic skills. There are too many decimals and too few fractions, too much estimation and too little calculation. The number π seems not to exist, and of course the word “theorem” is never mentioned, not even old reliable Pythagoras.

Prime numbers appear to have occurred only once, in a very silly question asking for the factorisation of 2009. (If it’s not clear why this is a silly question, here’s a hint: imagine that this year students are required to factorise 2011, and see what happens.)

If it weren’t otherwise obvious, the functional numeracy bias of ACARA is made clear in their including the use of calculators in the year 7 and year 9 tests. This appalling decision is entirely consistent with the techno-fetishism perverting mathematics education in Australia. Learning for Mathematics, for example, mentions “theorem” three times while the word “technology” occurs 85 times. The document can be more accurately titled Learning to Use a Calculator.

For the teaching of mathematics, and for the basic message of what mathematics is, the NAPLAN numeracy tests are extremely poor. More generally, the NAPLAN tests are a clumsy and misdirected attempt at accountability. Americans are seemingly now coming to the conclusion that, lacking the foundation of clear and good standards, their national testing has been a failure, that it is anathema to a true culture of discovery and learning. When Australians come to a similar conclusion, it won’t be a minute too soon.

Puzzle to Ponder: Below is a question from the year 5 numeracy test from 2008. What would you answer? What if the skates cost $38 rather than $42? Why not just add it up?

Burkard Polster teaches mathematics at Monash, and is the University's resident mathemagician, mathematical juggler, origami expert, bubble-master, shoelace charmer, and Count von Count impersonator.

Marty Ross is a mathematical nomad. His hobby is smashing calculators with a hammer.