Wednesday 9 April 2014

Beware the Polls of March


Note: This Post is unfinished. Aiming for an in-depth analysis, but find I now don't quite have the time available to do it justice. So I've temporarily abandoned it about a third of the way through. I may well come back and finish it off at a later date assuming I have enough spare time. But by all means have a read - you should get the gist of my central argument  - swordfish.

The Gazette begins its 20th year of publications

How did so many of New Zealand's leading political journalists and bloggers manage to get the latest round of Opinion Polls so hopelessly wrong ?

And did this profound misreading of public opinion - reaching a crescendo around late March/early April - create a bandwagon effect partly responsible for the swing back to the Right in the latest Roy Morgan ?

It speaks volumes about the current state of political journalism in this Country that in the very week  that new guidelines on the reporting of opinion polls were released, leading journalists continued to grossly misinform the voting public about the latest trends in party support.

Despite four consecutive opinion polls in March revealing a clear swing away from National and the broader political Right  (with the percentage point gap between Left and Right Blocs almost halving), most mainstream political journalists remained stuck in a kind of timewarp: continuing to repeat the February narrative of a "soaring" National Party, "defying political gravity" and "able to govern alone". All of which was starkly contrasted with a "luckless" Labour Party sinking ever deeper into the mire.

From mid-March to mid-April, as successive polls began to confirm: (1) a significant fall in National and Right Bloc support, (2) a somewhat milder lift in Left Bloc support, and (3) a Labour Party remaining pretty much where it had been throughout the previous 8 weeks (on a steady 31% average), the mainstream media was busy suggesting the polar opposite. Labour's support was apparently "slipping", "sliding", "dipping", "dropping", "eroding", "trending down", "going backwards", had "taken a dive", was "unappetising", "morale-sapping", "woeful", "in the doldrums" and "plumbing the depths". As a result of this alleged downturn, the Party's tide  remained "well and truly out", so much so that it had "a mountain to climb before September." Overall, according to the media consensus, if there was a core dynamic underpinning the March polls it was that they were "bad news for Labour" (a phrase finding favour with more than one commentator over this period).

National's poll ratings, meanwhile, were almost always cast in either positive or neutral terms, despite the seemingly obvious loss of support. Well after the comparatively poor results started coming in, a number of senior journalists were still insisting it was all "more good news for National." The Party was apparently maintaining "a commanding lead", was "riding high in the polls", had "a big lead over Labour", "a chasm", "too big a gap to bridge", "a one-sided affair", enjoyed "stunning poll ratings", was "going up", "climbing", "going forwards", was "looking strong and confident" - indeed was "in rampant form" - and continued to be "on target for a clear majority." The Government, we were told, suffered "precious little breadth or depth of anger", indeed they were "enjoying something of a second honeymoon with the voters." Clearly, the public were "sticking to the devil they know" and "not wanting to mess with success."

The Oravida and Te Kohanga Reo sagas had certainly "not dented" National support, had "done nothing to harm Key's and National's popularity", and "hadn't affected National at all." Labour's efforts to stoke these scandals had "demonstrably failed" and caused "little damage", with the Party unable to "drag down National's popularity." As the Nats continued to take a rather conspicuous hit in the polls, journalists and bloggers insisted the Party was "widely regarded as competent and clever" and continued to have "that vice-like grip on the centre voters." The poll trend, we were assured, was "in National's favour", so much so that "life on Planet Key couldn't be rosier."

Certainly there were a few exceptions (discussed below) to all of this gross misinterpretation (bordering at times on hagiography). Exceptions that multiplied after the release of the fourth consecutive bad poll for National on April 3. Even after that point, however, far too many in the media and blogosphere remained blind to the obvious swing in public opinion and to the fact that this trend had been manifest for a number of weeks.

In this post, I want to take a forensic approach in order to clarify the precise mechanics of how and why so many in the media got things so badly wrong. And towards the end, I'll ask if the swing to the Right recorded in the latest Roy Morgan is the corollary of a relentless media narrative that saw National wrongly cast as the "stunning" winner - and Labour the "woeful" loser - of the battle for public popularity in March. Did the media's misplaced cheerleading create what political scientists would call a bandwagon or snowball effect ?

Poll Trends of February and March


No one, as far as I know, has firmly established (in solid numbers) the poll trends of the last 3 months. Anecdotal evidence suggests many journalists and others in the commetariat rely largely on National-aligned blogger, David Farrar, to do their number-crunching for them. Farrar is certainly happy to oblige when the numbers favour National. When they don't, however, I would suggest his number-crunching often borders on obfuscation.

More analysis here. Details of Farrar's problematic analysis of poll trends etc

(Note to anyone who's managed to get this far: I set out some of my criticisms of David Farrar's recent monthly poll analyses (Kiwiblog) in a comment I made on The Standard here.......... http://thestandard.org.nz/open-mike-31052014/#comment-822766)




Table 1   Monthly Poll Averages (2014)
(Gap =  (1) Nat/Lab (2) Right/Left )

January     Nat 45 Right Bloc 49  Lab 33  Left Bloc 46  NZ First  5  Gap  (1) 12  (2) 3
                                                                                                                 

February     Nat 50  Right Bloc 53 Lab 31 Left Bloc 42  NZ First  4   Gap  (1) 19  (2) 11
                                                                                                                   

Diff                          + 5                     + 4             - 2                       - 4              -1

March      Nat 45  Right Bloc 50  Lab 31  Left Bloc 44  NZ First  5   Gap (1) 14  (2) 6
                                                                                                                

Diff                    - 5               - 3              =          + 2              + 1




More analysis here. Summary/Overview of Monthly Poll Averages.




Table 2     Individual Polls (2014)
(Gap = (1) Nat/Lab (2) Right/Left) 

February (5 Consecutive Polls Feb-Early March)

Fairfax         Nat 49  Right Bloc 53  Lab 32  Left Bloc 42 NZ First 4

Roy Morgan      Nat 48   Right Bloc 51   Lab 30  Left Bloc 43  NZ First  6

One News          Nat 51  Right Bloc 54  Lab 34  Left Bloc 42  NZ First 3

Roy Morgan      Nat  49  Right Bloc 54   Lab  31  Left Bloc 42  NZ First  5

Herald-Digi          Nat 51  Right Bloc 53  Lab 30  Left Bloc 43  NZ First 4

February                        50             53                       31                     42                4
Average


March (4 Consecutive Polls Early-Late March)

Roy Morgan         Nat 46  Right Bloc 50  Lab 32  Left Bloc 46  NZ First  4

One News        Nat  47   Right Bloc 50   Lab  31  Left Bloc  42   NZ First 7

3 News          Nat 46   Right Bloc 51   Lab  31  Left Bloc 44   NZ First 5

Roy Morgan      Nat  43   Right Bloc 48    Lab  32   Left Bloc 46  NZ First  6

March                  45                        50                 31                       44                      5
Average


More analysis here. Summary of Individual Poll trends.

Overview of MSM's analysis of February Poll trends (4-6 paragraphs)

Overview of MSM's gross misinterpretation of March Poll trends and the way they employed a contrasting methodology (relative to their February approach). In each case, the methodology underlying their analysis favoured National. In February, the method of analysis was legitimate and rightly highlighted the swing to National and the Right. In March the highly restrictive mode of analysis chosen by leading journalists was illegitimate, resulting in a significant silence on / downplaying of National and the Right Bloc's clear decline and the Left Bloc's partial revival.

Outline my theories on how they got things so wrong:

- The Echo effect

- The Time-lag effect

- The Media Exclusive effect

- The Partisan effect.

Analyse Early March Roy Morgan, then Late March One News, then Late March 3 News.

We've seen, then, that the Late March 3 News Reid Research Poll confirmed the trends highlighted in the previous two polls (Early March Roy Morgan and Late March One News). Support for National and the Right Bloc clearly fell relative to their average over the 5 consecutive polls of February - early March, while Labour remained steady and the Left Bloc and (to a somewhat lesser extent) NZ First were up. Clearly, the Oravida and Te Kohanga Reo scandals had taken a toll.

So, how did TV3 interpret the results ? Announcing the 3 News Reid Research Poll on the 30 March 6pm bulletin, Gower told viewers: "National are up to 45.9%, but Labour are down - quite a bit more actually - to 31.2% - down 2.3. The Left Bloc slipping, the Greens they are down 1.2 as well...National and its support partners would have a clear majority. The Left though would not get close - 56 seats - they don't have a shot on that."

All of which, according to Gower, meant the Oravida scandal "has done nothing to harm Mr Key's popularity" (Gower quotes Key to the effect that 'It's been a bad few weeks for National but no impact). Cunliffe, Gower concludes, "got no traction from Mr Key's woes whatsoever. In fact, it is the opposite." Cunliffe's "troubles with the trust" have him "going backwards." Conversely, "the Oravida saga has simply not dented Mr Key. He is going forwards" and continues to have a "vice-like grip on the centre voters." Overall, "Labour and Mr Cunliffe are going in the wrong direction." (Gower also posted an on-line article on the 3 News website headlined 'Poll, National Up despite Oravida saga').

3 News followed up this core report on the following day's 6pm News (and on their website) with 'What's keeping National high in the Polls ?'. According to the on-line version, "The latest 3 News Reid Research Poll has more good news for National and bad news for Labour...So after five-and-a-half years as the Government, what is keeping National riding high in the polls ?" In the 6pm bulletin, News anchor Michael Wilson introduces the item by asking Gower: "Now National's riding high isn't it ?...not quite as high as the Herald Poll...but our Poll's still showing a rise for National." Gower: "Yeah, National going up...And that is a pretty astonishing result given that during the polling period Key was on the back foot almost the entire time over the Oravida controversy with Judith Collins and all those other connections. But it hasn't affected National at all !" The explanation, according to Gower, was pretty simple: "Voters obviously see things like Oravida as a bit of a distraction. The economy - as you know, Michael - is picking up and people are pretty happy with the way the National government is going."