Evaluation – Did it really work? Effective public

Evaluation - Did it really work? Effective public relations measurement and evaluation Presented by Dr Tom Watson, Associate Professor in Communication, Charles Sturt University

Flying on one instrument?

Agenda The communications process What is evaluation? Models of evaluation Brief case studies

How PR is practiced Grunig’s four models One-way Press agentry Informational Two-way Asymmetrical Symmetrical

Communications process Presentation Attention Comprehension Acceptance Retention Action - Outcome McGuire (1984)

Communication and evaluation Presentation Attention Comprehension Acceptance Retention Action Output (L1) Out-growth (L2) Outcome (L3) Lindenmann 1993

Types of evaluation Summative – final impact Formative – “in progress” assessment Should be integrated into all stages of programs

What can be measured? Success or failure of strategy Message distribution and reception Understanding, favourability and acceptance Trends

What else? Internal Staff attitudes; value of newsletters, video, team briefings SWOT External Sponsorship impact Issues management and prediction Stakeholder attitudes

Case studies PricewaterhouseCoopers (PwC) accountants and biz advisers DIMIA – Harmony Day Polestar – major printing group Volvo - XC90 launch

PwC Why aren’t we getting coverage? – asked the bosses Merger of PW with C&L not seen as equal – a “takeover” Change in approach

PwC lessons Charts and graphics improve story acceptance and favourable write-up CEO in interviews increases favourability – “voice of the organisation” Benchmark coverage to find “who’s writing about what”

DIMIA Monitored media coverage of events Moved from celebrity to community basis More outside metros Increased participation

Polestar Europe’s biggest print group Going through major changes Monitored media coverage Able to see strategy and tactics working

Polestar – mid-2001 6 Numberofofitems itemscarrying carryingmessage message Number 4 2 0 6 4 2 0 -2 -2 -4 -4 Negatively Negativelyscoring scoringitems items Positively scoring items Positively scoring items -6 -6 -8 -8 -10 -10 Investment Investment Innovation Innovation Employer Employer ForwardForwardthinking thinking Financial Financial security security CTP CTP Message Message Demographics Customer Demographics Customer service service Value Value Environment Environment

Polestar – early 2003 9 8 Number of items carrying message Number of items carrying message 7 6 5 4 3 2 1 0 9 Negatively Negativelyscoring scoringitems items Positively scoring items Positively scoring items 8 7 6 5 4 3 2 1 0 Investment Investment Innovation Innovation Employer Employer ForwardForwardthinking thinking Financial Financial security security CTP CTP Message Message Demographics Customer Demographics Customer service service Value Value Environment Environment

Volvo – new car launch

UK Launch program Competing against BMW and Mercedes Strategy was to have continuing campaign instead of launch “blip” Closely targeted – 25 to 44, mid to high income, degree educated, adventurous Total of 609,000 people (1% of pop.)

PR-led campaign Media relations drove enquiries to internet and dealers More interest than cars to sell; All ordered before sale date announced Advertising ran for one month and cancelled – saving of c. 6m “PR helped sell the required number of cars”

PR works! Volvo XC90 enquiries vs PR impact Volvo XC90 combined metrics PR Impact 600 No. of enquiries 800 4500 700 4000 500 3500 600 400 3000 500 2500 300 400 2000 300 1500 200 200 1000 100 100 500 00 J a n u aJan-02 ry F Feb-02 e b ru a ry Mar-02 Apr-02 May-02 Jun-02 M a rc h A p ril M a y Jul-02 JAug-02 une - PR impact - Total enquiries PR impact 0 Sep-02 Ju ly Oct-02A u Nov-02 gu st SDec-02 e p te m b eJan-03 r O c toFeb-03 ber NMar-03 o v e m b e r Apr-03 D ecem ber Month Total enquiries (personal and web) Orders Advertising spend

Evaluation Three methods of evaluation used: Responses to internet and then dealers Reach of coverage to target audience 96% of target group saw it once; 90% twice Sales

Evaluation – your future Forget the “Holy Grail”, it’s about relationships (Hon & Grunig 1999) Most PR campaigns don’t influence behaviour. (McCoy & Hargie 2003) Avoid dead-ends like management language for evaluation, e.g. ROI (Watson 2005)

References and Reading Hon LC & Grunig JE (1999). Guidelines for measuring relationships in public relations. www.instituteforpr.com Lindenmann, W.K. (1993) An “effectiveness yardstick” to measure public relations success. Public Relations Quarterly, 38 (1) McCoy M & Hargie (2003) Implications of mass communication theory for asymmetric public relations evaluation. Journal of Communication Management 7 (4) McGuire, WJ 1984 Attitudes and Attitude Change in Lindzey, G. & Aronson, E. Ed. The Handbook of Social Psychology Vol.2 3rd Edn, Random House Walker, G (1997) Public relations practitioners’ use of research, measurement and evaluation, Australian Journal of Communication 24 (2) Watson T (2005) ROI or evidence-based PR: the language of public relations evaluation. PRism 3, http://praxis.massey.ac.nz/issue 3.html (forthcoming) Watson T & Noble P (2005) Evaluating PR, Kogan Page, London Watson T & Simmons P (2004) Public Relations Evaluation – Survey of Australian Practitioners, ANZCA04, July 2004

Web sources www.instituteforpr.com www.carma.com.au

Up and away!

Key thinking “PR is concerned with ill-defined problems. It cannot be evaluated by a single method or metric because it does not deal with simple issues ” “Likewise, business in general recognises that a simple, single financial measure does not give an accurate reflection of a company’s true worth.” Gregory & Watson 2006: 1

Policy and resources Research in the UK sponsored by CIPR Produced by Henley Management College Looked at practitioner and academic literature.

CIPR policy 1 PR is part of the management task and is subject to the same disciplines, such as the need to set direction, allocate and manage resources, and monitor progress.

CIPR policy 2 Measurement and evaluation are problematic in all areas of management. Complexity is a key factor and in business, it is difficult to separate out the effect of one area of management such as public relations.

CIPR policy 3 However, the situation can be eased by better planning and objective setting where precise, measurable objectives allow for enlightened judgements of progress in campaigns and their management.

CIPR policy 4 PR can be measured and evaluated in terms of: Contribution to social and economic development Contribution to management, leadership and organisational performance by aiding better decision-making and avoiding mistakes Being a process and part of programme development and implementation The contribution of individual practitioners

CIPR policy 5 Available methods, research based, provide information that is good enough for decision making for PR programmes Existing methods are adequate for measuring the contribution of PR. Each method has limitations, so a raft of measures appropriate to particular situations needs to be employed.

The CIPR resources Go to www.cipr.co.uk/research

Most Admired Companies A range of evaluation metrics were used – between four and eight. Most frequent were informal and/or qualitative such as journalist feedback and discussions with stakeholders. “ ‘Just checking’ everything is on track”. Gregory, Morgan & Kelly 2005