Do You Believe in Polls After All? An Experimental Study on Credibility in Political Opinion Polls

Table of contents

Bibliographic information

Cover of Volume: M&K Medien & Kommunikationswissenschaft Volume 73 (2025), Edition 4
Open Access Full access

Medien & Kommunikationswissenschaft

Volume 73 (2025), Edition 4

Authors:
Publisher
Nomos, Baden-Baden
Publication year
2025
ISSN-Online
2942-3317
ISSN-Print
1615-634X

Chapter information

Full access

Volume 73 (2025), Edition 4

Do You Believe in Polls After All? An Experimental Study on Credibility in Political Opinion Polls

Authors:
ISSN-Print
1615-634X
ISSN-Online
2942-3317
Preview:

This study examines the factors influencing the perceived credibility of political opinion polls in Austria. Using a 3x2 between-subjects design, we conducted an online experiment with 1,600 participants, quota-sampled to match key demographic characteristics of the Austrian population (age, gender, and federal state). The design was validated through an a priori power analysis, indicating a robust 95 % statistical power. Mean comparisons revealed no significant differences in poll credibility across media source conditions; however, perceived competence and warmth significantly mediated this relationship. The public service broadcaster (ORF) was rated as more competent and warmer than both the tabloid medium (OE24) and the control group. Interestingly, providing detailed survey information reduced perceived poll credibility, highlighting the need for improved poll literacy. Voting intention (motivated reasoning) and perceived media competence emerged as strong predictors of poll credibility. This study offers novel insights into the credibility of political polls from a Central European perspective.

Bibliography

  1. Abele, A. E., & Wojciszke, B. (2007). Agency and communion from the perspective of self versus others. Journal of Personality and Social Psychology, 93(5), 751–763. https://doi.org/10.1037/0022-3514.93.5.751 Open Google Scholar
  2. Abele, A. E., & Wojciszke, B. (2013). The Big Two in social judgment and behavior. Social Psychology, 44(2), 61–62. https://doi.org/10.1027/1864-9335/a000137 Open Google Scholar
  3. Appelman, A., & Sundar, S. S. (2016). Measuring message credibility: Construction and validation of an exclusive scale. Journalism & Mass Communication Quarterly, 93(1), 59–79. https://doi.org/10.1177/1077699015606057 Open Google Scholar
  4. Behr, D., Braun, M., & Dorer, B. (2015). Messinstrumente in internationalen Studien (GESIS Survey Guidelines). https://www.gesis.org/fileadmin/admin/Dateikatalog/pdf/guidelines/messinstrumente_internationale_studien_behr_braun_dorer_2015.pdf [12.11.2025]. Open Google Scholar
  5. Bhatti, Y., & Pedersen, R. T. (2016). News reporting of opinion polls: Journalism and statistical noise. International Journal of Public Opinion Research, 28(1), 129–141. Open Google Scholar
  6. Cuddy, A. J., Fiske, S. T., & Glick, P. (2008). Warmth and competence as universal dimensions of social perception: The stereotype content model and the BIAS map. Advances in experimental social psychology, 40, 61–149. https://doi.org/10.1016/S0065-2601(07)00002-0 Open Google Scholar
  7. Christofoletti, R. (2024). Trust in media and journalism credibility in the sea of misinformation. The International Review of Information Ethics, 33(1). Open Google Scholar
  8. Curry, A. L., & Stroud, N. J. (2021). The effects of journalistic transparency on credibility assessments and engagement intentions. Journalism, 22(4), 901–918. https://doi.org/10.1177/1464884918807591 Open Google Scholar
  9. Denner, N., Brocke, J., & Joeckel, S. (2016). Same result, different effect: The credibility attribution to public opinion research results. SCM Studies in Communication and Media, 5(2), 173–196. Open Google Scholar
  10. Fawzi, N., & Krämer, B. (2021). The media as part of a detached elite? Exploring antimedia populism among citizens and its relation to political populism. International Journal of Communication, 15(2021), 3292–3314. Open Google Scholar
  11. Fiske, S. T., Cuddy, A. J. C., Glick, P., & Xu, J. (2002). A model of (often mixed) stereotype content: Competence and warmth respectively follow from perceived status and competition. Journal of Personality and Social Psychology, 82(6), 878–902. https://doi.org/10.1037/0022-3514.82.6.878 Open Google Scholar
  12. Fiske, S. T. (2018). Stereotype Content: Warmth and Competence Endure. Current Directions in Psychological Science, 27(2), 67–73. https://doi.org/10.1177/0963721417738825 Open Google Scholar
  13. Gadringer, S., Holzinger, R., Nening, I., Sparvier, S., & Trappel, J. (2019). Digital News Report Network Austria. Detailergebnisse für Österreich. Universität Salzburg. https://digitalnewsreport.at/wp-content/uploads/2023/05/DNR_2019.pdf [12.11.2025]. Open Google Scholar
  14. Gadringer, S., Sparviero, S., Trappel, J., & Reichenberger, P. (2023). Digital News Report Network Austria: Detailergebnisse für Österreich. Universität Salzburg. https://zenodo.org/records/8008752 [12.11.2025]. Open Google Scholar
  15. Hanitzsch, T., & Vos, T. P. (2018). Journalism beyond democracy: A new look into journalistic roles in political and everyday life. Journalism, 19(2), 146–164. https://doi.org/10.1177/1464884917725163 Open Google Scholar
  16. Hayes, A. F. (2018). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. Guilford Press. Open Google Scholar
  17. Hellmueller, L., & Trilling, D. (2012). The credibility of credibility measures: a meta-analysis in leading communication journals, 1951 to 2011. WAPOR Hong Kong 2012: paper presentation. World Association for Public Opinion Research/Public Opinion Programme, The University of Hong Kong. http://wapor2012.hkpop.hk/doc/papers/ConcurrentSessionsV/VD/VD-3.pdf [12.11.2025]. Open Google Scholar
  18. Jackob, N., & Hueß, C. (2016). Communication and persuasion. In M. Potthoff (Ed.), Schlüsselwerke der Medienwirkungsforschung (pp. 49–60). Springer Fachmedien Wiesbaden. Open Google Scholar
  19. Kalogeropoulos, A., Suiter, J., Udris, L., & Eisenegger, M. (2019). News media trust and news consumption: Factors related to trust in news in 35 countries. International Journal of Communication, 13(22). Open Google Scholar
  20. Kervyn, N., Bergsieker, H., & Fiske, S. (2012a). The innuendo effect: Hearing the positive but inferring the negative. Journal of Experimental Social Psychology, 48(1), 77–85. Open Google Scholar
  21. Kervyn, N., Fiske, S. T., & Malone, C. (2012b). Brands as intentional agents framework: How perceived intentions and ability can map brand perception. Journal of Consumer Psychology, 22(2). https://doi.org/10.1016/j.jcps.2011.09.006 Open Google Scholar
  22. Kim, T., & Ball, J. G. (2021). Unintended consequences of warmth appeals: An extension of the compensation effect between warmth and competence to advertising. Journal of Advertising, 50(5), 622–638. Open Google Scholar
  23. Krause, W., & Gahn, C. (2023). Should we include margins of error in public opinion polls? European Journal of Political Research, 160(3), 1–23. Open Google Scholar
  24. Kuru, O., Pasek, J., & Traugott, M. W. (2017). Motivated reasoning in the perceived credibility of public opinion polls. Public Opinion Quarterly, 81(2), 422–446. https://doi.org/10.1093/poq/nfx009 Open Google Scholar
  25. Kuru, O., Pasek, J., & Traugott, M. W. (2020). When polls disagree: How competitive results and methodological quality shape partisan perceptions of polls and electoral predictions. International Journal of Public Opinion Research, 32(3), 586–603. https://doi.org/10.1093/ijpor/edaa017 Open Google Scholar
  26. Laustsen, L., & Bor, A. (2017). The relative weight of character traits in political candidate evaluations: Warmth is more important than competence, leadership and integrity. Electoral Studies, 49(4), 96–107. Open Google Scholar
  27. Madson, G. J., & Hillygus, D. S. (2020). All the Best Polls Agree with Me: Bias in Evaluations of Political Polling. Political Behavior, 42(4), 1055–1072. https://doi.org/10.1007/s11109-019-09532-1 Open Google Scholar
  28. Mavridis, C., & Ortuño-Ortín, I. (2018). Polling in a proportional representation system. Social Choice and Welfare, 51(2), 297–312. https://doi.org/10.1007/s00355-018-1177-1 Open Google Scholar
  29. McAllister, I. (2016). Candidates and voting choice. Oxford Research Encyclopedia of Politics. https://doi.org/10.1093/acrefore/9780190228637.013.38 Open Google Scholar
  30. Metzger, M. J., Flanagin, A. J., Eyal, K., Lemus, D. R., & McCann, R. M. (2003). Credibility for the 21st century: Integrating perspectives on source, message, and media credibility in the contemporary media environment. In P. J. Kalbfleisch (Ed.), Communication yearbook 27 (pp. 293–335). Lawrence Erlbaum Associates Publishers. Open Google Scholar
  31. Moy, P., & Rinke, E. M. (2012). Attitudinal and behavioral consequences of published opinion polls. In C. Holtz-Bacha & J. Strömbäck (Eds.), Opinion polls and the media: Reflecting and shaping public opinion (pp. 225–245). Palgrave Macmillan UK. Open Google Scholar
  32. Newman, N., Fletcher, R., Eddy, K., Robertson, C. T., & Nielsen, R. K. (2023). Reuters Institute Digital News Report 2023. https://reutersinstitute.politics.ox.ac.uk/sites/default/files/2023-06/Digital_News_Report_2023.pdf [12.11.2025]. Open Google Scholar
  33. Oleskog Tryggvason, P. (2020). How mediated opinion polls influence political parties: Revisiting the arena framework. International Journal of Public Opinion Research, 32(2), 243–265. https://doi.org/10.1093/ijpor/edz036 Open Google Scholar
  34. Oleskog Tryggvason, P., & Strömbäck, J. (2018). Fact or fiction? Journalism Studies, 19(14), 2148–2167. Open Google Scholar
  35. Peter, C., & Ponzi, M. (2018). The risk of omitting warmth or competence information in ads: Advertising strategies for hedonic and utilitarian brand types. Journal of Advertising Research, 58(4), 423–432. https://doi.org/10.2501/JAR-2018-044 Open Google Scholar
  36. Seethaler, J. (2024). Austria: Losing long-term media market stability. In A. K. Schapals & C. Pentzold (Eds.), Media compass: A companion to international media landscapes (pp. 8-17). Hoboken, NJ: Wiley Blackwell. Open Google Scholar
  37. Stadtmüller, S., Silber, H., & Beuthner, C. (2022). What influences trust in survey results? Evidence from a vignette experiment. International Journal of Public Opinion Research, 34(2), 229. Open Google Scholar
  38. Self, C. C. (1996). Credibility. In M. B. Salwen & D. W. Stacks (Eds.), An integrated approach to communication theory and research (pp. 421–441). Lawrence Erlbaum Associates. Open Google Scholar
  39. Stocké, V. (2003). Einstellungen zu Umfragen. Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS). https://doi.org/10.6102/zis218. Open Google Scholar
  40. Strömbäck, J. (2012). The media and their use of opinion polls: Reflecting and shaping public opinion. In C. Holtz-Bacha & J. Strömbäck (Eds.), Opinion Polls and the Media: Reflecting and Shaping Public Opinion (pp. 1–22). Palgrave Macmillan UK. Open Google Scholar
  41. Tsfati, Y. (2001). Why do people trust media pre-election polls? Evidence from the Israeli 1996 elections. International Journal of Public Opinion Research, 13(4), 433–441. Open Google Scholar
  42. Wojciszke, B. (1994). Multiple meanings of behavior: Construing actions in terms of competence or morality. Journal of Personality and Social Psychology, 67(2), 222–232. Open Google Scholar

Citation

Download RIS Download BibTex

Downloads

Download chapter (PDF) Download complete (PDF)

Share via email

Media