The dynamics of ideological polarization in Russian-language telegram channels: modelling with machine learning methods

The study focuses on the application of modern machine learning methods for analyzing textual data in the context of the dynamics of ideological polarization in Russian-language political Telegram channels during the first half of 2022. This work proposes an approach to classify text messages basing on ideological orientation – conservative, liberal, and communist – allowing researchers to utilize resources more efficiently.

Based on the developed approach, an ideological orientation classifier using ChatGPT was created, demonstrating a high level of consistency in responses between humans and the large language model by evaluating the ideological stance of texts. This indicates that the proposed approach can reduce resource expenditures when conducting textual data analysis.

In the next phase, a sample of 559 popular political Telegram channels, which published 50,000 messages, was analyzed for the dynamics of ideological polarization following the onset of the special military operation. Several models were compared: changes in opinion distribution, group composition, and changes in the proportionality of ideological texts within channels. We concluded that following the initiation of the special military operation, there was a change in ideological polarization, primarily manifested in the strengthening of conservative views, and to a lesser extent, liberal views. Communist views are virtually absent from the popular Telegram space.

This work not only captures the dynamics of ideological polarization but also offers a method for analyzing complex socio-political processes in the Russian-language online environment using large language models. This method is suitable for studying polarization as well as for analyzing other processes based on textual data, significantly reducing the costs of research that require a large number of expert evaluations.

1. Antonova D.E. Ideological rift in the political segment of Russian social media: problems and prospects for overcoming. Vestnik Moskovskogo universiteta. Series 12. Political Science. 2017, N 5, P. 84–89. (In Russ.)

2. Baranov N.A. Main directions of ideological discourse in modern Russia. Politex. 2012, N 2, P. 19–35. (In Russ.)

3. Benkler Y., Faris R., Roberts H. Network propaganda: manipulation, disinformation, and radicalization in American politics. Oxford: Oxford university press, 2018, 472 р.

4. Bright J. Explaining the emergence of political fragmentation on social media: The role of ideology and extremism. Journal of сomputer-mediated communication. 2018, Vol. 23, N 1, P. 17–33.

5. Carmines E.G., D’Amico N.J. The new look in political ideology research. Annual review of political science. 2015, Vol. 18, P. 205–216. DOI: https://doi.org/10.1146/annurev-polisci-060314-115422

6. Corner J. Fake news, post-truth and media–political change. Media, culture and society. 2017, Vol. 39, P. 1100–1107.

7. Del Vicario M., Bessi A., Zollo F., Petroni F., Scala A., Caldarelli G., Quattrociocchi W. The spreading of misinformation online. Proceedings of the national academy of sciences. 2016, Vol. 113, N 3, P. 554–559.

8. DiMaggio P., Evans J., Bryson B. Have Americans' social attitudes become more polarized? American journal of sociology. 1996, Vol. 102, N 3, P. 690–755. DOI: https://doi.org/10.1086/230995

9. Esteban J.M., Ray D. On the measurement of polarization. Econometrica. 1994, Vol. 62, N 4, P. 819–851.

10. Farrell J. Corporate funding and ideological polarization about climate change. Proceedings of the national academy of sciences of the United States of America. 2016, Vol. 113, N 1, P. 92–97. DOI: https://doi.org/10.1073/pnas.1509433112

11. Fiorina M.P., Abrams S.J. Political polarization in the American public. Annual review of political science. 2008, Vol. 11, P. 563–588.

12. Gilardi F., Alizadeh M., Kubli M. ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the national academy of sciences of the United States of America. 2023, Vol. 120, N 30, P. 1–3. DOI: https://doi.org/10.1073/pnas.2305016120.

13. Hirschman A.O. National power and the structure of foreign trade. Berkeley: University of California press, 1980, 194 p. DOI: https://doi.org/10.2307/jj.15976659

14. Huddy L., Feldman S., Taber C., Lahav G. Threat, anxiety, and support of antiterrorism policies. American journal of political science. 2005, Vol. 49, N 3, P. 593–608.

15. Jost J.T., Glaser J., Kruglanski A.W., Sulloway F.J. Political conservatism as motivated social cognition. Psychological bulletin. 2003, Vol. 129, N 3, P. 339–375.

16. King G., Schneer B., White A. How the news media activate public expression and influence national agendas. Science. 2017, Vol. 358, N 6364, P. 776–780. DOI: https://doi.org/10.1126/science.aao1100

17. Korgunyuk Y.G. Structure of electoral divisions in the elections to the regional assemblies of the subjects of the Russian Federation (2016–2020): trends of transformation. Electoral politics. 2021, N 1, P. 2–15. (In Russ.) Krippendorff K. Content analysis: an introduction to its methodology. Beverly Hills, CA: Sage, 1980, 441 p.

18. Lelkes Y. Mass polarization: manifestations and measurements. Public opinion quarterly. 2016, Vol. 80, N S1, P. 392–410. DOI: https://doi.org/10.1093/poq/nfw005

19. Lowe W. Understanding wordscores. Political analysis. 2008, Vol. 16, N 4, P. 356–371.

20. Margalit Y. Explaining social policy preferences: evidence from the Great Recession. American political science review. 2013, Vol. 107, N 1, P. 80–103.

21. Martyanov D.S., Bykov I.A. Ideological segregation and digital inequality in the Russian internet community. Sociodinamika. 2018, N 4, P. 43–54. (In Russ.)

22. Matalon Y., Magdaci O., Almozlino A., Yamin D. Using sentiment analysis to predict opinion inversion in Tweets of political communication. Scientific reports. 2021, Vol. 11, P. 1–9. DOI: https://doi.org/10.1038/s41598-021-86510-w

23. Németh R.A. Scoping review on the use of natural language processing in research on political polarization: trends and research prospects. Journal of computational social science. 2023, Vol. 6, P. 289–313. DOI: https://doi.org/10.1007/s42001-022-00196-2

24. Siegel A., Badaan V. #No2Sectarianism: experimental approaches to reducing sectarian hate speech online. American political science review. 2020, Vol. 114, N 3, P. 837– 855. DOI: https://doi.org/10.1017/S0003055420000283

25. Slapin J., Proksch S.O. A scaling model for estimating time-series party positions from texts. American journal of political science. 2008, Vol. 52, N 3, P. 705–772.

26. Stukal D.K., Akhremenko A.S., Petrov A.P. Affective political polarization and the language of hate: made for each other? RUDN journal of political science. Seriya: Politologiya. 2022, Vol. 24, N 3, P. 480–498. (In Russ.)

27. Sunstein C.R. Republic.com. Princeton: Princeton university press, 2001, 240 p.

28. Törnberg, P. Large language models outperform expert coders and supervised classifiers at annotating political social media messages. Social science computer review. 2024. Vol. 41, N 6, P. 1–15. DOI: https://doi.org/10.1177/08944393241286471

29. Zeitzoff T. How social media is changing conflict. Journal of conflict resolution. 2017, Vol. 61, P. 1970–1991.

30. Zhigadlo K.V. The problem of division between the left and right in modern Russia through the prism of social constructivism. RUDN journal of political science. Seriya: Politologiya. 2023, Vol. 25, N 2, P. 423–433. DOI: https://doi.org/10.22363/2313-1438-2023-25-2-423-433 (In Russ.)