Understanding bias in language models: comparing the cultural distributions of basic colour terms in English, German, Japanese, and Russian

Abstract Colours have accompanied us through evolution and have therefore one of the longest biological and cultural history. With different cultures forming, the use of colours has also diverged and adapted. This is underscored by the long and rich qualitative and quantitative research that has been conducted about colours, their use, and their terminology. With the advent of natural language processing and more recently (large) language models, an extra dimension has entered the research arena. This would in principle give researchers new ways to investigate colour names as well as their cultural representation, as these models are now trained on large text corpora in different languages. While they should fundamentally reflect cultural use through individual languages, a homogenisation can be observed. This paper investigates the cultural distribution of basic colour terms in English, German, Japanese, and Russian using text gathered from Twitter in 2015 and the multilingual pre-trained large language model mBERT. The bias in this language model is used to infer cultural tendencies and linguistic skews.