Challenges for the development of fair language-based assessments of health, education, behaviour, and beyond
Damian Blasi (Harvard University, USA – Max Plank Institute for the Science of Human History, Germany – Higher School of Economics, Russia), Joseph P. Dexter (Harvard University, USA), Amber Gayle Thalmayer (University of Zurich, Switzerland), Adolfo Martin Garcia (University of Santiago of Chile, Chile)
Linguistic behavior serves as a reliable, cheap, and increasingly automated resource to assess different aspects of individuals and societies, ranging from education to health and cognitive states. Speech samples might help detect incipient health issues, newspaper corpora are used to reveal what we collectively think about minority groups, and wordlists are the basis on which we determine verbal development - among a vast number of examples. However, these developments - which we label language-based assessments or LanBAs – were concocted, tested, and deployed primarily only on a handful of large and commercially central languages, with English dominating the scene. Since the 6,500 extant languages can and do vary substantially (and in every dimension), transferring LanBAs from English to them is often fraught with important technical and linguistic challenges, which might not be appreciated by practitioners in need of them. The consequences of this bias, which we only start to understand, is that speakers and signers of minority languages have at their disposal more expensive, less efficient, and potentially biased LanBAs.
The project addresses this complex and multifaceted issue by gathering a diverse set of experts covering cognitive neuroscience, digital humanities, comparative linguistics, developmental science, and cultural psychology with three main tracks of activity. First, it critically synthesizes the scientific evidence revealing the Anglophone bias in LanBAs, aiming at finding differences and commonalities across our fields of practice. Second, it engages policy-makers, experts on language technologies, and other non-academic agents with the purpose of building a clear course of action to transfer findings into practical recommendations amenable to impact the current state of the art in the development of LanBAs. Third, it engages with the general audience through diverse media strategies, including filming a mini-documentary on the topic of the group visits to the different IAS.