SCAS News - 28 May, 2024

Visiting CAT Scholars Address Challenges for the Development of Fair Language-based
Assessments

This week, SCAS is hosting another group of early-career researchers within the NetIAS Constructive
Advanced Thinking (CAT) programme. The research project being focussed on is titled Challenges for
the Development of Fair Language-based Assessments of Health, Education, Behaviour, and Beyond

and offers an highly interdisciplinary approach:

"Linguistic behavior serves as a reliable, cheap, and increasingly automated resource to assess different
aspects of individuals and societies, ranging from education to health and cognitive states. Speech samples
might help detect incipient health issues, newspaper corpora are used to reveal what we collectively think
about minority groups, and wordlists are the basis on which we determine verbal development - among
a vast number of examples. However, these developments - which we label language-based assessments 
or LanBAs – were concocted, tested, and deployed primarily only on a handful of large and commercially
central languages, with English dominating the scene. Since the 6,500 extant languages can and do vary
substantially (and in every dimension), transferring LanBAs from English to them is often fraught with
important technical and linguistic challenges, which might not be appreciated by practitioners in need of
them. The consequences of this bias, which we only start to understand, is that speakers and signers
of minority languages have at their disposal more expensive, less efficient, and potentially biased LanBAs.

The project addresses this complex and multifaceted issue by gathering a diverse set of experts covering
cognitive neuroscience, digital humanities, comparative linguistics, developmental science, and cultural
psychology with three main tracks of activity. First, it critically synthesizes the scientific evidence
revealing the Anglophone bias in LanBAs, aiming at finding differences and commonalities across our 
fields of practice. Second, it engages policy-makers, experts on language technologies, and other non-
academic agents with the purpose of building a clear course of action to transfer findings into practical
recommendations amenable to impact the current state of the art in the development of LanBAs. Third,
it engages with the general audience through diverse media strategies, including filming a mini-documentary
on the topic of the group visits to the different IAS."

The members of the group include Damian Blasi (ICREA & Pompeu Fabra University, Barcelona);
Joseph P. Dexter (Harvard University); Adolfo Martin Garcia (University of Santiago of Chile); and
Amber Gayle Thalmayer (University of Zurich). Everyone but Garcia is present at the Collegium this
week.


------
SCAS is one of twelve participating institutes of the CAT initiative, which was developed within the
framework of the Network of European Institutes for Advanced Studies (NETIAS), and launched in
2019. The programme aims at supporting small groups of excellent early-career researchers conducting
fundamental research dedicated to developing new ideas to understand and to tackle current or emerging
societal challenges. The selected groups work on their project for a duration of up to three years and
during this time they are invited to meet for short stays at some of the participating institutes, where
they also get the opportunity to get in contact with the Fellows in residence at the institutes as well as
local research communities.


Read more about the CAT programme >>
Read more about the Network of European Institutes for Advanced Studies (NETIAS) >>