When we create an English assessment, we constantly ask ourselves, ‘is it fair?’ But what does fairness really mean and how can you achieve it?
Fairness is about making sure everyone has an equal chance to demonstrate their ability, regardless of gender, age, nationality or life experience.
Because EnglishScore is a truly global test, factors like this are vitally important to recognise and understand, to ensure our test is truly fair, valid and reliable.
And if a test is fair, test takers will be motivated and engaged, and therefore want to perform as well as they can. We use lots of different approaches to ensure our tests are fair – here are three examples of the kinds of things we do.
1. No background or specialist knowledge required
Take a look at this listening item and try to answer the question that follows:
What sport is the woman describing?
A: basketball
B: football
C: cycling
Although there are lots of clues to anyone who knows or follows football (Euros, World Cup, stadium, penalties, pitch, Wembley, Qatar) the item relies on this knowledge, and so wouldn’t be a fair question for those who aren’t interested in football or know nothing about it.
When developing items, our writers and reviewers ensure that the question can be answered by anyone, regardless of background, nationality, or experience.
This is particularly important for EnglishScore as a test of English for employment. We use lots of workplace language and scenarios in our test items to ensure they are authentic and represent the test construct.
However, we also need to make sure that test takers who haven’t worked before, or work in different sectors, aren’t disadvantaged. And if test takers happen to be very familiar with a particular sector or role, we also don’t want them to have an unfair advantage.
So although our items relate to workplace scenarios, they don’t assume test takers have specialist or technical knowledge to answer them.
2. Reducing bias in the Speaking Test
One of the unique aspects of the EnglishScore Speaking Test is the combination of AI and human scoring.
This gives us the best combination of validity, reliability and fairness.
We use AI scoring where it works well (for example, assessing pronunciation and fluency) and real people to score features of communication that are important in the workplace e.g. ‘Did they answer the question?’ or ‘Did I understand them?’
The EnglishScore Speaking Test uses 16 people to contribute to a test taker’s score from a range of different countries and backgrounds. Not only does this reduce bias (and therefore make it more fair) it also reflects the multicultural and multinational workplaces our test takers are working in.
3. Avoiding sensitive topics
Another important part of fairness is avoiding content that might be sensitive or upsetting for certain groups of test takers. This could be content related to gender roles, religion, socio-economic groups, age and ethnic groups.
As part of our item development process, all items go through a sensitivity review, carried out by a cross-cultural panel of experts experienced in analysing item content.
The review process will either accept items, flag potentially problematic items that require revision, or ultimately reject items if they cannot be revised. Findings from this review also feedback into the item specification and training for item writers to support the continued validity, reliability and fairness of the test.
We want our test takers to perform to the best of their ability, which means they should stay motivated and engaged.
Overall, EnglishScore is designed for everyone – no matter their age, gender, ethnicity, belief or background. By taking steps to carefully create and review our test items, we aim to minimise bias, distractions and distress that could impact how people experience the test, or how they answer the questions.
If you’re interested in finding out more about how we’ve created an English test for the global workplace, you can find more information on our resources page.
Want to see how EnglishScore can help you assess the English level of your students and staff? Click the button below.
