The Common European Framework of Reference for Languages (CEFR) has had a major impact on language education worldwide. Aligning EnglishScore to international standards such as the CEFR is a way of establishing test validity and fairness, and ensures we are following best practice in English language proficiency assessment standards.
Assessment validation is an ongoing process of refinement to ensure test reliability and accuracy as time goes by. Below we discuss our approach to monitoring and improving EnglishScore’s validity over time and our latest CEFR Panel Report validating the relationship between EnglishScore and the CEFR.
What is the CEFR?
CEFR was developed by the Council of Europe in 2001 as a guideline to describe a language learners’ ability across Europe and, increasingly, in other countries. The CEFR is also intended to help educational institutions and employers to evaluate the language qualifications of candidates for education admission or employment.
Since its original publication, it has been regularly updated, most recently in 2020 with the publication of the Companion Volume.
Why is it important for EnglishScore to align to the CEFR?
Linking EnglishScore to an external framework such as the CEFR and using expert judgements from a CEFR panel helps ensure our assessment is reliable and accurate.
Using a 6 point scale it identifies an individual’s English level – with A1 being the score for beginners and C2 for those who have mastered English. Find out more about how EnglishScore aligns to the CEFR Scoring Scale.
Aligning to the CEFR also helps test takers and English language teachers interpret test results using a common framework. Moreover, it helps in monitoring and validating assessment quality to achieve reliable and accurate test results for our test takers, English language teachers and employers.
How does EnglishScore align with the CEFR?
When developing the EnglishScore test, our assessment experts adhere to a common validation framework as recommended by the Council of Europe (CoE) in its Manual for Relating language examinations to the CEFR:
- Familiarisation: expert judges with robust knowledge of the CEFR
- Specification: expert judgement on how the test purpose and content of EnglishScore aligns with the CEFR.
- Standardisation: achieving consensus from the panel of judges on level performance and test tasks associated with each CEFR level
- Standard setting: determining pass marks for each CEFR level and cut scores between levels
- Validation: feedback about standard setting from the panel of judges and collecting validity evidence
In linking our assessment to CEFR, we have also clearly defined the EnglishScore test purpose and content in our Validity Report. This outlines how we develop, monitor and maintain the quality of our test content over time.
Aligning EnglishScore to the CEFR through standard setting
Our latest CEFR Panel Report describes the standard setting study carried out to validate the relationship between EnglishScore and the CEFR. The study followed the Familiarisation, Standardisation and Standard setting elements of the CoE validation framework as outlined above. The findings from the study determined the test ‘cut scores’ i.e. the levels of performance in EnglishScore that align with CEFR levels A2 to C1.
For the study, the Centre for Research in English Language Learning and Assessment (CRELLA) conducted a standards-setting panel using the Bookmark method. The panel was comprised of 11 experts, including assessment experts, language testing specialists with knowledge and experience of using the CEFR and educators who worked with learners at teaching centres in countries with large numbers of EnglishScore test takers (China, Qatar, Thailand and Vietnam).
The majority of panellists were qualified teachers of English holding a wide range of prestigious qualifications and certifications in teaching English, with over 100 years of teaching experience combined.
The Familiarisation and Standardisation steps were covered before and during the panel sessions, with pre-session tasks and alignment activities during the three days. This ensured the panel members had a shared understanding of the different CEFR levels and could apply them consistently. The remaining time was used for Standard setting – following the Bookmark method, one of the most popular methods for large scale educational assessments, to determine cut scores for Grammar, Vocab, Reading and Listening. This ensures that the reported test results meaningfully link to the different CEFR levels.
The findings from the standard-setting workshop represents an important step forward in understanding the relationship between EnglishScore and the CEFR. As the test continues to develop, we will keep the alignment to the CEFR under constant review, in line with the Council of Europe guidelines on best practice.