A randomised controlled trial (RCT) conducted by researchers at the Faculty of Education University of Cambridge in 2019–20 showed that pupils who take part in The Scholars Programme report significantly higher levels of self-efficacy for university-style learning, compared to pupils who do not take part in the programme. University self-efficacy is the belief in one’s ability to develop the knowledge, skills and behaviours needed to succeed at university, and is known to increase young people’s sense of belonging to university, especially highly-selective institutions.
The type of evaluation that we commissioned, an RCT, is associated with the highest level of evidence. By randomly allocating each individual pupil to The Scholars Programme intervention, or a control group who take part in the programme after the evaluation is completed, it allows the trial to attribute any differences in outcomes to participation in The Scholars Programme.
What makes the findings from this RCT interesting is that The Scholars Programme is first and foremost an academic enrichment intervention, where pupils work with a PhD researcher to complete a university-style course, focused on developing skills such as critical thinking and meta-cognition. Yet, what we see from the findings is that our programme is having a significant and substantive impact on self-efficacy for university-style learning. We already know, from four years of independent match-control group analysis by UCAS, that pupils who take part in The Scholars Programme are significantly more likely to progress to a highly–selective university. So, is university self-efficacy the magic ingredient of The Scholars Programme?
Truthfully, it is hard to say with absolute certainty, as with any complex intervention delivered against a backdrop of varying factors it will always be difficult to isolate one specific mechanism alone. But what we do know is that university self-efficacy is an important outcome that supports pupils to progress to highly-selective universities. Now, from the RCT, we know that self-efficacy for university-style learning is – at the very least – one of the key intermediate outcomes supporting the higher university progression rates for Scholars Programme graduates.
In the RCT, we were interested in whether our programme improves pupils’ critical thinking skills in relation to a new topic, what is known as ‘general critical thinking’. However, attrition from the trial led to a sample size that was too small to establish whether the slightly higher levels of general critical thinking observed amongst participants in The Scholars Programme, compared to the control group, were statistically caused by participation in the programme. We would require a much larger sample to understand whether our programme affects this particular outcome. In general, our programme data shows improvements in subject-specific critical thinking related to topics taught by Scholars Programme tutors. Therefore, it is the connection to general critical thinking and wider meta-skills that remains uncertain based on the current trial.
What we do know from the trial is that The Scholars Programme positively impacts university self-efficacy. There is existing evidence showing that university self-efficacy both directly and indirectly improves progression to, and success at, university, with young people with higher self-efficacy having better outcomes both at school and at university.
The importance of university self-efficacy also speaks to everyday experiences across our programmes, that a difference between pupils from underrepresented backgrounds and their wealthier peers is their self-belief about succeeding at school and university. The way in which The Scholars Programme cultivates university self-efficacy is by engaging pupils with academic content and university-style learning to develop academic knowledge and skills. Through working with a PhD researcher and producing university-style final assignments, young people can directly develop and explicitly see the knowledge and skills that they need to progress and succeed at university, and realise that they are capable of this style of learning.
Brilliant Club Director of Research and Impact, Dr Lauren Bellaera, said:
“With this trial we really wanted to do something that was going to challenge us – both empirically and programmatically. Over the past five years, under our Path to Outcomes strategy, we have invested extensively in understanding the impact of our programmes. So, we know already that The Scholars Programme supports progression to highly-selective universities. With this new trial, we wanted to better understand exactly how our programme works and to do this using the most robust standard of evidence available, a randomised controlled trial. The findings from this trial show the importance of promoting university self-efficacy in young people, and will help to shape the charity’s future research and impact work.”
Chief Programmes Officer for The Brilliant Club, Richard Eyre, said:
“Young people from the most privileged backgrounds sometimes seem to have this unbelievable self-confidence. If only we could bottle that and share it more equitably. Well, we can. This research shows how an academic intervention helps pupils in ordinary state schools develop a genuine, earned sense of self-confidence in their ability to go on and succeed at university – even the most competitive ones.”
Principal Investigator from the University of Cambridge, Dr Sonia Ilie, said:
“This RCT is a good example of how we can build evidence around what is effective in supporting young people’s progression to university. The results of the trial show a positive impact of The Scholars Programme on self-efficacy in relation to university study. The trial was too small to see if the programme’s effect on general critical thinking was statistically significant, so a larger study may tell us more about how The Scholars Programme affects critical thinking and other cognitive strategies associated with better outcomes for our young people.”
Next steps for our evaluation work
Conducting an RCT has strengthened our evidence base and enabled us to understand how The Scholars Programme is supporting progression to highly-selective universities. This evidence has come at a crucial time as we start to plan for and embark on our next five-year strategy. What we now know, based on the trial, is that our programme is having a significant and meaningful impact on self-efficacy for university-style learning. It remains unclear the extent to which we are impacting general critical thinking. These findings have important implications for what we set out in our new five-year strategy:
Acknowledgements: We would like to thank the participating schools, teachers and pupils for their support with this trial, as well as Drs Sonia Ilie and Ashton Brown for all of their work in conducting this external evaluation on behalf of The Brilliant Club. If you would like to learn more about the design of the RCT, please click here.