The s:fl teaching methodology was piloted in tandem foreign language/science classes over a period of two years in six schools in four countries: Bulgaria, Greece, Poland and Romania.
In the first year, the s:fl methodology was used only in 5th grade classes, and in the second year it was extended to 6th grade classes, meaning that most 6th grade students participating in this project did so two years in a row.
Our careful evaluation of the methodology has produced these findings:
Do students achieve a higher and deeper level of knowledge in STEM-subjects and better fluency in their chosen foreign language than those not involved in this project?
Are students more interested in STEM subjects and in learning foreign languages (FL) than non-participating students?
Do students acquire useful skills?
Do teachers find their classes more interesting, fulfilling and motivating?
Is the project’s methodology feasible and implementable?
Is this methodology useful and successful?
Is it feasible to promote this methodology in more schools, including secondary schools and in more countries?
82% of students would like to continue using the s:fl methodology as would nearly all teachers, who would recommend it to their colleagues.
We will continue to promote this methodology in more schools and in more countries.
The University of Athens was responsible for evaluation the Schools: Future Labs (s:fl) Project. The evaluation strategy was planned according to the Project’s purpose and target group: to implement a methodology that will transform the teaching and learning of Language and Science, technology, engineering and mathematics (STEM) subjects so that students become active, self-directed, exploratory participants in their own learning. It was expected that with appropriate support, guidance and access to basic tools of scientific exploration, students would discover that STEM subjects and foreign languages are interesting, useful and open doors to their future. The target groups were upper primary school students and their teachers.
The Research Questions
The project’s research questions concerning the students were:
- Do students achieve a higher and deeper level of knowledge in STEM-subjects and better fluency in their chosen foreign language than those not involved in this project?
- Are participating students more interested in STEM subjects and in learning foreign languages (FL) than non-participating students?
- Do students acquire useful skills?
With regard to the teachers, the research questions addressed were:
- Do teachers find their classes more interesting, fulfilling and motivating?
- Are teachers’ profiles strengthened by the acquisition of additional teaching skills?
Finally, with regard to the project in general the questions asked were:
- Is the project’s methodology feasible and implementable?
- Is this methodology useful and successful?
- Is it feasible to promote this methodology in more schools, including secondary schools and in more countries?
The tools that were used for the evaluation (qualitative and quantitative) were questionnaires, observations and interviews during on-site visits:
- Pre- and post-implementation attitudes/motivation/perceptions questionnaires both for students and teachers;
- Post-implementation questionnaires seeking to obtain feedback on students’ and teachers’ satisfaction with the s:fl methodology, including also self-assessment questions;
- Comparison of participating students’ attitudes and performance with non-participating students before and after the implementation;
- Review of lesson plans and feedback from participating teachers.
The questionnaires were developed based on a review of relevant international literature and surveys (science learning, Foreign Language learning, STEM programs and Content Language Integrated Learning (CLIL) programs), on the study of research methods, tools and questionnaires, and on the basis of our prior experience.
Taking into consideration the questions and goals of this project, we compiled the appropriate attitudes/motivation questionnaires for students and teachers, consisting of seven factors for students (interest/enjoyment, competence, effort/importance, tension/pressure, value/usefulness, science skills, FL skills) and six factors for teachers (interest/enjoyment, competence, tension/pressure, value/usefulness, science skills, FL skills). We selected and adapted several questions from among the most commonly used in similar surveys and added some more to meet the goals of our research. In order to enhance the questionnaires’ reliability, for many questions we also added their reverses.
The questionnaires were tested for validity and reliability prior to finalisation. Three pre-implementation questionnaires were developed:
- Students’ questionnaire with 58 questions,
- STEM teachers’ questionnaire with 22 questions,
- FL teachers’ questionnaire with 18 questions.
The same questionnaires were re-used after the project implementation, with additional questions measuring participants’ satisfaction with the project and evaluating it:
- Students’ questionnaire with 78 questions,
- STEM teachers’ questionnaire with 47 questions,
- FL teachers’ questionnaire with 41 questions.
Both pre and post questionnaires were sent to the schools participating in the s:fl Project to be completed by all the students and teachers, whether they participated or not (control group).
Use of the questionnaire in the first year showed that they were functional, applicable and could measure the mentioned factors. For this reason, no major changes were made for the second year of the project, however minor modifications were made, especially to reduce the overall number of questions. The students’ questionnaire was also modified so as to include an assessment (by the teachers) of students’ knowledge level and skills. The new pre questionnaires consisted of 55 questions for students and 19 questions for STEM/FL teachers.
The Results – First year
|Participating students (5th grade)||171|
|Non participating students (5th grade)||99|
|Participating STEM teachers||11|
|Participating FL teachers||6|
The data from the questionnaires were coded into a database. After the data processing and the statistical analysis we summarized the descriptive results and analysed the results for each factor, separately for each school/country and also overall.
These analyses and results were very positive and encouraging, since we found:
- Statistically significant improvement of participants’ attitudes/motivation in almost all factors both in STEM subjects and in foreign languages;
- Satisfaction with the methodology.
The evaluation of the learning outcomes/students’ performance (based on students’ self-assessment and their assessment by the teachers) was also very satisfactory since improvement was reported in participants’ performance both in STEM/FL.
Some characteristic results based on participating students’ answers were that 72% of them consider the s:fl lessons more interesting than traditional science and FL lessons, and 69% found lessons combining FL and science more fun. However, the most important result is that 78% of participating students answer that they would like to participate again in the s:fl Project.
As for the teachers, almost all of them answered that the s:fl methodology raised students’ interest and improved their cooperation and students were able to respond adequately. They also said that they managed to make the lessons more fun; they would recommend this methodology to their colleagues and almost all of them would like to participate again.
The results – Second year
In the second year of the project, the s:fl methodology was implemented in the 6th grade in addition to the 5th grade. Most of the 6th grade students had already been exposed to this methodology in the previous year, when they were in the 5th grade.
|5th grade||6th grade||Total|
|Non participating students||55||45||100|
|Participating STEM teachers||9|
|Participating FL teachers||6|
The results were again very positive and encouraging, since we found:
- Statistically significant improvement, compared to the first’s year pre-participation measures, of participants‘ attitudes/motivation in almost all factors both in STEM subjects and foreign languages;
- Satisfaction with the methodology;
- Improvement in participants’ performance both in STEM subjects and foreign languages;
- Improvement in participants’ skills.
With regard to attitudes/motivation, we found statistically significant differences between participating and non-participating students in all factors with participating students achieving higher scores.
Comparison of participating – non participating students after the implementation
Interestingly, in this second year for some factors we found statistically significant differences between the two groups even before the implementation. This meant that the two groups didn’t start from the same point. We hypothesized that this was due to the 6th graders because the great majority of them had participated also in the s:fl project the previous year. Therefore we examined the students of the 5th and 6th grade separately, and this confirmed our hypothesis:
- With regard to the 5th graders, we found no significant differences between participating and non-participating students before the implementation, but after the implementation the participating students attained statistically significant higher performance;
- With regard to the 6th graders, we found that participating students had already a raised level of interest in Science, a raised level of interest in FL, a higher competence level in Science, a lower level of tension in FL.
This result confirms that the previous year’s implementation of s:fl had positive impacts on students and these impacts remained evident one year later. Moreover, after the second year of implementation, these results were further strengthened, since we found statistically significant improvement in participating students’ performance in all factors after the implementation.
With regard to the satisfaction of participants and their assessment of the project:
- 62% of them consider the s:fl lessons more interesting compared to traditional lessons;
- 61% did not find it difficult to understand the concepts of science in the foreign language;
- 70% did not feel stressed by the presence of two teachers in the classroom;
- 62% found the combination of science and foreign language lesson more fun
However, the most important result again is that 82% of participating students answer that they would like to participate in the s:fl project again.
Students’ learning outcomes/performance and skills were evaluated both through self-assessments and through teachers’ assessment. The great majority of students answered that after the implementation of the s:fl project they were doing better both in science (71%) and in FL (62%), they could conduct experiments more easily (76%) and express themselves in the FL more easily (74%). These results are in accordance with their assessment by the teachers who confirm that students’ performance both in FL and in science was improved.
As for the skills, the comparison between participants’ scores before and after the implementation showed statistically significant improvement after the implementation.
As for the teachers, almost all of them answered that this way of teaching raised students’ interest and improved their cooperation and students were able to respond adequately. They also reported that they managed to make the lesson more fun, they would recommend this methodology to their colleagues and almost all of them would like to participate again.
On site visits
Regarding the evaluation of the s:fl projects’ feasibility, success and usability in general, we made on-site visits to the schools to observe the implementation of the project. We visited all participating schools in each country and the general impression of the whole team of the University of Athens was that the s:fl program was implemented with great success in all schools:
- Students showed interest and participated willingly;
- They showed great enthusiasm and seemed to enjoy themselves throughout the lessons;
- They responded successfully to the classroom activities (comparing, identifying, predicting …), conducted the suggested experiments with great enthusiasm and successfully completed the worksheets;
- They used the German / Spanish language most of the time during the lessons. They used known words with ease and had no difficulty compiling proper sentences and expressing their opinion freely in the foreign language;
- They seemed to understand the instructions they were given in the German/Spanish language and it was not difficult for them to answer or present their work in front of the rest of the class using that language;
- They could cooperate harmoniously in small or larger groups .
With regard to the teachers, apart from the very careful preparation of the lesson (with worksheets, the necessary vocabulary and appropriate visual materials), they also showed considerable enthusiasm for the methodology and especially love for their work, which contributed decisively to the success of the program. The very good cooperation between the teachers also played an important role.