The Evidence

Summary

The s:fl teaching methodology was piloted in tandem foreign language/science classes over a period of two years in six schools in four countries: Bulgaria, Greece, Poland and Romania.

In the first year, the s:fl methodology was used only in 5th grade classes, and in the second year it was extended to 6th grade classes, meaning that most 6th grade students participating in this project did so two years in a row.

Our careful evaluation of the methodology has produced these findings:

Do students achieve a higher and deeper level of knowledge in STEM-subjects and better fluency in their chosen foreign language than those not involved in this project?

YES.

Are students more interested in STEM subjects and in learning foreign languages (FL) than non-participating students?

YES.

Do students acquire useful skills?

YES.

Do teachers find their classes more interesting, fulfilling and motivating?

YES.

 

Is the project’s methodology feasible and implementable?

YES.

Is this methodology useful and successful?

YES.

Is it feasible to promote this methodology in more schools, including secondary schools and in more countries?

82% of students would like to continue using the s:fl methodology as would nearly all teachers, who would recommend it to their colleagues.

We will continue to promote this methodology in more schools and in more countries.

Evaluation Report

The University of Athens was responsible for evaluation the Schools: Future Labs (s:fl) Project. The evaluation strategy was planned according to the Project’s purpose and target group: to implement a methodology that will transform the teaching and learning of Language and Science, technology, engineering and mathematics (STEM) subjects so that students become active, self-directed, exploratory participants in their own learning. It was expected that with appropriate support, guidance and access to basic tools of scientific exploration, students would discover that STEM subjects and foreign languages are interesting, useful and open doors to their future. The target groups were upper primary school students and their teachers.

The Research Questions

The project’s research questions concerning the students were:

  • Do students achieve a higher and deeper level of knowledge in STEM-subjects and better fluency in their chosen foreign language than those not involved in this project?
  • Are participating students more interested in STEM subjects and in learning foreign languages (FL) than non-participating students?
  • Do students acquire useful skills?

With regard to the teachers, the research questions addressed were:

  • Do teachers find their classes more interesting, fulfilling and motivating?
  • Are teachers’ profiles strengthened by the acquisition of additional teaching skills?

Finally, with regard to the project in general the questions asked were:

  • Is the project’s methodology feasible and implementable?
  • Is this methodology useful and successful?
  • Is it feasible to promote this methodology in more schools, including secondary schools and in more countries?

The Tools

The tools that were used for the evaluation (qualitative and quantitative) were questionnaires, observations and interviews during on-site visits:

  • Pre- and post-implementation attitudes/motivation/perceptions questionnaires both for students and teachers;
  • Post-implementation questionnaires seeking to obtain feedback on students’ and teachers’ satisfaction with the s:fl methodology, including also self-assessment questions;
  • Comparison of participating students’ attitudes and performance with non-participating students before and after the implementation;
  • Review of lesson plans and feedback from participating teachers.

The questionnaires

The questionnaires were developed based on a review of relevant international literature and surveys (science learning, Foreign Language learning, STEM programs and Content Language Integrated Learning (CLIL) programs), on the study of research methods, tools and questionnaires, and on the basis of our prior experience.

Taking into consideration the questions and goals of this project, we compiled the appropriate attitudes/motivation questionnaires for students and teachers, consisting of seven factors for students (interest/enjoyment, competence, effort/importance, tension/pressure, value/usefulness, science skills, FL skills) and six factors for teachers (interest/enjoyment, competence, tension/pressure, value/usefulness, science skills, FL skills). We selected and adapted several questions from among the most commonly used in similar surveys and added some more to meet the goals of our research. In order to enhance the questionnaires’ reliability, for many questions we also added their reverses.

The questionnaires were tested for validity and reliability prior to finalisation. Three pre-implementation questionnaires were developed:

  • Students’ questionnaire with 58 questions,
  • STEM teachers’ questionnaire with 22 questions,
  • FL teachers’ questionnaire with 18 questions.

The same questionnaires were re-used after the project implementation, with additional questions measuring participants’ satisfaction with the project and evaluating it:

  • Students’ questionnaire with 78 questions,
  • STEM teachers’ questionnaire with 47 questions,
  • FL teachers’ questionnaire with 41 questions.

Both pre and post questionnaires were sent to the schools participating in the s:fl Project to be completed by all the students and teachers, whether they participated or not (control group).

Use of the questionnaire in the first year showed that they were functional, applicable and could measure the mentioned factors. For this reason, no major changes were made for the second year of the project, however minor modifications were made, especially to reduce the overall number of questions. The students’ questionnaire was also modified so as to include an assessment (by the teachers) of students’ knowledge level and skills. The new pre questionnaires consisted of 55 questions for students and 19 questions for STEM/FL teachers.

The Results – First year

The sample

Participating students (5th grade) 171
Non participating students  (5th grade) 99
Participating STEM teachers 11
Participating FL teachers 6

The data from the questionnaires were coded into a database. After the data processing and the statistical analysis we summarized the descriptive results and analysed the results for each factor, separately for each school/country and also overall.

These analyses and results were very positive and encouraging, since we found:

  • Statistically significant improvement of participants’ attitudes/motivation in almost all factors both in STEM subjects and in foreign languages;
  • Satisfaction with the methodology.

The evaluation of the learning outcomes/students’ performance (based on students’ self-assessment and their assessment by the teachers) was also very satisfactory since improvement was reported in participants’ performance both in STEM/FL.

Some characteristic results based on participating students’ answers were that 72% of them consider the s:fl lessons more interesting than traditional science and FL lessons, and 69% found lessons combining FL and science more fun. However, the most important result is that 78% of participating students answer that they would like to participate again in the s:fl Project.

As for the teachers, almost all of them answered that the s:fl methodology raised students’ interest and improved their cooperation and students were able to respond adequately. They also said that they managed to make the lessons more fun; they would recommend this methodology to their colleagues and almost all of them would like to participate again.

The results – Second year

In the second year of the project, the s:fl methodology was implemented in the 6th grade in addition to the 5th grade. Most of the 6th grade students had already been exposed to this methodology in the previous year, when they were in the 5th grade.

The sample

5th grade 6th grade Total
Participating students 107 139 246
Non participating students 55 45 100

 

Participating STEM teachers 9
Participating FL teachers 6

The results were again very positive and encouraging, since we found:

  • Statistically significant improvement, compared to the first’s year pre-participation measures, of participants‘ attitudes/motivation in almost all factors both in STEM subjects and foreign languages;
  • Satisfaction with the methodology;
  • Improvement in participants’ performance both in STEM subjects and foreign languages;
  • Improvement in participants’ skills.

With regard to attitudes/motivation, we found statistically significant differences between participating and non-participating students in all factors with participating students achieving higher scores.

Comparison of participating – non participating students after the implementation

Interestingly, in this second year for some factors we found statistically significant differences between the two groups even before the implementation. This meant that the two groups didn’t start from the same point. We hypothesized that this was due to the 6th graders because the great majority of them had participated also in the s:fl project the previous year. Therefore we examined the students of the 5th and 6th grade separately, and this confirmed our hypothesis:

  • With regard to the 5th graders, we found no significant differences between participating and non-participating students before the implementation, but after the implementation the participating students attained statistically significant higher performance;
  • With regard to the 6th graders, we found that participating students had already a raised level of interest in Science, a raised level of interest in FL, a higher competence level in Science, a lower level of tension in FL.

This result confirms that the previous year’s implementation of s:fl had positive impacts on students and these impacts remained evident one year later. Moreover, after the second year of implementation, these results were further strengthened, since we found statistically significant improvement in participating students’ performance in all factors after the implementation.

With regard to the satisfaction of participants and their assessment of the project:

  • 62% of them consider the s:fl lessons more interesting compared to traditional lessons;
  • 61% did not find it difficult to understand the concepts of science in the foreign language;
  • 70% did not feel stressed by the presence of two teachers in the classroom;
  • 62% found the combination of science and foreign language lesson more fun

However, the most important result again is that 82% of participating students answer that they would like to participate in the s:fl project again.

Students’ learning outcomes/performance and skills were evaluated both through self-assessments and through teachers’ assessment. The great majority of students answered that after the implementation of the s:fl project they were doing better both in science (71%) and in FL (62%), they could conduct experiments more easily (76%) and express themselves in the FL more easily (74%). These results are in accordance with their assessment by the teachers who confirm that students’ performance both in FL and in science was improved.

As for the skills, the comparison between participants’ scores before and after the implementation showed statistically significant improvement after the implementation.

 

As for the teachers, almost all of them answered that this way of teaching raised students’ interest and improved their cooperation and students were able to respond adequately. They also reported that they managed to make the lesson more fun, they would recommend this methodology to their colleagues and almost all of them would like to participate again.

On site visits

Regarding the evaluation of the s:fl projects’ feasibility, success and usability in general, we made on-site visits to the schools to observe the implementation of the project. We visited all participating schools in each country and the general impression of the whole team of the University of Athens was that the s:fl program was implemented with great success in all schools:

  • Students showed interest and participated willingly;
  • They showed great enthusiasm and seemed to enjoy themselves throughout the lessons;
  • They responded successfully to the classroom activities (comparing, identifying, predicting …), conducted the suggested experiments with great enthusiasm and successfully completed the worksheets;
  • They used the German / Spanish language most of the time during the lessons. They used known words with ease and had no difficulty compiling proper sentences and expressing their opinion freely in the foreign language;
  • They seemed to understand the instructions they were given in the German/Spanish language and it was not difficult for them to answer or present their work in front of the rest of the class using that language;
  • They could cooperate harmoniously in small or larger groups .

With regard to the teachers, apart from the very careful preparation of the lesson (with worksheets, the necessary vocabulary and appropriate visual materials), they also showed considerable enthusiasm for the methodology and especially love for their work, which contributed decisively to the success of the program. The very good cooperation between the teachers also played an important role.

At the third year the SFL was implemented in the 5th and the 6th grade and in one school even in the 7th grade. Most of these students participated also at previous year(s).

 

5th grade 6th grade 7th grade
Participating Students 120 97 65

 

Participating STEM Teachers 9
Participating FL Teachers 6

 

At this final year of the project the implementation was evaluated in general by using only post questionnaires for the participating students and teachers.

The questionnaires were mostly focused on: self-assessment, evaluation of the program, difficulties encountered, students’ assessment by teachers, assessment by the teachers of the training and the support provided. The results were very positive and encouraging, since participants were satisfied with the methodology and their performance in both STEM and FL had progressed.

With regard to the interest / enjoyment of participating students and their assessment of the project:

  • 77% considered the S:FL lesson more interesting
  • 69% considered the science lesson more interesting
  • 71% did not find tiring the lessons combining FL and STEM
  • 72% enjoyed the presence of two teachers in the classroom
  • 97% enjoyed working with their classmates
  • 99% enjoyed conducting experiments
  • 74% enjoyed reading the instructions in FL
  • 71% found lessons much more fun

The most important result is that 89% of participating students answered that they would like to participate again in the S:FL project.

Participating students’ answers to the question “I want to participate again in this program”

 

Concerning students’ difficulties during the implementation of the project only 22% said that they had difficulty in understanding the concepts of science when using the FL. Only 7% said that they felt stressed by the presence of two teachers in the classroom at the same time. Only 6% of them said that they had difficulty in communicating with the members of their team.

Concerning the students’ self-assessment:

  • 74% believed that after participating in the program they were doing better in FL
  • 79% believed that after participating in the program they could express themselves and understand texts in the FL better than before
  • 79% were satisfied with their performance in FL
  • 71% believed that after participating in the program they were doing better in science
  • 78% believed that after participating in the program they could carry out experiments better than before
  • 76% were satisfied with their performance in science
  • 75% believed that after participating in the program their teamwork spirit was enhanced.

The teachers confirmed the students´ estimations concerning their performance and they stressed that students’ performance had indeed improved in both STEM and FL subjects. The teachers were also asked to assess each student’s performance in FL and STEM as well as their skills indicating whether there was no improvement, slight improvement, significant improvement or great improvement after participating in the program. The results are shown in the graphs below.

Percentage of students’ performance Improvement in the foreign language

Percentage of Students’ Performance Improvement in Science

Percentage of Students’ Skills Improvement

 

Almost all of the teachers answered that this way of teaching raised students’ interest (90%) and improved their cooperation (92%) and they were able to respond adequately (100%). Most teachers also said that they managed to do the lesson more fun (80%) and all of them said that the methodology improved the teaching process and they would recommend it to their colleagues. Moreover, almost all of them (90%) would like to participate again. With regard to the difficulties they encountered during the implementation of the program, many of them (60%) said that it took more time to complete a lesson, but 90% of the teachers enjoyed teaching in tandem.

Concerning the training courses most of the teachers answered that they were very useful (80%), the duration was adequate (70%), the material used was useful (80%), while 70% of them said that it would have been better if the training courses were not so focused on theory. With regard to the lesson plans, most of the teachers (90%) answered that they had adequate support, and considered the lesson plans very useful, However, 50% of them said that they were tiring and 70% found it hard to compose them. Teachers found it very interesting to cooperate with other colleagues. They enjoyed implementing the methodology and giving the opportunity to the students to think independently and creatively. Some useful comments from the teachers were that they would have liked to have more time in each lesson and a bigger variety of topics. In this way, they could have worked with smaller groups of students and could have carried out more experiments.

 

The on site-visits

The in situ visits in all participating schools gave us the opportunity to receive feedback directly from the teachers and students as well as to observe how the methodology was implemented and proved the S:FL projects’ feasibility, success and usability. With regard to students, they: participated willingly, showed great enthusiasm and seemed to enjoy the course, responded successfully to the activities of the course, conducted the suggested experiments with great enthusiasm, completed the worksheets successfully, used the German/Spanish language during most of the time of the course, had no difficulty presenting their work in front of the rest of the class using the foreign language and cooperated harmoniously.  With regard to the teachers, despite the meticulous preparation of the course, they also showed considerable enthusiasm for the course and love for their work, which led to the success of the project. The smooth cooperation between the two teachers played a crucial role as the lesson was taught mostly in German and Spanish. The interventions in the mother tongue were few.

The in situ visits in all participating schools gave us the opportunity to receive feedback directly from the teachers and students as well as to observe how the methodology was implemented and proved the S:FL projects’ feasibility, success and usability. With regard to students, they: participated willingly, showed great enthusiasm and seemed to enjoy the course, responded successfully to the activities of the course, conducted the suggested experiments with great enthusiasm, completed the worksheets successfully, used the German/Spanish language during most of the time of the course, had no difficulty presenting their work in front of the rest of the class using the foreign language and cooperated harmoniously.  With regard to the teachers, despite the meticulous preparation of the course, they also showed considerable enthusiasm for the course and love for their work, which led to the success of the project. The smooth cooperation between the two teachers played a crucial role as the lesson was taught mostly in German and Spanish. The interventions in the mother tongue were few.

 

 

The Dissemination of the S:FL Project

 

 

The majority of students answered that they had talked about this program to other students (in total more than 1.000) from their school as well as from other neighbor schools. Most of the teachers also talked about this program to other teachers (in total more than 350) from their school as well as from other neighbor schools. Moreover, during the science fairs organized by the schools more than 150 teachers and 650 students were informed about the S:FL project.