TOrch Project

From October 2016 - April 2018, I will be working on the TOrch project in collaboration with the Ruhr-Universität Bochum in Germany.

TOrch is short for "Teacher Orchestration". The aim of the project is to design and test the effectiveness of Teacher Dashboards for orchestrating math education in primary schools.

This project is supported by NWO through a Rubicon grant [grant number 446-16-003/1276].

Summary of TOrch project

When supported by computer software, collaboration leaves “digital traces”. The collection and analysis of these traces (i.e., learning analytics; LA) has the potential to help teachers move from overload to overview of students’ collaboration. Moreover, LA cannot only aid teachers by mirroring student activities, but also automate evaluation and intervention. The question is which level of LA-aid is optimal to enable adaptive teacher support of collaboration. The goal of this project is to investigate what information teachers need, how teachers perceive the usefulness of dashboards, and whether dashboards can influence teacher support of collaboration.




Research team

Principal investigators: dr. Anouschka van Leeuwen and Professor Nikol Rummel (Ruhr-Universität Bochum, Germany).

To support student collaboration and the teacher dashboard, we make use of the CTAT and MathTutor software developed at Carnegie Mellon University. Associate professor Vincent Aleven takes on an advisory role in the TOrch project.

Publications related to TOrch

Van Leeuwen, A., Rummel, N, et al. (2018). Orchestration tools for teachers in the context of individual and collaborative learning: what information do teachers need and what do they do with it? In Kay, J. and Luckin, R. (Eds.). Rethinking Learning in the Digital Age: Making the Learning Sciences Count, 13th International Conference of the Learning Sciences (ICLS) 2018, Vol. 2, pp. 1227-1234. London, UK: International Society of the Learning Sciences, Inc.

Van Leeuwen, A., & Rummel, N. (2017). Teacher regulation of collaborative learning: research directions for learning analytics dashboards. Proceedings of the 12th international conference on Computer-supported Collaborative Learning (CSCL) 2017, Volume 2, pp 805-806.

Van Leeuwen, A., Van Wermeskerken, M., Erkens, G., & Rummel, N. (2017). Measuring teacher sense making strategies of learning analytics: a case study. Learning: research and practice, 3(1), 42-58.

Van Leeuwen, A., Van Wermeskerken, M., & Erkens, G. (2016). Eye tracking and thinking aloud procedures to study how teachers use learning analytics. Presentation at the EARLI SIG27 meeting in Oulu, Finland.

News

03/2018 End of project

The end of the TOrch project has already come! The main goals of the project - to investigate teacher perceptions of and pedagogical interaction with various types of dashboards - have been accomplished by carrying out several empirical studies. Also, I had a wonderful time in Bochum!

Although the project has officially ended, this website will continue to be updated with publications related to TOrch as we analyze and report on the findings from these studies.

03/2018 Data collection with dashboard prototypes - part 2

Based on the previous prototype study (see earlier posts), we tried to manipulate contextual factors to simulate the classroom environment. In this case, we varied the amount of time teachers had to interpret the dashboard prototypes, as teachers under normal classroom circumstances would not have time to do so for an unlimited amount of time. We thereby try to gain a better understanding of what dashboard role functions best in which circumstances.


02/2018 Symposium was a success!

On February 21st, we organized the symposium with exciting talks by Vincent Aleven and Christa Asterhan. Please find the full program and a photo impression on this page

Vincent Aleven


01/2018 Official symposium announcement

To mark the near end of my Rubicon project, we are hosting a symposium in Bochum on february 21.We would like to cordially invite everyone to attend! Further information is shown on a separate page on this website, including the link to sign up.

12/2017 Happy Holidays!

Happy holidays and best wishes for the new year from Bochum!


11/2017 Data collection with dashboard prototypes

Based on the interviews I conducted in June, we built three dashboard prototypes and investigated teachers' reaction to them in an experimental study. By using fictitious class situations, we could systematically vary the dashboard type and investigate how teachers interpret the information on the dashboard. Hopefulle we will be able to share the results soon!


10/2017 Symposium announcement: "Using traces of digital activity to study and support learning and teaching"

Through the Center of Educational Studies, we received funding to host a symposium near the end of my Rubicon project. The symposium is planned for February 2018 and will combine talks that all illustrate a case study of how to use traces of digital activity to study and support learning and teaching. The symposium therefore offers a chance to discuss challenges and opportunities associated with this particular approach, both conceptually and methodologically. More information soon!

Center of Educational Studies

09/2017 Summer holidays

A late summer break for us this year (in Venice, beautiful place!). Hope everyone had a good summer!

08/2017 Conference in Finland

The biannual EARLI conference was in Tampere, Finland this year (land of Moomins). Although I did not present work on the TOrch project, it was good to see related work from other researchers. 

07/2017 Interview sessions

Another round of data collection this month, By means of elaborate interview sessions with primary school teachers, the aim was to find out what information teachers use in their daily practice to make decisions. One element of the interviews was that teachers were shown storyboards of how dashboards could be used in the classroom, see the picture on the right. Teachers were prompted to voice their opinion on different dashboard roles. Results of the interviews are used as input for our following study, in which we examine intial versions of the dashboard. 

06/2017 Presentation at CSCL conference

The CSCL2017 conference was hosted in Philadelphia this year. The TOrch project was represented with a poster, which outlines an agenda for future research in the area of teacher dashboards.

To read the paper with further details, please see the proceedings of the conference (pp 805-806), which can be found here


05/2017 Short promo on Dutch MathTutor

We created a short video with an impression of the Dutch translation of MathTutor, which we call 'BreukenMeester'. Please note all text in the video is in Dutch. 

05/2017 Article online

The article detailing our eye-tracking study is now online. This paper provides a short discussion of the hypothesised affordances of LA for supporting teachers, and subsequently, a case study is presented in which a teacher’s sense making strategies of LA were investigated within a digital environment, which enabled the teacher to monitor and regulate the activities of five collaborating groups of students. Besides automatically logged teacher behaviour, eye tracking was used to examine eye movements, and cued retrospective reporting was used to evaluate the choices and decisions made by the teacher. The case study is used to discuss directions for future research concerning learning analytics to support teachers.

You can find the full article here


04/2017 Data collection with MathTutor

The first classroom study was conducted this month. We translated existing fraction tasks into Dutch, and had students in gade 6 collaborate on them. I think the students had quite some fun, and after the first session became quite apt with the program. 


04/2017 Visit to Pittsburgh

To discuss the architecture of the dashboard we're developing, I visited the Human-Computer Interaction Institute at Carnegie Mellon University in Pittsburgh. It was an excellent opportunity to gain more understanding of the MathTutor software, and to get started on an initial Dashboard prototype. At the end of the week, we held a small "testathon", where the prototype was subjected to critical tests by the collegues at CMU. 

Pittsburgh

11/2016 Data collection with eye tracker

We have conducted a first pilot study, investigating whether eye-tracking methodology may help us to better understand how teachers use learning analytics/dashboards. An impression of the data collection procedure can be seen on this picture.


10/2016 Start of the project

The TOrch project has started! Looking forward to an inspiring time in Bochum.