On Saturday I could sleep in a bit, because my first session started at 13:00 local time.
My first session on Saturday was on Scaffolding, and the first paper was Exploring Algorithm Comprehension: Linking Proof and Program Code. The authors studied how people understand and correct algorithm proofs by running an eye-tracking study. The material consisted of three components: a discription of what the algorithm solves, Java code, and a proof. There were two algorithms, and in the code and proof of both, the researchers embedded an (easy to overlook) error. After the participants identified the error and tried to fix it, they held a retrospective interview where the participant also saw their recorded eye-movements. The authors identified three levels of abstraction in the participants' explanations (mapped to the SOLO taxonomy): 1. pre-structural understanding, 2. uni- and multi-structural comprehension, and 3. relational level. Some implications of this research for teaching are that the code and proof should be structured in a similar fashion to optimize understanding, and that the names of the elements should reflect the meaning of the elements in the proof.
The second paper was on Visual recipes for slicing and dicing data: teaching data wrangling using subgoal graphics. Data wrangling, for most students, is their first foray in programming. But, tabular data is inherently visualizable, and this visualization can then function as a scaffold. The authors tested this hypothesis by creating an online tutorial in two versions: with and without subgoal graphics (but both including subgoals). They did not find any statistically significant differences between the conditions in time on task, number of attempts, number of exercises completed. However, in interviews, the participants did indicate that they found both subgoals and subgoal graphics useful.
The final paper on scaffolding was Stepwise Help and Scaffolding for Java Code Tracing Problems With an Interactive Trace Table. The authors introduce an interactive trace table. Tracing can be done either holistically, or in steps. The downside of holistic tracing, is that when a student has an incorrect answer, they might not know where they went wrong. The tool introduced in this paper allows to trace and predict variables step by step. The students are first given the holistic option, but if they have made a mistake, they can go step-by-step. An evaluation shows that the students' accuracy increases with the use of the trace table.
The work auto generates the trace table, so it is applicable to many different problems.
The evening started with a social program, in which approximately 10 attendees played Sketchful together online. It was a lot of fun, but unfortunately there was not as much discussion in the voice channel. I did get to know more of some of the attendees by the way they participated in the drawing and guessing.
Then there was a final session of research papers, on the topic of Pedagogy.
The first paper was Towards a Framework of Planning Collaborative Learning Scenarios in Computer Science. The authors argue that we need a better implementation of teamwork, because it will lead to better results and increased motivation. But, we can't just tell our students to work well in teams, as teamwork is a complex task. The authors are working on the development of a framework for the design of collaborative learning activities. There are two parts: didactic analyses and instructional design.
Regarding the didactic analyses, the teacher should consider the learning objectives, learner characteristics and framework conditions (such as how to form the groups, and the number of students). For the instructional design, aspects to consider include (pre-)structuring of teamwork, group formation (which should not be left to chance), and what the learning activities of students are.
With all these elements planned, the collaborative learning is more likely to be successful, as there is a positive interdependence between the students. This in turn means that the individuals learn more.
Interestingly, the authors found that novices work better in homogeneous groups, whereas more advanced students work well in heterogeneous groups.
The second paper was Flipped Classroom Teaching in a Mathematics for Technology Course: Recommendations for Success, in which the author discussed how they had experienced running a flipped classroom. Jordan concluded that two things that worked well are high-quality tutorial videos, and lectures with a whiteboard. What did not well included students using a resource database, and use of a Q&A forum. He recommended to have tutorial videos with intermittent questions, weekly quizzes, and (weekly) student feedback.
The final research paper was Evaluating a Pedagogy for Improving Conceptual Transfer and Understanding in a Second Programming Language Learning Context. The authors started by introducing three types of programming language transfer:
In case of TCC, students will learn easily. In case of FCC and ATCC, the authors argue that students need explicit instruction, and then to reflect on their learning. In a between-group study in which the treatment group received such instruction, they found that the intervention group performed significantly better on FCC and ATCC. The students themselves were enthousiastic about this too: they understand the first language better too, as well as notice differences better. The lecturers were also enthousiastic, but one issue is that they may not be familiar with the students' first programming languages. Another practical problem could be that the students could have different programming backgrounds.
In the Doctoral Consortium, the theme was AI education, which is not so close to my own topic. However, I learned some interesting things from the presentations. Some topic were closer to data education, and might be relevant for our workshop proposal at SIGMOD. They are certainly candidates to keep an eye on.
Then it was time for the official closing. The best papers were announced, as were the PC awards. Normally, the new location for next year's conference would be announced, but of course Koli Calling is always in Koli! There was an announcement that Koli 2022 will hopefully be hybrid. Also interesting was the pulling forward of the submission deadline. So, thankfully, the deadlines for Koli Calling and SIGCSE are no longer as close together.
The unofficial closing was done through Dr. Nick's Wine Show. Although for Nick it was 6AM in Australia, he told us a lot about wine tasting in general, and the three bottles he opened specifically. He even included a graphic that could help us discover new wines, based on the ones we liked already. Unfortunately, I don't really like wine, and would have no idea what to do with three opened bottles of wine, so I just listened in.