Last week I attended the LearnLab summer school at Carnegie Mellon University. The focus of the summer school was to teach students about learning sciences theories and methodologies, both through readings and hands-on work. Over the week, I learnt a lot about theorethical foundations in the learning sciences. As a note-to-self, and maybe a resource for you, below, I'll elaborate on the theories presented and how they can be incorporated in research.
Now, let's move onto the theories.
source: Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012).
Knowledge Components are part of the KLI framework which is discussed in more detail below. However, I want to reflect on them individually too, as they return in many of the theories discussed in this blog.
According to Koedinger et al., Knowledge components are defined as:
Koedinger et al. identify types of KCs over two dimensions: the dimension of application, and the response dimension. Both of these can be constant or variable. Constant means that the KC is associated with a single unique pattern, whereas variable means that it might fit to multiple patterns. The three defined KC types are:
Knowledge Components are central to the KLI framework, but how do we identify individual ones? This is a really hard task. For example, one might think multiplication and subtraction are elementary elements and thus should be KCs. However, there are other factors that influence this difficulty: can we just multiply left to right, or should we incorporate order of operations rules. Is our subtraction one that is positive, or will our result be in the negative numbers?
More specifically for Computer Science, code tracing can help us understand understanding. The core feature that can help us identify KCs are learning curves. These are graphs made based on student performance over a sequence of exercises. If the set of exercises captures only one KC, the performance should go up. If we see strange patterns in the learning curve, it is possible that we have not decomposed our KCs enough.
Conclusion: Trying to model Knowledge Concepts for programming languages seems like a very interesting challenge. During the summerschool, my partner Jawwad and I started trying this for simple SQL query formulation. I look forward to working on this in more depth, and would be happy to find collaborators to do so.
source: Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012).
The article on the KLI framework was one of the largest preliminary readings that was offered for the summer school. With over 40 pages, it is quite the task. However, it was one of the most interesting things I learnt about this week.
One thing we should realize about learning is that it is done in the brain, and we cannot observe this process directly. Instead, we can build models that connect learning to real-world, observable, phenomena. The KLI framework is built surrounding this principle.
The framework identifies 30 different learning principles related to teaching and instruction. These are in three categories:
Given all these different principles, there are millions of unique combinations of instruction: instructional complexity. The KLI framework aims to identify those should always work well: instead of targeting by domain, they target based on the Knowledge Component type as discussed above, as well as on which Learning Event we want to reach.
Interestingly, the framework shows that applying more advanced learning techniques may have a negative influence on students' learning of simpler KC types. The most optimal learning effect is accomplished when the involvement of the learning event matches the difficulty of the knowledge component.
Conclusion: This seems like an important finding to keep in mind: we can design highly involved instructional materials, but if they do not match the difficulty of the concepts we are teaching, this work is wasted. So, appropriate educational design requires us to know more of the knowledge components that we are trying to teach.
Cognitive task analysis means that we try to understand the cognitive processes of a learner while they are completing some task. There are many different ways in which we can do this, for example: structured interviews of experts, theoretical analysis, think alouds, difficulty factors assessments, learning curve analysis.
Theoretical task analysis specifically is very interesting, as it allows us to create a direct model which we can then test. The steps are as follows:
We can also use think-aloud studies to develop a model, or use think-aloud to try to confirm the model we developed before. To do so, we try to mark each of the lines in a transcript with subgoals for the CTA. Are there any goals that students are skipping? Are there extra subgoals that we have not included? If we can iterate over the model enough, we can build a theory of cognition for the task.
Conclusion: CTA might be able to help us identify the appropriate KCs to focus on in education. All of these theories are related and interconnected.
Source: Aleven, V., McLaughlin, E. A., Glenn, R. A., & Koedinger, K. R. (2017). Instruction based on adaptive learning technologies. In R. E. Mayer & P. Alexander (Eds.), Handbook of Research on Learning and Instruction (pp. 522-560). New York: Routledge.
Instruction is the most effective when it takes into account that learners are different, and that they change as they learn. There are many aspects to learners and learning that we can adapt to: knowledge, strategies and errors, affect and motivation, self-regulation fo learning, learning styles. This, in combination with what we can adapt: design, tasks, and steps, leads to the adaptivity grid.Examples of adaptivity include:
Conclusion: There are many more ways to adapt to student learning than I had ever thought possible. There is a wealth of things to still learn in this area.
The idea behind worked examples came from the application of cognitive load theory in education. If we know that students struggle to understand both the problem and the problem-solving method at the same time, we can provide them with worked examples: step-by-step solutions and explanations for a problem. The idea is that using worked examples, students should require less extraneous load, which means that they have more resources to process information, helping them learn more.
The concept of faded scaffolding takes this further. Students are supposed to learn from the worked examples, building some sort of step-wise solution model related to the task. Faded scaffolding means that the worked examples stay, but instead of showing the same thing over and over, they start dropping the explanation for final steps. This means that the student needs to perform the final steps themself. Withdrawing support further incrementally, means that by the end, the student fully knows how to solve the problem, without suffering from high cognitive load.
Conclusion: I like how worked examples and scaffolding take away some of the problems that students might have given cognitive load. Cognitive load is a very commonly mentioned program surrounding my work in SQL query formulation as well. During the LearnLab week, Jawwad and I drafted two worked examples for SQL. I see much promise in this direction for providing strong, clear examples to our students. In my misconception work, I found that students may have learnt query templates that they then apply incorrectly. If we can develop enough appropriate worked examples, this problem may also be reduced.
In conclusion, I have learnt a lot during the week of summer school. I fully recommend students and others working in CER to attend LearnLab. The new insights, both regarding theory and practice will hopefully prove valuable in the coming years. I leave Pittsburgh with my brain full of new ideas and research questions, as if I did not have enough of those already... To be continued!