Novice Java Programming Mistakes: Large-Scale Data vs. Educator Beliefs
Neil C. C. Brown
Amjad Altadmri

2017

ACM Transactions on Computing Education, Volume 17, Issue 2

Teaching is the process of conveying knowledge and skills to learners. It involves preventing misunderstandings or correcting misconceptions that learners have acquired. Thus, effective teaching relies on solid knowledge of the discipline, but also a good grasp of where learners are likely to trip up or misunderstand. In programming, there is much opportunity for misunderstanding, and the penalties are harsh: failing to produce the correct syntax for a program, for example, can completely prevent any progress in learning how to program. Because programming is inherently computer-based, we have an opportunity to automatically observe programming behaviour -- more closely even than an educator in the room at the time. By observing students’ programming behaviour, and surveying educators, we can ask: do educators have an accurate understanding of the mistakes that students are likely to make? In this study, we combined two years of the Blackbox dataset (with more than 900 thousand users and almost 100 million compilation events) with a survey of 76 educators to investigate which mistakes students make while learning to program Java, and whether the educators could make an accurate estimate of which mistakes were most common. We find that educators’ estimates do not agree with one another or the student data, and discuss the implications of these results.

Study Information
Manually extracted from the paper by the Progmiscon.org team

Programming Languages

Java

Method

Quantitative systematic research analyzing 100+ M compilation events and surveying educators perceptions about 18 selected mistakes

Subjects

900+ students and 76 educators

Phenomena Studied

Errors
A Blackbox dataset of student-generated compilation events over a 2-year period.

Artifact

We are not aware of an artifact supporting this paper.

Related Misconceptions
Referenced by 4 Misconceptions