37 Million Compilations: Investigating Novice Programming Mistakes in Large-Scale Student DataAmjad AltadmriNeil C.C. Brown
Amjad Altadmri
Neil C.C. Brown
2015
Proceedings of the 46th ACM Technical Symposium on Computer Science Education
Previous investigations of student errors have typically focused on samples of hundreds of students at individual institutions. This work uses a year's worth of compilation events from over 250,000 students all over the world, taken from the large Blackbox data set. We analyze the frequency, time-to-fix, and spread of errors among users, showing how these factors inter-relate, in addition to their development over the course of the year. These results can inform the design of courses, textbooks and also tools to target the most frequent (or hardest to fix) errors.
Study InformationManually extracted from the paper by the Progmiscon.org team
Programming Languages
Java
Method
Repository mining study (BlueJ Blackbox)
Subjects
250000+ students across the world
Phenomena Studied
Mistakes
Misunderstandings, type errors, other semantic errorsArtifact
https://bluej.org/blackbox/Related MisconceptionsReferenced by 9 Misconceptions
AssignCompares — Java
= compares two valuesAssignCompares — Python
= compares two valuesNoReservedWords — Java
Every sequence of letters and digits starting with a letter can be used as an identifierNoReservedWords — Python
Every sequence of letters and digits starting with a letter or an underscore can be used as an identifierNoShortCircuit — Java
&& and || always evaluate both operandsNoShortCircuit — Python
and/or always evaluate both operandsNoSingleLogicAnd — Java
& is only a bitwise ANDParenthesesOnlyIfArgument — Java
() are optional for method calls without argumentsParenthesesOnlyIfArgument — Python
() are optional for function calls without arguments