Learning from instantaneous feedback designed for experts, provided by a machine

I remember reading Matt Jadud‘s thesis and being struck by a paragraph on punched cards, repeated here with kind permission:

In 1967, the Univac 1107 system running at Case University (later to become Case Western Reserve University) in Cleveland, Ohio, boasted an impressive turnaround-time of 12 hours on most jobs. Lynch reports that “A user can submit a deck at 8:00 a.m., have it on the input tape by noon, his job completed by 12:35, and the output returned to him by 5:00 p.m. in the evening.”[Lyn67] Compared to students today (who work in laboratories equipped with many machines, each thousands of times more powerful than the Univac 1107), the 9-hour turnaround time seems to approach infinity; but Lynch goes on to say that “It should be noted that only 10-15 percent of the runs are the first of the day, 85-90 percent are repeats, and about a third of the runs have a circulation time of less than 5 minutes. It is often possible to debug a moderate size program in less than an hour by gaining frequent access to the computer.”

This early report on programmer behaviour seems to imply that having access to rapid feedback from a computer regarding the syntax and semantics of a program was a valued interaction style. Not only is this behaviour exhibited by students today, but tools like Eclipse continuously re-compile the programmer’s code, highlighting errors as the programmer develops their code in real-time; this is the “rapid compilation cycle” taken to its natural limit.

It wasn’t that the above contains anything I didn’t pretty much know, but I had never thought about it in quite that way. Imagine having to wait hours or days to find out if a program compiled successfully! Perhaps some readers can remember those days. Anyway, this got me to thinking: what if compilers didn’t return error messages (or an indication that the compilation was successful) immediately? How would programming, and teaching programming, be different? This quickly led to thinking about how learning, and teaching, programming is different to other disciplines – and what that means.

In 1986, Perkins, et al. noted that under normal instructional circumstances some students learn programming much better than others. Investigations of novice programmer behavior suggest that this happens in part because different students bring different patterns of learning to the programming context. Students often fall into varying combinations of disengaging from the task whenever trouble occurs, neglecting to track closely what their programs do by reading back the code as they write it, trying to repair buggy programs by haphazardly tinkering with the code, or having difficulty breaking problems down into parts suitable for separate chunks of code. The authors categorized programming students into groups: stoppers, who quickly gave up when faced with errors; movers, who would work their way through, around or away from errors; and tinkerers, who poke, tweak, and otherwise manipulate their code in a variety of small ways, sometimes making progress towards a working program, sometimes not. Either way, the fact that the compiler provides instant feedback is what allows these behaviors to exist.

Today’s students must write code differently than their counterparts decades ago due to this instant feedback. the behaviors described by Perkins et al. wouldn’t be possible (or would be very different) if students had to wait hours or days for compiler feedback. I have witnessed students use the compiler to tell them where their errors are. I have done the same myself. Why bother pouring over your code with spectacularly aching scrutiny when the compiler will instantly, and with little concrete penalty, tell you what you did wrong (syntactically)? All you have to do is click this little button! Note that when I say instantaneous feedback, I am not addressing Eclipse-like red squiggly line behavior. That is of course ‘more instantaneous’ than having to click a button. But traditional push-the-button feedback is more or less instantaneous compared to the way it used to happen. As for the red squiggly lines, watch this space, there will be something to come there soon. Also, in this post I am not addressing the point that most compiler feedback is poor. See this post for an example.

Is there something lost in the present scenario? Was there value to thinking like a compiler, going over your code to ensure that when the real compiler does, there are no errors? Is there something that students are not learning, not experiencing, or not taking away from their  programming practice in not doing so?

I will leave that question where it is, lest we disappear down the rabbit hole. It is however worthy of future thought.

Before this post ends however, what about the feedback that compilers provide? Feedback is of course regarded as one of the most important influences on learning and motivation, noted by Watson et al. in the context of programming. Unique to computer programming is the fact that much of the feedback that programming students receive comes:

Edit Jan 30 2017: Also see this post for a discussion and an example of what can be done about this.

Clearly, programming students are in a different world when they are in CS1, compared to students in Phys1, Med1, or many (all?) other 101/introductory courses where feedback comes:

  • from a human
  • often delayed
  • designed for them, and (hopefully) in terms that they can fully understand

What does this mean for those that are learning to program? They get instant, difficult to understand feedback from a machine! For one thing, as they are in a different world to so many other students, their expectations, their outcomes, and the way that they are taught should probably be different too.

I’ll avoid the second rabbit hole by leaving that thought there… for now.

Becker, B.A. (2016) An Effective Approach to Enhancing Compiler Error Messages. In Proceedings of the 47th ACM Technical Symposium on Computer Science Education (SIGCSE 2016), (pp. 126-131). ACM

Jadud, M. C. (2006). An exploration of novice compilation behaviour in BlueJ. PhD Thesis. University of Kent

Perkins, D. N., Hancock, C., Hobbs, R., Martin, F., & Simmons, R. (1986). Conditions of learning in novice programmers. Journal of Educational Computing Research, 2(1), 37-55

Traver, V. J. (2010). On compiler error messages: what they say and what they mean. Advances in Human-Computer Interaction, 2010

Watson, C., Li, F. W., & Godwin, J. L. (2012). BlueFix: using crowd-sourced feedback to support programming students in error diagnosis and repair. In International Conference on Web-Based Learning (pp. 228-239). Springer Berlin Heidelberg


One thought on “Learning from instantaneous feedback designed for experts, provided by a machine

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s