Learning from instantaneous feedback designed for experts, provided by a machine

I remember reading Matt Jadud‘s thesis and being struck by a paragraph on punched cards, repeated here with kind permission:

In 1967, the Univac 1107 system running at Case University (later to become Case Western Reserve University) in Cleveland, Ohio, boasted an impressive turnaround-time of 12 hours on most jobs. Lynch reports that “A user can submit a deck at 8:00 a.m., have it on the input tape by noon, his job completed by 12:35, and the output returned to him by 5:00 p.m. in the evening.”[Lyn67] Compared to students today (who work in laboratories equipped with many machines, each thousands of times more powerful than the Univac 1107), the 9-hour turnaround time seems to approach infinity; but Lynch goes on to say that “It should be noted that only 10-15 percent of the runs are the first of the day, 85-90 percent are repeats, and about a third of the runs have a circulation time of less than 5 minutes. It is often possible to debug a moderate size program in less than an hour by gaining frequent access to the computer.”

This early report on programmer behaviour seems to imply that having access to rapid feedback from a computer regarding the syntax and semantics of a program was a valued interaction style. Not only is this behaviour exhibited by students today, but tools like Eclipse continuously re-compile the programmer’s code, highlighting errors as the programmer develops their code in real-time; this is the “rapid compilation cycle” taken to its natural limit.

It wasn’t that the above contains anything I didn’t pretty much know, but I had never thought about it in quite that way. Imagine having to wait hours or days to find out if a program compiled successfully! Perhaps some readers can remember those days. Anyway, this got me to thinking: what if compilers didn’t return error messages (or an indication that the compilation was successful) immediately? How would programming, and teaching programming, be different? This quickly led to thinking about how learning, and teaching, programming is different to other disciplines – and what that means.

In 1986, Perkins, et al. noted that under normal instructional circumstances some students learn programming much better than others. Investigations of novice programmer behavior suggest that this happens in part because different students bring different patterns of learning to the programming context. Students often fall into varying combinations of disengaging from the task whenever trouble occurs, neglecting to track closely what their programs do by reading back the code as they write it, trying to repair buggy programs by haphazardly tinkering with the code, or having difficulty breaking problems down into parts suitable for separate chunks of code. The authors categorized programming students into groups: stoppers, who quickly gave up when faced with errors; movers, who would work their way through, around or away from errors; and tinkerers, who poke, tweak, and otherwise manipulate their code in a variety of small ways, sometimes making progress towards a working program, sometimes not. Either way, the fact that the compiler provides instant feedback is what allows these behaviors to exist.

Today’s students must write code differently than their counterparts decades ago due to this instant feedback. the behaviors described by Perkins et al. wouldn’t be possible (or would be very different) if students had to wait hours or days for compiler feedback. I have witnessed students use the compiler to tell them where their errors are. I have done the same myself. Why bother pouring over your code with spectacularly aching scrutiny when the compiler will instantly, and with little concrete penalty, tell you what you did wrong (syntactically)? All you have to do is click this little button! Note that when I say instantaneous feedback, I am not addressing Eclipse-like red squiggly line behavior. That is of course ‘more instantaneous’ than having to click a button. But traditional push-the-button feedback is more or less instantaneous compared to the way it used to happen. As for the red squiggly lines, watch this space, there will be something to come there soon. Also, in this post I am not addressing the point that most compiler feedback is poor. See this post for an example.

Is there something lost in the present scenario? Was there value to thinking like a compiler, going over your code to ensure that when the real compiler does, there are no errors? Is there something that students are not learning, not experiencing, or not taking away from their  programming practice in not doing so?

I will leave that question where it is, lest we disappear down the rabbit hole. It is however worthy of future thought.

Before this post ends however, what about the feedback that compilers provide? Feedback is of course regarded as one of the most important influences on learning and motivation, noted by Watson et al. in the context of programming. Unique to computer programming is the fact that much of the feedback that programming students receive comes:

Edit Jan 30 2017: Also see this post for a discussion and an example of what can be done about this.

Clearly, programming students are in a different world when they are in CS1, compared to students in Phys1, Med1, or many (all?) other 101/introductory courses where feedback comes:

  • from a human
  • often delayed
  • designed for them, and (hopefully) in terms that they can fully understand

What does this mean for those that are learning to program? They get instant, difficult to understand feedback from a machine! For one thing, as they are in a different world to so many other students, their expectations, their outcomes, and the way that they are taught should probably be different too.

I’ll avoid the second rabbit hole by leaving that thought there… for now.

Becker, B.A. (2016) An Effective Approach to Enhancing Compiler Error Messages. In Proceedings of the 47th ACM Technical Symposium on Computer Science Education (SIGCSE 2016), (pp. 126-131). ACM

Jadud, M. C. (2006). An exploration of novice compilation behaviour in BlueJ. PhD Thesis. University of Kent

Perkins, D. N., Hancock, C., Hobbs, R., Martin, F., & Simmons, R. (1986). Conditions of learning in novice programmers. Journal of Educational Computing Research, 2(1), 37-55

Traver, V. J. (2010). On compiler error messages: what they say and what they mean. Advances in Human-Computer Interaction, 2010

Watson, C., Li, F. W., & Godwin, J. L. (2012). BlueFix: using crowd-sourced feedback to support programming students in error diagnosis and repair. In International Conference on Web-Based Learning (pp. 228-239). Springer Berlin Heidelberg

‘Supercomputing’ in the curriculum

A recent article on ComputerWeekly.com is calling for supercomputing to be put ‘in the curriculum’. In it, Tim Stitt, head of scientific computing at the Earlham Institute, a life science institute in Norwich, UK, says children should be learning supercomputing and data analysis concepts from a young age.

Although I agree in principle, the article doesn’t specify a particular curricula although it does seem to be aimed at pre-university ages. In the article, Stitt claims that current initiatives such as the new computing curriculum introduced in the UK in 2014 which makes it mandatory for children between the ages of five and 16 to be taught computational thinking, may “compound the issue”, as children will be taught serial rather than parallel programming skills, making supercomputing concepts harder to learn later on. Again, I can agree in principle, but the extent to which learning parallel programming after learning ‘normal’ sequential programming is debatable, and will certainly vary considerably from student to student.

I have mixed feeling about the word supercomputing. I can imagine someone saying “Really? You are going to teach supercomputing to kids? Don’t you think that’s a bit much?” I couldn’t blame them for being skeptical. The word itself sounds, well, super. Personally I think that High Performance Computing (HPC) is more down to earth, but I also concede that that may still sound a little ‘super’. I have some experience with this. I am one of many that maintain the Irish Supercomputer List. That project didn’t start off as the Irish Supercomputer List, but we changed the name in order to, quite frankly, be more media ‘friendly’. (Side note – interesting discussion on disseminating scientific work to the media here).  Additionally, the Indian and Russian lists also have the word supercomputing in their names and/or URLs. The Top500 list also used the word supercomputing before they rebranded a few years back. Anyway…

So, what we are really talking about is putting Parallel Computing (or parallel programming) in the curriculum, and therefore opening the door to supercomputing, as almost all HPC installations require parallel programming. In fact the current Top500 Supercomputer List is composed entirely of clusters (86.2%) or Massively Parallel Processors (MPPs – 13.8%). Clusters are parallel computer systems comprising an integrated collection of independent nodes, each of which is a system in its own right, capable of independent operation and derived from products developed and marketed for other stand-alone purposes [1]. MPPs (such as the IBM Blue Gene) on the other hand, are more tightly-integrated. Individual nodes cannot run on their own and they are frequently connected by custom high-performance networks. They key here is that in both cases memory is distributed (as are the cores), thus requiring parallel algorithms (and therefore parallel programming).  Before switching gears I would like to return to the point I opened this paragraph with – we are talking about parallel programming – not necessarily supercomputing – although learning parallel computing is indeed the essential requirement to eventually program supercomputers.

At the university level, there is more than an awareness of the issues that form the core of the argument which is the focus of the article that I started this post with. In particular there are two conferences/workshops that directly address HPC education at university level:

  1. Workshop on Education for High-Performance Computing (EduHPC-16), held in conjunction with SC-16: The International Conference on High Performance Computing, Networking, Storage, and Analysis
  2. The Parallel and Distributed Computing Education for Undergraduate Students Workshop (Euro-EDUPAR 2016), held in conjunction with Euro-Par 2016, the 22nd International European Conference on Parallel and Distributed Computing.

[1] Dongarra, J., Sterling, T., Simon, H. and Strohmaier, E., 2005. High-performance computing: clusters, constellations, MPPs, and future directions. Computing in Science and Engineering, 7(2), pp.51-59