Viewpoint

Crossing the Software Education Chasm

By Armando Fox, David Patterson
Communications of the ACM, Vol. 55 No. 5, Pages 44-49
10.1145/2160718.2160732
Comments (6)

Via a remarkable alignment of technologies, the future of software has been revolutionized in a way that also makes it easier to teach. Cloud computing and the shift in the software industry^a toward Software as a Service^b (SaaS) using Agile development has led to tools and techniques that are a much better match to the classroom than earlier software development methods. In addition, modern programming frameworks for Agile development like Ruby on Rails demonstrate that big ideas in programming languages can deliver productivity through extensive software reuse. We can leverage such productivity to allow students to experience the whole software life cycle repeatedly within a single college course, which addresses many criticisms from industry about software education. By using free-trial online services, students can develop and deploy their SaaS apps in the cloud without using (overloaded) campus resources. Our enthusiasm for Agile, SaaS, and cloud computing is not based just on improving students' employability; rather, we are excited that they can learn and practice software engineering fundamentals, and that they use them even after graduation. Our 50,000 online students are a testimony to the popularity of the material and the scalability of our SaaS "courseware."

Back to Top

The Complaints and the Defense

While new college graduates are good at coding and debugging,¹ employers complain about other missing skills that are equally important. A standard faculty reaction to such criticisms is that we are trying to teach principles that prepare students for their whole careers; we are not trade schools that teach the latest superficial fads.

To understand the complaints in more depth, we spoke to representatives from a half-dozen leading software companies. We were struck by the unanimity of the number-one request from each company: that students learn how to enhance sparsely documented legacy code. In priority order, other requests were making testing a first-class citizen, working with nontechnical customers, performing design reviews, and working in teams. We agree that the social skills needed to work effectively with nontechnical customers, work well in teams, and perform effective design reviews are helpful for the students' whole careers—the question is how to fit them into a course. Similarly, no one questions the value of emphasizing testing—the question is how to get students to take it seriously.

Instructors respond that even if we agreed with the recommendations, time constrains how much one class can cover. A typical undergraduate workload of four courses per term and a 50-hour workweek gives students about 12 hours per week per course, including lectures, labs, exams, and so forth. This works out to approximately 120 hours per quarter to 180 hours per semester, or just three to four weeks for a full-time developer!

Back to Top

A Classroom Opportunity: Agile Development

The Agile Manifesto signaled a paradigm shift for many software applications. This approach embraces change as a fact of life; small teams of developers continuously refine a working but incomplete prototype until the customer is happy with the result, with the customer offering feedback with every iteration. Agile emphasizes Test-Driven Development^c (TDD) to reduce mistakes, which addresses industry's request to make testing a first-class citizen; user stories^d to reach agreement and validate customer requirements, which addresses working with nontechnical customers; and velocity^e to measure progress. The Agile software philosophy is to make new versions available every two weeks. The assumption is basically continuous code refactoring over its lifetime, which develops skills that can also work with legacy code. Clearly, small teams and multiple iterations of incomplete prototypes could match the classroom.

---

The only hope for addressing all the concerns from industry within the course time constraints is to use a highly productive programming framework.

---

Note that we do not tell students that Agile is the only way to develop software; indeed, we explain Agile is inappropriate for safety-critical apps, for example. We believe that new programming methodologies develop and become popular in response to new opportunities, so we tell students to expect to learn new methodologies and frameworks in the future. Our experience is that once students learn the classic steps of software development and have a positive experience in using them via Agile, they will use these key software engineering principles in other projects no matter which methodology is used (see Figure 5).

Back to Top

A Classroom Target for the Post-PC Era: "I Do and I Understand"

To motivate students, it is helpful to use a platform that allows them to create compelling apps. In this emerging post-PC era, mobile applications for smartphones and tablets and Software as a Service (SaaS) for cloud computing are both compelling. (50,000 students are evidence that SaaS is indeed compelling.) As you can teach the principles with either target, given the time constraints mentioned earlier, why not pick the platform that has the most productive tools? Our view is that the only hope for addressing all the concerns from industry within one course is to use a highly productive programming framework.

Our experience is that the Rails ecosystem has by far the best tools to support test-driven development, behavior-driven design, and Agile processes, many of which are made possible by intellectually deep Ruby language features such as closures, higher-order functions, functional idioms, and metaprogramming. The accompanying table shows critical topics from the SWEBOK (software engineering body of knowledge),⁵ the Agile technique for that topic, and the Rails tool that implements that technique. Because these tools are lightweight, seamlessly integrated with Rails, and require virtually no installation or configuration—some are delivered as SaaS—students quickly begin learning important techniques directly by doing them. We agree with Confucius: "I hear and I forget. I see and I remember. I do and I understand." Students see and use tools that we can explain and check, rather than just hear lectures about a methodology and then forget to use them in their projects.

For example, the Cucumber tool automates turning customer-under-standable user stories into acceptance tests. Figure 1 is an example feature for a cash register application and one "happy path" user story (called a scenario in Cucumber) for that feature (see http://en.wikipedia.org/wiki/Cucumber_%28software%29). Note that this format is easy for the nontechnical customer to understand and help develop, which is a founding principle of Agile and addresses a key criticism from industry. Cucumber uses regular expressions to match user stories to the testing harness. Figure 2 is the key section of the Cucumber and Ruby code that automates the acceptance test by matching regular expressions.

Such tools not only make it easy for students to do what they hear in lecture, but also simplify grading of student effort from a time-intensive subjective evaluation by reading code to a low-effort objective evaluation by measuring it. SimpleCov measures test coverage, saikuro measures cyclomatic complexity of the code, flog measures code assignment-branch-condition complexity, reek comments on code quality by highlighting "code smells,"^f Cucumber shows the number of user stories completed, and Pivotal Tracker records weekly progress and can point out problems in balance of effort by members of teams. Indeed, these tools make it plausible for the online course to have automatically gradable assignments with some teeth in them.

Compared to Java and its frameworks, Rails programmers have found factors of three to five reductions in number of lines of code, which is one indication of productivity.^6,8 Rails also helps with a criticism of Agile in that TDD and rapid iteration can lead to poor software architecture. We do teach design patterns (see Figure 3). Indeed, the Rails framework follows the Model View Controller (MVC)^g design pattern to simplify development of the classic three-tiered applications of cloud computing. In addition, because of cloud computing, deploying their projects in the same horizontally scalable environment used by professional developers is instant, free for small projects, and requires neither software installation nor joining a developer program. In particular, it frees the course from instructional computers, which are often antiquated, overloaded, or both.

One criticism of the choice of Ruby is its inefficiency compared to languages like Java or C++. Since hardware has improved approximately 1,000X in cost-performance since Java was announced in 1995 and 1,000,000X since C++ was unveiled in 1979,⁷ the efficiency of low-level code matters in fewer places today than it used to. We think using the improved cost-performance to increase programmer productivity makes sense in general, but especially so in the classroom. Note that for cloud computing, horizontal scalability can trump single-node performance; deploying as SaaS on the cloud in this course lets us teach (and test) what makes an app scalable across many servers, which is not covered elsewhere in our curriculum. Once again, without using the cloud to teach the class, we could not offer students the chance to experiment with scalability.

Back to Top

An Example Course

We use this approach to teach a software course, which currently has more than 100 juniors and seniors, at UC Berkeley. We also are offering the first part as a massive open online course (MOOC) to 50,000 online students, most of whom work in the IT industry and graduated from college five to 10 years ago.³ (The MOOC tests the scalability of the tools and automatic grading of programming assignments.) Students follow steps shown in Figure 3, which turns the concepts and tools listed in the table into a logical process. Additional tools facilitate tying together the steps of this process; for example, Autotest monitors the source code tree in the background, automatically rerunning tests and user stories via RSpec/Cucumber in response to even minor code changes, giving immediate feedback if something breaks. Following the Agile philosophy, they deploy on Amazon Web Services (AWS) (via Heroku) multiple times during the course. The teaching staff evaluates iterations by interacting with the deployed app on AWS and by using Pivotal Tracker to check velocity and stories remaining to be implemented.

A typical faculty reaction to the request for students to deal with poorly documented legacy code is that it should not exist if students are taught good practices. However, despite decades of well-meaning instructors expounding on the importance of properly factored and highly documented code, our industry colleagues assure us the legacy code problem will continue to persist. Thus, on this point we quote Shimon Peres: "If a problem has no solution, it may not be a problem, but a fact—not to be solved, but to be coped with over time." Hence, if we can find principles that teach how to cope with legacy code, they would be appropriate for the classroom since it is a long-lasting challenge.

To learn to deal with legacy apps, our students learn and enhance a large, popular, well-written (but poorly documented) open source app written in Rails. We use Typo, a blogging framework containing 30,000 lines of Ruby and 15,000 lines of JavaScript, which suggests that it has the functionality of a Java program of 100,000 to 150,000 lines of code. We choose a Rails app rather than a Java app to maintain the programmer productivity we need to fit within our time budget. Feathers² argues that the first step is to write tests for legacy code to ensure understanding before modification, so enhancing legacy code fits surprisingly well with the test-driven development of Agile.

To learn to communicate with nontechnical customers, for the Berkeley course we asked for projects from nonprofit organizations. The projects are typically less than 3,000 lines of code, with typically two to three times more code for testing than for the app. Teams of four or five students meet with customers on each Agile iteration for feedback on the current prototype and to prioritize the next features to add. (Moreover, we encourage students to connect these applications to Facebook or Twitter, which gives students the chance to deal with users as well as nontechnical customers within a single course.) Teams members do design reviews for each other as part of a biweekly class laboratory section, as Agile favors frequent code check-ins and design reviews. (Online students do not do projects.)

Back to Top

Using the Course Content Afterward

Figure 4 shows the survey results of Berkeley students from two earlier course offerings. Just 22 of the 47 respondents had graduated, and just 19 had done significant software projects. The percentages indicate the results of their 26 software projects. We were surprised that Agile software development was so popular (68%) and that the cloud was such a popular platform (50%). Given that no language was used in more than 22% of the projects, our alumni must be using Agile in projects that use languages other than Ruby or Python. All the class teams had four or five students, which directly matches the average team size from the survey.

Once again, Agile development and Rails were not selected because we expected them to dominate students' professional careers upon graduation; we use them to take advantage of their productivity so we can fit several critical ideas into a single college course in the hope they will use them later no matter what methodology, programming language, and framework.

Figure 5 shows the students' ranking of the topics the table and Figure 3 in terms of usefulness in their industrial projects. Most students agreed that the top nine topics in the course were useful in their jobs. Once again, we were pleased to see that these ideas were still being used, even in industrial projects that did not rely on Agile or on Rails. The two lower ranked topics were Lo-Fi User Interface Mockups, which makes sense since few developers work on the UI of a user-facing project, and Velocity, as progress can be measured in other ways in industry.

Although a small sample and not a conclusive user study, our survey offers at least anecdotal evidence that students of this course do continue to use successful software development techniques in later software projects of all kinds.

Back to Top

Conclusion

Using Agile to develop SaaS apps via highly productive tools like Rails and deploying them using cloud computing cultivates good software practices and pleases many stakeholders:

• Students like it because they get the pride of accomplishment in shipping code that works and is used by people other than their instructors, plus they get experience that can help land internships or jobs.
• Faculty like it because students actually use what they hear in lecture, even after graduation, and they experience how big CS ideas genuinely improve productivity. Virtual machines reduces hassles for faculty plus the cloud allows for more interesting programming assignments—which the testing and code evaluation tools of Rails can help grade—thereby allowing us to offer a MOOC with 50,000 students.
• Colleagues in industry like it because it addresses several of their concerns. An example is this quote from a Googler: "I think what you're doing in your class is amazing. I'd be far more likely to prefer graduates of this program than any other I've seen. As you know, we're not a Ruby shop, but I feel this is a good choice for the class to be able to get real features done. Java or C++ would take forever."⁴

We received similar comments from colleagues at Amazon, eBay, and Microsoft, none of which are "Ruby shops." As we expected, leading software companies prefer students learn important ideas rather than steer us to teach specific languages and tools used inside those companies.

We believe Agile+Cloud+Rails can turn a perceived weakness of the CS curriculum into a potential strength. If you are a potentially interested instructor, we would be happy to help you cross the long-standing chasm between what industry recommends and what academia offers.

Back to Top

Back to Top

Back to Top

Back to Top

Back to Top

Back to top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2012 ACM, Inc.

---

Comments

---

vbxplxbfv

April 25, 2012 01:44

I'm also teaching a software engineering course mostly based on the course materials including the coming book - http://beta.saasbook.info/

I'm also getting a very positive feedback, (well - from most of the students and even some alumni who joined the course). The students enjoy the challenges of learning new but very relevant ideas while practicing the materials hands on.

I would like to thanks the authors for their open attitude, for all sharing and even supplying technical help. The course's supplied virtual machine (bookware) saves a lot of valuable time spent in practicing and learning instead of setup and configurations.

I believe a big challenge for this effort is keeping it updated, since many of the course tools and practices are still forming as we talk.
Reuven Y,
JCE-IL

---

David Patterson

April 25, 2012 09:28

Thanks for the nice comments!

We agree with the challenge of keeping up to date of computer science in general, but especially for software related courses, since the software tools change more quickly than traditional textbooks do.

That is one of the main why we were intrigued by the potential of three relatively new technologies:
1) Electronic textbooks
2) Virtual Machines (where old is new again)
3) Massive Online Open Courses (MOOCs)

1) Ebooks (and print-on-demand paper books) means there is no warehouse full of books that must be sold before you can make a revision. Hence, dedicated authors can track changes to software and include great new software tools by releasing a new version of the book, say, every year. Moreover, instead of errata, for ebooks you can release new versions with corrections immediately and readers with old editions can automatically get corrections to their copies. This option is particularly important when you're writing about software, as its easy to get some detail wrong that might frustrate readers trying to do what you say in the book.

2) Virtual Machines means the authors can put together a stable set of software tools that all work together, and hence all students can use the same version of all the software that goes with the book without configuration hassles. We just need to update the VM with each edition of the book.

Note that we worried whether VMs would really work with 50,000 students around the world, but amazingly, it did. We got no complaints from students who downloaded the VM and installed our image.

3) One challenge for textbook writers is how do you teach the teachers about relatively new ideas, new tools that they haven't seen, and perhaps even programming languages and frameworks that they haven't used. The MOOC means professors willing to learn new ideas can do it in their spare time. Moreover, instead of just seeing Powerpoint slides and trying to guess what the authors meant, they can watch the lectures themselves to see what they want to incorporate in their classes.

The more radical version of solving the "how do you educate the educators" is the "flipped classroom," where students watch the online videos before class and the lecture becomes a recitation where the professor works with students in small groups, clearing up misunderstandings and answering questions. We've never tried this, and have no strong opinions one way or the other, but some are strong advocates and it's an option for the even more adventuresome to try.

Dave Patterson
UC Berkeley

P.S. Our next MOOC starts May 18, in case anyone is interested. See www.saas-class.org

---

Anonymous

April 25, 2012 05:35

I very much enjoyed this article. It was refreshing to see how Professors Patterson and Fox have updated this important course at Berkeley (CS169), which I think has often been a hard course to teach in a relevant way. Although an human-computer interaction researcher by training, I had the opportunity to teach this once at Berkeley. Given my lack of expertise in the area I was forced to follow a textbook, which seemed overly theoretical. Having this type of material available at the time would have made the course much more practical and impactful on my students' future software careers.

As an aside, I'm not surprised that the graduates have not found the low-fi prototyping techniques a common part of their job as I think the UI part of projects are often more likely given to more experienced engineers or even usability/design specialists (at least in larger companies).

James Landay
CSE Department
University of Washington

---

Klaus-Erik Schauser

April 25, 2012 05:35

Hi Dave and Armando,

Congratulations on the the success of the class, book, paper, and video !!! It is AMAZING that you were able to teach that many students. I am proud of you. We wish that all computer science students had taken such a course.

At AppFolio we have been using Rails, TDD, Pivotal Tracker, Selenium, etc for the past 5 years and our team loves it. We now probably have one of the largest commercial Rails application.

We are onto our third product (latest is www.securedocs.com) and still do releases every 2 weeks, which our customers LOVE !! Being agile is even part of our company values www.appfolio.com/about/values :-).

Good luck with the next class.

Warm regards,

Klaus

---

Armando Fox

April 26, 2012 06:41

James, thanks for posting, and my anecdotal experience about design jobs being given to experienced UI/UX designers accords with yours. We plan to continue covering this material if only to instill the virtues of a user-centered process, and indeed, students reporting back on their experience working with customers told us that they now appreciate the value of lo-fi UI prototypes as a way of reaching agreement on what the customer actually wants. Equally important, though, is we want to reinforce the case for including exposure to this material in what is considered a "core" software engineering class, and influencing others to do the same.

I'd be very appreciative of suggestions on how to better approach this material in the course or book. (Our mutual colleague Marti Hearst has already pointed me in some good directions for this.)

Cheers,
Armando

---

David Patterson

April 30, 2012 01:53

By the way, CACM asked us to record a 5-minute video to demo some of the Rails tools that enabled the autograding of the thousands of programming assignments for the MOOC. The plan was to put it on the CACM web site, but it wasn't ready yet.

Here it is:

http://vimeo.com/saasbook/cacm

---

Displaying all 6 comments