Scrum is a software development process most commonly associated with agile software development projects. The name "Scrum" derives from how the different phases of a project overlap, with the entire process handled by a cross-functional team across the different phases, similar to a game of rugby, where the whole team acts as a unit, passing the ball back and forth.
Often described as "a process skeleton", Scrum methodology breaks development work into "sprints" (two-week chunks) and the team then works together to determine which "user stories" (requirements) from the "product backlog" make it into each sprint.
Scrum also includes a set of predefined roles, which include the ScrumMaster, who maintains the processes and acts like a project manager, the Product Owner who represents the stakeholders, and the team which includes the developers.
John Scumniotales, currently the vice president of research and development and application lifecycle management (ALM) products at Serena Software, was the first dedicated ScrumMaster back in the mid 90s. Scumniotales worked on the Smalltalk IDE product at Easel Corporation, where Scrum was incubated under Jeff Sutherland, along with assistance from Ken Schwaber. Computerworld US's own ScrumMaster, Lorraine Pauls Longhurst, recently caught up with Scumniotales during his visit to Australia to present a series of Agile development roadshows on behalf of Serena.
Computerworld: You were involved in creating Scrum, what is your connection with Jeff Sutherland and Ken Schwaber?
Scumniotales: Back in the early 90s I was working at a company with them called Easel Corporation in Massachusetts. I reported directly to Jeff Sutherland and my team was building a tool that was targeted at the client server/object-oriented space. We knew that we needed some kind of a rapid, incremental approach where we could build out some features, validate they were correct and then organically extend the growth and the product offering. There was a lot of market pressure — we wanted to get something out quickly, so again that lent itself to an incremental model.
So time-to-market is partly what initiated it?
Time-to-market and incorporating stakeholder feedback were some of the drivers that forced us to take the incremental approach. We recognised if we were going to build something fast, we had to build testing in, and we had to have the product manager or customer rep embedded in the team, so we created a collaborative team result.
How did you come up with some of the more structured rules to Scrum, things like the 15 minute meetings and that kind of thing?
The daily stand-up meetings are what you are referring to. Fifteen minutes or less — in fact 15 minutes is the max. Again it was communication; the process relies on direct communication with team members over formal documentation.
Thinking back to my time as a product manager, I was always pressuring the developers to get as many features into the release as possible.
The healthy tension between development and product management...
Yes, of course. How to do you think Agile would have helped in that kind of scenario?
In Agile, the product owner (or product manager) engages with the whole team during sprint planning. They come in there with the user stories for that sprint and the team collaboratively produces estimates — "Here is our capacity, and you can put whatever work you want the team to accomplish within that sprint, but there is only so much the team can do". It becomes very visible, as opposed to a product manager dumping a 200-page requirements document on the team and saying "I need it in six months".
I found that when the developers attempted to get more done than was possible, the quality would suffer. How does Scrum address this?
One of the critical success factors for agile is applying what is called a "test-first" approach. When we define the user stories — again the user stories are defining the requirements for what is going to be built in the sprint — a critical element is the acceptance test. In order to consider a user story as completed within a sprint, the product owner has to declare that the acceptance criteria have been met.
You've described the testing process throughout the sprint, and by following that process you can ensure that the acceptance criteria are met. I've heard Agile critics ask "What about the cycles of system integration testing (SIT)?"
Yeah, absolutely it needs to happen. System level integration testing is what I'd call traditional software quality assurance. Part of the delivery of the sprint should be to deliver appropriate system integration tests as the stories start to get assembled. But the key is not to think of it as if there's another quality assurance organisation delivering it. That's work that should be very explicit and planned within the sprint.
So you're not saying that SIT is gone, it's always part of the process. And so is user acceptance testing [UAT] because the product owner has built the acceptance tests into the sprint.
A good user story that's been delivered is one that has acceptance tests identified and that aligns completely with UAT.
What about changing requirements? That was another thing that would annoy the developers. How does Scrum deal with that?
Certainly a lot better than waterfall deals with it. With more traditional approaches, changing requirements can be extremely costly. Agile is built around the premise that requirements change. So the requirements, again, are typically captured using user stories.
As we start planning a sprint, the product owner, the ScrumMaster and the team members start to identify the highest priority elements from the backlog and move them into the sprint. You want to try to keep those requirements relatively stable within the sprint. If you can't keep requirements stable for two weeks, there's probably some other issue.
Every two weeks we'll go back to that sprint backlog and what was high priority in the previous sprint may be less of a priority now, so we'll move other new requirements or user stories to a higher level from a priority perspective and move those into the new sprint. That's the flexibility.
In your experience have you found it difficult to break down the requirements into small enough chunks that can be completed within the sprint? Would you suggest that this has to be done before the project even starts?
That's one of the classic challenges — especially when teams are transitioning into Agile.
Typically, before the project starts you have some kind of conception phase where you're going to lay out "here's the vision and direction of the release". You will build out high level epic stories — when I think of epic stories I think of a two-page marketing glossy with high-level bullets of what a product provides.
Then the hard work begins. That's when the Product Owners needs to roll up their sleeves and break those epic stories down into user stories. User stories must be: 1; atomic so they can stand on their own and, 2; implementable within a sprint. If you build a user story that can't be completed within a sprint, the team should never accept it into the sprint. But there's no doubt that is one of the things that teams struggle with, especially during the forming stages.
Lorraine Pauls Longhurst is a certified project manager with expertise in the use of agile software development methodology (Scrum/ADM) with LPL Software Consulting, based in Sydney.