In a previous article I discussed the day-to-day challenges facing the Product Manager in the Agile world and remarked that the debate surrounding the role of the Scrum Master and Product Owner was worthy of its own discussion.

A colleague of mine called my bluff the other day by asking me “What’s the issue with Scrum Masters and Product Owners?” This got me thinking about why one would go down the whole Agile route in the first place; what had we set out to do – and how close did we get? This article takes a further “eye of the storm” look at Scrum and its DNA, from a Product Manager’s perspective, and offers some further guidance on four of Scrum’s most critical elements.

Note: I offer no guarantees concerning the scientific accuracy of any DNA or any biological references herein; the references are purely indicative and metaphorical. Scientists need not feel obliged to correct any errors!.

Scrum – the DNA

Scrum, an Agile process framework, is one of the most highly-regarded ways of managing development projects faster, with better outcomes. It has gained an enormous traction in the software world, and is used by a large number of major organizations in the industry.

So, what characterizes an organization that is using Scrum effectively?

The first thing you might notice is that the actual Scrum model being used is already vastly different from the original implementation. Scrum adoption, according to its definition, allows for a lot of wiggle-room (adaptation) and this allows it to be refined to different scenarios and company cultures. Scrum is therefore – as an internal process – likely to evolve over time.

Additionally, there are basic principles of Scrum adoption, above and beyond “inspect and adapt” that give it its uniqueness. As Wikipedia tells us, “Scrum is a process skeleton that includes a set of practices and predefined roles”.

• The Scrum Master. Scrum tells us that for every team of developers working together to build something, there’s someone who cares only about the success of that task and is prepared to do whatever it takes to remove road-blocks, promote action and reach the end-goal.
• The Product Owner. A significant attribute of Scrum was to acknowledge the need for constant stakeholder input and validation, which is where Waterfall had so often failed (big, late projects with requirements that had completely changed during the project lifecycle, rendering the final deliverable useless). The Product Owner therefore acts as “the voice of the business” and sets original direction (requirements) and validates iteration outputs against target. Importantly also, the Product Owner is perfectly allowed to change the requirements – which then change the requirements list (or backlog).
• The meetings – Scrum defines itself in terms of the levels of interaction required between team members. Roles are clearly identified, and success criteria are well established.
• End-Goals. Scrum is quite modest in the chunks it suggests you manage – basically the concept of the ‘iteration’ of a few weeks is a rule-of-thumb duration for a scrum team. Small team, small time-box, simple (read: well-defined and agreed) list of things to do.

You will find more books on Scrum, Agile and derivatives than any other software project methodology in the last couple of decades, and the number of accredited scrum experts and trained scrum staff appears to be growing exponentially. To me it feels like there are less people in the industry who don’t know about Scrum, and on that note I will leave any definitions there.
Let’s now tear apart some molecules of the Scrum DNA and take a more critical look in some of more curious elements.

And why did we invest in that? The ROI molecule

Scrum is so popular it appears to be breaking all records in terms of adoption. For a de-facto industry standard, one might expect the measurement of its value, to be straightforward and unequivocal. As it is, Scrum’s Inspect and Adapt model makes statistical measurement of its business benefit illusive. Remember, “Scrum” today in a given organization is a very different beast to that of another, which is different again from another, and so on.

Guru Ken Schwaber gave us his view in his 2006 FAQ, “The benefits of Scrum are a competitive advantage if you are an ISV or your ability to respond more quickly to your competition if you are a business organization. The benefits are the ability to develop and implement high value, high priority software quickly to increase your return on investment, to reduce your staff turnover, to increase your staff satisfaction and to increase the quality of your product.”

Schwaber’s words give no hard facts at the time to support his claim; we are forced to search elsewhere for proof. When first I was educated in Scrum, I kept hearing “typically two to three times greater productivity” for Scrum-based projects (presumably compared with traditional waterfall models). That’s some claim. I remember one software company setting a target of 40% greater productivity over a year as a result of Scrum adoption. But while there is some truly independent work available (IEEE’s case studies, for example), much of the ‘noise’ about the success of scrum is generated by vendors of training, tooling and other media. A cynic might expect those who are selling services around Scrum to promote the measurable benefits. So, the “accepted truth” about Scrum is inferred, perceived and suggested, or promoted by those with a vested commercial interest.

Gilb and Brodie’s paper asserts this is not good enough, and the industry needs to stand up and be counted as to why this brave new world is better than the old one, statistically. Why doesn’t the Scrum Alliance promote itself by offering quantified benefits, rather than a purely qualitative discussion?

If we look at the molecular structure of Scrum, it appears not to include any self-promotion gene. Or at least not one that is quantitative. All this achieves is making the adoption harder to sell to sceptics. If the adoption flounders at any point, the business may well want to “look at the numbers” to justify continued investment. Those numbers need to be there.

The Scrum Master molecule

Another molecular exploration I feel is required here is that of the Scrum Master. The role is so pivotal to the success of the framework, it warrants further scrutiny.

Ominously, you only have to look at the size of the market of services and training available that there is clearly a skills “situation” out there. Not only do standard “Scrum Master” courses exist, but there is formal certification, and then, just in case, there are “Advanced Scrum Master” training courses also. The role is worryingly portrayed as “a challenging one”; “a tough job”. Is there something wrong with the DNA?

First, let’s look at the requisite skills – listener, mentor, facilitator, worrier, communicator, organiser, motivator… that sounds like a fairly clear definition. Now, let’s consider the typical resource pool from which Scrum Masters are picked – the Development organization, where incumbent skills are different –individual contributor, technologist, analytical, thinker, creator, one who works alone on limited orders or interaction.

Of course, I am creating a stylized and sweeping generalization about development organizations and incumbent skills, and it is not meant as a disparaging indictment, it is designed to illustrate a point. The point is this, if I wanted a good developer, I would not go and pick a non-technologist or someone with no attention to technical detail or analytical skills; conversely if I wanted a good organizer and motivator and leader, I would not pick from resource pool where most people have, generally speaking, limited significant communication or leadership experience.

And that is the problem in a nutshell; a couple of days on a course do not turn a technician into a Scrum Master. Furthermore it appears most of the industry is recognizing that problem, if we simply look at the growing availability and provision of Scrum Master and related training courses, materials and blogs. It suggests that organizations are trying to “skill up”, to rectify a shortfall. As we poke about in the DNA of the Scrum Master – the worrying signs are that while we might have a reasonable blueprint, our target gene pool might not be the right initial mix.

The Product Owner molecule

Looking at the chemistry of the Product Owner role, we are presented with an entirely different cell structure, but the task remains the same – to check whether the existing gene pool is sufficient to support the structure we need to replicate across the organization.

The primary attributes of the role, according to one view at least, are “thorough understanding of the customer needs, an active stakeholder management, and a basic knowledge of how software is developed and deployed”. The clear implication here is that the incumbent needs business savoir-faire and technological understanding in equal measure, and be “highly available”.

Concern is raised at the availability inference. One Senior Product Manager of a mid-sized ISV said to me that the lack of product owner resources was his “single biggest bottleneck” in getting products released. Availability was a problem, he said, because “product owner” skills are so specialized due to the duality of the role (business experience and technical prowess). The lack of obvious candidates for this role means companies either use inappropriately-skilled staff to attempt to perform the function, or “do without” and allow bottlenecks to appear.

The most logical gene-pool for this role is Product Management, but often Product Management is staffed from a strategic operational level, at a macro solution level – such that there would be one Product Owner for multiple products. With Scrum, there is the need for a “product owner” for each scrum team, which could be many for each product. From one-to-many to many-to-one means Product Management does not and cannot exclusively provide the resources for the embryonic product owner role. The math just doesn’t work. Worse still, Scrum (and its funding) was only ever a Development organization initiative: chances are there was no consideration or funding allocated to additional “product management” resource to support it.

Some observers have taken this battle back to the scrum architects. For example, one particular blog entitled “The mythical Product Owner”, castigated: “I’ve seen many flavors of Product Owners and none of them really worked as they should have”. Perhaps skills availability and bottlenecking was a key issue here too – the blog makes for an interesting insight.

Another blogger asserted that the availability issue means that, quite simply, the Product Manager cannot be the product owner “One big mistake a lot of people make is assuming that the Product Owner has to be the Product Manager. While conceptually that may be true, the Product Manager cannot, and in my opinion, should not attend … daily scrum meetings.”

What has happened is that the very unique skill requirement of the product owner role has precluded other options being considered, and Product Management is inexorably sucked in to “plug the gap”, and perform as much of the product owner role as time will allow. Often, this isn’t enough for the demands of Scrum, which is where cracks appear.

Enlightened organizations are seeking to nurture leadership potential in the development organization by moving high performers into a junior scrum-level product owner role, and above that developing a hierarchy such that the Product Manager acts as a “Senior” or “Chief” Product Owner.

But it remains a common ailment of the software industry that the product owner role is often extremely poorly resourced. Again the genetic blueprint looks fine, but the gene pool is the wrong type.

The Reporting to the Business molecule

There is one more strand of the DNA I want to explore here: the molecular fabric of reporting to the business. Remembering that Scrum is a Development discipline, its direct involvement with the business, at least in terms of how it is defined, is left up to the individual organization, and expertise of key architects.

It is perhaps therefore unsurprising that, sometimes, the metrics a business might normally expect to see as a matter of course from other functions (mid-term plan, resource spend, productivity, status, costs, impediments) are either difficult to come by or simply not available from Scrum.

Scrum is – helpfully – built on the management of information (requirements, backlog, teams, dates, issues), but it is the outcome that is more important than any reporting thereof, and this orientation can mean that only the scrum teams themselves actually know the status at any given time. For a major organization planning major external activities on the back of major software projects, this is wholly untenable. In fact, ultimately, the only outcome that truly matters in Scrum is the outcome that the business wants to present to its customers. It is this holistic view that the DNA should evolve to embrace. The promising news is that much of what would normally be expected in such reporting is already being monitored and recorded as part of a Scrum implementation. Productivity, Velocity, Quality and even Value (there should be benefit weighting in the original requirements) are values and metrics that support the Scrum methodology, and by that virtue could find themselves in any reports.

What you might often find though is that the task of reporting ‘owned’ by an internal governance role, where the responsibility falls outside the scrum team. It is arguable whether this is truly appropriate given the level of visibility and transparency required. At the very least, there should be some very real and direct link between the governance engine and the scrum master role.

Conclusion – Inspect and Adapt, and Evolve!

Product Management as a discipline has, for a long time, understood the fluidity of organizational requirements that change with the wind, and the need to respond rapidly to such changes, in the context of a wider business strategy. Furthermore product managers the world over have undertaken the relentless role of translating between business needs and technical requirements, between the field and the lab, and back again.

A recent McKinsey survey on building nimble organizations stated 89% of the global executive respondents ranked agility as ‘very’/‘extremely’ important to their business success. Any method that promotes business agility or is agile by its own definition (Scrum) and any roles that are by nature flexible, agile, responsive (Product Management) are therefore set to remain at the forefront of organizational innovation. Striving to derive the best possible results may require things to evolve to be more focused on business outcomes. The better Product Managers and Scrum architects, and by definition the more enlightened organizations, are well-placed to embrace the evolutionary demands that their businesses will inevitably require.

Derek Britton has worked in a variety of Development, Project and Product Management positions in the IT industry over the last two decades. Currently, he is a Senior Director, Product Management at Micro Focus, responsible for the portfolio management product lines.