An Agile Perspective Why Bother

I thought I would step back a bit for this blog post and revisit the reason why we are even doing this. Why completely change the way we deliver software?  So, in this article, we will discuss some of the aspects that make Agile successful and introduce some important ideas that are at the heart of the Agile methodology.

First, it is essential to understand that Agile, as with any methodology, is not a silver bullet.  There are plenty of examples where Agile failed.  While the essence of Agile is straightforward, the challenges lie in understanding the business need(s), sticking to the principals you believe in, being patient, and focusing on transparency, communication and accountability.

Agile Manifesto

In 2001, a group of software experts got together at a ski lodge in Utah and developed the Agile Manifesto. This group identified a short set of values that helps guide much of the agile movement.

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

They believed that while there is value in the items on the right, the focus should be on the bold faced items on the left.  Agile has gotten a bad rap for a misinterpretation of these values. I hope to help set the record straight as I explore Agile software delivery.

One of the primary pillars of Agile is to do the right amount of something, and no more.  This varies greatly based on the circumstances of a given project.  A start-up with a small budget and a short time to market will require much less formal process and documentation than a larger organization implementing a product regulated by the FDA.  Each of these projects will benefit from using the principals of Agile.

Individuals and Interactions

Many projects using sophisticated tools and processes have failed.  Much of the Agile process was designed to be implemented using 3x5 cards and a white board.  In fact, many Agile professionals prefer this approach.  By meeting regularly with the Product Owner and developing the features they want delivered in the priority they specify, we will be on a path for success.  These features are maintained in a prioritized list called the Product Backlog.  The Product Backlog is then broken down into 2-4 week Sprints.  I will talk about Sprints in more detail below in the Working Software section.

During a Sprint a daily standup team meeting is conducted every day.  The meeting is time limited to 15 minutes and focuses on reporting progress on the features included in the current Sprint.  Each team member reports what they did yesterday, what they have planned for today, and any impediments they face in completing their work on time.

The trick, as with much of Agile, is knowing the proper level of detail for your given project.

 Working Software

Agile believes that working software is the primary measure of progress and that the project owner (usually the business owner) will learn more through a set of short delivery cycles called Sprints.  Sprints are typically two to four weeks in duration and deliver the Features the Product Owner has identified.

Features are identified in a way for them to be self-contained and small enough to be completed in a single Sprint and to be Potentially Shippable.  This means that each feature is coded, reviewed, tested and deploy-able at the end of each Sprint.

This is one of the most powerful aspects of Agile.  It is very common for developers to start working on one thing, get sidetracked, and begin working on something else that is cool.  Using Agile, the team completes the features the Product Owner wants 100% in each Sprint.

Customer Collaboration & Responding to Change

Many projects that have very elaborate and long specifications, detailed contracts and project plans have failed  Experience has shown that it is virtually impossible to spend weeks or months developing long specifications and deliver what the business needs.  I was once told, "You delivered exactly what I asked for but, that is not what I need."  Even if it were possible to develop a perfect specification, it is likely that the business needs have changed since the time the requirements were gathered.

As we progress Sprint by Sprint, the Product Owner learns things they need just by seeing the product evolve.  Often we discover features late in the game that were never even considered early in the project.  As long as the new features add quantifiable business value, this is a good thing.  If we discover a set of features we never imagined and it saves our company more than it costs, we are happy.

Sometimes we actually stop development before we complete the product.  During Release Planning we carefully review the Product Backlog and deliver the high value features first.  We all know that most people typically only use a small percentage of the features in a given product.  Microsoft Word is a good example of this.

By following the rule of doing only the right amount of something and no more, we can often save money by delivering only a portion of what we originally intended.  Of course it is up to the Product Owner to make that call.

The Agile Manifesto and the simple elegant ideas it professes are the most important change to how we deliver software that I have seen in my decades of experience in this field.  I will never deliver software any other way!

The Product Backlog and How to Manage Your Way Out

The Product Backlog contains a prioritized list of the Features we wish to build. The Product Owner works with the Customer to develop and prioritize the backlog.

Features are broken into User Stories of various sizes. A user story is small enough to be completed in a single Sprint.
The Product Owner must work to define the Business Value of the features and user stories. This is often done with an eye toward the reduced cost or increased revenue the story generates. The size of the user story is also a consideration in backlog priority. A feature with a small size and a large business value are typically high in priority.
It is not always simple to assign priority. When this is the case, we need to use a more sophisticated technique to prioritize the backlog. This could be Weighted Shortest Job First (WSJF) , Kano Analysis, Theme-Screening, Theme-Scoring or Relative Weighting.

Release Before the Product is Complete

We should work to identify a set of Release Candidates. A release candidate is a set of features that delivers an operational product to our Customer. We want to ship as soon as possible, even before the product is complete.
It is up to the Product Owner and Customer to identify the Release Schedule. A common practice is to deliver two or three sprints of user stories followed by a hardening sprint that results in a release candidate.
The amount of work the team can deliver is based on their velocity and capacity.
The Scaled Agile Framework uses the term Potentially Shippable Increment or PSI.
This gives us a cadence of quarterly releases. This is a very good thing. The customer learns to expect valuable software every three months.

It is all About Customer Value and Regular, Flexible Delivery

A number of important benefits arise from this agile approach. They are:

• The Product Owner and Customer set the priority of the Features they want.
• Features are delivered and completed in short sprints so progress is obvious.
• Before each sprint, the Product Owner has an opportunity to change her mind and modify the schedule.
• We are never more than one sprint (2 weeks) away from delivering new features which, in turn, gives us frequent opportunities  to change and adapt.
• We are never more than one release (three months) away from a release of software to our customer.

This builds confidence in the product with our customers and in the delivery team with senior management. This cadence can continue forever or until we decide to stop.

Learn More

• ScaledAgileFramework.com
• Mike Cohn (Blog)
• Release Planning Guide (Rally Software)

Books

• Agile Estimating Planning (Mike Cohn)
• Agile Software Requirements (Dean Leffingwell)

Servant Leadership

We have moved from the command and control project manager approach of delivering software to a more coaching role. Here are a few guiding principles that are moving us in this direction:

• People generally want to do a good job. With proper guidance, support and clear direction a team will be successful.
• When a team believes in what they are doing, they will be more productive.
• A leader's job is to serve the team, not tell them what to do. The expert team knows what is best.
• A leader asks proper, sometimes difficult questions of the team and should do so with respect.
• It is acknowledged that an independent observer will notice things that individuals involved in day to day interactions may not notice.
• The servant leader is not the expert, they are the facilitator of experts.
• The team has the expertise and with the proper guidance will excel.

It can be challenging to move from a typical project manager role to that of a servant leader. A typical project manager "knows" things the team does not know and is frustrated when things do not go as planned. He knows that if he were doing the work, it would be different.
The reality is, the team is comprised if individuals each at different places in personal, professional and emotional development. The team is comprised of individuals who are experts but are not "like" the project manager wishes they would be. The challenge for the Servant Leader is understanding where each team member is in their growth development and work with them to make them successful based on where they are, not where we wished they would be.
Sometimes things do not work out. If a team member is truly not working out, it is best for the team and the individual to end the relationship.
A servant leader must use communication and negotiation skills to be a facilitator, teacher and problem solver in an indirect manner. The approach of “Be a Flashlight” is appropriate here. Ask probing questions and facilitate conversation.
A key skill is to become comfortable with silence. Sometimes when probing questions are asked, an immediate answer is not apparent. Be comfortable with the silence a discussion will eventually occur. The team will figure it out. They do not need you to supply the answer. They may only need you to facilitate the conversation for the answer.
Learn More

• What Is Servant Leadership? (Center for Servant Leadership)
• Coaching Agile Teams Website (Lyssa Adkins)

Books

• Coaching Agile Teams (Lyssa Adkins)

The Agile Manifesto is Misunderstood

The Agile Manifesto was authored in 2001 at a ski lodge in Utah. A group of software experts got together and re-imagined how software should be delivered. They came up with four general guidelines and twelve principles that guide the delivery of software using an agile approach.
Since that time, agile has had tremendous success. However, the simple elegant words of the Agile Manifesto has caused misunderstanding and in some cases caused projects to fail.

Sometimes, folks in the agile community are our own worst enemy. Every time I hear someone say “That is not true agile” I cringe. There are also folks who are reluctant to change and feel that agile is the approach that has no documentation or goes on and on and never ends. This also gives me a headache.
So, I thought I would write some words about agile from the direct perspective of the Agile Manifesto. If we keep the overall idea these people had in mind and allow for flexibility we can increase our chances of success.
I will start with the four main points of the Manifesto. They are:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

Directly following these points is the following sentence.
“That is, while there is value in the items on the right, we value the items on the left more.”
The phrase “while there is value in the items on the right” is often left out of our thoughts as we deliver agile solutions.
I will explore each of the four points and provide my insight as to what they mean to me.
Individuals and interactions over processes and tools
People are required to deliver software. The people who want and need the product must explain it to the people who will deliver it.
This is very hard to do. The people who need the software are overworked at their normal day-to-day job. When it is decided that a software product is required, these people must take the time to think about what is needed in addition to their regular job.
As detailed processes and sophisticated tools were developed, the focus shifted from the people to the tools and process. Vendors would sell a process or tool as a solution. All along, it was still people.
What this point means is the focus should be on the people and the communication between them. The process and tools should be the minimum needed for a given situation.
For a startup firm, building a product with a short time to market, a simple lightweight process and little or no tools may be the proper solution. Here a large whiteboard and 3x5 cards may be best answer.
For firms that are regulated and audited, more process is necessary and more formality is needed. This will be different based on the situation. An objective should be to use the minimum amount of process and tools and no more. Use just enough to get the job done.
Start with the people and then decide what level of process and tools are necessary for a given circumstance. Use established agile techniques to facilitate individuals and interactions.
Working software over comprehensive documentation
Many projects have failed that have thick binders of software requirements. This is the model for the typical waterfall approach. Under this approach, the delivery team interviews the customer and documents the requirements.
The customer is asked to read the requirements and sign off on them. Then the delivery team builds and tests the product described in the requirements.
We have learned that it is virtually impossible for a customer to document the computer system they need. If they could do this, it is likely that the needs will change between the time the requirements were signed off and the product ships.
The agile approach of building potentially shippable increments of the product in short periods of time is one of the most tangible changes and benefits an agile process provides. This demonstrates progress and builds confidence with the customer and delivery team because useful software is constantly being delivered. The customer also sees an evolving product and will notice things they never would have using a thick binder of requirements.
So what level of documentation is proper? The philosophy of agile is to do the least amount of something and no more for a given situation. A regulated industry will require much more documentation and trace matrices than an unregulated one. Agile process can accommodate this whole spectrum of needs.
Customer collaboration over contract negotiation
This is one of the most misunderstood points of agile. No one is saying that we do not need contracts and all collaboration is informal. I take this to mean that the iterative delivery approach will have a better chance of delivering a product that provides the customer with a competitive advantage than a signed contract early in the lifecycle that is difficult to change.
When we focus on a contract, we get a false sense of security and think everyone agrees to what is being done. After all, we have a signed contract–we must be okay.
In reality, we do not know what we do not know. An iterative approach is the best solution to this problem.
We need contracts, they just need to be flexible and have the proper change control built in to accommodate an agile process. There is a “sales” effort that is required to convince the customer of the benefits of a more “flexible” contract approach. We need to explain that while there will be many opportunities for change, the customer will be in complete control.
What if we are forced to do a fixed price contract? Does this mean we cannot do agile? I believe we can. Although this is not the preferred approach, agile still provides process that increases our chance for success. With fixed price contracts we need to include proper language in the change control section and include this in our process. We can specify acceptance criterion in our contract and review changes in our sprint review meetings to accommodate this type of situation.
Responding to change over following a plan
Many projects have failed that have elaborate project plans with detailed Gantt Charts. We make the plans so complex and detailed that it is difficult to modify when changes occur.
Agile process replace project plans with release schedules and burn down charts that accommodate change. We can still track progress and in fact progress is more transparent than in a typical waterfall project plan.
Agile is actively transparent. This means that the progress on the project, good or bad occurs as a result of the process. There is little need for separate updating of progress reports.
In conclusion, it’s the results that matter
The point of all of is to focus on results that add business value and competitive advantage for a firm. Of course, this was always the intention of waterfall projects, it simply did not happen very often.
All too often we focused on the process and documentation that resulted in a false sense of security and we did not communicate with the people who in the end, are the most important.
However….
There are also examples of agile projects forgetting the importance of the “Items on the right”.  This is also a recipe for failure. Delivering the proper software product is still very hard. Agile does not change that fact.
Using an agile approach give us permission to lessen the need of formality but we should not forget it. We will close with one of the twelve principles of agile that is pertinent to this discussion.

Simplicity--the art of maximizing the amount of work not done--is essential.

I will review these twelve points in my next few blog posts.

Use Skype Groups to Improve Team Communications

What I love about my job is that I have the opportunity to work with many teams. From time to time, I learn something that is so good; it changes the way I work. The team I am currently working with uses Skype Groups to ask questions, make project announcements and, sometimes, banter back and forth. It really improves how the team works together.

A Skype Group is simply a set of Skype users saved under a name. To send a message to the group, you simple click on the group and type your message. An example of some of our conversations in our group is shown below. We call our group “Dev Chat”.


We use “Dev Chat” to broadcast any information that the group members will find useful. It has turned out to be kind of like a private Twitter for the project. The team knows to monitor the group, and look at the thread when a new message appears.

I have included some of the types of communication we use below:

• When team members have questions of each other, they post them on “Dev Chat”. This could be a tester asking a question or developers corresponding.
• When a person will be in late on a given day or needs to work from home, we “Dev Chat” to notify the team.
• If a team member will not be able to make the standup, they notify the team and mention what they have been working on. Note: If a person misses the standup without notifying the team, it costs them a quarter.
• When a server is down or there is some environmental problem, we post the issue on “Dev Chat”. Since the operations group monitors “Dev Chat”, they typically respond quickly.
• When operations needs to restart various environments, they use “Dev Chat” to request a window and schedule the restart.
• Jokes and sarcasm also make their way into “Dev Chat”. Sometimes the jokes are quite bad.

This fast and spontaneous communication mechanism really helps increase productivity as well as morale. Since Skype is available on most devices, it is easy to monitor activity no matter where you are. Also, the history of the conversation is available.

Setting up a Skype Group is easy. Select Create New Group from the menu.



Drag the contacts you want to include in the group.



Click on Save Group in Contacts.


Give the group a name.


Once the group is saved, anyone in the group will receive messages. You can read more about Groups here.

I hope your team will find this as useful as ours does.

Watch out for Smells It may be Limburger Cheese but …

As an Agile Coach or Scrum Master, a critical part of your job is to keep an eye or nose open for smells. A smell is an indication that something May be wrong. Everything may be ok, but then again, maybe not.

The most important advice I can offer is to trust your intuition and experience. If you have the feeling something is going wrong, it probably is. A major tenant of Agile is to find out about problems early and often. If you feel attention is needed someplace, address it early before it becomes critical.
This is often more about you, the Agile Coach, than it is about the team. You need to gain confidence in yourself and recognize that you have valuable experience to bring to the table. Your perspective as a facilitator and Servant Leader gives you a view of the projects others who are down in the weeds do not have.
You also need to have the confidence to possibly be wrong. Some of the smells you uncover will turn out to not be smells at all. It is human nature to hold back and be quiet because we are afraid of being wrong.  As an Agile Coach, you do not have this luxury.
Once you have identified a smell, it needs to be validated and possibly addressed. Keep these two steps in mind; validate first, and then address, if necessary. This requires tactful and respectful interaction with the party or parties who are related to the smell.
Agile Manifesto Tip  - Individuals and interactions over processes and tools
When the Agile Manifesto mentions “Individuals and Interactions” this is what they mean; respectful and transparent communication to uncover issues before they affect the project in a major way.
So you may say:

"Maybe I am missing something, but taking 45 hours to add two pages with static content seems a bit long to me.”

Your intuition and experience caused you to notice this smell. Suppose your team responds:

"That is true, Scott, but you do not understand. When we built this system, we were under a time crunch and told management it would take longer than normal for this type of change. We made our dates, but this was the cost.”

So you have validated the smell. You may respond like this:

"Thanks for explaining that to me. I’ll bet you were frustrated designing a system like that. I am sure you agree that we need to be better at responding to change than we currently are. I recommend that we bring this up at the next Release Planning Meeting. This will be a good architecture improvement to add to the backlog.”

You acted with respect and understanding.
Now the time will come, when you will be completely wrong as you validate a smell. You may even get hostile responses from the team. Suppose the head architect reacts like this”

"(Rolling Eyes) Scott, you do not understand the complexity in the system. I will send you an article where you can read more about this. I do not have time to explain it to you.”

This is a chance for you to mentor the architect and explain to him that we are a team and we should all feel comfortable voicing concerns. You may say:

"I can see you are frustrated and I understand you are under a lot of pressure. We are however a team and anyone should feel comfortable voicing concerns about any topic as long as it is respectful and constructive. Thanks for explaining your point and for providing me with the article. I hope you will take the time to explain things to me once I read the article.”

Moving from the command and control approach of a typical project manager to a respectful facilitator takes time for you, the Agile Coach, and the team.
Lyssa Adkins has an excellent website and blog at coachingagileteams.com. I also recommend her book titled “Coaching Agile Teams”. I also blogged about this topic in an article titled Be A Flashlight. You may find this article useful too.

Be a Flashlight

As the year ends, I thought I would like to stand back and review why agile is so powerful. Using an agile approach manages many of the challenges we face in the delivery of software. By following a simple set of principles and processes we can improve our chances of success.
The reason agile works is it focuses on adding the business value the customer wants based on her priority under an iterative delivery approach. It does this in a predicable manner without overworking the team. Progress is monitored using a small set of reports and frequent demonstrations of completed software.

The team remains productive because regular short meetings monitor progress and identify roadblocks. A switch from a Command and Control approach of a typical project manager to a more Self-Governing style improves productivity. The project manager role moves from a director to a facilitator.

The term Agile Coach has started to gain popularity. I prefer this to Scrum Master because my experience in the real world has shown me a mix of Kanban, Scrum and Extreme Programming is typically required.

So how can a team be Self-Governing? How can a group of people get the right thing done without being told what to do?

I believe most people want to feel good about their work and want to do the right thing. What causes most people to deviate from this is frustration and a feeling of despair. When people just show up for work and do what they are told, even when they know it does not make sense, productivity and quality suffers.

I view the job of an Agile Coach is to “Be a Flashlight”. Shine light on areas that “smell” and ask pertinent questions.  The reason this works is it affects aspects of human nature both positive and negative and leverages success. Let’s take a look at some of them next:

• Being Told What To Do – People do not like to be told exactly what to do. This is especially true with programmers. These creative people want room to grow their craft. By providing direction without being a micro-manager, people will be more motivated.
• Shared Accountability – Everyone moves off track from time to time. What typically causes frustration is when this occurs and the team feels nothing can be done about it. We need everyone to feel they can ask any question about anything. By asking these questions we shine light on uncomfortable areas. When a mistake is made, the person who made the mistake is accountable. Now this is not blame. Over time the team learns that mistakes are inevitable and eventually everyone will make one. We identify the error, correct it and move on.
• We are in this together – People respond and engage when they feel they are part of a larger thing. It is bigger than just the individual.
• Peer Pressure – people do not want to look stupid. They want the respect of their peers and their bosses. The complete transparency of agile causes everyone to know what is going on without having to point fingers at individuals.
• We always did it that way – when people feel they are trapped and helpless in a situation that will never change, they come to work, do their thing and go home. Both productivity and quality suffer.

It is the job of the Agile Coach to work through project challenges. Agile has process and guidelines to handle all of this. We will explore some of these next and highlight how being a flashlight and asking questions rather that giving orders manages the human nature in all of us.

Business Value

The whole team knows it is all about business value. We build features that add the most business value early. It is the job of the Product Owner to place the features in the backlog in the proper order.
The Agile Coach watches the backlog and looks for “smells”. She may say to the Product Owner

“I seem to remember that we discussed moving the forgot password feature later in the backlog but I still see it in the next sprint. Did something change?”

Just asking this question shines light on a situation and it will work itself out.

Daily Standups

Each day there is short meeting where each person reports progress and impediments. Everyone on the team knows this meeting happens and each day they report progress. The progress is reported to the team, not to a command and control project manager.
This meeting exercises all of the human nature points we discussed. Peer Pressure will cause people to be motivated to get their tasks completed. Openness and transparency causes people to raise issues when something is an impediment. When one team member has a problem and is helped out by another team member we see we are all in this together.
When a person reports the same progress on a task for a while and begins to run late, the Agile Coach may ask

“ May I help you on that task in any way, you seem to be struggling with it?”

There could be many answers to this question. Again by shining light we will get a solution and the team figures it out.

Iterations

We deliver a set of features that meets the definition of "done" in short periods of time. The team agrees to the following:

• Definition of done
• Length of iteration
• Features to be included in a given iteration
• New features can always be added, but not to the current sprint

The important thing is that the Team agrees to these things. It is the job of the Agile Coach to facilitate this agreement. If the team does not agree, most of the power of agile is diluted. The job of facilitator is critical. When the team does agree, things are easy. Let’s look at some examples.
The Product Owner says:

“I have a hot new feature we need to add to this sprint, please add it and let me know the effect”

The agile coach says:

“We agreed to two week sprints. We are 5 days into this sprint. Is it possible to wait 5 days for the new feature?”

Normally, requests like this come out of anxiety. The product owner was probably pressured by her boss to get this feature in right away. The boss may not even know the project is agile. Your job as an agile coach is to help the product owner, so her answer aligns with what the team agreed to. You may need to talk to her boss and explain why we need to wait five days and explain the disruption the request will cause. The agile coach helps the product owner in dicey situations. Normally, the Product Owner will see the logic of waiting and will agree.
During a standup meeting the Agile Coach Says:

“Thank you everyone for your updates. I noticed that there are no impediments, which is good. However, I notice we are above the ideal line on our burn down chart. Are we still confident that we can complete the features we all agreed to at Sprint Planning?”

A person from the testing team says:

“Well if the login feature is completed on Thursday, that only gives us one day to test it and my estimate was two days”

The agile coach continues the dialog, asking probing questions until the team discovers a solution.  The coach uses her analytical and negotiating skills to probe in areas the individual team members do not readily see.

Some Good Information

In closing out the year, I wanted to pass on three links that provide very useful information and an excellent book on learning to be an agile coach.
Coaching Agile Teams, by Lyssa Adkins is an excellent book for anyone who wishes to become a better agile practitioner. I use Lyssa’s tips and practice her advice every day. A link to Lyssa’s Blog is available at http://www.coachingagileteams.com .
An excellent video on Scrum is available by a vendor who sells the agile tool OnTime. I am not typically one to recommend a specific tool without understanding more about the given situation, but I can recommend the video Scrum in 10 Minutes. It is one of best short descriptions I have seen on Scrum.

I just learned about this video today after reading about it in an agile group on Linkedin. It is titled Agile Project Ownership in a Nutshell but really covers the agile philosophy as well as I have seen anywhere. You are also left with a very good diagram.
There is an excellent roadmap to all things agile hosted by the Agile Alliance. This clickable map is presented in the format of subway station stops.

You may find this excellent resource at the Agile Alliance web site here.
Finally, I want to wish everyone a great 2013.