Current Articles | RSS Feed RSS Feed

How 11 Companies are Scaling Agile

Posted by Jon Friedman on Tue, Jan 08, 2013
  
  

The Scalable Agile Design Group (SADG) is an informal group representing 11 companies plus Assembla, pulled together to share ideas about how Agile techniques can be scaled up to address large, complex software development projects.

Although the group is not a scientific sample, it does contain a cross-section of midsized and large companies in businesses ranging from sporting goods, to financial services, to online computer games, to custom software development.

Below are findings that provide insight into how Agile is currently being used and where it is going, based on interviews and discussions with 11 companies that have scaled Agile to larger projects. We also include links to some recent Assembla blog posts related to these findings.

Finding #1: There are three distinct Agile use cases

There is not one set of Agile best practices that fits all development organizations. In fact, our group showed three use cases that dictate very different priorities and methods. Development teams fell into three categories:

  • “Major Release Developers” work on client-server and locally installed products with releases every three to six months
  • “Cloud Application Developers” build web-based applications that are updated frequently, as much as several times per day.
  • “Multi-Project Developers” work on many concurrent projects for multiple customers

The group you belong to determines which Agile practices make sense for you and which don’t.

Get more details in: The Three Agile Use Cases: Where Do You Fit In?

Finding #2: Almost no large Agile organization is totally co-located

The vast majority of companies in the group have distributed development organizations:

Number of companies with development staff in:

  • 7+ locations: 27%
  • 3-5 locations: 55%
  • 1 location: 18%

And almost all of these companies also have employees working at home, freelancers, and virtual business partners at multiple locations.

Finding #3: Scrum is the default Agile methodology

Scrum is clearly the predominant Agile methodology in the group.

Number of companies using:

  • Exclusively Scrum or “scrumish” techniques: 55%
  • Scrum and Kanban or Lean techniques: 36%
  • Exclusively Kanban or Lean techniques: 9%

Finding #4: Kanban has its place, but not as a replacement for Scrum

Kanban is not seen as a new wave replacing Scrum. Rather, Kanban and lean techniques are seen as the right fit for web-based applications with frequent releases (Cloud app developers in finding #1), while Scrum is still viewed as optimal for client-server and locally installed applications with less frequent releases.

Finding #5: Teams are adding lean concepts to sprints (ScrumBan)

Companies are incorporating Lean concepts into their Scrum processes. This includes leaving room in the sprint plan for last-minute new tasks and limiting WIP within a sprint. These practices reduce some of the inflexibility of pure Scrum, while preserving advantages such as having defined releases at regular intervals.

We recently posted a blog article that discusses this hybrid approach. Check out: Scrum + Kanban = ScrumBan, an Easy Scrum Upgrade. See where ScrumBan fits in the progression toward Continuous Delivery: Assembla's Beyond Scrum Roadmap

Finding #6: Coordinating teams and dependencies is a major challenge for scaling Agile

Perhaps the greatest challenge in scaling up Agile projects is coordinating teams and managing dependencies across teams. As projects grow, development groups resort to ever more complicated and time-consuming rounds of meetings and calls. This is clearly a topic crying out for new ideas and fresh thinking.

Read: 3 Ways to Handle Dependencies

Finding #7: Continuous Integration and integrated testing are critical for scaling

The companies that have scaled most successfully are strong believers in Continuous Integration. They have invested in processes and automated build tools to the point where they create new builds daily or several times each day.

The organizations with the fastest release cycles have also worked hard on integrating testing into all parts of the development cycle. Testing early and often allows them to find and fix problems faster, without slowing down release cycles.

Assembla has published a lot about continuous integration and Continuous Delivery: 

Do these findings ring true for you?

Do these findings agree with your experiences, or contradict them? Please comment below.

0 Comments Click here to read comments

Scaling Agile with Continuous Delivery and Subversion [Video]

Posted by Jon Friedman on Wed, Nov 28, 2012
  
  

On November 15, Assembla and WANdisco presented a webinar Beyond Scrum: Scaling Agile with Continuous Delivery and SubversionFor those that did not get a chance to register or attend, the video and slides are below. Enjoy. 

 

View and download the slides

In this 30-minute webinar Andy Singleton of Assembla and Scott Rudenstein of WANdisco described how to go beyond traditional Scrum principles and scale to globally distributed teams with continuous delivery and Subversion.

They discussed how to:

  • Achieve continuous delivery using branch, merge and code review techniques.
  • Accelerate the work and coordination of multiple distributed teams without lengthy meetings.
  • Improve performance and security management, while replicating real-time repositories, within a centrally-managed Subversion infrastructure.

 Other Releated Resources from our Blog:

0 Comments Click here to read comments

The Three Agile Use Cases: Where Do You Fit In?

Posted by Jon Friedman on Tue, Oct 23, 2012
  
  

There are three distinct “use cases” for employing Agile techniques. They are based on different business challenges. They should be addressed by different Agile processes. If you don’t know which one you are, you are probably doing it wrong.

The three use cases are:

  • Long release cycles of 3, 6, or 12 months
  • Online apps (Web, SaaS, enterprise cloud, big data, centrally deployed)
  • Developers servicing multiple clients or projects

Background

Earlier this year Assembla formed the “Scalable Agile Design Group,” a collection of senior managers of development groups at 12 companies with extensive Agile experience. We created the group to learn more about the common issues facing Agile companies.

We were surprised to find that there is no one common set of issues. We found three distinct types of Agile companies. Each has a different set of pain points and priorities. Each focuses on different processes and tools to address those priorities. They were almost talking different languages.

The conclusion: knowing your use case will tell you what parts of Agile to emphasize and what parts to ignore.

Major Release Developers

“Major Release Developers” are development organizations that work on long-term projects, typically with release dates three or more months apart. Often they are building classic “enterprise” applications. Teams are usually assigned to one project for months, or even years.

These organizations can reap gains by using classic Agile techniques like sprints, scrum masters and product owners who provide customer input on a daily basis. They make extensive use of Agile training and Agile coaches to inculcate Agile principles and improve coherence within teams. Essentially they are focused on maximizing team efficiency across long-term projects that do not deal with frequently changing requirements, bug fixes, etc.

Most Agile textbooks are written for this use case.

So doesn’t this reflect almost all Agile development groups? No, not at all.

Cloud Application Developers

“Cloud Application Developers” maintain web- and cloud-based applications, and release new functionality frequently. They include organizations supporting web sites and web-based applications within an enterprise, SaaS companies, and major “cloud” services providers like SalesForce or Google.

These companies are driven to:

  • Release new functionality frequently, sometimes several times each day.
  • Reduce cycle times, to react quickly to customer and marketplace feedback.
  • Automate processes to reduce the burden of frequent releases.

Scrum is actually a bad model for these companies. It prevents them from releasing any more frequently than the end of each sprint (typically every 2-6 weeks). They can’t easily handle tasks introduced in the middle of sprints, no matter how high the priority. And they create a lot of overhead with planning meetings, scrum-of-scrum meetings and retrospectives.

Instead, these companies gain the most by emphasizing:

  • Kanban and lean techniques.
  • Continuous integration, including automated builds and automated tests, and continuous delivery. 
  • Collaboration tools for distributed teams, to take advantage of the global market for development talent.

Multi-Project Developers

“Multi-Project Developers” is the third use case. These groups work on many concurrent projects and move teams among them. Many are service providers and custom development shops, but they also include IT groups that service multiple business units inside an enterprise.

Because these organizations are constantly moving teams around and reconfiguring them, Scrum doesn’t buy them much. 

Their challenges revolve around resource management. They need to pay close attention to capacity planning and resource allocation across projects (especially for people with scarce skills). They benefit from frequent, low-friction communication with client organizations. They also need to onboard new developers easily.

The tools to address these challenges include ticketing and planning systems, collaboration tools, and tools that allow customers to provide rapid feedback and track the progress of their projects.

Some Have Two

We found several Scalable Agile Design Group members that had more than one use case within their company.

For example, part of the organization might be working on an enterprise accounting application (the Major Release Developer use case), while another part is working on web sites or data warehousing projects that need to make changes on a daily basis (the Cloud Application Developer use case). 

These companies found, sometimes to their surprise, that Scrum teams would take root in the first area, while Lean techniques thrived in the second.

What Does This Mean to Me?

If your group fits the Major Release Developer use case then you are probably in good shape staying with the classic Agile textbooks.

But if you are a Cloud Application Developer or a Multi-Project Developer wondering why Scrum hasn’t brought you very far, then you should look at some of our blog posts about Going Beyond ScrumContinuous Delivery, and Scalable Agile

0 Comments Click here to read comments

Beyond Scrum Roadmap

Posted by Andy Singleton on Wed, Oct 17, 2012
  
  


About six months ago, we started working internally on our "beyond Scrum" campaign to improve agile software development so that it works in a cloud-based world. As noted in my article, 7 Things I Hate About Agile, I don’t agree with Scrum, the dominant agile methodology.  I was called everything from "ignorant youngster" to “bitter old man” to a “Moron with a capital M.” However, the response was big.  50,000 people read the article. 1200 signed up for a “Beyond Scrum” webinar that we did with Perforce. It’s clear that a lot of people are ready to move on to the next step.

Here is our roadmap for the positive steps beyond Scrum.

Beyond Scrum Chart 3

In moving our own teams and development process along this path, we were to increase our release frequency from twice a month to almost 60 releases in September - all while fixing issues faster, releasing more features, and overall happier team members. It worked!

Let’s look at this roadmap in more detail.

Distributed Teams and Scrumban

Why?

Because modern software teams really are distributed around the world. Once you accept that fact, your project will run much more smoothly at any scale bigger than one person, and you will find the people you need.

A simple first step is a Scrumban process that works with co-located and distributed teams. Scrumban will save you precious time by removing upfront release planning and allow your sprints to be more flexible for bugs/issues, chaging requirements, new features, etc.

Key Principles

  • People work where they are, which is usually distributed
  • Unite the team with strong social features and activity stream
  • Tasks and code are shared online
  • Team members pull one task at a time.  Manage WIP (work in process)
  • At the end of the iteration, freeze new tasks and stabilize what is ready
  • Release on schedule


Assembla Product Features

This could be a really long list. Assembla Workspaces was designed to support distributed teams. Workspaces provide a place to create, plan, and manage tasks, share code (Subversion and Git and Perforce), and collaborate on project specs/requirements. We find that a key driver of team performance is the activity stream, which allows you to see what other team members are doing, and we have enhanced this with popular social features like @mentions.

Continuous Delivery

Why?

Continuous delivery is the art of releasing software whenever you have a valid change, often many times per day.  It’s a natural goal for online services that are centrally deployed and updated. If your product gets released less frequently, you can still run continuous delivery in the development group.

1) It's hot

2) Moving to multiple releases per day resulted in big improvements for us, and we’d like to help more people get the same benefits. It dramatically reduces release planning and release related stress. You can do more because each task takes the right amount of time. It simplifies the interactions between teams and team members. And, it’s faster. If your competitor is running continuous delivery, you will eventually have to do it, or be overrun.  It also helps with scaling, since it removes the big meetings for iteration and release planning.  As we hear more about what is happening in Silicon Valley, we are becoming convinced that continuous delivery might be the only effective way to organize a big agile effort.

When I first heard about , I thought it was crazy, assuming that every release of a serious product needs testing phase.  When we started studying it, we found a few specific tactics that will move any project to continuous delivery.

Key Principles

  • Multiple test environments, on-demand, or one for each contributor or contributing team
  • Continuous integration with automated tests
  • On-demand test and stage environments
  • Code review to enforce test coverage and other requirements
  • Accept and merge changes when they are ready
  • Release on-demand, whever you want. Alternatively, deliver to batch testing “when ready”

Assembla Product Features

Continuous delivery requires code-related features to support multiple test environments, like the merge request workflow, various types of continuous integration, and the new SSH build tool. It also requires a new Kanban style approach to task management. We’ve redesigned our ticket workflow to fit.

  • Agile Planner and virtual Cardwall 
  • Merge requests with code review
  • Cumulative flow diagram and other metrics to measure velocity and throughput
  • SSH build tool
  • Jenkins integration

(Get these features for free with Assembla Renzoku)

We have also fired up some services that are not related to the product.  We are working with customers that need help with consulting and continuous integration, build and test scripts, and other DevOps stuff.  We hope that we can help them get to the next step with whatever tools and process they already have.

Scaling

Why?

Scaling to get more development done, faster, with bigger teams, is the most expensive problem in software.  We can use Internet technology to make it less expensive and much less painful. We have found tactics for working with globally distributed teams, breaking one backlog down into multiple teams, and adding and evaluating teams.  We can help them work together by removing the big meetings for iteration planning, dependency planning, and release planning so each team can work at full velocity on its own schedule. If necessary, you can go big – big like projects we studied at Facebook and Google Android.  At smaller scale we can use the same simple mechanisms to scale up AND make projects more agile.

Key Principles

  • Eliminate batch processes and big meetings
  • Add people or complete teams
  • Distribute tasks from a shared backlog
  • Evaluate and report on contributors

Assembla Product Features

We built Assembla Portfolio specifically for companies that want to scale and make large projects more agile, and we are making major enhancements to it.

  • Multi-team topologies so multiple teams can work on a project
  • Large backlog management
  • Fork and merge
  • User reports and team activity reports
  • Portfolio (roll-up) reports
Get Assembla Portfolio for your spaces starting at $79 here.

2 Comments Click here to read comments

The Future of Cloud Development: Where is it going? [Video]

Posted by adam feber on Thu, Oct 11, 2012
  
  


Below is a recording from the September 26th event, "The Future of Cloud Development," where Andy Singleton, CEO of Assembla, digs deep into the changes affecting cloud-based application development. 

Andy focuses in on the trend of teams moving away from a traditional Scrum process and more towards scalable Agile practices that leverage cloud-based resources like continuous integration and on-demand test environments to achieve continuous delivery.

To learn more about how Assembla can help you achieve continuous delivery, check out Assembla's professional services.

0 Comments Click here to read comments

Beyond Scrum: Introducing Simple Scalable Agile Development [Video]

Posted by Jon Friedman on Thu, Sep 27, 2012
  
  


On 9/25/2012, Assembla and Perforce presented a webinar titled "Beyond Scrum: Introducing Simple Scalable Agile Development." It is very clear that this is a hot topic with over 1,000 people registering for the webinar. For those that did not get a chance to register or attend, the video and slides are below. Enjoy. 

 View and download the slides.

Webinar Overview:

Scrum is the most popular Agile process, and aspects like the daily meeting are familiar to most developers and businesspeople. But it's not for everyone.

Scrum was designed for small collocated teams working on simple projects. Larger Scrum projects can experience hierarchical planning, integration nightmares, and inefficient use of resources.

In this webinar you'll see an Agile framework that avoids these problems by recognizing the concept that managing code is often easier than managing people.

Other Releated Resources from our Blog:

0 Comments Click here to read comments

Webinar: Beyond Scrum - Introducing Simple Scalable Agile Development

Posted by Jon Friedman on Fri, Sep 07, 2012
  
  


On September 25 Assembla and Perforce are presenting a new webinar: 

Beyond Scrum: Introducing Simple Scalable Agile Development

Scrum is the most popular Agile process, and aspects like the daily meeting are familiar to most developers and businesspeople. But it's not for everyone.

Scrum was designed for small collocated teams working on simple projects. Larger Scrum projects can experience hierarchical planning, integration nightmares, and inefficient use of resources.

In this webinar viewers will see...

  • An Agile framework that avoids these problems by recognizing the concept that managing code is often easier than managing people 
  • A demonstration of this framework that uses Assembla's hosted planning tools and Perforce repositories. 

Presented by:

Andy Singleton, CEO of Assembla, and Randy DeFauw, Technical Marketing Manager at Perforce

Tuesday, September 25 | 19:00 CEST | 1:00 pm EDT | noon CDT | 11:00 am MDT | 10:00 PDT

Screen Shot 2012 09 06 at 11.15.41 PM 

You can also try Perforce on Demand, hosted by Assembla here.

webinar logos         

0 Comments Click here to read comments

Intro to Scalable Agile: Scale Your Teams and Release More Frequently

Posted by Andy Singleton on Thu, Sep 06, 2012
  
  

This article provides an introduction to Scalable Agile, a non-Scrum process that is designed to help you scale a development effort from 2 contributors to 2000 without abrupt transitions, work with diverse and distributed contributors, and release whenever you want.

Table of Contents:

Why do we need it?
Useful principles for the design of Scalable Agile
How we do it - the Scalable Agile Process Chart
Advantages of Scalable Agile
4 Key Technologies
Putting it together

Why do we need it?

When I talk to corporate people about agile software development, they are usually talking about "Scrum."  10 out of 12 of the software managers that we brought together for our Scalable Agile Design Group describe their process as Scrum or Scrum-like.  Here's a description of Scrum:

A team gets together in a short planning session and decides what they will work on in the next iteration.  Then they work for a fixed period of time – often several weeks – and they end up with a "potentially shippable release."  Here's a link to a picture.  There's more - you'll hear about three roles, three meetings, three artifacts - but not too much more.  If you have a team of 5 to 10 people, all in one place, releasing at known intervals, Scrum can help you achieve the agile goal of frequent, high-quality releases.

If your development effort, like most, does not exactly meet the description of a "team of 5 to 10 people, all in one place, releasing at known intervals," Scrum will leave you with some problems.  If you have only two or three people, the cycle has too much overhead.  If you have more than ten people, you will need multiple teams, and you will need to make an effort to coordinate them for an iteration plan.  You won't be co-located.  Most modern development projects, and all big ones, include people from multiple locations.  As you get close to a release, you merge stuff together for testing, and there is never enough time for that.  If you run an online service, or you have a lot of components to update, you face pressure to release more frequently.

Most development efforts will face some of these issues.  Cloud development projects face most of these issues.  On Assembla projects, we always have all of these issues.

Problems cluster around two things that happen in a batch – iteration planning and testing for release.  These activities don't scale well.  Iteration planning is a problem for any distributed team or multi-team project that is not meeting in one place.  Test and release in a fixed time interval is a problem if your system is big, or if your developers are exceptionally productive.

So, one of our goals it so eliminate these batch activities and set up a continuous process that allows us to plan every day and release every day.  We want to relieve the stress that builds up around a fixed release schedule.  That goal is especially relevant in the age of cloud software development.  Our research shows that cloud and online product developers are very concerned with continuous integration and rapid release cycles.

Another goal is to include diverse types of contributors.  Training people to work together in self-managed Scrum teams is a laborious process.  If you have such a team, that's fantastic.  If not, if you are a lone hacker, or you work in a hierarchical organization, we still want you to be an effective contributor.

A related goal is to include contributors from any location.  Distributed work is a requirement for any modern project.

Finally, we want to scale.  The value of process and methodology is much greater for a large group, as getting efficient work from such a group is a difficult and expensive task.  An agile process should be designed to help us with that task.

"Scaling" doesn't only mean going to larger project sizes.  Most software projects are smaller than a cross-functional Scrum team.  An effective process for 2 or 3 people can be summed up as "prioritize and communicate."  For these groups, the transition to a Scrum process with "3 artifacts" and "3 rituals" can be harsh.  Shouldn't we instead try to bring easy and informal cooperation into larger groups?  A truly scalable agile process should provide some guidance to a team of 2, and allow 100 more contributors to join, without abrupt transitions.

Useful principles for the design of Scalable Agile

I applied some basic principles for managers interested in scaling any effort:

  • Recognize reality.  For example, teams are distributed, specialists are shared, and projects will cost as much time and effort as they are worth, and not what you estimate or recommend.
  • Let contributors decide what to do and get the resources they need.  Most management problems occur because managers have limited knowledge.  They just don't know as much as the people doing the work, and it is unreasonable to expect them (or expect yourself) to make a perfect project plan, or to know what each of 20 other contributors can do or should do.  21 people are smarter than one person.
  • Expand the resource base.  Add new contributors when you need them.  Try to get everything on-demand.  The Internet is very large and it connects you to a lot of people and things that can help you.

We're going to start with what is basically a Kanban process.  Here is one of my articles with a picture of Scrum, Kanban, and a recommended variant I call "ScrumBan."  I try to remember three important things about Kanban:

1) People "pull" work.  This is a simple way to make sure that work never piles up in the wrong place, and it saves a lot of time and effort for managers who would otherwise have to figure out who to assign things to, and when.  See my discussion under "empower individual contributors."

2) It's a continuous process.  Instead of planning a batch of tasks, each contributor takes one task, works on it until it is finished, and then gets one new task.

3) It's based on a Lean principle which says that you should not keep "inventory."  In a software project, inventory is the stuff you worked on that isn't finished.  In principle, you should finish each task before you start a new one.  The software Kanban board takes care of this in a very simple way by limiting "Work in Progress", or WIP.  WIP is similar to a sprint backlog in Scrum.

Scalable agile adds specific recommendations which will help you with continuous delivery - releasing every day or whenever you want.  Scalable agile also provides a structure for including diverse types of contributors.

How we do it  - the Scalable Agile Process Chart

In a Scalable Agile process, you plan your release at the end, and not at the beginning.  You just skip all of the iteration planning and other batch processes, which is an immediate productivity booster.  People work at their own pace, which you measure for forecasting and incentive purposes.  Then, when you are ready to release, you take what is ready, and you release it.

Here is a chart that shows the process.

 scalable agile process

(1) We make a prioritized backlog of features, tasks, and bugs that you want to work on.  This is the same backlog you will use for Scrum or any other agile process.  The "product owner" selects a subset of items that we are ready to work on, which we call "Current" work.  "Current" is very similar to a sprint backlog, with the difference that you don't have estimate its size.

(2) Contributors grab tasks out of Current and start work.  They can grab a task of any size (tasks have very different sizes – that's one of the realities we need to recognize).  In keeping with Kanban principles, they should only grab tasks that they are sure they are going to work on it in the very near future, and they should never be allowed to hoard tasks just because they are experts in a particular subject.  We haven't specified what a "Contributor" means.  It could be a lone hacker, or the person in the next desk.  It could be an established high-performance Scrum team, which can grab a sprint backlog, and commit to delivering it on their normal sprint cycle. This process is 100% Scrum compatible.  A contributor could also be an outsourced team, or a big partner, that uses hierarchical management and grabs individual tasks.  They can all work with their own process as long as they plug in correctly at the next step.

3) Each contributor has a separate environment for building and testing.  You can do this step manually on developer workstations if you have a small team and a highly skilled tech lead or "maintainer".  However, this is normally the step where you pull out the high technology, such as automated tests and on-demand cloud based copies of your production environment.  You will also want to reorganize your QA professionals to serve as resources that can test on any of these environments.  This type of organization is a necessary (and probably sufficient) step toward releasing every day.  You can read about it in more detail in my article "Continuous Delivery and Scalable Agile - Find Out the Secret."

4) Contributors submit code (and other changes) for review and release.  There is a centralized release manager or release team, which has ultimate power at this stage.  The system should  be set up to handle changes in the form of specific merge requests (Assembla) / changes (Gerrit) / pull requests (Github), which can be reviewed and accepted or rejected.  This review process is usually used to put changes into the contributor test builds, and then again to move changes into the production release.  When a change makes it through both reviews, we assume that it is ready to release.

Advantages of Scalable Agile

It's faster.  It's continuous delivery.  We enable releases "on demand" by having multiple "on-demand" test environments and accepting changes when they are ready.

Statistically, we have found that smaller changes result in fewer bugs with smaller bug impact.

We make the process work for distributed groups by eliminating mass meetings and the requirement for mass consensus.  We do this with "the principle of pull" where contributors pull and contribute when ready, knowledgeable tech leads who can manage both code and people, and clear online visibility of all work.  This allows many discussions to take place asynchronously, or one-to-one.

We make the process scalable to an indefinite number of contributing individuals and teams by eliminating centralized iteration planning and centralized testing.

We open the door to many types of contributors, not just Scrum teams, or people trained in a particular process. At Assembla we like to think about "blended teams" that can include our tech-lead driven distributed teams, partners or outsourcers with hierarchically managed teams, shared functional specialists (like design and ops), lone hackers, and individual experts.  It's Scrum compatible!  Established Scrum teams can grab a sprint backlog, and work as contributors with no process change.

If we are bringing in contributors we don't know, how do we control them?  We aren't necessarily managing them.  They are managing themselves in unknown ways!  Will our project fly out of control?  Actually, we have two very strong points of control.  First, we control the backlog, so we control what people work on.  Second, we control the code changes and other changes that we accept.  So, we have an absolute control over what goes in the product.  The review and acceptance process actually gives us a much stronger way to enforce standards for quality, architecture, and testing than we get in most other methodologies.  Finally, the continuous process prevents contributors from causing problems if they get stuck.  Work from other contributors will just flow around them.

4 Key Technologies

There are modern projects, mostly cloud-based, that use variations on the Scalable Agile process that I describe here.  However, it is a recent development, because it is dependent on four key technologies, and some of those technologies are new.

manage merge

1) Ticketing and task management

In its simplest form, the list of tasks is a stack of cards, or post-it notes, or lines in a spreadsheet.  The quaint stack of cards is idealized as a good focal point for Scrum or Kanban teams, but it's not what we want for scalable agile.  I'm going to describe it as a technology because a distributed team always has an online ticket or issue list that is globally visible.  It may contain some of the following features:

  • Customers and stakeholders adding requests
  • Ways to organize large backlogs
  • Requirements and mockups added with increasing detail
  • Ongoing real-time discussion and collaboration
  • Workflow steps, including testing
  • "Traceability" linking a ticket to related code commits, tests, and deployments

2) Global team collaboration and management

Internet workspaces for distributed teams (like Assembla) and related management tools.

3) On-demand build and test systems

Continous delivery requires separate integration test environments for each contributor, so you can test each change.  To make it scalable, you want to satisfy "the principle of pull" that a contributor should be able to get what he needs without going through some centralized system for building and testing.  Fortunately, there are now many suppliers of on-demand cloud servers. 

You will want to invest in automated testing and continuous integration.  Test and integration environments are specialized for each project.  Fortunately, there are also many new systems and platforms for automated testing, continuous integration, and automated deployment.

4) Code management and merge

Source code management and code review tools are an essential part of the process.  New merge workflows have given a big boost to the type of continuous delivery process that I describe here.  Code management tools were neglected in the last generation of agile methodologies, which focused on interactions between people.  However, the reality is that managing code is a lot easier than managing people.  It's difficult to personally manage 20 developers, and it gets more difficult if they are in 20 time zones, but you can scale a code contribution and review system to include thousands of contributors.

Putting it together

The Assembla product will provide tools to help you run "Scalable Agile" and other continuous delivery processes.  Here is an overview of relevant new and planned features:

Continuous flow for tasks:  Traditionally, the Assembla ticket system organized tickets into multiple "milestones," or iterations.  In the new Planner view, there is only one Backlog and one Current container (WIP or Sprint Backlog), so work can flow through Current in a continuous process.  You can see Current on the cardwall, a traditional Kanban view.  This week we will put a continuous flow diagram on the metrics page to provide a measure of progress that is useful in the continuous flow system.

Large backlog management (unreleased):  We found that when we put everything in one backlog, the list was too long to visualize, and we needed some new tools. We are compressing it with epics, stories, and tasks.  We are also testing in filtered views, so that you can see only the tasks that apply to your team or epic on the Planner and Cardwall.

Multi-team task management (unreleased):  You will be able to throw a filtered view of tickets out into a related "child space" for a new team to work on.  You will also be able to report on the progress of the complete backlog, or any contributor or team.

Merge requests / code review:  A few months ago, we released advanced merge requests for Git.  They allow code contributors to package up a set of changes, and submit them for code review and merge.  We have used and refined this feature as we moved our release interval from 2 times per month to many times per day.  We have improved the integration with tickets so that changes can move from idea to release in one smooth process.  We added "tickets affected" by a marge request- a type of automated release note makes my life much better - and in the new ticket layout, a view of merge requests related to a ticket.  We have implemented merge requests for Perforce, and we have been working with the Perforce team on their own highly scalable form of continuous delivery.  We are currently working on merge requests for Subversion, which will fit with the "symmetric merge" improvements coming in Subversion 1.8 later this year.

Continuous integration plugins for Jenkins:  Jenkins can help you build and test every proposed change.  These plugins help you configure Jenkins to run continuous integration with tests for any merge requests, authenticate using Assembla accounts, and even include it as a space tab.

Professional services, under the Studios label: These services range from basic DevOps services, to setting up scripts for continuous integration and continuous delivery, through full management and project delivery.

Talent: You should be able to expand with new individuals and teams.  We're working to make it easier.

0 Comments Click here to read comments

Scrum + Kanban = ScrumBan, an Easy Scrum Upgrade

Posted by Andy Singleton on Tue, Aug 28, 2012
  
  


These simple charts show how we can combine Scrum (with periodic releases) and Kanban (with very frequent releases) and combine them in a “ScrumBan” process that has periodic releases, and is an improvement on the normal Scrum process.  It's better for distributed teams, and closer to the way many of them actually work now.  It's a simple step toward the fully scalable process that we will propose later.

Scrum

Here’s what a Scrum process looks like in a “continuous flow diagram:”

idealized scrum

You select your current work as a “Sprint backlog.”  Then, you burn up the completed  tasks until (hopefully) you are ready to release.

In actuality, you don’t usually see sprints that match this chart.  Tasks get added near the beginning, and removed near the end.  You can get a lot of bottlenecks when you get near the end of a release.

Kanban

In a Kanban process, you select a list of "Work In Progress (WIP)."  As you finish one task from this list, you select a new task.  This continues without stopping at special release points.

idealize kanban

Note that WIP in a Kanban process is usually not expressed in terms of size or points.  This is because Kanban uses a “lean” philosophy, where the idea is to avoid having tasks that are waiting around for someone to work on them. Kanban limits the number of tasks, regardless of size, so that each current task has someone that is working on it.  A task that will take 1 day and a task that takes 9 days require the same amount of attention today.  In contrast, a Scrum team will consider a task that takes 9 days to be similar in size to 9 tasks that take one day, sequentially.  The Kanban WIP limit is a simple and effective way to ensure that each current task is finished as rapidly as possible, it doesn’t require estimating, and it provides a natural way to work on tasks that take longer than one iteration.

The Kanban process has a lot of advantages, and it is great if you update an online service with daily releases.  However, most teams can’t release every day.  They send things to the appstore once per month, or have enterprise clients that will accept quarterly updates, or burn CD’s or PROMS every year.  They need periodic releases that are stable.  For those teams, we suggest the ScrumBan process.

ScrumBan

In our idealized ScrumBan process, you start your sprint with the normal Kanban process.  When you get closer to the end of the sprint, you do some things to get ready for a release which I have labeled here with my own vocabulary.

idealized scrumban

Triage:  Find tasks that you are not going to finish by release time, and get rid of them by moving them to a future release.  This is actually an important part of any process that delivers releases at a fixed time.  I have diagrammed all of the triage at one time, but  you should  actually do it whenever you find a task that is not likely to be  completed in time.

Feature Freeze: Stop adding new tasks to the list of current work.  Then, you  can stabilize your release by burning down the list of incomplete tasks and problems (shown here as burning up the closed tasks).

Stabilization: Finish or remove tasks, and fix problems, until you have a stable release.  The cumulative flow diagram converges to the end of the iteration, One hint for the stabilization phase is to not allow developers to move on to new tasks until the release is finished.  This creates psychological pressure to finish a good release.

I find that the stabilization process often takes about 1/3 of the release cycle.  A good rule of thumb for your first release cycle is to do an aggressive triage and  a feature freeze by the  time  you have used up 2/3 of your  time to release.  If you can stabilize in a shorter time, you might be ready for a straight Kanban process, using this testing secret.

If you are using Scrum sprints, I recommend that you try this Scrumban process for your next sprint.   You get two immediate advantages:

1) You don’t have to do estimating and sprint planning.  Hooray!  You get an extra day for design and programming.  If you have one or more distributed teams (like the Constant Contact project shown below), you avoid the logistical challenges of agreeing on a sprint plan.

2) You can add bug fixes at any time.  In the idealized Scrum process, it’s annoying how bug fixes go into a separate work queue.

Compare the ideal Scrumban diagram to this real continuous flow diagram presented by Gil Irrizarry of Constant Contact.

constant contact diagramGet Cumulative Chart Diagram and Cardwall to help you out with ScrumBan for free with Assembla Renzoku.

4 Comments Click here to read comments

Seven Things I Hate About Agile

Posted by Andy Singleton on Wed, Aug 22, 2012
  
  
Tags: 

Last week Assembla posted a bunch of “agile” words on our home page.  I resisted doing this for years, because the word “agile” doesn’t necessarily have a good feel to it.  It’s a refuge for a lot of organizations and people that aren’t very agile.  The original idea is great.  It stands for releasing software frequently, with short lags for the implementation of valuable new features and ideas. The productivity of software development increases every year, and in theory you could use the word "agile" to describe many of the things that are light and bright, great and good, fast and fun about our new world.  But, if we want to use the word “agile” for this, we have to burn off the stink of stagnation that surrounds the old “agile.” So, here are seven things I hate about “agile.”

Old people: Agile has the smell of death on it.   If you go to an “agile” event you will see few people under the age of 40 and many over 50. These attendees are on average much older than the average age of programmers, and often older than the people running today’s hot software companies.  Why aren’t more younger people grasping at agile straws?

Stagnation: "My development guys don't think 'agile' is interesting," reported the president of a major software tools vendor, sitting among cubes covered with agile task cards. The modern agile movement is chained to an aging orthodoxy based around Scrum. What's new in Scrum this decade? Most organizations which use the word "agile" are using the word precisely because they are NOT agile, and want to be more agile. This has created a demand for a whole industry of people who try to push "agile" further into non-agile organizations amplifying the link between "agile" and really not agile. Making matters worse, they are seen as management stooges attempting "process change," which is a boondoggle that nobody wants applied to his real-world job. Most victims would rather go to the dentist.

Imposing values:  Read the agile manifesto here and think about whether this is a good tool to get a diverse group of people to work together.  We value this.  We value that.  I don’t think values have anything to do with productivity or collaboration.  The Americans and the Russians didn’t share many values, but they teamed up into a mighty fighting force during WWII because they shared a goal.  The people that make an iPhone make a beautiful product efficiently.  They live in the US, Japan, China, and many other countries.  They live in mansions, boats, and dormitories.  Do they share values?  I doubt it.  They don’t have to.  They share a supply chain.  Productivity is comes from shared GOALS, not shared values, and from a process that is structured to allow people to do their separate jobs, and work together.  As long as they share a goal and have room to work, they can ignore each other’s values.  Team productivity research shows that a shared end goal is the best predictor of high performance teams.  If you are going to insist on shared values, you are going to have a very short supply chain – probably just 5 or 10 people sitting around a campfire singing Kumbaya.  That sounds like a Scrum team!  I’m going global.  Get the hell out of my values.  And while you are at it, let me cram what I value down your throat - code that we can release.

Scrum Master roles:  Some scrum coaches rank among the nicest and most thoughtful people that I ever met.  However, the general concept is very suspicious.   A scrum master is not supposed to lead the team, code, or take responsibility for deliverables.  What do they do?  Massage?  Get a real job.  We use tech leads, who can understand the code that flies around a distributed team, and take responsibility for deliverables. I shed more light on this back in Novemeber with a popular post - Tech Leads will Rule the World

Dilbert Cartoon

Estimating:  A good, detailed estimate with task breakdowns is very reassuring if you are using it to make a budget decision or a go/no-go decision.  If you already know what you want to do (eg the top stuff on your prioritized backlog),  then estimating is a waste of time.  It reduces productivity by wasting time.  That’s true in a startup and it is also apparently true for the bigger projects in our “Scalable Agile Design Group” survey.  Our analyst talked to 12 of these project leaders and came back with this summary:

Some Agile textbooks recommend estimate development times for tasks, then measuring actual results against estimates. But members of this group were cautious about making estimates, and downright opposed to evaluating people and groups based on actual versus estimate comparisons.  Most managers expressed one or more of these views:

  • Estimates of software tasks are inherently unreliable.
  • The benefits of estimates are outweighed by the time required to make them.
  • Most developers won’t take the time to enter actual hours.
  • Measuring people against estimates sends the wrong signals and will distract people from being flexible and innovating.
  • Measuring people based on estimates causes them to game the system.

Pair programming:  Great for vendors, bad for customers.  Pair programming is like those girls that go to the restaurant bathroom together.  What are they doing?  If you are a vendor selling “pairs”, you have an awesome situation where you can charge twice as much, and you can easily churn guys on and off the pairs, one at a time, to steal talent for turnover or new projects.  If you are customer, you pay twice as much and you get churn.  I recommend a more efficient alternative – review pairs.  Person A and person B use a code review system to review each other’s code before release.  You get the same benefit without the neck cramp and the BS.

Scrum reality warp:  Let’s look at a few of the fantastical things that scrum advocates believe should be true about a Scrum team.

  • According to the Scrum manuals, development teams should be co-located (everyone in the same place for daily discussion).  Where are the foosball tables that these guys are crowding around for their daily standup?  I haven’t seen a fully co-located team in years. Even if they are co-located in theory, there is always someone who stayed home or moved to Vermont.
  • Self-contained, cross functional teams.  In this world, you have a full-time person on your team for every function – design, database, etc.  I love the idea that a team can deliver a complete feature end to end.  However, reserving talent for one team isn’t a practical way to do it.
  • Teams with the same capacity every week.  People don’t move to new projects, get hired and trained, or even move over to  help other teams  that are further behind.  Apparently that would be idiotic, because you would get work done, but you would screw up your precious velocity calculation.
  • No bugs.  You plan your next iteration, and then you don’t get any extra work because your last release has no bugs.

I was at an event where Alex Brown, COO of Scrum, Inc. was talking about his work, and when someone said “that doesn’t sound like Scrum,” he said “We’re a Scrum company, so we call everything Scrum.”  We can all recognize reality and lighten up.

We're going to be calling our stuff "Scalable Agile".  If you are interested in new solutions to some of the issues listed above, please follow this blog over the next few weeks.

60 Comments Click here to read comments

Previous Page | All Posts | Next Page

Follow Assembla

twitter facebook youtube linkedin googleplus

Get Started

blog CTA button

Subscribe by Email

Your email: