Agile Bugs Q&A: The experts weigh in

Lital Barkan on July 19, 2016


In a recent live chat about handling bugs within agile frameworks, I had the opportunity to ask two technical experts my questions around this topic. Hundreds signed up to listen in to their answers and when we opened the floors to questions, well . . . it turned out mine were just the tip of the iceberg!

Raygun CEO JD Trask and Assembla CTO Jacek Materna have led dev teams in companies and through projects of all sizes. You can read the recap of their answers to my questions here. After the event, we continued to address the audience questions we did not have the opportunity to answer in the live chat.

We have addressed the questions in a multi-part blog series. This first edition is all about estimating. Please see our other posts for answers about:

Part 2:


Part 3: 
  • Differences in dev and production environments
  • Bugs that stop the sprint goal
  • Third-party bug reporting in Assembla
Part 4:
  • Tips for designing use cases
  • Automating testing
  • Preventing regression
  • Using Assembla for failed test cases
Part 5:
  • Bugs after the sprint
  • Bug-handling tools
  • Bugs when developing for clients
  • Differences in methodologies 



Tom wondered, "How do you estimate tasks? We currently use Small, Medium, or Large, but this hasn't been accurate for us during sprint planning."

JD.pngRaygun CEO, JD Trask, replied, "Everyone estimates differently. We use "Story Points," where a point is the equivalent of a person-day of effort. We then allocate each person 8 story points for a two week sprint. This gives them 2 story points of buffer for support work, defect work, or paying down technical debt. Worst case, they need that time because the work was mis-estimated.

"We then discuss how long things will take. Some engineers will always overestimate, others will underestimate. Learning who does what is important. When I was still writing code, I often found estimates were most correct when I started to feel slightly embarrassed about how much time I said it would take me :-)

Jacek.pngAssembla CTO, Jacek Materna explained, "We have this feature in Assembla, points (high, medium, low, hours, etc.) but our own team doesn't use it alot. Why? Because it rarely works - it’s the illusion of being in control.

"Akin to the the old adage of 'if you have a Gantt chart with everything on it, a year from now it will happen exactly as planned.' Nope. In agile you have to culturally sign up for a level of entropy in the system and build people and processes around the entropy to deal with issues in a dynamic manner. Tweet: In #agile, build people & processes around entropy to deal with issues in a dynamic manner @jacekmaterna @Assembla

"The best that has worked for us is that we say we’ll do something and then we start doing it - as it gets closer to completion, we know if it’ll be out this month or next. It may sound funny but in reality, that's how it always ends up! Bugs are different - but with features, the reality is that you don’t really know until 80% into the task how long it will take to finish it."

Clayton also wanted help with his issue, "As a PO, how can I try to get my developers to estimate their time better? For example, when we find a bug and the developers estimate during planning, they can't know how many hours it will take to find that bug. Is there some way to get better estimates?"

JD.pngJD offered his take, "I try to share the data. 'You estimated 4 days, it took 6 days - were you distracted or was the estimate just incorrect?' To be honest, it can be hard to discuss this in a culture that feels any discussion of mistakes is an attack, so encouraging candor and a willingness to use data to drive decisions is important.

"It often comes down to one-on-one work between the development manager and the team members. I personally found my estimates only got close to right when I felt they were awkwardly large estimates. This is what causes many to under-estimate the time.

"Be honest with the data, and let the team members know it’s not a bad thing to say ‘you know what, I get this wrong by 25% almost every time, let me add a buffer.’ It’s better than being wrong all the time! :-)"

Jacek.pngJacek's take was "Hard to master, impossible to perfect. I think the culture must be there to build in entropy into the model. If entropy, failure, slips, bugs during dev are found and the process accepts those are part of the “norm,” then the the culture can focus all of its energy on handling the reality versus building a culture of blame.

"Hey, blame is good, it’s actually productive during code reviews but at a PO level, I think the focus should be on creating the infrastructure to deal with the reality. I digress, in terms of the specific question, it’s really about seniority and past experience and common sense. If a new guy is working on a thing that he’s done 10 of in the last year, the confidence level that he will estimate well... is pretty good, whereas a senior guy doing something brand new may not be so good - even if he’s senior. I typically ask one question - 'OK, John is doing that feature? Great - how many times has he done something like this before? Oh first time? What’s his estimate? 4 weeks? Double it!' :)"

Rafael asked, "What's your suggestion on how to proceed when a bug needs to enter the current sprint but the effort (story points) is unknown? Should we take the bug adjust story points later?"

JD.pngJD explained, "We add the points later and reduce the workload in the rest of the sprint for the impacted engineer(s) so they are not behind on their work or feel pressured to work over time to deliver on the original sprint goals they had."

Alex wanted a ballpark, "Approximately how much time in a sprint do you spend on bug fixing?"

JD.pngJD estimated about estimates, "It varies. We allocate 20% of the story points in a sprint to fixing bugs, repaying tehcnical debt, or handling estimate mistakes. That works pretty well for us."

Sendhilraj asked, point-blank, "How can I estimate the time required for defects fixing?"

JD.pngJD obliged, answering "It can be hard to estimate. What we do is have the original developer who worked on that part of the code base look at the issue. They have the most context to make a good estimate of the time to fix. Even then however, we have to just get started on the fix and measure how long it took. Then we adjust their feature workload to compensate."

Marie Louise explained the most-asked about subject of estimations like this, "When planning a sprint, some teams use an estimating technique for stories. A common dilemma is - should they estimate the bugs that are in the backlog, which are a result of another story being deployed in a previous sprint? One assumes when the other story was estimated, it took into consideration testing and fixing bugs. So the question is, do you estimate those outstanding bugs in the new sprint, to ensure your velocity is accurate? Or is that double dipping, since they should have been covered by the past sprint velocity/estimate?"

JD.pngJD shared his experience, "At Raygun, we add 20% story points to a sprint for team members to work on bugs, repay technical debt, or to handle estimate issues. It works pretty well. That seems to help reduce overruns and other issues."

Please stay with us through the next part of this series, a we explore the questions about defining and logging defects, as well as internal team communication.

We know the audience members who asked questions during our event were speaking for many who are working in agile framworks and are unsure about how to handle the bugs. We thank those who asked questions for speaking up for everyone! 

Moreover, we thank our experts, Jacek Materna (@jacekmaterna), CTO of Assembla, and JD Trask (@traskjd), CEO of Raygun - for your eye-opening answers. We all learned a lot from this event and from the questions that followed. Thanks JD and Jacek for being our teachers!