Agile Bugs Q&A: Part 5

Lital Barkan on July 19, 2016


If you have been following along with the earlier posts in this series, you already know that it all started with a recent live chat, in which I asked questions about #AgileBugs. THe tech experts Raygun CEO JD Trask and Assembla CTO Jacek Materna answered and then dozens of attendees sent in their intelligent, thought-provoking questions about handling bugs within agile methodologies.

We created a series of blog posts with the answers.

This fifth and final post in the series is about bugs after the sprint, bug-handling tools, bugs when developing for clients, and differences in methodologies. Please see our other posts for answers about:

Part 1:
  • Estimating   


Part 2:
Part 3: 
  • Differences in dev and production environments
  • Bugs that stop the sprint goal
  • Third-party bug reporting in Assembla
Part 4:
  • Tips for designing use cases
  • Automating testing
  • Preventing regression
  • Using Assembla for failed test cases


Bugs after the sprint

Lijo asked, "What should I do if a functionality bug is caught after sprint?"

Both experts agreed, "If it’s critical, jump on it. If it’s minor, just schedule it for another sprint."

Bogdan asked, "Who decides if a bug found during the sprint is release-blocking or not?"

Both experts agreed on this too, explaining "Typically if a bug is blocking, it’s pretty obvious (e.g. user cannot perform a function they want to), and anyone on the team can raise a bug as being severe enough to block release.

"However, we do release feature branches many times a day, so a block is relatively low impact because it stops the release of only the feature that it impacts, not the rest of the team."  Tweet: Critical bugs = low impact. They stop the release of the impacted feature, not the rest @jacekmaterna @traskjd

Sumant asked us, "How do we replan when we have bugs after the sprint has been closed?

JD.pngJD explained, "We just roll the fixes into the next sprint. If they are critical bugs, then we break with the sprint model and just focus a team member on resolving the issue as soon as possible. We may then reduce the workload in the next or current sprint for the engineers who had to work on resolving the bug. I hope that helps!"

Jacek.pngJacek said, "Our focus is on working product. Whatever it takes to ensure the product works and the customer is happy becomes the center of attention - the path you take to make that happen is variable. JD’s approach is a common one in a sprint model.

Bug-handling tools

David wanted to know, "What tools would you suggest for managing bugs?"

JD.pngJD obliged, "Firstly, I’m obviously a big believer in automatic bug reporting so that you get as much information as possible about the system.

"Secondly, you want to use such a tool that integrates with your choice of issue tracker (Assembla is a great choice here!). That way, you can automatically pick up bugs from your systems and then connect them into your issue tracker that your entire team (even non engineers!) can use and see the full health and roadmap of the system.

"That’s what we do at Raygun - automatically report bugs, then promote them into the issue tracker so that the director of engineering and director of product can manage them from the issue tracker."

Bogdan wanted to know, "Why is Raygun better than Sentry?"

JD.pngJD put it simply, "Sentry only does crash reporting, while Raygun is a platform encompassing crash reporting and real user monitoring. I could chat for a long time about all the differences, but the key difference that impacts customers is that Sentry only samples data, not storing everything that happens. They do not make this clear to their users but it does mean you’re still flying somewhat blind when using Sentry, while Raygun gives you everything."

Sendhilraj asked for a recommendation, "Is there any collaborative code review tool?"

Jacek.pngJacek explained, "The Assembla collaborative code review and merge features enable contributors to submit code from a branch, and their team members can pull and test changes, view changesets and affected files, submit a new version, vote on the request, and ultimately merge or reject the request - all from one simple UI.

"Merge Request (i.e. Pull Requests) are first class citizens in Assembla. Meaning you can comment, link to tickets as a relation, view all commits, view code diffs, look at a commit and check the Blame history." 

Bugs when developing for clients

Melissa asked, "How do you explain to a difficult customer that it is normal that bugs appear after a deployment?"

JD.pngJD suggested, "I usually relate them to reality. I talk about how buggy Windows or Microsoft Office is. If a 300 billion dollar company can’t make perfect software, then why exactly are they expecting perfection from you? :-)

"But seriously, often times, if a customer is very angry about such things, it means they are annoyed about something else. I’d talk to the customer to find out what’s really the matter and if there’s a way to make them happy. If there’s not, then I usually suggest that they should look at other options - sometimes they are not worth the hassle (I appreciate this is a lot easier for the CEO of the company to say!)"

Jacek.pngJacek agreed, "Rarely is someone mad about bugs - only missing features that they expected to get (expectation problem) or something else. The goal, of course, is to detect bugs before clients see them and have a great process to fix them in production proactively - this where Raygun helps.

Andrew asked, "Do you involve testers in talking to customers?"

JD.pngJD explained, "No. We engage a lot with customers through our engineering team, and our Director of Product. We encourage all team members to talk to customers though as it helps them understand how customers see the world.

"Having said that, the Raygun product is one that is primarily used by engineering teams, and they usually appreciate being able to speak directly to the developers on the team who worked on a feature -- that may not apply to the products you’re building."

Differences in methodologies 

Ram wondered out, "What would I do differently (than what I did in waterfall) when following agile methodology?"

JD.pngJD explained, "How much you buy into agile methods is up to you. As mentioned, at Raygun we use a hybrid of aspects from several methodologies because that’s what works best for us. We have stand ups because it helps the team communicate, we use sprints just to ensure that the team is focused most of the time on a small feature set to work on, and we use story points so we can get better at estimating the effort for things.

"Where I think we do things differently to a waterfall approach is that we don’t let the methodology replace critical thinking. If an issue arises mid sprint and impacts customers, we’ll focus on fixing the issue and not blindly follow what our methodology is.

"If we realise part way through a feature development that it’s a bad idea or could be done a better way, we change it. We mold the process around what makes sense, rather than letting the methodology dictate how to work." Tweet: Mold the #agile process around what makes sense, rather than letting the methodology dictate how to work @traskjd

Sendhilraj asked our experts, "How can we use XP techniques in large teams to reduce bugs?"

JD.pngJD replied, "Extreme programming (XP) is a software development methodology that helps with bug reduction at any scale by giving you a second brain to think about potential issues that could arise.

"As I mentioned in the webinar, communication often will ensure bugs don’t make it to production in the first place. In particular, some people are very good at thinking about how to exploit software and so will witness something being developed and speak up with “Whoa - hey, I could totally break this by doing xyz”. XP helps with that."

Before we bid you farewell (for now), a big thank you to the hundreds of you who listened in, asked questions, and keep coming back to the recording of the webinar to learn more about how to deal with bugs.

And, to our experts, Jacek Materna (@jacekmaterna), CTO of Assembla and JD Trask (@traskjd), CEO of Raygun - thank you for your time, your experience, your patience, and in general - thank you for being brilliant!