If you have been following along with the earlier posts in this series, you already know that it all started with a recent live chat, in which I asked questions about #AgileBugs. THe tech experts Raygun CEO JD Trask and Assembla CTO Jacek Materna answered and then dozens of attendees sent in their intelligent, thought-provoking questions about handling bugs within agile methodologies.
We created a series of blog posts with the answers.
This fifth and final post in the series is about bugs after the sprint, bug-handling tools, bugs when developing for clients, and differences in methodologies. Please see our other posts for answers about:
Bugs after the sprint
Lijo asked, "What should I do if a functionality bug is caught after sprint?"
Both experts agreed, "If it’s critical, jump on it. If it’s minor, just schedule it for another sprint."
Bogdan asked, "Who decides if a bug found during the sprint is release-blocking or not?"
Both experts agreed on this too, explaining "Typically if a bug is blocking, it’s pretty obvious (e.g. user cannot perform a function they want to), and anyone on the team can raise a bug as being severe enough to block release.
Sumant asked us, "How do we replan when we have bugs after the sprint has been closed?"
JD explained, "We just roll the fixes into the next sprint. If they are critical bugs, then we break with the sprint model and just focus a team member on resolving the issue as soon as possible. We may then reduce the workload in the next or current sprint for the engineers who had to work on resolving the bug. I hope that helps!"
Jacek said, "Our focus is on working product. Whatever it takes to ensure the product works and the customer is happy becomes the center of attention - the path you take to make that happen is variable. JD’s approach is a common one in a sprint model.
David wanted to know, "What tools would you suggest for managing bugs?"
JD obliged, "Firstly, I’m obviously a big believer in automatic bug reporting so that you get as much information as possible about the system.
"Secondly, you want to use such a tool that integrates with your choice of issue tracker (Assembla is a great choice here!). That way, you can automatically pick up bugs from your systems and then connect them into your issue tracker that your entire team (even non engineers!) can use and see the full health and roadmap of the system.
"That’s what we do at Raygun - automatically report bugs, then promote them into the issue tracker so that the director of engineering and director of product can manage them from the issue tracker."
Bogdan wanted to know, "Why is Raygun better than Sentry?"
JD put it simply, "Sentry only does crash reporting, while Raygun is a platform encompassing crash reporting and real user monitoring. I could chat for a long time about all the differences, but the key difference that impacts customers is that Sentry only samples data, not storing everything that happens. They do not make this clear to their users but it does mean you’re still flying somewhat blind when using Sentry, while Raygun gives you everything."
Sendhilraj asked for a recommendation, "Is there any collaborative code review tool?"
Jacek explained, "The Assembla collaborative code review and merge features enable contributors to submit code from a branch, and their team members can pull and test changes, view changesets and affected files, submit a new version, vote on the request, and ultimately merge or reject the request - all from one simple UI.
"Merge Request (i.e. Pull Requests) are first class citizens in Assembla. Meaning you can comment, link to tickets as a relation, view all commits, view code diffs, look at a commit and check the Blame history."
Bugs when developing for clients
Melissa asked, "How do you explain to a difficult customer that it is normal that bugs appear after a deployment?"
JD suggested, "I usually relate them to reality. I talk about how buggy Windows or Microsoft Office is. If a 300 billion dollar company can’t make perfect software, then why exactly are they expecting perfection from you? :-)
"But seriously, often times, if a customer is very angry about such things, it means they are annoyed about something else. I’d talk to the customer to find out what’s really the matter and if there’s a way to make them happy. If there’s not, then I usually suggest that they should look at other options - sometimes they are not worth the hassle (I appreciate this is a lot easier for the CEO of the company to say!)"
Jacek agreed, "Rarely is someone mad about bugs - only missing features that they expected to get (expectation problem) or something else. The goal, of course, is to detect bugs before clients see them and have a great process to fix them in production proactively - this where Raygun helps.
Andrew asked, "Do you involve testers in talking to customers?"
JD explained, "No. We engage a lot with customers through our engineering team, and our Director of Product. We encourage all team members to talk to customers though as it helps them understand how customers see the world.
"Having said that, the Raygun product is one that is primarily used by engineering teams, and they usually appreciate being able to speak directly to the developers on the team who worked on a feature -- that may not apply to the products you’re building."
Differences in methodologies
Ram wondered out, "What would I do differently (than what I did in waterfall) when following agile methodology?"
JD explained, "How much you buy into agile methods is up to you. As mentioned, at Raygun we use a hybrid of aspects from several methodologies because that’s what works best for us. We have stand ups because it helps the team communicate, we use sprints just to ensure that the team is focused most of the time on a small feature set to work on, and we use story points so we can get better at estimating the effort for things.
"Where I think we do things differently to a waterfall approach is that we don’t let the methodology replace critical thinking. If an issue arises mid sprint and impacts customers, we’ll focus on fixing the issue and not blindly follow what our methodology is.
"If we realise part way through a feature development that it’s a bad idea or could be done a better way, we change it. We mold the process around what makes sense, rather than letting the methodology dictate how to work."
Sendhilraj asked our experts, "How can we use XP techniques in large teams to reduce bugs?"
JD replied, "Extreme programming (XP) is a software development methodology that helps with bug reduction at any scale by giving you a second brain to think about potential issues that could arise.
"As I mentioned in the webinar, communication often will ensure bugs don’t make it to production in the first place. In particular, some people are very good at thinking about how to exploit software and so will witness something being developed and speak up with “Whoa - hey, I could totally break this by doing xyz”. XP helps with that."
Before we bid you farewell (for now), a big thank you to the hundreds of you who listened in, asked questions, and keep coming back to the recording of the webinar to learn more about how to deal with bugs.