The live chat about handling bugs within agile frameworks had experts answering my questions with a combination of knowledge, experience, and fun (surprising as that may seem). I recapped some of their answers here.
During the event, we were asked dozens of questions from the audience. The questions were interesting and our experts, Raygun CEO JD Trask and Assembla CTO Jacek Materna took time to respond to each personally.
We have addressed the questions in a multi-part blog series. This second edition focuses on defining & logging defects, as well as internal communications. Please see our other posts for answers about:
Defining & logging defects
Marvin wanted clarity, "There is the idea that in the context of agile software development, we do not find bugs, we find defects. Do you think they are the same thing? How we should call a misbehavior of the software we are producing?"
Our experts agree, "We personally care very little about the labels -- we care that the customers have a great experience with our software. So when something is wrong we usually say it’s a bug just because that’s what we’ve always called them. If they’re really bad, then we’ll use curse words too ;-)"
Sendhilraj asked, "How can we deliver MVP to customers with zero defects?"
JD suggested, "Zero defects?! Sounds like some incredibly expensive software to build. If Microsoft, a $300 billion company, spend billions on software quality, and still has bugs, then maybe it’s best to accept that bugs will occur, and the best solution is to build a process to quickly iterate and resolve bugs as they’re found. Actual zero defect software would be unaffordable to develop with the tools we have today."
Brandon wanted to know, "When you go into sprint planning, do you keep a certain percentage or point value open for defects found during the sprint?"
JD replied, "Not specifically for defects, but we don’t fill the sprint entirely. Our story points are a day each, and in a 2 week sprint (10 business days) we’ll allocate 8 story points of effort. This allows for time in fixing bugs, repaying technical debt, or handling estimates being optimistic."
Marie Louise wondered, "Are bugs considered Technical Debt?"
JD shared, "We don’t consider bugs technical debt. Anything that impacts the customers is considered a bug. In our situation, we consider technical debt things like out of date libraries, database upgrades that are needed, removing old legacy data structures that are potentially still used, code tidiness, etc. Things that don’t impact the customer, but that make software development slower and less enjoyable (or less maintainable)."
Giada asked, "Do you formally log defects during the sprint or you won't consider the story completed until the MVP the story delivers is working?"
JD responded, "Because Raygun gives us insights into defects, we usually discover problems very quickly when a deployment is done containing a defect. This means we either role back the deployment (not completing the story), or we rapidly patch the defect and push out a new build (thereby completing the story). That’s our process, but I would add that if something is knowingly buggy, I wouldn’t call it completed."
Jacek's take is that "working product is when something is done that meets quality level in my opinion - not critical, gets pushed out to next “release” cycle. At Assembla, we don’t have a fixed cadence really. We focus on quality over a firm date - again there are exceptions to the rule, but that's my general view. Focus on releasing quality so customers are happy over meeting some artificial self-inflicted schedule :)"
Taylor asked us, "For new features of an existing product, what is the best method of communicating changes internally (for all affected team members)? How early in the development process are those outside of research and development looped in?"
JD said "This is a common challenge for us at Raygun as well. We have a weekly all hands team meeting where the Director of Product will say what features are shipping that week.
"We created the Director of Product role partly because of the challenges of keeping the team in sync, and specifically between marketing and engineering. Features would ship out and miss marketing, which wasn’t great. Sprint planning is done so all of the engineering & design team know what everyone is working on, and the daily standups keep everyone in the know also. It’s an ongoing challenge, but communication is always the key."
Jacek explained, "At Assembla, we keep our roadmap and continuous delivery backlog in our system and everyone can see at a high level what is coming down the pipe. We push out weekly product updates internally as well. We are a small team and we all see what’s going on daily inside our Assembla spaces, so the Assembla app keeps the Assembla company in sync! We loop folks in at day 0 via the app and its communicated broadly out to everyone once it becomes more defined."
Thanks to Marvin, Sendhilraj, Brandon, Marie Louise, Giada, and Taylor for the thought-provoking questions. Thanks to our experts, Jacek Materna (@), CTO of Assembla and JD Trask (@), CEO of Raygun for the result-focused answers.
Please stay with us on our next post as we address questions about differences in dev and production environments, bugs that stop the sprint goal, and third-party bug reporting in Assembla.