We created a series of blog posts in which our experts addressed all of your questions.
This third post is about differences in dev and production environments, bugs that stop the sprint goal, and third-party bug reporting in Assembla. Please see our other posts for answers about:
Differences in dev and production environments
Sid wanted to know, "How do you control code release? Do you let the software developer release directly in production or do you release in a sub-production environment and then into production?"
JD answered that "It depends on the organization. At Raygun, anyone on the development team can deploy code to production. They do need to follow the process of deploying to Office, then Beta, then Production."
Jacek agreed, "Same at Assembla. QA can be looped into the pipeline if the developer tags the merge request a certain way, that way the release of code becomes more of a QA driven deal versus done automatically by staging tests only."
Amarjot sent this question, "How do you handle bugs if they get deducted in production environment, but not in the dev environment?"
JD explained, "I don't want to be too much of a sales person, but this is exactly why we built Raygun. In development, you often have less data, use one browser, use the software exactly as intended, and already have a debugger on the machine.
Production is a hostile environment. You need the tooling support to automatically capture as much diagnostic data about a bug, record it, and alert in quickly. Raygun does that for you :-)"
Jacek added "Your production environment and staging must be 100% the same from an infrastructure and configuration point of view in terms of runtime. That’s the first step. We use Chef to ensure this happens.
The second step is understanding that the wild has many variable you will NEVER be able to test for - so like in today’s security environment, it’s not about preventing the attack anymore, its about realizing the attackers will get in, and having the tools and processes in place to detect, report, mitigate as fast as possible. Thats where something like Raygun comes into play as a key piece of this loop."
Bugs that stop the sprint goal
Bret inquired, "What if high priority Defects are disturbing our Sprint goal?"
JD addressed the issue, "We fix issues first, always. Our customers usually appreciate that having what’s already built work well is more important than getting a new feature (we sell to engineering teams, so they get that there are tradeoffs).
"We are also quite fast-moving, so often resolving a defect can be done within a few hours or a couple of days, at the worst end of the spectrum. This means that any delay in feature work is not substantial.
"It’s important to remember that the methodology is a guide and not set in stone - at the end of the day, we get to come to work and build great software because of our customers pay us money. It’s therefore our foremost goal to ensure that they have a great experience and the system works well. Any methodology that is so set in stone as to prevent this behavior is a bad one in my view."
Nirupama inquired, "How do you prioritize a blocker bug in a new feature versus a critical regression bug before a release?"
JD explained, "We do feature level branching and merge into Master for deploy. The releases we do are not ‘big bang’ releases, therefore this doesn’t come up so much with our process.
"If a bug is known in a feature branch, then it won’t be merged. If it does get merged before the bug is discovered, then we will either reverse the merge or make the fix quickly and merge it into master.
"In terms of prioritising bugs at the team level, anything that prevents the customer from performing an action is raised as critical and becomes a priority-1 thing to solve. While we haven’t had multiple priority-1 issues, we’d just have other team members own them if that was to occur."
Third-party bug reporting in Assembla
Juan wanted to know, "It was mentioned that bugs are being reported either manually or automatically to Assembla. For clarification purposes, are third-party systems being integrated with Assembla to create bugs automatically? And, can we, as Assembla users, integrate our Assembla Spaces with third-party bug reporting systems currently?"
Jacek says, "Assembla has a lot of REST based API’s you can use. Many third party systems allow you to build hooks to push events out. One of our deployment monitoring tools can be pointed to Assembla to push tickets directly when certain errors occur in production."
Thanks to those who asked these important questions. Your inquiries are helping the many others who listened in and are still tuning in to the recording. And thanks to our experts, Jacek Materna (@), CTO of Assembla, and JD Trask (@), CEO of Raygun.
Please stay with us for the next post, in which JD and Jacek tackle issues of designing use cases, automating testing, preventing regression, and using Assembla for failed test cases.