This post was written and contributed by Freyja, Marketing @ Raygun, a software intellegince platform for web and mobile.
Shipping code both on time and within budget can be difficult when software errors rear their ugly heads.
Software veterans will tell you there’s no silver bullet approach to keeping your end users protected from errors. It’s pretty much impossible to write flawless code and catch every bug before releases end up in the hands of customers.
It’s more about understanding the overall health of your software so you can address software errors quickly and effectively.
So how can we get a realistic picture of the health of our software so we can face issues proactively?
As leaders in the software intelligence space, we recognize three main ways you can build an accurate picture of your application health:
- Use the right KPIs to benchmark your efforts
- Foster a healthy company attitude toward software errors
- Automate the process of finding and resolving errors so you can fix them quickly
1. Use the right KPIs to benchmark your efforts
Which KPIs should you be tracking to monitor the health of your software? We know that what gets measured gets managed, but which metrics give you the most accurate picture of what is actually happening in your application after deployment?
The team at Raygun are accountable to four KPIs that we know will:
- Improve user experience with less crashes and faster software
- Reduce technical debt
Users affected by software bugs
This is a much better metric than total error count. It’s easy to forget there are people behind the numbers. Reducing your error count still means end users are experiencing a slow buggy app, therefore it’s better to measure by user.
If you have 20,000 errors affecting one customer, it’s not as critical as 100 errors affecting 500 customers. Be smart where you allocate your developer resources.
Median application response time
The median response time is what 50% of your customers are experiencing (or faster). Get your team to collaborate around making this time faster. Afterall, poor app performance costs money!
P99 application response time
We also need to appreciate our upper 99th percentile of users. This will usually be pretty slow - but we are aiming for 5 seconds or less, not 30 seconds slow!
Resolved bugs vs. new bugs
Instead of managing crash counts (instances of bugs encountered), we find it best to measure bug count. That way, your team is fixing errors as quickly as they are being created.
We use our own software, Raygun, to track these metrics in real time. The key is to make them visible across your whole staff.
Raygun has the ability to create custom dashboards, so whichever operational metrics you use they can be held front of mind across your team. (And also makes it easier to show your key stakeholders your progress.)
2. Foster a healthy company attitude toward software errors
Using the benchmarks above, you can start to accept that software bugs will happen, and zero bug software doesn’t exist.
If Microsoft can be a 300 billion dollar company, spend billions on software quality, and still have bugs, then it’s best to accept them as part of the development process.
The best solution is to build a process to quickly iterate and resolve bugs as they’re found. Your development team are in your code base all day long and probably have a great nose for when ‘something isn’t quite right.’
Encourage your team members to take an investigative approach to errors and performance issues. A crash reporting tool can make this process enjoyable for your team, as they can dig right into the diagnostic details of an error rather than the frustrating process of digging around in log files trying to replicate issues.
This small switch in attitude can work wonders for a team. A collaborative approach emerges and software errors are no longer an afterthought. Code quality becomes a priority and your end users become much happier for it!
3. Automate the error resolution process
Because zero-bug software is an unattainable goal, the best solution is to build a process to quickly iterate and resolve bugs as they’re found. Automating the error resolution process allows you to move faster and reduces the risk of large errors disturbing sprint goals.
For example, at Raygun, we make fixing issues our priority over releasing features. We recognize customers would rather have a great experience than be on the receiving end of buggy features.
So, why not prioritize your most important asset - your customers?
Tools like Raygun and Assembla are designed to do just that.
They help you automate your bug reporting so you can build an accurate picture of the health of your software.
Taking a proactive approach to solving errors in your applications will help put your end users first, boosting user engagement and satisfaction.
This is what this process looks like in detail:
At a high level, Raygun automatically reports bugs then promotes them to the issue tracker so fixes can be assigned a time frame and fixed accordingly.
(At Raygun we assign 20 story points to fixing errors and managing any technical debt.)
Here’s how the error resolution looks with Raygun and Assembla (you can see how this workflow looks in Raygun and Assembla in more detail here)
A error arrives in Raygun and is presented in your dashboard:
You can even search for the specific user who reported an issue:
When you click on an error group, you can view the diagnostic details for that particular occurrence:
Now for the resolution.
Using Assembla, assign team members, loop in support staff and put this into your workflow within your team’s Assembla account. You can then prioritize fixes easily in your Assembla account:
You can also easily click to return to Raygun at any stage and view the error diagnostic information again:
Once a fix has been deployed, Raygun will scan for any problematic deployments including the version, so you can detect whether or not your fix was effective:
Taking a proactive approach to solving software errors will help you to automate the entire process.
Understand the health of your software with Assembla and Raygun
Shipping more features that your customers love is what’s important, not chasing software errors.
Tools like Assembla and Raygun are designed to work together to automate your error resolution process. They make it easier and faster to fix errors affecting real people.
If you rely on healthy software to bring in revenue, it makes sense to understand your software health inside and out.