Why Relying on Your Users to Report Errors Is the Dumbest Thing You‘ll Ever Do

As a full-stack developer, I dream of building software that is flawlessly architected, elegantly coded and blissfully free of pesky bugs. But then I wake up to reality. The cold hard truth is, no matter how skilled or rigorous you are, errors are inevitable. Especially once your application reaches any level of sophistication and scale.

The real measure of quality engineering is not preventing bugs entirely, but how quickly you detect and resolve them when they do occur. And one of the worst mistakes I see developers make is relying on users to report errors. It‘s the equivalent of a smoke alarm that only works if your neighbor happens to notice your house is on fire and gives you a call. Not exactly ideal!

The Pitfalls of User-Reported Errors

At first glance, user-reported errors seem like an obvious and cost-effective approach to monitoring application health. Let your customers be your QA team and support-ticket their way to a better experience! Sounds good in theory. But there are some serious practical flaws with this approach.

First and foremost, the vast majority of users will never actually report an error. A 2017 study by SmartBear found that only 1 out of every 26 users who encounter a problem actually reports it. That means 96% of errors remain invisible to your team!

The users who do make the extra effort to send feedback rarely provide enough detail to be actionable. Their reports often lack basic diagnostic information like browser version, operating system, steps to reproduce or screenshots of the issue. Following up to extract these details costs your team precious time and slows down resolution.

Even more concerning, a portion of users who hit errors won‘t just fail to report them. They‘ll rage quit your app and never come back. According to a 2015 Dimensional Research survey, 61% of users said they were unlikely to return to a mobile app that crashed. 40% said they would consider switching to a competitor. Can you really afford to bleed users like that?

Drowning in Email Alert Noise

Some developers do recognize the need to proactively detect errors rather than depend on sporadic user reports. Often their first instinct is to add email alerts that fire off whenever an unhandled exception occurs in production. This quickly devolves into the coding equivalent of the boy who cried wolf.

With no way to intelligently filter or aggregate errors, your inbox will be flooded with an alert for every single exception instance. In an app of any significant usage, that means waking up to thousands of emails about trivial issues drowning out the critical errors. Imagine trying to dig through 50k emails about a benign browser warning to find the one email about a billing system failure.

The noise level becomes unmanageable and error emails start getting ignored or filtered into oblivion. I‘ve even heard horror stories of developers accidentally sending error alerts to customers because of overzealous emailing! Talk about a poor user experience.

The Limitations of Logging Frameworks

The next step up the error monitoring maturity model is plugging into logging frameworks like Log4j, Serilog or LogStash. These certainly provide richer diagnostic detail and better aggregation than email spamming. But they still fall short for true application health monitoring.

At their core, logging frameworks are designed to passively collect log events for later analysis and auditing. They don‘t provide the real-time alerting, impact analysis or workflow integration needed for proactive error resolution. Some key gaps include:

• Lack of alerting when new or critical errors spike
• No concept of user-impacting errors vs low priority bugs
• Limited context on affected browsers, devices, locations, etc.
• No de-duping or aggregation of similar errors
• Decoupled from bug tracking tools and processes

Here‘s a simplified code example illustrating these limitations:

try {
  // Some bug prone code
} catch (Exception ex) {
  logger.error("Something broke!", ex);
}

With the above approach, an exception will get logged with a stack trace. But you won‘t know what user impact it had, whether it spiked in frequency after a release, or how to prioritize it against other bugs. There‘s no clear next step from logging to resolution.

That‘s where true error monitoring tools come in. They go beyond basic logging to provide end-to-end error management workflows. A code snippet using an error monitoring tool would look more like:

try {
  // Some bug prone code 
} catch (Exception ex) {
  Raygun.send(ex, tags: ["critical", "checkout"], customData: { userId: 123 });
}

Now the exception is captured with rich metadata on severity and impact. It‘s aggregated with similar errors and alerted on if it exceeds thresholds. An issue is auto-created in your bug tracker with all the diagnostic details needed to prioritize and resolve. That‘s a night and day difference from passive logging.

The Benefits of Proactive Error Monitoring

Implementing a robust error monitoring tool fundamentally transforms how your team detects, triages and resolves bugs. Instead of hearing about errors anecdotally through the grapevine, you have quantified insight into the real-time health of your application. Some key advantages include:

• Visibility into all errors, not just the small fraction users report
• Automatic alerting when critical errors spike or new exceptions appear
• Rich context to assess user impact and priority, with breadcrumbs, environment details and affected user counts
• Seamless integrations that sync errors to your project management and communication tools
• Release and deploy tracking to identify version-specific issues
• Trends and analytics to measure stability and improvement over time

This proactive, data-driven approach to error resolution yields meaningful results. Teams adopting error monitoring consistently see major gains in quality and agility metrics. Raygun reports their customers on average experience:

• 95% reduction in mean time to resolution (MTTR) for errors
• 90% reduction in customer-reported issues
• 25% increase in Net Promoter Score (NPS)
• 15% reduction in churn rate

A great case study comes from HotSchedules, who reduced error resolution time by 80% after rolling out error monitoring. By proactively identifying and resolving bugs before users complained, they increased NPS by 20 points and saw a significant uptick in user retention.

Observability: The Bigger Picture

Error monitoring is really just one piece of the observability puzzle. To have complete visibility into the health and performance of your application, you need a holistic approach that includes metrics, logging and tracing as well.

Forward-thinking development teams are embracing observability as a core tenet of their software strategy. They instrument their code with telemetry that allows them to answer any question about application behavior. Questions like:

• How many users are impacted by this error?
• What‘s the 99th percentile latency for this API?
• How many orders were dropped due to this checkout bug?
• Which database queries are slowest and why?

With this observable system in place, they can be proactive rather than reactive. They identify and resolve issues before users are broadly impacted. They have the insights needed to continuously optimize performance. They make data-driven decisions about where to invest development resources.

This is the future of software quality. Not shipping and praying everything works, but having quantified confidence in the real-user experience. Observability is how elite engineering teams operate.

Conclusion

If you take away one thing from this post, let it be this – relying on users to report errors in your application is a massive mistake. You‘re letting critical issues fall through the cracks, frustrating customers and bleeding revenue. In today‘s hyper-competitive software market, that‘s just not something you can afford.

Investing in error monitoring tooling and observability pays massive dividends. You‘ll identify and squash bugs before they drive users away. You‘ll have happier, more productive developers who aren‘t constantly firefighting. You‘ll deliver the stable, performant experiences needed to stand out in a crowded market.

Don‘t leave your error detection strategy to chance or expect your users to QA for free. Take control with proactive monitoring and set your team up for long-term success. Your customers (and engineers) will thank you.