Striking the bug report balance
When your high quality standards are a blocker
Challenge
As a product owner, one of my key jobs is to look over the quality of the (SaaS) product we put out there.
Before customers put their hands on a new update or feature, I must make sure it fits our standards. This includes, obviously, working properly but also usability, copywriting, consistency, documentation, and more.
One of the challenges I've been dealing with is coming up with what I like to call the 'bug report balance'.
The bug report balance
There are two extremes of product reviewing. On one hand of the spectrum, a product owner could simply not review anything at all. They could trust the developers, let 'em merge branches, and come what may.
On the other hand, a product owner could be overbearing and meticulous. Every pixel needs to be perfect, every piece of copy optimised, every potential use of the new tool/feature tested backwards, forwards, and sideways.
While it may seem that going for option #2 here is the right approach, my experience as a product owner has taught me otherwise.
The dangers of not reviewing anything are pretty self-evident. But, the dangers of over-reviewing and bug-reporting are very real, too.
Slow production down
Facebook famously said, "go fast and break things". While that might not be a motto you want to adopt for yourself, it is absolutely true that agonising over every detail of a release is going to slow you right down -- something no SaaS business should want to do.
Bring down developer morale
I'm all for high shipping standards. As someone who owns the product, markets it, and presents it to customers, trust me, I want features I can be proud of and trust.
But there is something almost sadistic about searching for the needle-bug in the haystack of amazing work your devs are putting out there. This 'gotcha' attitude will soon demoralise your team(s) -- something you definitely don't want.
Stopping the truly valuable feedback loop
Whether you develop, market, or own the product, know one thing: you know nothing.
The customer, the end-users: they know what they want. You merely have a conceptual representation of the features you'd like built and how they'll use them. By nitpicking early in the process, you block the valuable feedback from real users from hitting your desk. It's a sacrifice you should never be willing to make.
Decisions
That's all nice and good, but how do you strike a balance?
In all honesty, I think it's a work in progress. The right balance changes depending on many factors, such as company stage, target audience, market vertical, features, and more.
I've taken a specific look at my behaviour to try and avoid the three dangers listed above. Here are my decisions.
1. Does it work?
Often, this is the only question I ask myself. Does it work?
If it does, great. Sure, there might be minor things to report, but overall if the feature works as intended, I believe it should be thrown into the real world.
If it doesn't, then I have an easy decision. Report the issues and get them fixed.
2. Thorough scoping and testing
As a product owner, it is (part of) my job to instruct the dev team on what should be built and how. It is also my job to ensure thorough testing gets done, either by me or a team of qualified individuals.
With this in mind, it’s easy to realise that bugs in production are often as much my fault as it is theirs.
Without a proper system that specifically lays out a feature's requirements, user stories, and goals, it's nearly impossible for the dev team to produce satisfying work.
Even if, miraculously, something gets built, without testing you’re begging to ship poor work — and it’ll be all your fault.
I take it upon myself to describe the work I need as clearly as possible. Then, once shipped in testing, I allocate time to test as much as possible. I believe this to be a crucial part of striking the right balance.
3. Descriptive, helpful, and empathic bug reports
I've never built anything in my life, other than a few crappy HTML/PHP websites 20 years ago. But, I've worked with developers (and humans) long enough to know this.
If I have to write a bug report, I make sure I give as much as possible. This is essentially done in three parts:
Lots of information on the issue.
Lots of information on your journey to the issue ('I clicked here, went there, did that').
Supporting material (ideally videos).
In my experience, a poorly crafted bug report has too many negative repercussions. You'll suffer, at best, a 'quick call to discuss' (-1hr of your life). At worst, you'll have one or several devs spending hours of their time trying to understand and replicate your issue because they don't want to (or are afraid to) ask you more details.
Finally, I try to wrap the bug report into an empathy layer. This is generally good management advice, as well. This one I often struggle with. It's easy to be frustrated and blunt but, at the end of the day, I know the intention was never to ship a bug.
There is no secret to striking the right balance here. It's too dependent on your team and your product (and the factors listed above). However, as product owners, there are measures you can take to ensure you at least attempt a balance that benefits all parties.
Experiments
This week I'm excited to kick off an experiment I've been meaning to work on for years. I'm moving an important part of my website from a subdomain to a folder.
There is no shortage of evidence that doing so could increase SEO traffic.
I've set the entire experiment up this week and will kick it off in the next few days. Excited to see what happens!