Comments
Comment by Nicholas Shanks:
¶
I find that if you don’t release any software you get much fewer bug reports :-)
2177 words on Bugs
Brent Simmons has a great piece of advice on writing bug reports and feature requests. While he only claims that the techniques he describes work to coerce him into solving the problems you have with his software, I believe that they are pretty close to generally useful advice. It’d be good if everyone reporting bugs or requesting features read that text.
I’ve been reporting bugs and requesting features for years. And I’ve done so with many different developers. The bug reporting mechanisms have varied considerably: From freeform e-mails straight to the developer to filling complicated forms. And from that I’ve had many different experiences: From my report apparently being ignored to a pre-release copy of an upcoming version in my mailbox for testing within a few hours.
In what follows I’d like to reverse Brent’s perspective and make a few – mostly obvious – notes on how sending bug reports or feature requests can be a better experience for the user.
Most people will agree that getting feedback from users to developers is a good thing. Bugs will be found, possibilities for improvement will be pointed out and completely new ideas can be discovered. The downside is that dealing with all those requests can take considerable time and effort which a developer may not be able or willing to take and make.
I will assume you agree with the statement that getting feedback from users is a good thing™ in what follows. If you don’t agree with that point of view, you may as well stop reading here. Starting with that assumption, it will be good for the developer to encourage users to submit their feedback. The communication should be smooth and easy.
To me that is a key point. At the end of the day sending feedback is an effort. And if it is a big effort with many hoops to jump through for the user, sending feedback may just not be done. If the user has to go to a web site, complete a complicated registration process, find the correct button in the midst of technical lists and then enter a bug report split up in a number of text fields on a web form that will refuse accepting what he entered and lose half of the information on clicking the back button to correct that ‘error’, the user isn’t encouraged.
Compare this with a feedback approach where you find a menu item to send feedback right in an application’s Help menu. This will be perfect for the task in many situations. It can direct you right to a relevant web page for sending feedback or create an e-mail message to the correct address. [In my opinion e-mail is the better option as it doesn’t rely on the user being on-line and allows a more free form and illustrative expression with the easy inclusion of screenshots.] Even better, the application can automatically gather information like its own version number or that of the system it is running on and include them into the feedback report it prepares for the user.
And, best of all, this is rather simple to do for programmers! So there’s hardly any excuse not to do it if you’re serious about gathering feedback from your users. [And I told you to not read on if you aren’t…].
If you want or absolutely need some information from your users, let them know! Usually users have a hard time guessing which information will be essential for the developer to understand the problem. If you want crash reports or a particular form or report – well, the users won’t just guess that either, you better tell them as well. Ideally, your application can gather all the relevant information and then prepare the bug report for the user.
Many may shrug at this issue, but I think it’s important: Only request the information from the user which you absolutely need and – if possible – give the user a hint why you need that information if it is beyond the usual. Don’t request excessive information. Like the popular Apple System Profiler report, say.
While the information in the saved profiles many help, it is just too much. With many software developers being in the U.S.A, a country that gives a rat’s ass about foreigners or data protection, I am extremely reluctant to send those reports around.
What’s even more problematic is that the reports are so large that actually checking whether you’re prepared to share their content with others cannot be considered a reasonable effort. Did some application dump one of your passwords into console.log in the past days? Do you really want everybody to know about your network setup? Do you trust Apple enough to be sure they don’t dump any private information in those logs – particularly with the system profiler modules being updated from time to time? Do you participate in some pre-release software tests where sending around version numbers could mean you’re violating some NDA?
Another point that is obvious but unfortunately needs to be mentioned: Actually look at the feedback you receive. Even if it could be considered malformed and imprecise. Even with whatever little feedback we get for our software, we have received a number of requests that were downright inintelligible. But usually any report which isn’t just an insult has a some valid point to it. So even if it looks like it has been written by an analphabet imbecile, it may well be worth to ask people to elaborate their point because you didn’t get it the first time around.
And guess what? It may suddenly make sense and you can understand the issue that led to those reports once people make a greater effort to write them down.
When reading feedback, try to actually understand it. Again, this sounds obvious, but I have experienced the weirdest things in that respect… the oddest probably being that whoever read the feedback not knowing what a Mac SE is and just assuming I referred to an iMac SE.
While this point sounds so trivial, I assume that it is the hardest for software makers to stick to. Once their enterprises grow beyond the size of a few people all of whom know most things, there may be intermediaries handling the user feedback. It seems essential that those are much more competent and trained than their supposedly minor role suggests.
While this isn’t an advantage for developers right away, it’s a very good communication strategy to keep the user in the loop as far as their feedback is concerned. While improvements can be made upon that feedback without telling the person who reported it about the progress, getting that acknowledgement of their input makes users who gave feedback feel that their effort was worth it. This, in turn, may make it more likely that users who are happy with the feedback handling by the developer will give further and possibly even better feedback in the future.
Ideally I think developers should acknowledge they received the feedback right away. Frequently some system for handling the feedback is involved in this and each item in there gets a unique number which can be used in future communication. Just to make things easier for later reference, I’d suggest that the acknowledgement should include both the original feedback as given by the user and the relevant reference number. This has the advantage that the user can easily look up what he reported earlier on and have the reference number right with it.
Having those references ready, enables users to add further notes to whatever they first remarked later on. For things like problems which aren’t easy to reproduce for the developer, the user may get a better idea about what is involved in reproducing the problem reliably. Those notes will be helpful. And even more so if the original report can be easily amended.
Sounds a bit convoluted, right? Basically it gives users a good feeling to know their feedback has been seen and to have an idea on whether it will cause any changes. And, guess what! Even ‘No’ is an acceptable answer in that game. And giving it is much better than not answering at all. It will let the user know that his suggestion has been thought about and that it has been deemed too complicated to realise or otherwise not fitting in the the scheme of the tool in question. For bonus points, give a hint of why you can’t / don’t want to follow the suggestion.
Or, if the the user wanted a feature for something an application can already do just fine, just let the user know how to achieve this goal and make a mental note that your way of implementing or exhibiting that feature may not be quite as obvious as you thought it is.
Finally, don’t insult the user. Unfortunately that needs saying. But replies on feedback which have the undertone of ‘you’re just too stupid to use the application properly’ don’t cut it. And, yup, the infamous Behaves as specified
falls in that category as well.
Most of what I’ve written is completely trivial. Be polite and helpful and your users will love you.
Of course not all of the points I describe above hold in all situations. For bigger enterprises the whole area of bug tracking and handling may be very complex and I don’t expect them to deliver the same speed and attention to issues as smaller developers do. That is, I don’t expect their communication skills to match those of their smaller competitors. I don’t think that’s a good thing, but I assume it just can’t be avoided.
Similarly, there may be other situations where the feedback on applications may be handled differently due to the author’s interest or disinterest in it. In all of those cases it may help to be upfront and clear about the status of the tool you’re offering for download. People who eagerly submit feedback or bug reports deserve to know beforehand that their effort will be fruitless because you’ve given up maintaining that tool.
I’ve left examples out of the preceding sections because I realised that I’d end up using the same ones over and over again. So I’ll just gather the examples here. An example that fails to follow most of the points I made is feedback with Apple computer. Their ‘normal’ feedback process is well hidden and word in the street is that it is mostly of statistical significance to the company – to be pointed to problems that many people are seeing. I’d guess that giving detailed reports there is a waste of time. (And it may be worth thinking a while about how most of the software feedback that people give may end up at large companies and how people may be discouraged from following Brent’s advice by a complete lack of acknowledgement of their feedback in schemes like this one.)
Then there is the Apple Developer Connection bug reporter. You’ll need a (free) developer account with Apple to use it, so it can be considered to be not accessible to most people. It’s probably better than the ‘general’ bug reporter as you can see a list of the problems you have filed previously (and only those) and – in case your report isn’t marked Duplicate
right away – it may give you a very rough idea of what people at Apple think about the problem you reported. That said, the bug reporter is mostly a source of frustration.
Yet I am torn whether it is the worst bug reporting process. Because there’s still the wonderful world of open source software which comes with bugzilla or some similar web based software of gazillion form fields and that wonderful open source attitude of user hostility. Those systems may work wonderfully for developers but they are also the places where users first have to make an effort to file a bug at all and then may have some geek come up and tell them how they’re all wrong and stupid. That’s not very encouraging. But apart from people stopping to make snide remarks there’s probably not much that can be done within the open source scene, a scene where even developer friendly acts like writing documented code aren’t particularly common.
To finish on a high note, let me say that I find most small Mac developers to be rather good at dealing with feedback. Often you’ll receive a reply very quickly and chances to see your problem solved usually aren’t too bad. And while I’m at it I feel that particular kudos should go to Thorsten Lemke and Panic who consistently manage to be friendly and helpful in reaction to your feedback. And all that while having products with a non-trivial user base and feature set.
I find that if you don’t release any software you get much fewer bug reports :-)
