Beta Testing

_{» home » Rants}

Beta testing is a way for a (software) developer to get large scale testing for their product. For this rant I'll only deal with software beta testing because that's what I know about. Also, be warned that this is written from a software developers point of view so it may not be as balanced as it could be.

What exactly is a "beta"?

A "beta" product is one that is not yet fully tested or ready for consumers. This might mean it is lacking functionality, hasn't been fully approved, or that the vendor just wants some reassurance that his product works on a wide variety of systems.

Beta testing, to me at least, is the act of using software that the developers don't yet trust. For this reason neither should you. The 'quality' of software beta releases varies from vendor to vendor. I like to think my beta versions are pretty close to release quality but your mileage may vary.

Is Beta testing for you?

If you even ask yourself the question then it's probably not a good idea to beta test software. At the very least you should know how to use Explorer, a browser, attach files to an email, compress files and communicate effectively.

Beta testing can be frustrating. Using badly broken software is not a pleasant experience for anyone. Avoid it at all costs. However, beta testing can be incredibly valuable to both the consumer (tester) and the developer. Think long and hard whether you want potentially badly broken software on your machine.

DigiGuide, doesn't install any JUNK throughout your computer. Also, it doesn't do anything low level. Therefore it's a pretty safe bet that installing a DigiGuide beta on your system will have no negative side-effects - but there's NO guarantee.

A side benefit to using the latest beta of any product is that you get to have the latest whizz-bang features before anyone else. Which is nice.

What a customer should and shouldn't do

• Remember that you're using this software so you can provide valuable feedback to the developer. If the software crashes then SAY SO.
• Be prepared to work with the developer to get problems resolved. Under no circumstances should you ignore a problem. This is one of those cases where you are either helping to solve the problem or YOU are the problem. Feedback about issues you have is the most important thing.
• Always have the latest release of the software.
• Know how to report issues.
• Know how to ask for advice. For DigiGuide betas we use our forums, other software vendors may have email lists. Know how to seek advice when things go wrong. Don't email the developer direct unless you're on first name terms already, and don't email their support team unless they have expressly told you to.
• If a developer or support staff member asks you for some files then don't just send the files. I get lots of email every day and when someone just sends me some file I often have no clue why. In your email put a brief explanation of why you are sending them. Refer to a bug or forum post. Give some clues at least.
• Use the community to get issues reproduced. Getting the steps to reproduce a problem is often the hardest and most time consuming task. This will free more time for the developers to actually work on the product. Help them to help you.
• Don't whine and bitch about the product. Nothing turns the support or development team off more than a customer that whines and claims a product or feature is useless or broken. Try to be constructive. Just try to imagine how YOU might feel having created something only to be told it's useless.
• Do make suggestions. It doesn't matter if they seem a little bizarre. When making a suggestion please provide as much detail as possible. Also, explaining why you want the feature as well as how you'll use it are incredibly important. These two seemingly minor things help the developers incorporate your suggestions into their plans for the product.
• Show some appreciation for the hard work that goes into the product. It takes an enormous amount of effort to create great software and the developers need to know that their effort has not been wasted. During a beta it's way too easy to become focused on the negative side of things, a little positive won't hurt.

What a developer should and shouldn't do

• Provide as much information as possible. This will allow your volunteer beta testers to get on with testing much quicker than simply having the product installed on their computer. Yes, they can 'discover' the new features but an explanation of what they do will help everyone.
• Provide frequent updates - daily if possible. Fix bugs as soon as possible and get it released. Customers will feel like they are actually being listened to, that their issues are getting resolved. Too many companies fail to respond to customer queries, or are so slow that they may as well not bother.
• Build numbers should always be increased no matter how minor the change. You beta testers can then rely on the build number as the number one piece of information to use when reporting issues. You can also use it to determine whether a customer is using the latest build. Life without build numbers can be very frustrating.
• A public bug list is amazing. It allows consumers to directly contribute to the project. They can report a new bug, see when it's status changes to 'fixed' and then either confirm the bug is gone or send it back to the developer. For DigiGuide we wrote our own system that integrates with other components.
• Automate as much as possible. This will reduce the burden of releasing a beta and will give you more energy for the entire process. It's common to release a new build as a matter of urgency when something goes wrong, and this is a lot easier if it's just a "click of the mouse" to do it.
• Show some appreciation for the hard work that goes into beta testing. Good beta testers are hard to find and, just like everyone, they need to know they are doing a good job. A simple "thank you" is often enough.

When to release a beta?

Release early. It's that simple. Release your software at the earliest possible convenience. If you're operating a zero-defects strategy where you attempt to maintain a constant working version then you'll basically need to do very little.

Releasing early gives you an enormous amount of feedback. This can result in you adding more features or doing more work than you intended (Feature Creep). Try to stay focused on what you planned for the beta version but remain as flexible as possible when it comes to change requests. Small changes are a bonus during the beta cycle but they can play havoc with your time-scales.

Also, releasing early allows you to collect a lot of suggestions for future releases. Keep these suggestions and when you begin planning for the next release you can incorporate as many of these as possible - this is an easy way to listen to your customers, and your customers see the benefits of being a beta tester..

If there is one, a DigiGuide beta can be found here