Product Testing

Having worked in different types of technical support, on and off, for many years, I enjoy reading the stories at Tech Tales (  Recently, I saw a tale that has a direct bearing on our engineering practices:

Wrong Plug

Posted 04/01/2002 by Greg

I used to work for a large electronics retailer, selling TV’s and VCRs. Basically, salesmen were responsible for doing phone support for their customers (if a customer called us instead of the retailer’s toll-free help line) since we got dinged if a customer returned a product we sold them (both a loss of commission and a negative impact on our performance rating).

I had one customer issue that was a bit difficult.

For some reason, their VCR clock kept re-setting itself to blink “12:00” overnight as if it had lost power. Instead of calling me, they called Circuit City’s help line and were told to return it because it was defective.

I spotted them when they brought it back in and asked what was wrong. They told me and I asked one simple question: “where do you have it plugged in?” They said “we plugged it into the outlet in the back of the cable box.”

Sadly, the cable boxes used in that area had a fun little trick. When you shut off the cable box, it cut power to the outlet, cutting power to the VCR every time they turned off the cable box. They were smart enough to set the clock on their VCR, but this power loss kept making it blink “12:00”, which was very frustrating for them.

They’d plugged the VCR into the back of the cable box because they didn’t have a spare outlet.

I sold them a power strip for $8, had them unplug the cable box from the wall, then plug the box and VCR into the strip.

Problem solved. And how did I know to ask this question? Because a few months earlier I’d plugged my bookshelf stereo (poor man’s home theater system) into the back of the cable box and watched it reset its clock a couple of times before I figured out why it was happening.

So was my customer a dummy? I never think someone’s stupid because something that’s obvious to me isn’t obvious to them (i.e. I figured out the cable box problem without help when it happened to me, but they didn’t). If that was the case, I’d have to consider myself a complete retard every time I needed the expertise of a lawyer, doctor, plumber…

I’d say the idiot was the guy who designed the cable box.

Even the best tech support is no substitute for a well-thought-out product. If every designer/manufacturer of hardware or software had to beta test it on their parents or grandparents before releasing it, we’d have a much better world all around.

Notice the last paragraph; Greg is absolutely right!  When you design a new product, don’t ignore the testing.  Your new product will undergo two different levels of testing: Alpha and Beta.  These levels are named for the first two letters in the Greek alphabet.  The first level of testing (Alpha) is the testing that you (and your team) perform on your new product.  After the Alpha testing is complete, but before you release your beast into the wild, you will need to do some Beta testing.  This is where you get hold of outsiders and have them test your product.  You use others because you cannot anticipate every way that a customer can (ab)use your product.

Ideally, your Beta testing program will encompass several steps.  First, you will want to select a couple of other engineers, or other technically oriented people (but make sure that the testers were not involved in the design of your product – that would still be Alpha testing) and ask them to check out your product.  Give each tester one of your products and ask them to spend some time with it and to list any problems or troubles and any questions that they come up with to which they cannot find the answers in your documentation.  After the Beta-1 testing is complete, compile the lists from all the testers and correct each issue, either in the product’s design, or in your documentation.

After you have correct each issue found by your technically oriented testers, hand out a couple of products to some decidedly non-technical people.  These could be friends or relatives, non-technical students or nearly anyone else who may want to purchase your product.  Children can be really good at finding weaknesses in a product (as long as this is an age-appropriate product.)  These people will test your product in ways never imagined by technically-oriented people.  They will find ways to make your product fail that no self-respecting tech would try.  Again, have each tester write a list of troubles and questions and resolve each issue in your design or in your manual.

It is traditional to pay Beta-Testers by providing one of your final products as a thank you gift for their work.  By following this test procedure, you can correct issues like the cable box outlet which kills power to the plugged in appliance before they make a customer angry.


2 thoughts on “Product Testing

Comments are closed.