As discussed in Practical Product Management, there are six methods of research: three are best for finding problems; three are best for evaluating solutions. An experiment is the best tool for evaluating usability—or is it? Check out www.interface-research.com. This site compares 10 different aspects of usability for a web site and asks which you prefer. At the end, they show you their results so far. Pretty clever. This form of research is called a choice model because it forces the respondent to choose. After all, you cannot choose both answers.

But here’s the question: is the survey telling us which is better or telling us which is popular? And is popular better? Britney Spears is popular. (For better, check out The Alternate Routes.) I’m not convinced that the popular choice is indeed better or if it is just what we’re used to.

If you have a new user interface or new web site design, why not use a method like this to test before you poke it out online? For that matter, you should test positioning, product names, your marketing collateral templates look-and-feel, and so on. Do you?

We at Pragmatic Marketing like to balance qualitative research from interviews with quantitative research from surveys. Every week we meet hundreds of product managers and technology executives; we hear their stories; we observe the magazines they read (such as our own Pragmatic Marketer) and what web sites they visit (such as my blog at productmarketing.com). We supplement this info with quantitative research from our annual product management survey.

Are all surveys created equal? Not according to Wharton. Read more in Polling the Polling Experts: How Accurate and Useful Are Polls These Days?

Product managers tell me that their execs don't value research. Actually I find that executives do indeed value research--it's just that they tend to value the qualitative over the quantitative. They want product managers to call face-to-face on customers while many product managers prefer to hire the work out to a third-party.

Of course, you need both. You need both stories and data. So go visit three customers and then survey a hundred. Deliver market facts to your executives in their preferred format.