Surveys are not always the right research tool. For the questions you put in your survey, ask yourself what YOU wanted to learn. Are you trying to justify features that were put in the release (or show others that features were put in that no one values)? Are you trying to find out whether your UI style guide works? You might get these answers quicker by calling a few customers and interviewing them. Look for patterns across customers.

If you are trying to validate something across a market of customers to justify future investments, a survey with the right sample size to the right target market is the right tool. For your next release, not this imminent one.

At this stage of release, you will get better feedback interviewing your pilot customers (as long as they are representative of your target market). But set expectations with them that the feedback will not necessarily impact the next release. Continue working with them into the next cycle.

Then, as part of your next cycle, go visit 3 other customers at their site. You will learn more market problems than you'll have resources. Before deciding what to build for the next release, survey the rest of the customers to validate the pervasiveness of the problems and how customers would rank solving the problems, not whether they value a feature they haven't used yet. (Keep surveys short with relevant questions to increase the number of people who will respond.) The survey is a great way to learn what features to put in the product in the first place rather than learning after you've built them that few people value them.

Once you have a solution concept, validate the solution with your target market. You can use mock-ups or html in the early stages and working code later to test the usability of the product and that the solution solves the problem. Representative users are more important than a large sample size for this type of research.

Answered by Barbara Nelson