Document Actions
Measuring Success of a Software Product
How do companies measure the success of a software product after it has been installed on the customer's site? I usually use customer surveys and questionnaires, however they do not provide clear empirical data on which features have been used and how often. Whenever the CEO and VP R&D want me to justify a new requirement or feature, I can never show them how many times the customer has used my previous features.
The answer isn't always easy or even possible. How much revenue will this feature generate? How many people use that feature? Would people buy without it? Sometimes we have to guess or interview some folks.
Research told Detroit that we didn't care about the spare tire so they put a cheap, temporary tire in the trunk. While this change did not affect sales, it did affect customer satisfaction. Talk to some customers about what they like and dislike and these issues will come out.
Researchers tell me that the only way to assign revenue to a feature is to do a form of research known as "conjoint analysis" -- search for it and you'll find loads of articles that can tell you more.
Have you ever noticed the "Report usage back to us" options in Microsoft and Google products? That's how they are able to gather information on how the product is used after the sale. Microsoft ran stats on how often the various commands and options were used in Office 2003 so that they could make a better Office 2007. Interesting stuff. Years ago a client of mine added a capability during the beta test that would log every function performed for exactly this reason.
Answered by Steve Johnson
Conjoint Analysis
I have been working with Conjoint Analysis and I cannot see how a Conjoint Analysis can help after a product has been installed. Yes, Conjoint Analysis is usually used to rank a set of features and find out how important or valuable those are from the customers' point of view. But to carry out a Conjoint Analysis you have to present the customers fictive products that consist of more or less random combinations of those features. I cannot see how this could help analysing one existing product.
Those who are working in the IT industry sooner or later wish to know what features the customers are using and how often. If you have a web-based software, you could analyse web logs to see what pages or what data has been requested. If your software is using a database, you can analyse the data that is stored in case your software has different modules that have separate data repositories. Be careful to respect the customers' privacy!
If that technology is not available, you will have to continue with interviews and surveys or you will have to re-think what "success of a software product" means. My guess is that there are many definitions of success for a software product and usage numbers are maybe not the most important ones.
René