How does is this process effected if there is an intermediary between customer and manufacturer (indirect sales model).

Thanks Elise. This is a great question. I speak with many vendors who really don't know why buyers buy their products or services. Of course, this will eventually lead to misalignment with buyer needs. Even worse, obsolete solutions. Therefore, I believe that W/L Analysis must become an iterative process for the owner or manufacturer of the product. One reason is that wholesalers are responsible for training and positioning. Indeed, this adds a more complex element to the management and the deployment of win/loss reviews. However, you have an excellent opportunity to become closer with your indirect distribution channel and prospective buyers at the same time. I have seen vendors offer a 1-5% higher commission rate if the indirect channel provides their contact data for competitive opportunities during the prior month (or quarter). There is a silver lining as well. The wholesaler can conduct the win/loss analysis (without outsourcing)with a high probability of success because you are removed from the salesmanship and, as a third party, are only trying to maintain alignment with the buyer's process, perception and expectations. I hope that this helps.

You mentioned that the interviews have to be done in a competitive situation. Would you consider interviewing lost opportunities who choose to do nothing or take on the initative themselves?

Great question Rachel. There are two disparate situations here. When a potential prospect decides to do nothing they have essentially concluded that the VALUE of your solution is not worth the investment of resources (money and time)to fix their established NEED. That means that a sales organization was unable to establish value & need, which is a preliminary step in the sales process when selling solutions. As such, a "vendor solution/buyer need" qualification was not established. You see, I invested a little more time in establishing the sales process pros and cons during the presentation because of this exact scenario. Therefore, alignment can't be achieved if value is not established and the sales organization should have established this before your organization invested another dime in the selling process. Furthermore, it should not have been listed as a "loss" in your CRM because the sales qualifications should not have made it beyond the DEMO process (e.g., no "proof," site visit, contract). Likewise, there is little value in conducting win/loss analysis for Action Not Taken prospects UNLESS it is an epidemic within your salesforce (more than 10% of the opportunities do nothing). If this is true, open-ended interviews about the sales process should quantify the need for additional salesforce training. I also recommend using these types of accounts to help verify the sales process attributes and actions that you need for your win/loss template. On the other hand, prospects electing to do it themselves can still offer insights into your metrics, performance, attributes and perception of a competitive opportunity. I wouldn't bother collecting a do-it-yourselfer's metrics because they will be biased. Even though not a true competition, its OK to have a one-side response to learn from. I always enjoy learning why a prospect would take the internal RISK of going it alone. It usually is more political and ego-driven than a logical process. Note: I have found that do-it-yourselfers will become prospects again in 2-3 years when the person who recommended that direction is no longer an obstacle.

Would you advise doing this win/loss interview on phone if in-person visits are not possible ? ?

Thanks for the questions Chandni. I conduct 100% of my interviews over the phone. Most interviews last 30-45 minutes depending on the interviewee's capacity to condense thoughts. It is cost-prohibitive for the interviewer and also more time-invasive for the interviewee if conducted on site. By scheduling teleconferences, the interviewee can select more convenient times and venues, which ensures your success at securing an interview. For example, many C-Suite (CEO, COO, CIO) interviewees schedule time during their commute to/from work which is productive time for both parties. I hope that this helps.

Will this work with an indirect sales model? Are there any modifications for that type of sales model?

In a company with multiple product lines sold in different channels, do you need to customize the surveys to each product/channel?

Thanks Irvin. This is a great scenario. Your situation is the norm, rather than the exception. I believe that W/L Analysis must become an iterative process for the owner or manufacturer of the product regardless of the distribution channel. One reason is that wholesalers are responsible for training and positioning as well as product updates to match prospect needs. Indeed, this adds a more complex element to the management and the deployment of win/loss reviews. However, you have an excellent opportunity to become closer with your indirect distribution channel and prospective buyers at the same time. I have seen vendors offer a 1-5% higher commission rate if the indirect channel provides their contact data for competitive opportunities during the prior month (or quarter). There is a silver lining as well. The wholesaler can conduct the win/loss analysis (without outsourcing)with a high probability of success because you are removed from the salesmanship and, as a third party, are only trying to maintain alignment with the buyer's process, perception and expectations. As far as multiple templates, this is a little more complex. You should maintain as much consistancy in the interview vehicle as possible. You will not need to change the template Decision Process, Decision Criteria, Organizational Statistics and Sales Performance UNLESS the differing product lines are sold into uncommon market spaces (e.g., healthcare IT, "Green" cleaning solutions)or personas with differing buying cycles (e.g., 12 month vs 2 weeks). Regardless, the Company Attribute responses will always remain the same. However, the sales cycle actions and the product differentiation may have to change in the templates by buyer persona and products. These will equate to 10-20% of the template's attributes and actions, so shouldn't be a resource drain,if needed, to change or manage. Do everything you can to prevent customization of your base vehicle. I hope that this helps.

Who should do the win-loss interview? We are a small company. It could be Marketing or we hire a consultant...

Thanks for the question Larry. A company's size is relative. For example, I have seen a revenue metric for a small company as being under $100 million. However, I believe $100M in my field is huge. Either way, it matters little who conducts the interview as long as an un-biased non-sales environment is achieved. The alternatives to weigh are:

How many interviews will you conduct? If 10 per quarter, conduct them internally. If you reach 20 interviews per quarter, consider outsourcing to a firm that does not have a conflict of interest, has a good interview:interviewee ratio, knows your business model and can deliver actionable results.

Do you have the resources to commit to constant buyer interviewing? If marketing will be extracted from other non-discretionary and critical responsibilities, consider outsourcing. You will spend approximately $350-750 per interview if going that route. You need not interview all opportunities, but just enough to validate your alignment (10-50% of all competitive opportunities). It is more cost effective to internally conduct interviews if you have 50 or fewer ANNUAL opportunities. I also like the idea of housing internal knowledge to help with bettering products, services and buyer alignment.

Feel free to send me some of your organizational statistics [e.g., employees, annual sales opportunities (total salespeople, opportunities per salesperson), revenue, marketing staff] and I will be glad to fine-tune a recommnedation for you.

1. What if your main competitor is the status quo, meaning most non-purchasers stay on their current solution rather than purchasing yours? Does the methodology still work well?

2. What do you think about the use of focus groups for getting qualitative feedback from lost leads? Is that likely to be useful or too biased / non-quantitative?

Great questions Susannah. The win/loss analysis methodology still works even when the competitor is the status quo. Matter of fact, the metrics will quantify the story that your products may need to be better positioned. On one hand, you may have dominant market share, so the only option is for a prospect to do nothing. However, when "most" non-purchasers stay on their current product rather than move to yours, there may be a more basic issue: Your company may be selling products, not solutions. To be a "solution" three qualifications must be present: A Buyer Need; Urgency to change; Differentiated product. Bob Schmonsees says, "If its not mapped, its not a solution." Therefore, if "status quo" is the result from most non-purchasers, sales people are investing expensive company resources into sales cycles that have not been properly qualified (mapped buyer needs to your unique solution with an associated urgency to change.) To fix this, be sure to invest into heavy qualification steps and/or solution sales training. There are several caveats when conducting win/loss interviews with status quo competition: 1) the result sometimes really is a simple economic decision; 2)political decisions skew the integrity of the win/loss metrics; 3)many times the sales process was not completed after the proposal was recieved, which justifies an internal budget; 4)Ratings will validate that the buyer need, vendor solution, risk adversion and price willing to pay was out of alignment with the buyer. Because of these four caveats, I would recommend conducting a post-decision interview with "status quo" lost opportunities using 8-10 open-ended questions rather than the structured post-decision interview template. This way, you can discover areas requiring improvement such as: Amount of pre-sales qualification performed; Internal power of decision maker; Real cost of status quo choice; Perceived value of your product; Buyer needs that you are solving; Buyer underlying issues; Any differentiating features; Why wasn't there an "urgency" to change.

As far as focus groups, they are great venues for extracting customer-related product enhancement knowledge and for quantifying product development "resonators" when non-customer groups are used. Conversely, the use of focus groups are not an optimal venue for quantifying or qualifying evaluators who did not select your products for several reasons. The first is that most buyers conduct unique decision processes, review differing competitors and have different sales people making it hard for any controlled continuity of research. Secondarily, loss analysis collects perception and perception can be easilly influenced when hearing other group viewpoints, experiences and context. Third, its a considerable continued expense (of which the "loss" may have been the product of faulty sales cycle positioning) when a telephone call could achieve a better result. Moreover, it would be less expensive to pay each prospective attendee $200 for a phone interview. For these reasons, I find that the integrity of findings and the investment is better served when post-decision interview information is collected autonomously and in an incubator-like environment. Conversely, the end result of an on-site, lost opportunity focus group would probably be the validation of attendees that they made the right decision given a room full of like-minded individuals.

I hope that these thoughts were helpful.

sounds like you don't want sales to do win loss...if I can't set up a new department, who do you recommend? Are teams useful verse one person?thoughts?

Thanks for the question Mark. You are correct. Sales personnel are natural objection-handlers and because they must sell "what's on the truck" are ill-equipped to be unbiased. Albert Einstein said, "The significant problems that we have can not be resolved with the same level of thinking which created it." That doesn't mean that a sales department can be ineffective because it matters little who conducts the post-decision interview as long as an un-biased non-sales environment is achieved. From my experience, as reviewed on Rule #4 of 8 Rules of Successful Win/Loss Analysis, there is a 60-75% response rate whenever an un-associated vendor calls to schedule a potential interview prospect. My findings conclude that the company name is the "defense" trigger more so than the salesperson themslves. Prospects become apprehensive when fielding calls from the company that they did not select in a sales process because they feel guilt and want to avoid conflict. On the other hand, common to the Tuned In Process, a marketing department can have success (not as high) by claiming that "they have nothing to sell and only want to gain the prospect's insights into their company's performance." Marketing is especially successful at securing prospects who have become customers. Yet, as mentioned in the presentation, the ratings are skewed higher due to the attention from a vendor that they do business with. Finally, marketing personnel are an excellent option as long as the resources aren't extracted from other non-discretionary and critical responsibilities.

If the resource task is too daunting or a company can't establish a non-sales environment, then outsource the prospect interviews to a firm that does not have a conflict of interest, has a good interview:interviewee ratio, knows your business model and can deliver actionable results. To outsource, or not, also depends on how many interviews you need to conduct and the expense. If 10 per quarter, conduct them internally. If you reach 20 interviews per quarter, consider outsourcing. You will spend approximately $350-750 per interview if going that route. You need not interview all opportunities, but just enough to validate your alignment (10-50% of all competitive opportunities). It is more cost effective to internally conduct interviews if you have 50 or fewer ANNUAL opportunities.

As far as a team concept vs. a single person, I really don't have any empirical data to support either. There is no reason that multiple marketing individuals couldn't divide interview responsibilities into a smaller series of prospects to enhance their knowledge of their buyers (actually a great idea to keep marketing Tuned In). However, I do believe that repetition breeds content familiarity, so the fewer people to interview will make the interviewee experience more efficient and accurate.

I hope that this helps.

Hi - do you think internal marketing staff can make the win-loss calls and not be sales-biased like the sale staff?

Great question Rich. As mentioned in a previous response, sales personnel are natural objection-handlers and they must sell "what's on the truck," so are ill-equipped to be unbiased. Albert Einstein said, "The significant problems that we have can not be resolved with the same level of thinking which created it." That sometimes includes marketing departments. To your point, I have encountered Product Managers that have a filial connection with the products that they manage and will defend to the death any suggestion that a competitor's solution is better. These are not the individuals that you want interviewing evaluators of your solutions. Instead, look to marketers responsible for buyer messages, branding and communications because they may be less familiar with product functionality to defend. A sales or marketing department can be effective because it matters little who conducts the post-decision interview as long as an un-biased non-sales environment is achieved. The internal risk is that prospects become apprehensive when fielding calls from the company that they did not select in a sales process because they feel guilt and want to avoid conflict. On the other hand, common to the Tuned In Process, a marketing department can have success (not as high) by claiming that "they have nothing to sell and only want to gain the prospect's insights into their company's performance." Marketing is especially successful at securing prospects who have become customers. Conversely, as mentioned in the presentation, the ratings are skewed higher due to the attention from a vendor that they do business with. Finally, marketing personnel are an excellent option as long as the resources aren't extracted from other non-discretionary and critical responsibilities.

As mentioned, if the resource task is too daunting or a company can't establish an unbiased environment, then outsource the prospect interviews to a firm that does not have a conflict of interest, has a good interview:interviewee ratio, knows your business model and can deliver actionable results.

I hope that this helps.

One of the questions we ask in loss interviews is, "If you could give our CEO one suggestion, what would it be?" Why, Roger, is a path to the CEO a negative thing in your mind?

I love your loss interview question Keith. It is a great qualitative response common to "What one word best describes our company?" I don't believe that I suggested that a path to the CEO is negative at all. My only mention of a CEO during the presentation was that the CEO, alone, is ill-equipped to respond to all of the questions in the post-decision interview template. Therefore, a singular contact involved in all decision processes and activities should be used as the "traffic cop" to gain all of the responses...including the CEO's perception and comments. A CEO has excellent insight into the company attributes at a high level, but rarely is involved with detailed or evaluating steps (e.g., attend site visits, attend all product demonstrations, make reference calls). Usually these steps are the sales process responsibility of decision influencers (recommenders), not decision makers.

I also may be confused by the question because I believe that your loss interview question is referring to your company's CEO in context to the opinion of the evaluator of your products. Therefore, the path with which you are referencing would be associated with the CEO of one of your prospective customers and not yours. Either way, I highly endorse any sales and marketing effort that includes a path to the CEO.

I hope that this helps.

What is the ballpark cost to perform 50 such interviews per year?

Thanks for the question Christine. If conducted internally, you would be investing 12-15 hours of interview time, 5-6 hours of outbound scheduling, plus 3-5 days to assimilate the data per quarter. If outsourcing, it would cost you between $350-750 per interview for 50 win/loss analysis interviews. SCAI charges $500 per interview, but it includes the findings and 4 quarterly PowerPoint presentations. Refer to some of the other Q&A topics re: determining the number of interviews required. Remember that the pool of interview candidates is reduced by non-competitive opportunities (e.g., RFP response only, demo only) and by prospects unwilling to be interviewed (50-75% response rate). Non-competitive opportunities should still be interviewed using a short-form survey vehicle with 8-10 open-ended questions. Therefore, you may not have 50 win/loss interviews per year if you have 50 annual sales opportunities.

I hope that this helps.

How does a company determine how MANY win/loss interviews are necessary? Is there some formula that ties to annual revenues or number of clients, for example?

This is a major question to answer and I am glad that someone asked it. The answer, of course, depends on the number of salespersons and the number of opportunities per salesperson. It is impossible to conduct win/loss interviews with 100% of the prospects who evaluated your solutions because it is not a mandate for your prospects to participate. Therefore, you can expect a 50-75% participation rate. Furthermore, the number of competitive opportunities (not just an RFP response or demo) will decrease the pool size for the number of interviews that you conduct. This implies a simple relationship equation: The smaller the number of annual opportunities, the greater the percentage of win/loss interviews you can conduct. Specifically, lets assume that each salesperson has an average of ten opportunities per year in a complex sales firm. Lets also assume that the size of the firm's salesforce is 25, which equates to 250 opportunities per year. That would be 125 win/loss analysis per year or roughly 30 per quarter assuming a 50% participation rate. This 50% rate would be an optimal validation sample. On the other hand, a firm with 100 salespeople could not afford, or need, to conduct as many interviews. They need only validate 10-20% of their opportunities or 200 per year/50 per quarter. Why? My findings show that there is a point (or glass ceiling) of diminishing returns. As such, the further statistical validation isn't worth the incremental investment increase. Specifically, I always cap win/loss proposals at 5000 opportunities (or 500 sales reps) per year, because your findings will not change significantly beyond 125 interviews per quarter/500 per year. You do, however, want to manage the interview pool to ensure that prospects are included from all sales regions. The bottom line is that a sliding scale exists depending on number of salespersons (10-500) and opportunities per salesperson (100-5000) that will help you to achieve optimum results for your win/loss analysis.

I hope that this helps. If you would like my recommendation of win/loss interviews to achieve optimum results at your firm, feel free to contact me.

Is this about W/L or the sales process? Thus far, this isn't meeting my expectations.

Thanks for the comment Laura. I hope that you stuck with us for slides 6-24 so that I could share the W/L Analysis findings. It is real hard to present to large audiences because of their differing experiences, expectations and maturity regarding certain topics. I am always torn in a brief 45-minute encounter between too much complexity and simplicity. Therefore, I try to strike a compromise between the two. Obviously, you are a person who understands that you can only conduct win/loss analysis on the backend of a sales process and why you can't rely on sales to be responsible for the alignment with your buyer. I have been requested in earlier maturations of this presentation to help the majority of attendees to "align" the win/loss analysis with the sales process, so elected to do so this time around. In order to "align," I felt it appropriate to review the pros and cons of sales process training. In future iterations or splinters of this presentation, I am sure I will drill down into more "meatier" examples and best practices. I hope that you will participate.

Again, I hope that you stuck with us for slides 6-24 and found some value in the "8 Rules of Successful Win/Loss Analysis." Feel free to contact me with any further thoughts or questions.

How do you breakdown sales staff barriers to contacting thier lost prospects?

Thanks for the question Becka. If I understand your issue correctly, your sales staff has mental barriers with calling prospect opportunities (lost to another competitor) to collect win/loss data. If so, relieve them of that responsibility and turn the win/loss interviews over to the marketing department or another option that can provide an unbiased and open environment. In the presentation I quoted Albert Einstein, who said, "The significant problems that we have can not be resolved by the same level of thinking with which created it." Therefore, how can a person who lost an opportunity be counted on to discover the real reasons why the account was lost? If win/loss analysis is conducted by the same salesperson conducting the sales process, the only reasons for loss will be price, product or lateness to the process. Sales people are great at objection-handling, so naturally, not unbiased. As such, I recommend re-listening to the presentation and reading the article to glean some nuggets to re-allocate the responsibility. If your environment prevents adoption of my recommendations or you really want the salespersons to collect prospect perception data AND they are reluctant to do so, consider with-holding commissions until they are completed. Similarly, doctors have admitting priviledges suspended until patient charts are completed. With sales personnel, money talks. You may also consider additional sales training or improved hiring practices if the salespersons have barriers against interaction with prospects.

Hope this helps. Feel free to contact me if you need further clarification.

Do you recommend interviewing 100% of qualified opportunities? How do you compare new products if competition doesn't have a similar product?

Thanks for question Scott. As mentioned in a prior question, you can expect a 50-75% participation rate by your prospects. Furthermore, the number of competitive opportunities (not just an RFP response or demo) will decrease the pool size for the number of interviews that you conduct. When deciding on how many interviews to conduct, use a simple relationship equation: The smaller the number of annual opportunities, the greater the percentage of win/loss interviews you can conduct. Specifically, a firm with 25 salespersons who each have 10 competitive opportunities per year would have 250 opportunities per year. That would be 125 win/loss analysis per year or roughly 30 per quarter assuming a 50% participation rate (optimum). On the other hand, a firm with 100 salespeople need only validate 10-20% of their opportunities or 200 per year/50 per quarter. Why? My findings show that there is a point (or glass ceiling) of diminishing returns. As such, the further statistical validation isn't worth the incremental investment increase. I always cap win/loss proposals at 5000 opportunities (or 500 sales reps) per year, because statistical validation does not change significantly beyond 125 interviews per quarter/500 per year.

As far as your second question, there is no need to reconcile dissimilar products during a win/loss analysis because sales processes naturally encounter competition with differing solutions to buyer needs. Therefore, win/loss analysis (post-decision interviews) is validating perception that doesn't change by solution. Instead, you are quantifying the buyer's: Decision Process; Decision Criteria; Sales Performance. Quantification is achieved by collecting ratings and comments regarding your corporate attributes, your sales process actions and your functional differentiation. Therefore, if your functional differentiation is well articulated, you should win the business as well as secure high ratings (8-10) on your Functionality, Innovation, Clarity of Solution Differentiation and Vision. Dissimilarity is good for win/loss analysis because competitive dissimilarity should equate into a better solution unless one of the vendors is out of alignment by trying to solve a square buyer need with a round product.

I hope that this helps. Feel free to contact me if you need further thought on these topics.

Do you feel that a non-commissioned product manager who may have been involved in the sales process is appropriate to conduct the analysis, or still too biased?

Thanks for the question Sue. It doesn't really matter who conducts the interview as long as a non-sales, open and unbiased environment is established. That said, it seems to me that any person associated with the account from the vendor would be considered a part of the sales process and would generate a similar prospect skepticism or reticence to be honest IF the prospect did not select the vendor. I have encountered many product managers with a filial protection of their solutions to the point of a death match if people have contrary views of the solution's competitiveness. Therefore, I don't believe that it would be the most unbiased approach that you could take. On the other hand, I have encountered an open environment with product managers with NEW CLIENTS, but it still generates higher ratings due to the attention that is being given. Try to focus on marketers with branding or messaging responsibilities because they probably are not familiar with the detail functionality of your solutions. In addition, it is a great umbilical chord to the buyer's needs to keep marketers Tuned In and validated!

I hope that this helps. Contact me if you have further questions!

Could you clarify the following: are you saying the win/loss interviews should be conducted only for opportunities where the company made vendor of choice? If yes, how do you get broader market intelligence - i.e., opportunities where you're not even considered or only briefly considered?

Thanks for the question Mariela. I believe that evaluating prospects should be interviewed using metric-based templates if there was a competitive opportunity. One of the elements of being a competitive opportunity is that you were either the vendor of choice (VOC) or runner-up in the competition. If we only interviewed VOC wins, we would be missing 33-50% of the opportunities which would greatly skew our findings. You could also have placed #3 or #4 as long as you completed the sales cycle (e.g., site visit, proposal submitted, demonstration, reference calls). I use the designation of "competitive opportunity" because performance ratings can not be collected on actions that were not taken. I make the assumption that the sales cycle process steps are being managed in a sales forecasting application and can trace the last stage completed prior to elimination. It is harder to quantify why your company's products were not considered or early stage eliminations because they are usually political in nature, driven by consultant/prospect bias, blatant functionality missing or a product of poor sales-related cold-calling. Don't get me wrong...there is still value in conducting an interview with this prospect. Therefore, I would still engage the prospect in an open-ended interview to capture the real reasons that your product was eliminated. I recommend the use of an 8-10 question template that would probe into issues, such as: What need was not met; What value was precieved; What competitor was more attractive; Viability perception; Brand perception; Price performance; Functional perception.

I hope that this helps. Feel free to contact me if you need further clarification!