In my last blog, regarding the 12 questions that separate PaaS leaders from laggards, questions 8 and 9 asked about what features developers wanted to see in PaaS products and what importance developers would attach to each of these features. The challenge was that I had a list of 20 features. Expecting a developer to reliably rank order 20 features is as likely as Larry Ellison getting married again. However, in the market research space, there is a type of discrete choice analysis named maximum-difference. Max-diff is an approach for understanding preference or importance. The reason max-diff warrants attention is that the scores generated are both linear and consistent. That means that if you have a list of features and feature A has a score of X and feature B has a score of say 2X, then feature B is twice as important as feature A. This added precision in understanding feature preference or importance makes portfolio analysis far easier, especially when comparing feature set costs with feature set preference. This precision also provides a reliable foundation for comparing the preference for one vendor relative to other vendors. For this reason, max-diff analysis is widely used to understand brand preference, customer satisfaction, feature preference, and message testing.
The reason why max-diff is able to provide this added precision is due to the rigorous methodology that is built into how the questions are asked. In the PaaS survey that ESG just completed, each developer was presented with 15 questions. In each question, they were asked to review 4 PaaS features and select the most important feature and least important feature. In each of the 15 questions (scenarios), features were randomly combined subject to each feature always being viewed 3 times. Showing each feature the same number of time eliminates the potential for sample bias, and carefully orchestrating what features are presented in each scenario ensures complete and consistent feature coverage. The net is that we can now compare developer feature preference for every leading PaaS vendor against all other vendors or a specific vendor. This will allow us to understand not only what features are most important to say Azure developers, but then also see how much more or less important these features are to Amazon developers, or Google developers, or Bluemix developers, or any one of the top 12 vendors we asked developers about.
ESG will continue to use this approach in our upcoming application development and deployment research as we look at mobile application development tools and application integration. While max-diff is a very effective approach for evaluating relative differences, when brand and price are factored into the analysis, it is possible to understand developer preference for specific products and features and how preference share changes based on changes to the product feature mix or price. For vendors who want to want to be data and market driven, take the guess work out of product planning, and optimize for revenue, profitability, or share; discrete choice analysis provides a science-based approach to product marketing.