ACBC can be the solution for your complex product design

iPhone

If you are developing new complex products, you should be thinking that conjoint analysis is a useful tool to evaluate your target market preferences. It is also well known that Choice Based Conjoint (CBC) analysis is very popular between the other conjoint techniques because it simulates quite well the real decision taking process.

However, when we are talking about complex products with many attributes (like computers, homes, cars or consumer electronics), choosing between options becomes a difficult task, especially when the process is repeated several times as it is done with the CBC interview.

The decision taking process of complex products includes non-compensatory steps where some options are rejected taking into account attribute levels that should be or not present in the choices. Similarly, some options are accepted only if they have certain attribute level. For example, for one consumer a car must have high acceleration and shouldn’t expend too much gas. Therefore, all the options shown to this consumer within an interview should have high acceleration and low gas consumption and change the levels for the other attributes.

The Adaptive Choice Based Conjoint (ACBC) Analysis takes into account the non-compensatory process to filter the options before applying the compensatory selection of traditional CBCs. It also centers the options to be evaluated around the interviewee’s ideal product, eliminating the time spend with options very different and therefore no relevant.

Next, I describe briefly the steps of the ACBC.

The first step of the ACBC allows the interviewee to tell the most important levels per attributes taking into account the incremental cost that higher levels add to the total value.

The second step of the ACBC identify the must have and un-acceptable attribute levels. To do this, choices close to the ideal product from the step one are generated and the participant is asked if each choice is acceptable or un-acceptable. From these answers, the must have and un-acceptable attribute levels are identified. After few screens, the participants are required to confirm the “must have” and un-acceptable attribute levels. These levels are used to create the product options that the participant will have to compare.

The third step present the choices in a tournament style comparison. That means that three options are shown in the first question and the winner is shown in the second question with other two different options. The process continues until a final winner is found.

The fourth step is optional and requires the participant to evaluate few models in a rating scale. The ideal product, the winner from the choice tournament and few more rejected and accepted concepts are shown. This information is used to estimate the “none” threshold and utilities of the concepts for further analysis.

Evidence of success: the following studies show examples were ACBC and CBC has been used.

Electronic Consumer Products[1]. The study included product concepts with 8 attributes with 2 – 5 levels. 400 personal computer users completed CBC and ACBC surveys. Although the ACBC survey required more time to complete, its preferences share predictions of a product A (33.0%±4.8%) were closer to actual market data (34.6%) than those derived from CBC (43.5%±5.0%).

Laptop Computers[2]. This study included options with 10 attributes. There were 277 interviews for the CBC and 282 for the ACBC. To evaluate the results of both methods, there were three holdout tasks per participant with four alternatives each and a fourth holdout task that included the winner of the first three. The ACBC survey had significant better hit rates (60.8%) than the CBC (50.0%) predicting the fourth holdout task.

Homes[3]. Two ACBC surveys with 24 and 32 concepts were compared with a CBC survey with 18 choice tasks of 4 concepts per task. The two ACBC surveys had 299 and 303 interviewees and the CBC had 314 interviewees. Four holdout tasks with four choices were assigned to the respondents. A final holdout task showed the first four choices. The two ACBC experiments got higher hit rate (40.5 and 48.8) than the CBC experiment (36.9) predicting the fifth holdout task. When the share of preference were summarized for the first four hold out tasks of all the respondents, the mean absolute error (MAE) from the share of preferences calculated with the ACBC methods was smaller (3.47 and 3.15) than the MAE calculated with the CBC method (5.47).

Summarizing, the ACBC is preferred when:

1)     The product concepts have 6 or more attributes

2)     The sample size is small (thanks to the additional information acquired during the interview)

3)     Individual level conclusion are required

The drawbacks of the methodology are:

1)     Longer time to complete the survey

2)     Requires computers to administer

3)     More complex to analyze and program

Sawtooth Software provides an useful software to execute ACBC studies and more information about the topic.

References

[1] Chapman, Alford JL, Johnson C, et al. CBC vs. ACBC: comparing results with real product selection [Sawtooth Software research paper series] Sequim, WA. Sawtooth Software, Inc. 2009

[2] Johnson RM, Orme BK. A new approach to adaptive CBC [Sawtooth Software research paper series]. Sequim (WA): Sawtooth Software, Inc., 2007

[3] Orme BK, Johnson RM. Testing adaptive CBC: shorter questionnaires and BYO vs. ‘most likelies’ [Sawtooth Software research paper series]. Sequim (WA): Sawtooth Software, Inc., 2008

Choice Based Conjoint Analysis as tool for New Product Development

choice

Do you want to increase your sources of revenue through new products? The conjoint analysis allows you to identify target market preferences about the products that you are developing. It has the following benefits:
The conjoint analysis simulates the way that the consumer makes decisions in the real life. The test presents all the relevant attributes together to the respondent.
The conjoint analysis estimates the interviewee’s evaluation of the attributes without asking directly each feature importance. This is especially relevant regarding the price because the interviewees will generally reduce the price that they are willing to pay if they are asked directly. Besides, respondents tend to evaluate high all the attributes. If you ask, what is important to you about a product, they are likely to say that everything is important, but the reality is that the products can not have all the benefits at maximum level. Some attributes can have a higher level, but other will have a lower level. A higher quality will be more expensive (lower benefit from price) for example.
Furthermore, the conjoint analysis can uncover hidden drivers that may not be conscious to the respondent themselves. The interviewees just make choices about products. Many times, these choices don’t have a detailed analysis of the information but are a general evaluation of the whole product. However, the analysis of the data will show which attribute is more important.
The choice based conjoint (CBC) is a particular type of conjoint analysis that has been popular for some time. It presents several options of the complete product prototype to the interviewee from which he has to select one or none. This process of evaluation is highly preferred to others because it is very similar to the way that buyers behave in the marketplace and it is a simple task that everyone can understand.
One advantage of the CBC analysis is that it allows considering interactions. They are especially important in pricing studies. For example, you may want to consider the effect of women responses in the estimated price of a product. Another example of interactions is the preference for a smaller size if the price increases.
To execute the CBC, the respondent will be asked to indicate their preferences for different bundles of attributes. For example, a prepared chicken food for sales in the supermarket has the following attributes: weight, style, package, and price. The weight levels of the test options can be 1 Kg, 750 gr, 500 gr, 250 gr. The style levels would be Honey Garlic, Dijon Mustard and Herbs and Teriyaki. There are 2 package designs and the following possible prices: $6, $8, $10, $12, $14, $16 and $18. One choice example would be 750 gr, Honey Garlic, package design 1 at $10. The result of the analysis will tell you that the interviewees prefer a product with 500 gr, Dijon Mustard, package design 2 at $12, for example. This choice will receive an evaluation called utility. The other choices also will have their evaluations, so you can rank the choices. The analysis also will tell you which attribute and level for each attribute are more important for the sample in the utility-scale.
Some CBC Softwares estimate the utilities using the conditional logistic regression. An example of how the logistic regression works can be found in the blog “Analytical model for Lead Qualification”.  The conditional logistic regression is an extension of the logistic regression and it is used because each choice generates several rows of data that should be grouped for the analysis.
The CBC analysis requires experimental design. The number of conditions for the design is equal to the number of levels for attribute 1 x number of levels for attribute 2 x number of levels for attribute 3… The total number of conditions implies too many questions for the respondents. Therefore, some conditions are selected for the test through the Fractional Factorial Design. A conditions’ selection that form an Orthogonal Array meets the experimental design goals. Most of the conjoint programs include the functions needed for the experimental design.
The aggregate utility for each choice let you calculate the share of preference. This indicator is very useful because it allows you to modify the level of the attributes and estimate the share of preference based on the data that you already have collected. Therefore, you can select an attributes’ combination that maximizes the share of preference. Following the example of the chicken package, you have that the preferred product (500 gr, Dijon Mustard, package design 2 at $12) has the higher share of preference. The simulation based on the collected data allows you to increase the weight to maximize the share of preference. The increase of the attributes has restrictions, like the price, which will also increase. Therefore, from a certain level, an increase of the weight will not increase the share of preference of the product.
The choice task of the respondents requires a bit of adaptation and too many questions can tire him. It is recommended to do warm up choices and limit the choices between 10 and 20.
The number of attributes should also be limited to a maximum of 6 in a full profile concept. If there are too many attributes, most probably the respondents will ignore unimportant attributes.
The model for the utility calculation can be validated through the method of Hit Rate. This method calculates the percentage of correct individual predictions of the model compared with the sample size.
As I said, CBC is one type of conjoint analysis. There are also Rating based Conjoint and Hybrid techniques. These basic categories include subdivisions techniques like Adaptive Conjoint Analysis, Hierarchical Bayesian (HB) CBC or HB Rating based Conjoint Analysis. These techniques have the following advantages over CBC Analysis: allow estimating preferences at an individual level, can handle more attributes, and are able to work with smaller samples. The use of them is recommended instead of CBC if the above requirements are necessary.
Summarizing, CBC analysis is a good way of testing new products because it presents a natural way of evaluation. It is especially useful when there are few attributes, there is a high interaction probability between the variables and the “None of the choices” option should be included. Besides, the CBC, as a particular type of conjoint analysis, allows you to evaluate the characteristics of the product simultaneously and can reveal hidden drivers of the respondent.

Overcoming Lack of Coverage & Non-Response Bias

coverage-nonresponse

Coverage and Non-response Bias are issues that you will find choosing the sampling and the data collection methods for a marketing research survey. When you decide to research a target population and select the methods to collect the data, you will find that the different methods face the problem of lack of coverage. There are several examples: telephone interviews don’t cover cell phone only households; the White Pages directory doesn’t include all the residents that moved recently; the access to the internet is lower for elders and less affluent people; many apartment buildings would not allow personal interviewers.
The problem of lack of coverage becomes relevant when the segments of the population that are not covered are within your target market and are different from the segments that are covered by the research method that you have selected. For example, use an online survey to measure the use of voice over IP (VoIP) in Canada. The population’s segment that has access to the internet may have higher use of VoIP than the segment without access because the VoIP use the Internet as principal media. Therefore, the survey will not show the correct level of use of VoIP in the total target market.
Even when the different methods have coverage problems, we can find methods that have enough coverage across all the target segments. For instance, use online surveys to get the opinion about a website that has music videos for university students and young workers in the Metropolitan areas. The use of Internet in both segments is enough to be sure that a sample drawn for an online survey will cover all the target market.
The issue of Non-Response Bias becomes relevant for online surveys, mail surveys and recently also for phone interviews. A well-designed research can reduce the Non-Response. However, the percentage of non-response may be higher than 50%. When facing the problem of Non-response, again it is important to find out if the Non-respondent differs significantly from the respondent. This difference can happen if there is a reason for the Non-response related to the study variable. For example, asking about the speed at the highway. The people who go over the speed limit may choose not to answer with higher frequency than people who drive below the speed limit. Therefore, you can underestimate the average speed.
To assess the Non-response Bias severity, you may need to compare a low-response result with a higher response result. If there is a high similarity between the results, then, the Non-response is not a serious issue.
Some authors say that response rate will have low impact in mass market products like banking, insurance, telecoms and consumer durable products. This argument has support in the low estimated difference between respondents and Non-respondents.
Summarizing, the research design should be done in such a way that the final sample resembles the population being studied achieving enough coverage of all segments and evaluating the influence of non-respondents.

Analytical model for Lead Qualification

pipeline

Once you have designed and finished your product. Would you like to know how probable a given prospect will buy your product before investing sales efforts in him? Would you value knowing an estimate of the people with a high probability of buying your product in a given segment?
The Logistic Regression allows you to classify a person or entity into groups based on their characteristics. For many marketing problems, these groups are buyers and no buyers. In the case of the sales department, they usually have a group of leads that may have shown some interest in the product. However, they may want to contact only those with the highest probability of buying given the characteristics of the company. On the other hand, the marketing department may want to evaluate the potential buyers’ proportion for different segments to assign marketing resources.
The Logistic Regression also allows you to know the relative weight of the characteristics taken into account when you classify the prospects. Let’s think about the following example: ready to eat pork that only needs to be warmed up. One variable that will determine the buying decision is the consumer’s occupation. Those consumers more busy with work will look for products that they can cook faster. Members of certain religions won’t buy the product. What about people with fitness awareness. Will they buy the product? May be the pork has more fat than other meats, but they still value the flavor. And what about people with health conditions. In this case, the fat may have more influence in the decision. With the logistic regression, we intend to give a weight to these and other variables, add them to the evaluation and get a conclusion about the probability of buying the product.
The formula for the model is the following:

logistic-regression

In this equation:
p = probability of buying the product (dependent variable)
xi = Independent variable (occupation, religion, fitness awareness, health condition, … )
β0 = constant
βi = parameter that indicates the variable xi’s importance
p > 0.5 indicates that the person will buy the product. The closer that p is to 1, the higher is the person’s probability of buying the product.
To find the parameters, it would be necessary information from a sample of prospects and a statistical package (SPSS, SAS, R). You can test the variables’ relevance and evaluate the goodness of the model with the sample. You can measure the prediction’s goodness with the percentage of correct predictions in relation to the total sample. You also can evaluate it comparing predictions and results of future buying behavior of prospects.
The advantages of the logistic regression compared with other classification techniques are:
1. There is no need of linear relationship between the probability and the variables.
2. The dependent variable doesn’t need to be normally distributed.
3. The independent variables can be intervals, ranking, nominal (names).
4. The independent variables don’t need to be normally distributed.
5. The group sizes of buyers and no buyers (dependent variable) can be different.
6. The means and variance of the dependent variable can be different.
7. The error doesn’t need to be normally distributed.
The use of fewer assumptions allows predicting a bigger number of situations.
The purpose of this article is to discuss the model’s application to the prospects’ qualification. However, the data structure is similar to problems in other environments. I performed a couple of tests to evaluate the model. The first test was about the service’s evaluation in a library. The dependent variable is the level of satisfaction (very satisfied or not very satisfied) and the independent variables are the evaluations given to different aspects of the library. I took answers with probability over 0.5 for the very satisfied group and below 0.5 for the other group. I obtained the predicted probabilities for the sample once that I estimated the parameters and I found that in 71% of 390 cases the model gave the correct prediction.
In the same problem, I applied the model to 390 different cases from the same interviews (these cases were not used to estimate the parameters of the model) and I found that 72% of the predictions were correct.
The second test is the admission in graduate school. In this model, if the probability is higher than 0.5, the candidatewill be admitted. The variables are GRE (Graduate Record Exam scores), GPA (grade point average) and prestige of the undergraduate school.
After estimating the parameter βs, I run the model over the sample and I found that in 71% of the 400 cases the model gave the correct prediction.
If we modify the cutoff probability and we make it 0.7 and over for the “success” or “admitted” and less than 0.3 for the “failure” or not admitted, the percentage of correct predictions increase. For this test, the percentage increased to 79%.
Is it interesting to you a tool that allows you to identify prospects with a higher probability of buying taking into account the multiple factors related to them? If your answer is yes, you will need prospect’s data and few hours of analysis. Contact Us.

5 tips to improve questionnaires

– Identify the profile of the respondents: the questionnaire must be written keeping in mind the characteristics of the population to be interviewed. Factors like the type of employment, the level of education, revenue, the city of residence.

– Avoid double barrel questions. The question “Do you think that the speaker was fun and resourceful?” is asking for two questions in one and the answer may refer to “fun”, to “resourceful” or both.

– Give preference to structured questions. In structured questions, the answer has specific response choice and format like “yes” or “no”. These questions are easier to answer, code and analyze and therefore reduce the cost of the project.

– Don’t ask questions that require lot effort from the respondent: questions that require complex calculations or lot of memory. Example: how much did you expend in the grocery in the last year?

– Select the type of question and scale based on the desired measure and analysis. For example: if you want to know the wiliness of the population to become members of a gym in the next three months, you should use a semantic differential question and scale:

Please indicate on the following scale how likely are you to become a member of a gym in the next three months:
Very unlikely    1     2     3     4     5     6     7     8     9     10     Very likely

Please feel free to contact us if you need further help – Contact Us