The reliability of the market research sample is often judged as inaccurate because of sampling errors causing the whole market research process to be put into question.
It is important for business decision-making to collect, record, process and analyze market research data. Thus, the research sample used in primary research should be as reliable as possible. It may not be absolutely accurate unfortunately, due to either intended or unintended mistakes, or both.
The only accurate method of market research is to ask the entire target population. But, this is not possible due to cost and time limitations.
However, it is important for users of the market research data to be certain that the market research data and information can provide valid guidance for management.
Results from a research sample should not be different from those that would have been obtained, if the entire target population had been researched.
Why can sample become inaccurate?
Sample can become inaccurate because of sampling errors.
Sampling errors occur when the market research findings from the sample are not the same as the findings from the actual population.
They are usually found when the sample is being prepared, in the results from data collection and when conclusions are drawn from the results.
2 errors when sampling
There are two kinds of potential errors related to sampling: sampling errors and non-sampling errors. Sampling errors are those mistakes that are directly related to the sample and none-sampling errors are those mistakes that are not directly related to the sample itself.
1. Sampling errors
What are they? Sampling errors are caused by mistakes made at the time of designing the sample. These can be considered as statistical errors and biases which occur most regularly.
Why they occur? They arise from the researchers’ mistakes in preparing to collect data – bad design of the final sample.
So? Sampling errors distort the data collected and used for the market research.
A. STATISTICAL ERRORS:
Statistical errors will happen within the sample itself, as well as between the sample and the population.
- Inappropriate sampling method. The wrong type of sampling method is chosen by a researcher to collect the data. This may be caused by lack of experience in conducting market research or poor knowledge of different sampling methods.
- Small sample. When the sample size is too small (<1,000 respondents), researchers will get statistically invalid results, or statistically insignificant. This will decrease the confidence about market research results. Small samples will have large sampling discrepancies – differences in results between the selected sample and the actual population.
- Unrepresentative sample. In addition to a small size, a researcher can make a mistake of choosing wrong respondents to participate in the research. Then, the sample will not represent the market that the business is planning to target. For instance, asking old men about pregnancy tests.
B. BIASES:
Bias will happen when prejudice and favoritism distorts data and information.
- Sampling bias. The researcher causes bias as he chooses the sampling method that he prefers, or is the most convenient for him to use to carry out the research. Simple random sampling method is the least likely to cause statistical errors as everyone in the population has an equal chance of being selected for the sample. Non-probability sampling methods including convenience sampling, snowballing sampling and judgmental sampling will be the most biased. Also, the researcher may inflict bias by selecting who makes up the sample such as family members, colleagues or friends.
- Survey/Questionnaire bias. The questions in the survey/questionnaire can cause bias as they may lead respondents to choose one particular answer. Those leading questions will distort what people really think. Also, any misleading questions may trigger bias as well.
- Interviewer bias. The interviewer himself causes bias when asking questions in a certain way that encourage interviewees to give a certain response. Also, the language used by the researcher may be hard to understand causing many questions to remain unanswered or be irrelevant.
- Respondents bias. The people who are filling in the survey/questionnaire and who are being interviewed cause bias when lying in their answers. This may happen either by not answering the questions at all to which they know the answer, or by not answering the questions in the truthful way.
- Other forms of bias. This includes source bias, language bias, culture bias and other biases.
2. Non-sampling errors
What are they? Non-sampling errors are caused by mistakes made by humans when carrying out market research, or arising as a result of human behavior.
Why they occur? They arise from the researchers’ mistakes in handling data (recording, processing, analyzing, etc.) and respondents’ wrongful answers.
So? Non-sampling errors distort the final results of the market research.
Examples of non-sampling errors include:
- Incorrect data recording. This usually happens when a researcher records and/or inputs data from primary market research data incorrectly. Or, when numerical analysis of the results is carried out incorrectly.
- Wrong data collection. This usually happens when a researcher gathers incorrect data for secondary market research. For example, any information that is out-of-data, incomplete or does not answer the research question.
Any of these two factors can result in the market research data being inaccurate. If this is the case and the researcher decides to move on with his or her research, then using the faulty data is most likely going to produce wrong results leading to incorrect business decisions being made.
When is sample reliable?
Ideally, the market research results should reflect the intentions of the total population. However, data and information collected cannot be 100% accurate. Sampling errors and non-sampling errors will almost certainly occur and market researchers need to be aware of them
Usually in statistics the accepted confidence level should be no less than .95 (95%). This means that the results will be accurate 95 times out of 100 times. Statistics allows for the standard accepted margin of error to be no more than .05 (5%).
Scenario 1: Reliable sample (>95%): When the confidence level is higher than .95 (95%), it means that market research findings are considered to be certain. For example, four surveys could not have been filled in completely out of every 100 surveys handed in to respondents.
Scenario 2: Unreliable sample (<95%): When the confidence level is lower than .95 (95%), it means that market research findings are considered to be inaccurate. For example, the interview question was not answered by six people out of every 100 interviews conducted.
It is possible to estimate the sample as long as we know the sample size and the research method. The important thing to remember is that results, while not fully accurate, should be as accurate as possible.
Summary of sampling errors
In summary, the larger the sample size, the more statistically reliable the answers are. Meaning that they are closer to reflect the views of the entire population. In larger samples, the change of meeting the .95 (95%) confidence level is greater.
Generally, the more effort market researchers put into properly designing and selecting a good sample, the more chances that statistical errors and biases will not exist. And, carefully interpreting the research results will reduce any other sampling errors.
However, no one can ever be 100% confident that sample results are 100% accurate.
