Problems faced while conducting survey
I get this question all the time because I do a lot of research with online teenagers. The short answer is: because surveying children under 18 is expensive and complicated. But for those interested in the slightly longer answer, keep reading. Research with minors presents some unique challenges. By law, minors cannot consent; their parents must give consent for them in this case to participate in the research project.
The consent requirement is a legacy of the belief that people under the age of 16, 17 or 18 to complicate matters the age of consent varies by state are not fully equipped to make good decisions in their best interest and that youth may be unduly susceptible to coercion.
So, all this means we must obtain parental consent to interview minors younger than 16, and because of state differences, we typically seek parental consent for all youth under the age of This means that we must speak or interact with two individuals in each household in a specific order with the parent first rather than one, as in traditional surveys of adults. Interviewing two people increases the complexity of the project, and requires more phone calls or messages to reach eligible respondents in the proper order.
This means that in a random digit dial surveyonly one in every seven households will be eligible for the survey. Also, the increasing use of the cell phones over the past decade means that in order to reliably reach a representative sample of teens and families we must conduct research on mobile phones.
While it is now possible to survey a representative sample of cell phone usersthe cell phone presents its own challenges, for youth research in particular. Landline phones are attached to households; so as researchers we can call the same landline number to reach two different people parent and child and have a reasonable expectation that both might be available to answer the phone.
In contrast, cell phones are typically attached to individuals even though they are sometimes shared within a household. Completing two interviews with two individuals where the parent is reached on their cell phone often takes more time and effort and requires additional calls back to try to reach the teen, thereby increasing the already substantial cost of telephone surveys with youth. Even in online survey administrations where respondents take the survey on a websitemaking the jump from the parent consent to youth completion of the survey can prove challenging.
All of these complications make surveying children and families very expensive. About Pew Research Center Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research.
Pew Research Center does not take policy positions. It is a subsidiary of The Pew Charitable Trusts. If you have been invited to participate in one of our surveys, please click here. Home U. Main More. Share this link:. Sign up for our election newsletter. Research Areas U. We need to confirm your email address. To complete the subscription process, please click the link in the email we just sent you.Home Consumer Insights Market Research.
Online surveys are like that as well. Twenty or thirty years ago, you had to hire a research company to do surveys. By identifying what the most common problems are with online surveysyou can most easily overcome the challenges with surveys so that your business can enjoy the benefits they provide. The Questions Are Confusing or Misleading Survey questions generally must clearly ask a specific and pointed question if you want them to yield effective results. When an individual taking a survey is confused by the question, he or she will typically not answer the question in a way that is useful to you.
After all, if the respondent believes that the question means one thing and you believe that it means another, the response will not yield any information that is reliable or trustworthy for your needs. The Questions Are Too Long Another common issue with surveys is having questions that are too lengthy or wordy.
It is easy to lose meaning when the questions are too long. More than that, the respondent may feel overwhelmed or may even lose focus when the questions are too long. The best questions are those that are short, direct, and to the point. They have a specific purpose, and they generally have short answer options that are easy to understand.
Respondents are busy and may easily become distracted. They typically want to spend a few minutes at most answering questions, so they should have short, direct questions and answer options that help them to feel as though they are progressing through the survey with speed. After all, a survey will reveal limited or meaningless information if the respondent fails to answer all of the questions.
The Questions Do Not Identify Specific Issues or Problems When your survey relates to the pros and cons of a feature or a product, you want to obtain specific information about it.
Some questions may simply ask for broad or undefined information about the problems a consumer may have or why a consumer chose one product over another one. However, the information that is provided by the respondents through the survey may not adequately define what the problem specifically is or how a consumer may prefer the company to address the issue.
The Questions Use Ambiguous Rating Systems Many surveys that companies use today ask consumers to rate their experiences on a scale of one to five or one to This may give a general indication about whether a consumer feels neutral, positive or negative about a certain area that is being questioned in the survey, but the response is rather arbitrary.
For example, a consumer that rates an area positively on a scale of one to 10 may offer an 8 when they are completely satisfied with the experience, but he or she may not give a 10 as a response because of the belief that a company can always do better. On the other hand, a customer may give a 10 when they were not completely satisfied but because they do not want store staff or management to get fired or in trouble. They may indicate a positive or negative experience, but they do not provide any useful or real information for a company to improve upon.
Surveys Do Not Provide the Customer With the Ability to Clarify Answers Many surveys ask customers questions with a multiple choice response option multiple choice questionsbut they do not provide the respondent the opportunity to clarify his or her answers. For example, it may ask a consumer which feature was most important in the buying decision, but it may not ask the consumer why that feature was important.
There may be multiple reasons why a specific feature may be important in a buying decision, and obtaining better feedback from respondents is important for companies that want to improve products or improve their marketing message. Freely Written Responses Can Be Difficult to Quantify Through Analysis While it may be important for companies to obtain freely written or open-ended responses from their target audience through surveys, it can be difficult to quantify or analyze these responses.
For example, one question may be an open-ended response option that asks the customer to state why they selected one product over another or to describe their experience.
Software programs can be used to analyze the responses by picking up commonly used words in the answers, but when you have hundreds or thousands of unique responses, it can be difficult to fully quantify them in a manner that is beneficial to the company in any real way.
This is a contradiction to the fact that customers should be provided with space to clarify their responses freely, and it generally means that businesses may need to read through each of the responses carefully to get a better idea about what the customers actually believe or think.
Overcoming These Challenges As you can see, there are multiple challenges that companies may face when using surveys, but surveys are nonetheless beneficial and important. They can guide you in facilitating marketing decisions, product development or refinement, customer service and more.
Generally, the feedback that you receive through these surveys can be used to give your customers a better overall experience, and they can be used to give your efforts focus and clarity.
However, in order to accomplish the goals that you have, you must find a way to overcome the challenges that are common with poorly planned and crafted surveys. As you research and select a survey software platformconsider not only how they help you create, distribute and analyze surveys but also how they help you make it effective.
Best practice articles, how-to guides, in-depth help, blog articles, and training are all things to look for and utilize. Though you're welcome to continue on your mobile screen, we'd suggest a desktop or notebook experience for optimal results. Publisher of DIYMarketers.
Are you making these online survey mistakes? Happy surveying!The problem starts with the fact that the only reason you do a survey is to get information on which you can rely on for your decision-making. If you cut corners or put forth less than a full effort then questions will always remain. Can we rely on the data?
Should we use the survey data or go with our expert opinion? We should have asked. That brings you right back to the point that led you to consider a survey in the first place.
At the start of any survey you have a question or hypothesis about which you want information. The survey data and subsequent analysis helps you make a solid decision driving specific actions. This is where we will focus first. Survey software has come a long way, but there is much more required to produce usable information from a survey. Also, remember no survey is straightforward. When you run across one of these red flags take a step back before you waste your time and money on data that is likely to create more questions than it answers.
Your information will never be shared.All Rights Reserved. The material on this site can not be reproduced, distributed, transmitted, cached or otherwise used, except with prior written permission of Multiply. Hottest Questions. Previously Viewed.
Unanswered Questions. Sales and Customer Service. Marketing Advertising and Sales. What are the problems involved in conducting a survey? Wiki User There are many factors involved in the development and management of any properly conducted research project.
Different needs require different 'types' of research methodologies. First ask yourself; "What is the specific end use for the response data developed from the research? A: Does the purpose require quantitative data? Or qualitative data? In most cases, a "survey" will produce quantitative data. B: Is there a need for statistical validation of the research?
This means that the poll was based on a mathematical probabilities constant, with a sufficient number sample of randomly selected individuals polled to assure that the data developed is truly indicative of the specific beliefs or habits of the "universe".
A: Target The population to be queried. B: Sample The "sample" is the group of potential respondents to be queried. It is a sub-group of the C: Universe D: Bias Systematic error introduced into sampling or testing by selecting or encouraging one outcome or answer over others. E: Random That is more than enough for this answer, but if you want more, just ask. Related Questions Asked in Statistics When conducting a survey to see how people feel about certain products you are?
Asked in Area What is the procedure in conducting a linear survey on site? Asked in Surveys When conducting a safety survey you should not? Stand in a lion's cage in the zoo! A sample survey may be preferable than a census because it can be more comprehensive. While its research only involves a subset, it is typically more accurate. Asked in Statistics, Probability, Surveys What can you learn by conducting a survey?
If the survey is well designed then you can estimate some characteristics of the whole population relatively quickly and cheaply. Asked in Salary and Pay Rates What are some factors to consider when conducting a wage and salary survey? Some factors to consider when conducting a wage and salary survey are questions related to job performance.
A person could ask about punctuality, merit, and overall job performance. Asked in Geography, Elevations, Surveys Cost breakup of conducting a survey?
Cost of conducting a Survey depends upon : 1 Are you designing you survey of your own OR hiring a professional for that.Qualitative research can capture the meaningful experiences and life values of persons with dementia not reported in quantitative studies. This disease affects approximately 5. Little is known about the personal impact of the cognitive losses and how persons cope with their own cognitive losses.
Consequently, there is a growing interest in conducting qualitative studies that capture the personal perspective of AD Downs, Thus the purpose of this article is to address some of these challenges and to share some lessons learned from conducting a qualitative study that explored how persons with AD use spirituality as a coping mechanism. Specifically, it will discuss 1 obtaining informed consent, 2 effectively communicating with persons who have AD, and 3 ensuring credibility and dependability of data.
Although limited in number, qualitative research of lived experiences in persons with AD have revealed significant findings. Second, they can determine what is important to their quality of life Katsuno, ; Matano, This information is critical to researchers and clinicians who develop and test interventions for persons with AD to reduce stress, fears and anxieties and to improve their mental well-being and quality of life.
It is important for persons with AD to be included in research studies. Study participants shared that their contributions as research participants gave them a sense of purpose and usefulness Katsuno, ; Matano, Persons with AD are able to inform researchers of their feelings, desires, and preferences; albeit the AD affects how well they communicate their thoughts and feelings Bourgeois, Persons with AD are a vulnerable population because of their compromised decision-making ability.
Thus, safeguarding their rights is a key issue in conducting research. An important concern is determining their capacity to give consent or assent. The National Bioethics Advisory Commission recognizes this inconsistency and requires research protocols to include a method of determining capacity of the potential participants. In addition, the protocols must include a plan to reassess cognitive capacity if there appears to be a significant change of cognitive function.
If a potential participant lacks capacity to consent to a research study, then the researcher must seek consent from a surrogate or proxy decision-maker, usually a family caregiver, and seek assent from the participant. Since elderly family caregivers can also have some degree of cognitive impairment, the researcher must plan a cognitive capacity determination method for the proxy signing the consent.
Qualitative research requires effective communication to optimize participant responses. First, a researcher needs a caring and respectful attitude toward those with AD. Being an attentive listener conveys this attitude to participants Proctor, It is important to make eye contact, use a calm voice, and refrain from talking down to the participant, contradicting their statements, or quizzing about details Bourgeois, Second, researchers need to understand the communication challenges for those with AD.
Mild difficulties in word finding, abstract reasoning, and following complex conversations occur in the early stages of AD Warchol, Individuals also experience fluctuating awareness and attention and concentration lapses in the early stages Downs, In addition, it takes longer for individuals to register verbal stimuli such as interview questions.
As AD progresses, there are increased deficits of word finding, memory deficits and difficulty staying on a conversation topic Bourgeois. Researchers should be knowledgeable of communication challenges and need to develop strategies to address these concerns with the caveat that the same strategy may not work in all situations see Table 1.
One strategy is to conduct interviews in a place that is familiar and comfortable to the participant to reduce anxiety related to unfamiliar surroundings and to control some distractions.
In addition, the researcher must allow ample time for the participant to respond to a question or remark. If they are struggling to find a word, the researcher can reassure them Bourgeois, Providing cues such as photos or using reminiscence to trigger memories may encourage the participants to share information Robinson, A useful strategy to use when participants dwell on a topic, is to first validate the meaningfulness of their experience, and then gently redirect them to a different topic Davis, Other authors discuss similar techniques of communicating with people with dementia.
An important issue to consider with qualitative AD research is how to ensure credibility and dependability of the data. Samples sizes from published qualitative studies in the literature range from 7to 28 participants. In addition, multiple interviews may be necessary to obtain enough rich data, especially if the participants are in the more advanced stages of AD.
For example, one researcher stated she needed two to five sessions for her sample of people with mild to moderate AD Snyder, The third guideline directs attention toward the analysis of the data.Designing a research project takes time, skill and knowledge. With Qualtrics survey softwarewe make the survey creation process easier, but still you may feel overwhelmed with the scope of your research project.
Population specification errors occur when the researcher does not understand who they should survey. This can be tricky because there are multiple people who might consume the product, but only one who purchases it, or they may miss a segment looking to purchase in the future. Example: Packaged goods manufacturers often conduct surveys of housewives, because they are easier to contact, and it is assumed they decide what is to be purchased and also do the actual purchasing.
In this situation there often is population specification error. The husband may purchase a significant share of the packaged goods, and have significant direct and indirect influence over what is bought.
For this reason, excluding husbands from samples may yield results targeted to the wrong audience. How to avoid this: Understand who purchases your product and why they buy it.
Survey sampling and sample frame errors occur when the wrong subpopulation is used to select a sample, or because of variation in the number or representativeness of the sample that responds, but the resulting sample is not representative of the population concern.
Unfortunately, some element of sampling error is unavoidable, but sometimes, it can be predicted. For instance, in the presidential election between Roosevelt and Landon, the sample frame was from car registrations and telephone directories. The researchers failed to realize that the majority of people that owned cars and telephones were Republicans, and wrongly predicted a Republican victory.
Example: Suppose that we collected a random sample of people from the general U. This sample would not be representative of the general adult population and would influence the data. The entertainment preferences of females would hold more weight, preventing accurate extrapolation to the US general adult population.
Sampling error is affected by the homogeneity of the population being studied and sampled from and by the size of the sample.
How to avoid this: While this cannot be completely avoided, you should have multiple people reviewing your sample to account for an accurate representation of your target population. You can also increase the size of your sample so you get more survey participants. Selection error is the sampling error for a sample selected by a non-probability method. When respondents choose to self-participate in a study and only those interested respond, you can end up with selection error because there may already be an inherent bias.
Example: Interviewers conducting a mall intercept study have a natural tendency to select those respondents who are the most accessible and agreeable whenever there is latitude to do so. Such samples often comprise friends and associates who bear some degree of resemblance in characteristics to those of the desired population.
How to avoid this: Selection error can be controlled by going extra lengths to get participation. A typical survey process includes initiating pre-survey contact requesting cooperation, actual surveying, and post-survey follow-up. If a response is not received, a second survey request follows, and perhaps interviews using alternate modes such as telephone or person-to-person. This may occur because either the potential respondent was not contacted or they refused to respond.
The key factor is the absence of data rather than inaccurate data. Example: In telephone surveys, some respondents are inaccessible because they are not at home for the initial call or call-backs. Others have moved or are away from home for the period of the survey. Not-at-home respondents are typically younger with no small children, and have a much higher proportion of working wives than households with someone at home.
People who have moved or are away for the survey period have a higher geographic mobility than the average of the population. Thus, most surveys can anticipate errors from non-contact of respondents. Online surveys seek to avoid this error through e-mail distribution, thus eliminating not-at-home respondents. You can also use different channels to reach your audience like in person, web surveys, or SMS.Reflecting back on my legal studies, I often equate survey question development to direct examination and cross examination of witnesses during a trial.
Questions in a courtroom cannot be overly prejudicial to either side, so as to force an answer from a witness or prejudice the objective jury. In the courtroom, the ultimate goal in asking questions is to get to the facts and allow the jury to use those facts to present a course of action. Thanks to advancements in technology, just about anyone can design and issue a survey. Too many surveys start with a series of demographic questions name, title, address, phone number, email, etc.
Besides maintaining a well-organized contact list, look for survey tools that allow for demographic information to be populated. This helps to reduce respondent annoyance and improve your response rate. When you are asking a number of questions based on a similar rating scale, it is easy to overlook rating level inconsistencies.
Also, be sure the scales are reflected the same way i. Do not assume respondents know more than they do about your organization.
Ethical Issues for Online Surveys
Your survey should leave no room for ambiguity or incorrectly rely on presumed prior knowledge from respondents. An example of this would be utilizing acronyms or industry jargon that may not be readily known by all your customers, or could be easily confused, leading to inconsistencies in the data. What are your thoughts on the first class product? This is often unintentional and is a common mistake when a survey is designed by someone who is too closely associated with the project.
Assume that you ask ten open-ended questions and receive responses. On average, each respondent writes two sentences per question, resulting in two thousand sentences to be reviewed following the survey. An average written document contains 16 sentences per page, which means you will have to review pages worth of open-ended feedback.
Now think about the respondents —many respondents will not write out responses to more than just a few open-ended questions. Keeping your open-ended questions to three or four is generally sufficient. So, choose wisely and use open-ended questions thoughtfully.
The Problem with Surveys and How to Solve It
Time is money; ask only what you need to know today. Surveys that are too long, too complex, and too confusing will frustrate respondents, resulting in skipped questions or drop-offs. Limit your questions to subjects that you can act upon within the next six months. Demonstrate that you respect the time and effort respondents spend answering your survey. Save more strategic-oriented questions for phone outreach or Customer Advisory Board meetings with your highest priority customers.
Remember that just because technology allows you to create a survey, the ease with which surveys can be issued should not undermine the importance and quality of your survey questions. Satrix Solutions can help you to develop a survey that produces quality, actionable results by avoiding these common mistakes. Contact us today. Top Ten Common Problems in Designing Effective Survey Questions Reflecting back on my legal studies, I often equate survey question development to direct examination and cross examination of witnesses during a trial.How to Create a Likert Scale/Questionnaire in Word
Overcoming Survey Design Pitfalls Thanks to advancements in technology, just about anyone can design and issue a survey.