What you should know about Survey Research Design

Introduction

Conducting accurate and meaningful surveys is
one of the most important facets of market research in the consumer
driven 21st century.
Businesses, governments and media spend billions of dollars on finding out what people think and feel.
Accurate
research can generate vast amounts of revenue; bad or inaccurate
research can cost millions, or even bring down governments.
The
survey research design is a very valuable tool for assessing opinions
and trends. Even on a small scale, such as local government or small
businesses, judging opinion with carefully designed surveys can
dramatically change strategies.
Television chat-shows and
newspapers are usually full of facts and figures gleaned from surveys
but often no information is given as to where this information comes
from or what kind of people were asked.
A cursory examination of
these figures usually shows that the results of these surveys are often
manipulated or carefully sifted to try and reflect distort the results
to match the whims of the owners.
Businesses are often guilty of carefully selecting certain results to try and portray themselves as the answer to all needs.
When you decide to enter this minefield and design a survey, how do you avoid falling into the trap of inaccuracy and bias? How do you ensure that your survey research design reflects the views of a genuine cross-section of the population?
The
simple answer is that you cannot; even with unlimited budget, time and
resources, there is no way of achieving 100% accuracy. Opinions, on all
levels, are very fluid and can change on a daily or even hourly basis.

Despite
this, surveys are still a powerful tool and can be an extremely
powerful research tool. As long as you design your survey well and are
prepared to be self-critical, you can still obtain an accurate
representation of opinion.

Establishing the Aims of Your Research

This is the single
most important step of your survey research design and can make or break
your research; every single element of your survey must refer back to
this design or it will be fatally flawed.
If your research is too broad, you will have to ask too many questions; too narrow and you will not be researching the topic thoroughly enough.

Researching and Determining Your Sample Group

This is the next crucial step in determining your survey and depends upon many factors.
The
first is accuracy; you want to try and interview as broad a base of
people as possible. Quantity is not always the answer; if you were
researching a detergent, for example, you would want to target your
questions at those who actually use such products.
For a political or ethical survey, about which anybody can have a valid opinion, you want to try and represent a well balanced cross section of society.
It is always worth checking beforehand what quantity and breadth of response you need to provide significant results or your hard work may be in vain.
Before you start the planning, it is important that you consult somebody about the statistical
side of your survey research design. This way, you know what number and
type of responses you need to make it a valid survey and prevent
inaccurate results.

Methodology

How do you make sure that
your questionnaire reaches the target group? There are many methods of
reaching people but all have advantages and disadvantages.
For a
college or university study it is unlikely that you will have the
facilities to use internet, e-mail or phone surveying so we will
concentrate on only the likely methods you will use.

Face to Face

This
is probably the most traditional method of the survey research design.
It can be very accurate. It allows you to be selective about to whom you
ask questions and you can explain anything that they do not understand.
In addition, you can make a judgment about who you think is wasting your time or giving stupid answers.
There
are a few things to be careful of with this approach; firstly, people
can be reluctant to give up their time without some form of incentive.
Another
factor to bear in mind is that is difficult to ask personal questions
face to face without embarrassing people. It is also very time consuming
and difficult to obtain a representative sample.
Finally, if you
are going to be asking questions door-to-door, it is essential to ensure
that you have some official identification to prove who you are.

Mail

This
does not necessarily mean using the postal service; putting in the
legwork and delivering questionnaires around a campus or workplace is
another method.
This is a good way of targeting a certain section
of people and is excellent if you need to ask personal or potentially
embarrassing questions.
The problems with this method are that you
cannot be sure of how many responses you will receive until a long time
period has passed.
You must also be wary of collecting personal
data; most countries have laws about how much information you can keep
about people so it is always wise to check with somebody more
knowledgeable.

Structuring and Designing the Questionnaire

The design of your questionnaire depends very much upon the type of survey and the target audience.
If
you are asking questions face to face it is easy to explain if people
are unsure of a question. On the other hand, if your questionnaire is
going to include many personal questions then mailing methods are
preferable (but may violate local legislation).
You must keep your
questionnaire as short as possible; people will either refuse to fill
in a long questionnaire or get bored halfway through.
If you do have lots of information then it may be preferable to offer multiple-choice or rating questions to make life easier.

Cover Note

It
is also polite, especially with mailed questionnaires, to send a short
cover note explaining what you are doing and how the subject should
return the surveys to you.
You should introduce yourself; explain
why you are doing the research, what will happen with the results and
who to contact if the subject has any queries.

Types of Question

Multiple
choice questions allow many different answers, including don’t know, to
be assessed. The main strength of this type of question is that the
form is easy to fill in and the answers can be checked easily and quantitatively; this is useful for large sample groups.
Rating,
on some scale, is a tried and tested form of question structure. This
way is very useful when you are seeking to be a little more open-ended
than is possible with multiple choice questions. It is a little harder
to analyze your responses. It is important to make sure that the scale
allows extreme views.
Questions asking for opinions must be
open-ended and allow the subject to give their own response; you should
avoid entrapment and appear to be as neutral as possible during the
procedure. The major problem is that you have to devise a numerical way
of analyzing and statistically evaluating the responses which can lead
to a biased view, if care is not taken. These types of question should really be reserved for experienced researchers.
The
order in which you ask the questions can be important. Try to start off
with the most relevant questions first. Also friendly and
non-threatening questions put the interviewee at ease. Questions should
be simple and straightforward using everyday language rather than
perfect grammar.
Try and group questions about similar topics
together; this makes it a lot quicker for people to answer questions
more quickly and easily.
Some researchers advocate mixing up and
randomizing questions for accuracy but this approach tends to be more
appropriate for advanced market research. For this type of survey the
researcher is trying to disguise the nature of the research and filter
out preconceptions.
It is also a good idea to try out a test
survey; ask a small group to give genuine and honest feedback so that
you can make adjustments.
Common mistakes when doing the survey research design.

Analyzing Your Results

This is where the fun starts and it will depend upon the type of questions used.
For multiple choice questions it is a matter of counting up the answers to each question and using statistics to ‘crunch the numbers’ and test relevance.
Rating type questions require a little more work but they follow broadly the same principle.
For opinion questions you have to devise some way of judging the responses numerically.
The
next step is to devise which statistical test you are going to use and
start to enter some numbers to judge the significance of your data.