RPG Research Catalog
Taking Research Online
One of the most common questions we’ve been hearing over the last few years is, “what sorts
of research can be done online?” The answer, of course, is that any research project can be
modified to become an online study. But the question that first needs to be asked is whether
or not the research should be conducted online.
One of the biggest myths about online research is that it provides the same quality of data
for a much lower price. It is true that online studies can be cheaper due to the automation of
processes that software can provide. But the quality of the data is much harder to manage,
and it is wise to be skeptical of any marketing research provider that is pushing online
research as the preferred option. The truth is that the highest quality of data will always come
from the low-tech solution of one-on-one interviews. Online options must always
sacrifice some benefit of in-person interviews in order to provide value.
With that said, online research does present many new and exciting options that are worth considering. Some online techniques can even make use of social networking tools to provide data that is distinctly different from more conventional methodologies.
Qualitative Online Research
Perhaps the most exciting new frontiers in online research exist in the realm of qualitative research, where researchers are free to try creative (and sometimes unconventional!)
approaches to acquire data.
Earlier in this catalog, we’ve covered some of the many ways that Internet technology can be
used to enhance in-depth interviews and focus group research. But there are plenty of other options as well.
Online video diaries are one emerging technique that provide information that’s part interview, part ethnography. The research team sends each respondent an inexpensive digital
video recorder and ask the respondents to capture video footage of themselves using a
product or service or talking about it (often working through a list of questions provided by
the team). The respondents then upload their videos and the research team puts together a
video report that incorporates the comments of the respondents into a cohesive narrative.
One advantage of this style of research is that it provides an enormous amount of nonverbal and ethnographic information that is normally not captured by marketing research.
Content analysis is often used for secondary research, but it can be used for primary
research as well. The idea of content analysis is to index data from Web-based communities
(such as message boards or Twitter feeds) and use software to search for recurring words or
phrases. This can help researchers to pinpoint hot button topics and to look for connections
between behavior and attitude. Content analysis can also help locate vocal online detractors
for the purposes of customer recovery.
Social Network Panels involve recruiting a group of specialized panelist respondents who
participate in a private social network to offer their opinions on topics presented to them. To
deliver value, these panels need to be made up of members who have some unique and useful
characteristic in common (such as a panel of physicians, or a panel of young professionals).
But these panels can be very useful for obtaining quick opinions from a community of wellqualified
respondents, and the social media platform allows rich content (such as images, slide presentations, videos and audio clips) to be easily shared with members of the group.
Quantitative Online Research
Of course, the online technique most people are familiar with is the simple online survey, a technique that is becoming increasingly common due to its inexpensive nature and its relative
ease of deployment. Unfortunately, online surveys are often conducted in a “do-it-yourself ”
manner, often resulting in a low response rate and a questionable quality of data.
Our experience has taught us that online surveys must follow three rules to be effective:
They must be short. We recommend no more than 10 questions (not counting demographic questions at the end, of course).
They must be easy. We recommend no more than one or two open-ended questions. Everything else should involve selecting choices from a list or rating items
on a scale.
They must be relevant. As online surveys grow in popularity, more and more
people are being asked to take them, resulting in a higher tendency for these people
to ignore them or avoid them altogether. The one exception we have observed is
when an online survey is being conducted by an organization to which the respondent feels a strong connection or a desire to help. So long as the survey is relevant to the respondent’s interests, he or she is more likely to make an attempt to
One important limitation of online surveys is that it is extremely difficult to draw a random
sample. This means that the results of online surveys are unlikely to be representative of the population as a whole – an important limitation to keep in mind if you are conducting
As is true of any type of study, it is important that an online survey be well-designed and that
the questions be clear and concise. We recommend, at the very least, having a research
professional design the study, even if it will be administered by your own organization.
In February, 2011, we released a white paper on do-it-yourself (DIY) research. If you're considering designing your own study, we highly recommend checking it out!