Early in my career as a survey researcher, I made a mistake that I’ve since seen countless organizations make. When it was time to conduct a survey, I started thinking about the questions we should ask.
It seemed perfectly logical. After all, isn’t a survey just a collection of questions?
Clients often approached projects the same way. Some arrived at our first meeting with lists of questions they wanted included in their surveys. Others assembled teams representing different parts of the organization, each with its own ideas about what should be asked.
They weren’t being careless. They were doing what seemed perfectly reasonable. If you’re going to conduct a survey, you need questions. So why not start writing them?
The problem is that knowing what questions you’d like to ask isn’t necessarily the same as knowing what you need to learn.
And that distinction can determine whether a survey produces valuable business information or simply a collection of interesting answers.
A Lesson I Learned Early in My Career
I remember sitting in my office during one of my early projects, looking over a list of questions a client wanted to ask.
The list contained plenty of ideas, but something wasn’t coming together. I remember thinking, “What is all of this about? What are they actually trying to accomplish?”
I can’t pinpoint exactly when that realization became a turning point in my career. But over time, I came to appreciate something that fundamentally changed how I approached survey research.
Before deciding what questions to ask, everyone involved needs to understand why the survey is being conducted and what the organization needs to learn.
Until I understood that, I sometimes felt like a glorified notetaker during questionnaire-planning meetings.
I didn’t necessarily know the client’s business well enough to evaluate every question someone proposed. One manager wanted to ask about pricing. Another wanted feedback about customer service. Someone else was interested in product features.
My role often seemed to be collecting everyone’s ideas and helping turn them into a questionnaire.
But once I began concentrating on the survey’s purpose, my role changed.
I could ask why certain information was needed, how it related to the organization’s objectives, and what the team expected to do with the findings. I could help guide the discussion rather than simply record everyone’s suggestions.
I became a better researcher because I was beginning to understand that designing a successful survey starts well before questionnaire development.
When a Completed Survey Doesn’t Deliver
One experience particularly illustrates what can happen when that connection is missing.
I was working with a manufacturing company on a survey project. As was common, the planning team represented several areas of the business, and everyone brought questions they wanted included.
We worked through those requests and developed the questionnaire. The survey was conducted, the results were analyzed, and eventually I presented the final report to the group.
After I finished reviewing the findings and answering questions, someone spoke up.
“This isn’t giving me the information I need,” he said, or words to that effect.
Then someone else expressed much the same concern.
That was an uncomfortable moment.
We had completed the survey. We had collected data, analyzed the findings, and delivered the report. Yet some of the people expecting to use the information believed their needs hadn’t been met.
Looking back, I can’t say that a poorly defined purpose was the only reason those individuals were disappointed. There may have been other factors. But the experience reinforced an important lesson.
A survey can be completed successfully and still fail to accomplish what the organization needed.
The time to discover that a questionnaire won’t provide the information people need is before the survey goes into the field, not when everyone is sitting around a conference table discussing the final report.
Everyone Wants Something Different
Defining a survey’s purpose isn’t always easy.
Consider a company planning to survey its customers.
A senior executive may be thinking about market share and opportunities for growth. The sales director may be focused on sales performance. Operations may want to identify service problems and improve efficiency.
All three have legitimate interests. But they may have different expectations about what the survey should accomplish.
Simply combining everyone’s proposed questions doesn’t necessarily produce a survey that meets those expectations. It may instead create a lengthy questionnaire that attempts to address too many objectives without adequately addressing the most important ones.
The solution isn’t necessarily to choose one department’s priorities over another’s. It’s to bring those different interests together and determine what the organization needs to learn.
That means discussing the objectives before discussing individual survey questions. Which concerns are most important? Where do the information needs overlap? Can a single survey reasonably address them, or are some better handled separately?
The goal is to establish an agreed-upon set of objectives that gives the questionnaire a clear direction. Not every proposed question will make the final cut, but everyone should understand how those decisions are being made.
As my experience grew, I began approaching projects differently.
Before the first planning meeting, I would try to have a conversation with the leader commissioning the research. I wanted to understand what had prompted the project, what the organization hoped to learn, and what it expected to accomplish.
That gave me a better foundation for guiding the subsequent discussions.
It didn’t guarantee agreement. Different stakeholders still had different priorities, and occasionally those differences were difficult to reconcile.
But at least we had a starting point.
And I had learned that helping people clarify their information needs was just as important as helping them write good survey questions.
Start With a Different Question
Suppose a company president tells me:
“We haven’t surveyed our customers in several years, and we’d like to find out how we’re doing.”
That sounds like a reasonable idea.
But my first response would be:
“How you’re doing in regard to what?”
The president might be interested in several aspects of the customer relationship: product quality, customer service, responsiveness, communication, or overall satisfaction. Perhaps there are concerns about losing customers to competitors, or the organization simply wants a better understanding of its strengths and weaknesses.
All of these could be legitimate subjects for a single survey.
My question isn’t intended to narrow the survey to one issue. It’s intended to begin a conversation about what the president and others in the organization really want to understand.
We might discover that the primary objective is to assess the overall customer experience, identify areas of weakness, and determine where improvements are needed.
That’s a much clearer purpose than simply saying, “We want to know how we’re doing.”
And with that purpose established, we can begin identifying the specific information the survey needs to provide.
The important point is that a survey can address several topics while still having a clear, coherent purpose.
A Simple Test Before Writing Your Questions
Before drafting a survey, I would encourage organizational leaders and their teams to work through three questions:
1. Why are we considering this survey?
What prompted the need for information? Is there a business problem, an emerging concern, an important decision, or a need to monitor performance?
2. What do we need to learn?
What specific information is missing? What don’t we know that would help us better understand the situation?
A survey may address several related information needs, provided they contribute to the organization’s overall purpose.
3. How will we use what we learn?
Will the findings help us choose among alternatives, identify problems, establish priorities, measure performance, or determine whether further investigation is needed?
These questions won’t resolve every disagreement or guarantee a successful survey. But they can expose unclear objectives before those uncertainties become embedded in a questionnaire.
They also provide a useful test for every proposed question: Does the information this question will produce help us accomplish the survey’s purpose?
If the answer isn’t clear, the question deserves another look.
The Most Important Question Comes First
Over more than four decades in survey research, I’ve learned that many of the problems organizations encounter with surveys begin long before the first response is collected.
They begin when people move too quickly from deciding they want information to deciding what questions they should ask.
Writing good survey questions is important. So are sampling, data collection, analysis, and interpretation. Each contributes to the quality and usefulness of the findings.
But even a carefully executed survey can disappoint if the organization never established what it was trying to accomplish.
That’s why, before writing the first survey question, I would encourage every organization to answer one more fundamental question:
“What are we trying to accomplish with this survey?”
Once that answer is clear, the work of designing the survey can begin with a sense of direction and purpose.
Because the real objective isn’t simply to conduct a survey.
It’s to produce information worth acting on.