“How satisfied are you with delivery speed and the employee’s attitude?” The question looks concise, but a customer may appreciate the employee while being unhappy about a late delivery. With only one rating available, respondents must decide which part to answer. When the data are aggregated, the researcher cannot tell what that rating represents.
This is the problem of combining multiple issues in one survey item. Pew Research Center’s guidance on questionnaire design notes that asking respondents to evaluate several concepts at once can make answers difficult to interpret. The principle to preserve is that each question should produce an answer with a sufficiently clear meaning to be useful.
Separate aspects that can receive different evaluations
The example can become two questions: “How satisfied are you with the delivery time for your most recent order?” and “How satisfied are you with the attitude of the employee who delivered that order?” Both refer to the same order, but each focuses on one aspect.
This separation helps the business distinguish slow delivery from poor conduct. The problems may require different responses. A single combined rating gives managers little help in deciding whether to adjust delivery schedules, improve warehouse coordination, or coach employees.
Look beyond the word “and”
A statement such as “The app is easy to use, reliable, suited to my needs” still combines several evaluations without using “and.” A user may find the app easy to navigate but experience frequent errors. Another may trust it while having little need for the functions it provides.
Conversely, a sentence containing “and” is not necessarily defective. The useful test is whether a respondent could reasonably give different answers to its separate parts. If so, the requirement to provide one combined rating deserves scrutiny.
Specify the experience and time period
Splitting an item is insufficient if respondents do not know which experience they should evaluate. “Our service is good” is broad for someone who has used several services or had different experiences across purchases. Specify whether the question concerns the latest order, a particular support interaction, or experiences over a defined period.
The period should suit respondents’ ability to recall and the research objective. Asking about the latest purchase can clarify an event; asking about the overall experience serves a different purpose. Switching between these approaches and treating the results as the same indicator would be inappropriate.
Let people without experience answer accurately
A person who has never contacted support has no basis for rating support quality. Where appropriate, use a screening question or options such as “Have not used” and “Not enough information to evaluate.” Do not force respondents to choose the midpoint simply to proceed.
A neutral evaluation and a lack of experience mean different things. Combining them in one score makes it difficult to distinguish a middle-ground judgment from missing information. Response options should match both the question and respondents’ actual circumstances.
Test understanding before launching the survey
Before the main survey, ask some people from the intended respondent group to explain each question in their own words. Follow up with questions such as “What situation were you thinking of when choosing that answer?” or “Was there a part of the question you wanted to rate differently?” This checks interpretation; it does not establish that the measurement scale has been fully validated.
For a published scale, do not split or rewrite an item merely because it is long. Review the construct definition, original instructions, translation, and permissions, then examine the adapted version through an appropriate process. Readability is necessary, but it does not establish measurement validity by itself.
A concise questionnaire removes questions that are unnecessary while keeping answers clear enough to analyze. Combining two problems in one item may save a line on the screen while removing the ability to identify exactly what needs improvement.
Reference
Pew Research Center. (n.d.). Writing survey questions. https://www.pewresearch.org/writing-survey-questions/.
Illustrative photo: RDNE Stock project / Pexels.
