4.3 Ascending vs. descending order of response options

Another important design decision is whether to present response options in ascending or descending order because “the order in which response alternatives are presented to respondents may have a significant influence upon their selections” (Krosnick & Alwin, 1987, p. 202^[1]). This psychological phenomenon found in surveys is known as the response order effect.

4.3.1     Survey data impacted by various response order effects

Two common types of response order effects are the recency and primacy effects (Holbrook, 2008a^[2]).

• The recency effect typically occurs in oral surveys. When options are presented orally, respondents tend to select one of the options that they heard most recently. This phenomenon can be observed during an order at a restaurant when you listen to several salad dressing options and end up choosing the one that you heard toward the end of the list.
• The primacy effect occurs in visual surveys. When options are presented visually, respondents tend to select one of the options that they see at the beginning of the list. You may have observed this phenomenon when you administer surveys on paper or on the web where respondents see a list of options on the rating scale and select one of the options they see first. When text is written from left to right horizontally, they tend to select one on the left side of the list options, which is referred to as a left-side selection bias. This brings up an issue when you use an ascending order or descending order of the Likert response scale options.

Survey respondents may also be influenced by other biases:

• Respondents tend to agree more than they disagree with the statement provided—a.k.a., yea-saying bias. One explanation for acquiescence bias is the social norm to be polite (Holbrook, 2008b^[3]).
• This is explained as social desirability bias, the tendency for respondents to select among the options that they think are more socially acceptable or desirable, instead of a response that reflects their true feelings/opinion (Callegaro, 2008^[4]).
• People also tend to want to minimize their psychological costs by selecting solutions that are satisfactory or acceptable enough, which is known as a satisficing principle (Simon, 1957, as cited in Krosnick & Alwin, 1987^[5]).

4.3.2     Survey data impacted by psychological phenomena and environmental factors

The primacy effect, left-side selection bias, acquiescence bias, social desirability bias, and satisficing are psychological phenomena that survey respondents may exhibit when using response scales. As a survey designer, it’s important to consider how your choice of ascending or descending order of the Likert options—an environmental factor—might interact with the psychological tendencies.

In an ascending order (when presented horizontally), positive response options are presented on the right side of the scale:

In a descending order (when presented horizontally), positive response options are presented on the left side of the scale:

Compared to the ascending order, the descending order is more likely influenced by a combination of the primacy effect, left-side selection bias, acquiescence bias (yea-saying bias), and social desirability bias. Respondents tend to perceive that selecting one of the positive responses (e.g., Strongly agree or Somewhat agree) is more socially desirable and they see the positive options first on the left side. Therefore, they tend to select a positive option, potentially leading to inflated positive responses.

4.3.3     Inflated survey data from survey items with descending-ordered response scales

In a study by Nicholls, Orr, Okubo, and Loftus (2006)^[6] involving 292 college students in Australia, participants were assigned to complete a survey with a 5-point Likert scale presented in one of two orders:

In ascending order:

In descending order:

The group that completed the survey with a descending-ordered scale produced significantly higher scores than the group with the ascending-ordered scale.

A similar pattern wase found in a study by Hartley and Betts (2010)^[7]. Participants were randomly assigned to one of four conditions created by using different directions of worded labels (Clear to Unclear, or Unclear to Clear) and different directions of numeric ratings (10 to 0, vs. 0 to 10):

The first format—the descending-ordered scale that began with a positively worded label (Clear) and a highest numerical value (10)—resulted in significantly higher ratings in comparison to the other three response options.

A simple solution to the issue of getting inflated data obtained from descending-ordered scales is to present response scales in ascending order.