2. Understand the Basic Structure of Survey Items
2.3 Survey items measuring constructs
As discussed earlier, a survey is just one of the many data collection methods available to you. You choose to conduct a survey when it is difficult or impossible to directly observe the information you need or obtain it from other sources.
For example, if you need to know how well nurses are performing patient rounding, the most direct method for collecting this information would be to observe the nurses’ performance. Alternatively, you could review rounding checklists if they are formally implemented. You could also ask nurses to respond to a survey item asking how many rounds they completed during their shift, or how easy or difficult to perform patient rounding. However, self-reported data may not be as accurate as data obtained from direct observations or checklist reviews.
Though surveys have some weaknesses, they are not necessarily less useful than other data collection methods. In fact, there are situations where surveys are the most appropriate method, and sometimes the only way, to obtain the information you need. When the information you require is abstract in nature and resides in people’s mind—such as their perceptions of the quality or value of something, which cannot be directly observed or recorded—you would use a survey to retrieve that information.
These abstract pieces of information, which cannot be directly observed or easily assessed, are called constructs. Constructs are hidden until studied and revealed, and are often referred to as latent variables in research. Interestingly, even facts such as age, gender, or time can be considered socially-agreed constructs. Therefore, survey questionnaires that ask for facts and perceptions about the quality or value of something are also measuring constructs.
Constructs can be simple or complex. Simple constructs such as employment status or job title can be measured using a single survey item for each. For example:
Are you an employee of Sawtooth Health System?
○ Yes
○ No
What is your job title?
○ Instructional designer
○ E-learning developer
○ Graphic designer
○ Programmer
○ Describe: _____________
In contrast, more complex constructs, particularly those related to people’s inner thoughts, feelings, or perceptions, usually require multiple survey items to measure them accurately.
For example, if you are evaluating trainees’ satisfaction with a workshop, asking a single question such as “How satisfied are you with the workshop?” may not provide enough insight. Satisfaction is a multifaceted construct, and a single survey item does not capture the various elements that contribute to it. To measure satisfaction more effectively, you would use multiple survey items that assess specific aspects of the workshop, such as the quality of the content, the relevance of the material to their job tasks, the amount of information provided, etc. as shown in Exhibit 1.
How satisfied are you with the following aspects of the workshop?
| Question | Not satisfied |
A little bit satisfied |
Somewhat satisfied |
Quite a bit satisfied |
Very satisfied |
|---|---|---|---|---|---|
| Quality of the content | ○ | ○ | ○ | ○ | ○ |
| Amount of the content | ○ | ○ | ○ | ○ | ○ |
| Relevance to my job | ○ | ○ | ○ | ○ | ○ |
| Workshop length | ○ | ○ | ○ | ○ | ○ |
| Trainer’s preparedness | ○ | ○ | ○ | ○ | ○ |
For more complex constructs—such as intelligence and organizational culture—a single survey item is not sufficient. Instead, you need to use a battery of survey items to measure the multiple dimensions that make up the construct. For example, the Dimensions of the Learning Organization Questionnaire (DLOQ), developed by Marsick and Watkins (2003)[1], includes 43 survey items designed to measure seven dimensions of a learning organization and additional 12 survey items that measure learning organizational results. Exhibit 2 presents a few sample survey items from the DLOQ:
| Question | Almost Never 1 |
2 | 3 | 4 | 5 | Almost Always 6 |
|---|---|---|---|---|---|---|
| 1. In my organization, people openly discuss mistakes in order to learn from them. | ○ | ○ | ○ | ○ | ○ | ○ |
| 2. In my organization, people identify skills they need for future work tasks. | ○ | ○ | ○ | ○ | ○ | ○ |
| 3. In my organization, people help each other learn. | ○ | ○ | ○ | ○ | ○ | ○ |
Therefore, when designing a questionnaire, it’s important to consider constructs you intend to measure. Some constructs are simple, such as capturing factual information, while others—particularly those measuring people’s perceived quality or value of something—are more complex and require multiple survey items. To effectively measure these complex constructs, you need to ensure that the survey items you’re using are all aligned with the same construct. Additionally, it’s beneficial to use the same rating scale across these survey items. This allows for a consolidated summary of the results (e.g., reporting an average score based on the multiple survey items that measure the same construct). We will explore response scales and how they support this process in the following sections.
- Marsick, V. J., & Watkins, E. W. (2003). Demonstrating the value of an organization’s learning culture: The dimensions of the learning organization questionnaire. Advances in Developing Human Resources, 5(2), 132-151. ↵
