5. Develop Surveys for Different Purposes
5.4 Sample surveys used during instructional design
While working on an instructional design project, you will have opportunities to collect data from various groups during the job task analysis, content analysis, learner analysis, and formative evaluation. For example, you may collect data from subject matter experts and learners (actual performers) to:
- Determine specific tasks to be performed for the job
- Identify the desired level of knowledge, skills, and attitudes that learners should acquire
- Assess learners’ current level of knowledge, skills, and attitudes
- Determine the necessary instructional content
- Solicit feedback on parts of the instruction that may need revision
You may decide to conduct a survey in some of these cases.
For example, let’s say you are offering an annual mandatory lab safety course to all lab technicians. You discovered that some lab technicians feel confident in their lab safety knowledge and do not perceive a need to complete this course. To improve learner motivation, you designed the course using Keller’s ARCS model[1],[2], addressing the following four elements in instruction:
- Attention: If learners find the presentation of instructional content to be clear, interesting, and engaging
- Relevance: If learners find the instructional content to be relevant to them and address their personal and professional needs
- Confidence: If learners feel confident in using the information they learned
- Satisfaction: If learners feel satisfied with the overall learning experience and outcomes
After completing your course design, you plan to conduct a pilot test (formative evaluation) with a small group of lab technicians to check how well the instruction works and which parts of the instruction may need to be revised to maximize learning outcomes. After the pilot course, you decide to administer a survey to the pilot learners.
Since you designed the course with the ARCS model in mind, you wanted to evaluate whether the instruction effectively addresses each ARCS component. Therefore, you designed your survey items aligned with the four factors as shown in Exhibit 15.
Here, it is important to note that the ARCS components are interrelated and may not function as entirely separate constructs. For example, when instruction is easy to follow, interesting, and relevant, it not only holds learners’ attention but also enhances their learning, confidence, and overall satisfaction. Thus, the overall construct that you are measuring with a survey questionnaire such as the one presented in Exhibit 15 is the course’s motivational appeal to learners. The section headings—Attention, Relevance, Confidence, and Satisfaction—are presented only for organizational purposes in the survey design, not to imply separate statistical constructs.
Exhibit 15 A Sample Survey Questionnaire to Measure ARCS Factors in Instruction
Please rate each of the following aspects about the course based on your experience, by selecting a number from the 5-point scale where 1 = Not at all and 5 = Very much.
| Attention | Not at all 1 |
2 | 3 | 4 | Very much 5 |
|---|---|---|---|---|---|
| 1. I was able to recognize the course objectives. | ○ | ○ | ○ | ○ | ○ |
| 2. I found the content to be easy to follow. | ○ | ○ | ○ | ○ | ○ |
| 3. I was able to pay attention to the content throughout the course. | ○ | ○ | ○ | ○ | ○ |
| 4. I found the examples provided in the course interesting. | ○ | ○ | ○ | ○ | ○ |
| Relevance | Not at all 1 |
2 | 3 | 4 | Very much 5 |
|---|---|---|---|---|---|
| 5. I found the content to be relevant to lab safety in my job setting. | ○ | ○ | ○ | ○ | ○ |
| 6. I found the content to be useful for improving lab safety in my lab. | ○ | ○ | ○ | ○ | ○ |
| 7. I was able to follow and understand the content based on my previous knowledge of lab safety. | ○ | ○ | ○ | ○ | ○ |
| 8. I think the multimedia used in the course helped me understand lab safety issues better. | ○ | ○ | ○ | ○ | ○ |
| Confidence | Not at all 1 |
2 | 3 | 4 | Very much 5 |
|---|---|---|---|---|---|
| 9. I found the exercises to be challenging enough. | ○ | ○ | ○ | ○ | ○ |
| 10. I felt confident enough to continue the course. | ○ | ○ | ○ | ○ | ○ |
| 11. I now fully understand what I need to do to ensure lab safety. | ○ | ○ | ○ | ○ | ○ |
| 12. I will be able to apply the new knowledge to ensure lab safety. | ○ | ○ | ○ | ○ | ○ |
| Satisfaction | Not at all 1 |
2 | 3 | 4 | Very much 5 |
|---|---|---|---|---|---|
| 13. I am glad that I am knowledgeable about lab safety. | ○ | ○ | ○ | ○ | ○ |
| 14. I feel it was worth spending time to complete this course. | ○ | ○ | ○ | ○ | ○ |
| 15. I had a positive learning experience during the course. | ○ | ○ | ○ | ○ | ○ |
| 16. I think other technicians should complete this course. | ○ | ○ | ○ | ○ | ○ |
