Observing from a distance: Insights for conducting effective field-based observations in online courses

Introduction

Rural teachers tend to have fewer qualifications than those in urban centers (Malkus, et al., 2015), and quality professional development for rural teachers has been tied to enhanced student outcomes (Barrett, et al., 2015). This has led many programs to offer online graduate coursework — however, it is our responsibility as teacher educators to ensure these online courses are of high quality. Integral to teacher education is quality fieldwork with ongoing supervision and feedback, and it is therefore critical to establish strategies for conducting effective distance observations.

The 2017 International Literacy Association Standards require that teacher preparation programs include “multiple, ongoing, supervised field experiences.” Such fieldwork has long been a keystone of teacher preparation, and research has demonstrated the importance of teachers receiving quality feedback from their supervisors (Jones, et al., 2018). As literacy programs across the country move online, a common challenge is how to best conduct observations and provide feedback when the supervisor cannot physically be in the classroom.

Statement of the Problem

Prior to beginning this work, we anecdotally identified two areas of concern related to conducting “distance observations,” namely: 1) students often struggle to use technology to record their teaching, or report that uploading even short videos can take hours; 2) instructors struggled to provide effective feedback to online students. However, we lacked objective research detailing these concerns, which made it difficult to identify effective solutions.

Method

This research was conducted in two phases. First, we conducted semi-structured interviews with the students who had previously completed a course requiring distance observation. As most of our students have had traditional observations, we asked them to compare distance observations with their prior experiences. We also investigated what they liked about distance observations, as well as what challenges they faced. We recorded the interviews and analyzed the transcribed data using open coding to identify successful implementation strategies as well as areas for improvement. This provided us with baseline data for making programmatic improvements. Our interview questions are presented in Table 1.

Table 1

Semi-structured interview questions

Next, we used these findings to make changes to our courses’ distance observation protocols. This included finding solutions to technology problems as well as modifying how we provide feedback. We created a survey to be distributed to the students on Qualtrics at the end of the semester to learn how the distance observations impacted their teaching, and how we can continue to improve. Our survey questions are detailed in Table 2.

Table 2

End-of-semester survey

Results

Interviews

We conducted interviews with six students who had completed one or more of our previous graduate courses requiring distance observations. As the researchers on this team are also instructors in the program, we purposely interviewed students who had not been in our own course so students would not feel uncomfortable providing critical feedback.

Two themes related to challenges emerged from our analysis. First, as expected, students frequently expressed frustration over the technical aspects of conducting distance observations. While most had the ability to record themselves using a personal device (such as an iPad, phone, or laptop), sharing that file with the instructor was difficult. Students reported that uploading to Google Drive could take hours, and many had to attempt the upload multiple times. One student expressed that she spent more time and energy trying to share the recording with her professor than she did planning, implementing, and reflecting upon the lesson she recorded.

Another challenge related to the delay in feedback. In traditional observations, it is typical for the supervisor to provide feedback very quickly – written notes are often given at the end of a lesson, or the teacher and supervisor meet to discuss at the end of the observation. Because of the time lapse between when a video is recorded and feedback is provided, students felt that the feedback was less effective.

One positive theme that emerged was students felt that the feedback they received was very detailed. Especially when compared to traditional observations, students could tell their supervisor often paused the video to take notes and give specific feedback on how they asked questions or approached the children in the room.

Intervention: GoReact

While we were conducting our interviews and analyzing the resulting data, we also researched technical solutions. GoReact (www.goreact.com) emerged as a promising option, and following the results of our interviews, we hypothesized that this tool would help solve some of the identified problems. GoReact is an online tool that allows students to video record themselves directly to a cloud-based platform (thus alleviating challenges related to upload speeds). Students can use any device for recording, and technical assistance is available 24 hours a day. The instructor can see the recording as soon as it is complete, and use the online platform to provide comments as annotations or audio feedback. This feedback is shared with the student as soon as it is posted.

Survey Results

We first used GoReact in the Spring of 2020 in two classes (ED-LLC 545 Writing Processes, Instruction, and Assessment: K-8 and ED-LLC 551 Literacy Leadership). There were 10 respondents from ED-LLC 545, and six from ED-LLC 551. The quantitative survey results are detailed in Table 3.

Table 3

Quantitative Survey Results from Spring 2020 Semester

We quickly identified the fact that students in ED-LLC 551 were far more satisfied with the experience using GoReact than the students in ED-LLC 545. The qualitative feedback revealed that the two instructors used GoReact very differently. Due to last minute schedule changes, an adjunct instructor (who did not receive sufficient training in GoReact before the beginning of the semester) taught ED-LLC 545. While she tried to integrate GoReact into the course, comments such as “would have liked to receive direct feedback on my video” and “the only feedback I recall was with grade submissions” indicated that the instructor did not use all the features for providing feedback in GoReact. This made any challenges with technology more frustrating, as students did not see the benefit to using GoReact. However, further investigation revealed that the instructor was using GoReact to provide feedback, but the students were unaware of how to view that feedback.

By contrast, students in ED-LLC 551 (taught by Dr. Hannah Carter, PI on this project) provided comments such as “I was happy with how the feedback was provided for distance observations” and “I thought that GoReact was great!”. Together, these findings suggest that proper training and support (for both faculty and students) is essential for successful technology implementation.

Based upon these findings, we adjusted our summer course (ED-LLC 542 Best Practices in Literacy Improvement) to ensure both faculty and students were ready to use GoReact to its full potential. This course requires that students provide 15 hours of targeted literacy intervention, and receiving feedback on their teaching is an essential part of meeting the course standards. To address the concerns identified in the spring semester, the course was taught by Dr. Katherine Wright (PI on this project), who was very familiar with how to use GoReact. Additionally, we added additional instructions and supports in the course to help students navigate GoReact. Finally, as part of their coursework, students were required to evaluate their videos on GoReact to ensure they understood the platform and could access instructor feedback. Six students responded to the survey. Their quantitative results are summarized in Table 4.

Table 4

Quantitative Results from Summer 2020 Semester

Written feedback on the survey suggests that students found the feedback on GoReact videos meaningful and authentic. For instance, one student wrote:

I liked that you can insert comments as you watch the video and it shows the moment of time it occurred. I liked that both the professor and I could insert comments. After making these comments, it was good to have a meeting with my professor to discuss what went well and how I could improve.

Other students commented on the ease of using GoReact to share videos. This was especially relevant, as COVID-19 required they complete their tutoring in online environments. The added technology burden could have made sharing recordings challenging, but one student remarked that “Zoom immediately records while you teach and provides you the file immediately after the meeting ends. It is also easy to upload it onto Goreact.”

Conclusions

When we first applied for this fellowship, our goal was to improve our own practice providing feedback in online environments. Little did we know that this would become an essential part of teacher education in 2020. We are grateful for this opportunity to proactively investigate best-practices for distance observations, as these practices have become commonplace at all levels of teacher education.

In the following sections, we discuss implications for teacher education, as well as limitations to our research.

Implications

Moving forward, we offer the following suggestions for conducting distance observations using GoReact:

• Instructor training is essential. GoReact must be integrated into the class as an authentic instructional tool, not simply added on as an additional task for students. Helping instructors identify potential ways GoReact can support their instruction will also aid student buy-in.
• Students need sufficient support for using GoReact. While the learning curve for using GoReact seems to be much less steep than many technology tools, it is not non-existent. Early frustration can lead to students “tuning out” and not using GoReact to its full potential. We recommend including an assignment requiring students to interact with the platform (such as annotating a sample video) before they are required to record and annotate their own video.
• Students still value direct interaction with the instructor. In both ED-LLC 551 and ED-LLC 542 students had opportunities to discuss their videos and feedback with the instructor (either via email, phone, or videoconference), and students commented on how the use of GoReact with these conversations were meaningful in their development as teachers.

Limitations

A clear limitation of the current research is that we currently only have a small sample from one master’s degree program. Future research must expand this to identify how populations of students at different levels and from different fields react to distance observation protocols. Additionally, we only implemented one intervention strategy (GoReact), and other platforms exist that may be more effective. However, our cost analysis led us to believe that GoReact is the best option for programs without extensive funding. Students are able to pay for a subscription each semester, and the cost is equivalent to that of a low-priced textbook. As a department, we have committed to reducing the number of textbooks we require in our courses so we may use GoReact without increasing financial burdens for students.

References

Barrett, N., Cowen, J., Toma, E., & Troske, S. (2015). Working with what they have: Professional development as a reform strategy in rural schools. Journal of Research in Rural Education, 30(10), 1-18.

Jones, L., Tones, S., & Foulkes, G. (2018). Mentoring associate teachers in initial teacher education: The value of dialogic feedback. International Journal of Mentoring and Coaching in Education, 7(2), 127-138. https://doi.org/10.1108/IJMCE-07-2017-0051

Malkus, N., Hoyer, K. M., & Sparks, D. (2015). Teaching vacancies and difficult-to-staff teaching positions in public schools. Stats in Brief, 65. http://ies.ed.gov/pubsearch/pubsinfo.asp?pubid=2015065