6 Course Material Decisions and Factors: Unpacking the Opaque Box

^[1]

^[2]

^[3]

Prescriptive Perspectives and Processes for Review of Course Materials

Process matters and many scholars have opinions and suggestions regarding how learning materials should be selected. Again, this literature is weighted toward analysis of traditional print textbooks, though some concepts may be transferable to interactive electronic resources. By far, the most insightful one-volume handbook I located is Meredith Gall’s 1981 Handbook for Evaluating and Selecting Curriculum Materials. While out of print and far out of date for electronic, internet-hosted, or interactive content, the handbook, I believe, accurately and succinctly describes issues related to any era of course material selection. Especially helpful for those wanting an introduction to curriculum studies, Gall mentions the timeless issues of: curriculum quality and commercialization, roles for various actors in higher education, the wide range of types of curriculum objects, the propensity of instructors to limit their searches to what’s easily available, the lack of instructor time and expertise in selecting course materials, relationships between instruction and course materials, and differences in learning resources even when options appear to be equivalent. Several helpful tools are included in the book, including an inventory and description of dimensions for analyzing curriculum materials (Chapters 4–6), a high-level course material process relevant for any topic and level of education, even higher education, and an appendix of featured curriculum materials that may facilitate learning. While updates would be needed, this source is very helpful and takes seriously the importance of selection of instructional materials in light of the fact that students spend far more time using instructional material than anyone else (Gall, 1981). (See the note below for guidance on accessing this out-of-print resource.)4

Several other authors report on prescriptive course materials selection processes or report on processes they have created or use. Prosser offers a summary of text readability analysis processes, prominent in the literature in the 1970s and 80s, namely using SMOG (Simple Measure of Gobbledygook) readability and the cloze test (Prosser, 1978). Heye offers a tool and process for evaluation of textbooks for nursing that enabled her school to include input from faculty members not initially involved in course material selection. Implementation of this process eliminated the need for supplementation of a main outdated text, reduced costs to students, and resulted in the use of materials that included updated health care developments (Heye, Jordan, Taylor Harden, & Edwards, 1987). Novotny provided a checklist for selection of nursing textbooks and provided guidance in assigning textbooks and journal reading assignments (Novotny, 2011).

Several scholars suggest value in having different levels of review. Lawrence summarized suggestions that the best of textbook evaluation schemes adopt a “leveled” approach: an initial overview of the strengths and weaknesses of the book with regard to design and structure, sequence, visual attractiveness, and availability of ancillary materials, and a further evaluation which is more detailed and determines whether text, skills and activities meet syllabus and learner needs (Lawrence, 2011). Kato affirms multipart approaches, indicating that textbook evaluation conversations should consist of pre-use evaluation, in-use evaluation, and post-use evaluation (Kato, 2014). Arnold’s research adds the insight that faculty valued being part of (textbook) pre-publication review (Arnold, 1989). Multiple authors cited a need for more instructor training and knowledge regarding course material selection (Gall, 1981; Stein, Steuen, Carnine & Long, 2001).

Course Materials, Pedagogy, and Levels of Instructor Expertise

Open education advocates and librarians can benefit from understanding the intended role or purpose of a text within a course. Texts may be adopted as a course reference, because textbook adoption is expected even though the textbook is not well integrated into the course, to aid students in building a resource collection, for ease in scaffolding the course or countering an instructor’s self-perceived deficiencies (Lawrence, 2011, p. 7, Confrey & Stohl, 2004, p. 43-46), or a combination of these reasons. In theory, course material selection should support course objectives, instructor pedagogies, and efficacious student learning habits. New instructors or instructors with new courses are more likely to adopt “book in a box” course materials but as they become comfortable with the course or less risk averse to changing the course away from parts that are not working, they may become more open to alternate pedagogies and curricular materials. It is into this dynamic environment that the open education advocate steps. The open education advocate or librarian may encounter a wide range of instructor comfort or discomfort with teaching and learning processes. Some instructors may be experimenting with incremental or major course design changes. Others may be content with limited investment in teaching or feel obligated to focus most of their energies on research endeavors. The librarian may also observe the impact of the institution or department’s politics and practices, governance and budgeting constraints, relative importance of career advancement via tenure and promotion, and individual instructors’ tolerance of risk, comfort, and perceived available support; each of these factors can and do influence student learning outcomes, selection of pedagogical processes, course materials, an instructor’s decision to go without traditional or emerging course materials, or an instructor’s openness to experiment with open pedagogical approaches. Beyond pressures related to an instructor’s career advancement, the relationship between instructor risk tolerance, comfort, and support, pedagogical understanding and openness to pedagogical and assessment methods, and beliefs regarding the purpose of course materials should not be understated.

Usage and types of course materials have changed over time due to educational philosophies, legal environments, cultural expectations, availability of trained educators, and commercial and technological changes. The earliest and most traditional course materials were printed textbooks and readers for children, designed to lead to literacy using catechism (question and answer) as their instructional mode. In the late 19th century, changing educational philosophies, the increased availability of trained teachers, and orientation toward deductive approaches and generalized morals in response to high immigration resulted in changes in curriculum resources (Wakefield, 1998).⁵ One hundred years later, and in the scope of higher education we see continued evolution of educational philosophies, development of cultures of tenure-track and adjunct faculty, an increased proportion of the population expected to engage in higher education, and impacts of technological change on course materials and instructional practices. Print resources are supplemented or replaced by digital course materials and systems. Not dismissing persistent digital divide issues, course materials and learning processes are now embedded in closed learning management systems, blogs and wikis, ebooks, online discussion boards, online homework systems, adaptive learning and intelligent tutors, student-driven platforms for authoring, game-based learning, and all manner of tools, clickers, and software systems. All bring pedagogical assumptions, some evidence-based. Regardless, many are adopted for classroom use.

Technology-enhanced learning resources represent some of the most creative, interesting, useful, and potentially responsive but constantly changing options within learning spaces. Many provide student metrics, allowing instructors a view of student time on task, theoretical opportunities for early interventions, and a research platform in which to start to learn what works and what does not work. Some offer freemium services, with more advanced premium services available at a fee. Open source projects are also present in the mix, some with an open business model (where content costs nothing) and in which services are rented on a subscription basis. The options are constantly changing. Current and new instructors with limited prior exposure to digital instructional methods or constantly changing digital learning environments are likely to be overwhelmed and feel disrupted when changes are foisted on them—such as the change of an enterprise-wide learning management system or techno-pedagogical changes such as flipping a classroom or converting a course to a blended or online format, which are willingly undertaken to improve a course which is otherwise not working, or for department, institutional, or financial reasons. Instructors at research institutions may lack adequate incentives—or support—to envision or achieve these types of changes in their classrooms (Gregory & Lodge, 2015). While there are many reasons to experiment with emerging technologies, some choose not to but regularly update their course notes and are perhaps less engaged by new technologies. For those who embrace new technologies, there are several potential downsides: needing time to teach a new tool, neglect of student privacy,⁶ and setting students up to game a system rather than engage in deep learning and authentic reflection.

While this chapter does not cover in depth selection methods for educational technologies including software programs, audience response clickers, homework software access codes, or other electronic ancillary tools, sometimes thought of as instructor conveniences and sometimes thought of as student aids,⁷ I do want to note that implementation of educational technology tools seems to be growing. I have observed two main responses: instructors who select materials based on research evidence are often slow to adopt such tools when their effectiveness is insufficiently documented, and instructors attempting to manage very large courses tend to adopt them quickly out of convenience if not survival; sometimes they are abandoned just as quickly.

Incentives Influencing Design and Selection of Course Materials

To the innocent bystander, the presumed aim of course materials in mediated instruction within higher education is student learning. As discussed above, this goal can be muddled by various incentives. Several influential factors still remain:

• Perceived quality (sometimes signaled by a trusted brand name)
• Author authority/accuracy and currency of content
• Reliable scope, sequence, or structure for instructors and students to follow
• Perceived fit for the student’s level of expertise
• Perceived fit with instructor’s methods of teaching
• Use of emerging technology (this factor can encourage or discourage, depending on instructor comfort)
• Time savings for faculty (including pre-made lecture slides and assessments)
• The selector’s valuation of meaningful promised student analytics⁸
• Authoritative resource for student reference now and in the future (Arnold, 1989)
• Departmental/institutional expectations or requirements regarding always assigning a text even if the text is not heavily used

^[8]

For novice instructors, part-time adjuncts without preparation time, graduate teaching assistants, and faculty teaching a course for the first time, course resources fit best as a support structure for the instructor. Experienced instructors and those with a more comfortable grasp of teaching the content area are likely to not need to rely as heavily on course resources, may be more likely to teach with learning resources they developed themselves, might not require student acquisition of learning resources, but may still assign course materials for student benefit, because students expect it, or because assigning a text is just what you do.⁹ Sometimes, an instructor’s long-time habits dictate assigning a required textbook as a “resource” even when it will not be used very much in the course.

^[9]

Of course, disciplinary differences in pedagogy and student needs come into play. Course materials in different disciplines may have quite different functions. For example: student learning activities in literature, foreign language, and biology differ quite a bit and affect the types of course materials selected. Students studying literature may focus primarily on reading and writing activities; students in foreign language experience a much greater emphasis on listening, speaking, and, at the lower levels, grammar and basic sentence construction. Students studying biology are involved in learning the scientific method, maintaining a lab notebook, experiments, and hands-on activities in a laboratory.

Since purposeful learning materials were developed, they have helped instructors solve complex teaching problems (Wakefield, 1998). A generous example is the instructor who selects course materials that fit the course learning outcomes and offer students helpful problem sets, real-life applications, case studies, or other examples that help students transfer knowledge to other domains. A more cynical example is the likely overworked or unsupported instructor, perhaps with too many students, who assigns course materials primarily for the instructor’s own benefit. Course materials, especially commercial ones “have been written, edited and marketed as teaching and learning aids” (Wakefield, 1998, p. 23) and are often interpreted by instructors as such. Required homework software access codes and classroom response tools or clickers primarily for instructor time savings in grading or as an expensive experiment in innovative teaching are a prime example. In my experience, I’ve seen these decisions justified by having to teach a very heavy course load, large classes, or the promise that the tools will make students who don’t complete their reading assignments engage with the materials. These tools unfortunately pass on the burdens of an instructor struggling with getting students to engage in course material and/or trying to manage interaction and assessment of a very large class directly to students, often in the form of multiple required learning resources such as clickers, quizzing or classroom interaction tools, print or electronic texts, and/or homework software access codes.

This conflict between instructor and student needs is not condemnation of instructors who make these decisions, but a reflection a common problem cited in economics and political science literature, called a principal–agent problem. A principal–agent problem features a decision-maker (the agent), in this case the instructor, who to varying degrees reflects (or doesn’t reflect) the values and interests of the person or people she represents (the principal), in this case the students. When the agent or instructor is motivated to act in his or her own interest to the detriment of the interests of the principal or students, economists identify “moral hazard” as an outcome (Eisenhardt, 1989). One may argue that the instructor is indeed in a difficult situation, often teaching as a non–tenure-track professor, without leverage, and in a somewhat impossible situation where implementing all sorts of tools is the only solution. Whether intended or not by the department which created such large courses or by the instructor, moral hazard, or “harm,” is likely to occur when an agent puts their own needs above those of their principal—or when instructors assign course materials too expensive for students to access. Students may be harmed by losing the freedom to take a class or a major because it is too expensive, by taking on additional debt, additional work hours, or at minimum by additional financial stress. Further, students may lose consistent and reliable access by sharing course materials, or by participating in peer-to-peer copyright infringement in the forms of digitizing, sharing, and/or downloading illegal copies of learning materials. Again, this is not a condemnation that instructors or departments in these types of situations are malicious actors, but an observation that care needs to be taken to proactively identify and remediate situations in which incentives encourage an agent to act in a way that may be harmful to the principal.

Introducing a Paradigm Shift

What can an open education advocate do? For starters, engaging faculty in conversation regarding their particular contexts, what they like or do not like about their course materials. What kind of content do they wish existed? What kind of content (including questions and other artifacts for assessment) could they or students create? What freedoms do they have to pilot or repeatedly create small quantities of content or assessments over long periods of time? What do they wish was happening in their class that isn’t? These can be tender topics, so trust and diplomacy is called for. Six open educational practices or values may scaffold instructors in their early and late attempts in openness: sharing, early drafting, supportive feedback, studying licenses, giving credit, and putting students at the center (West, 2017). Understanding open practices and values as a paradigm shift, and introducing, discussing the relative merits, supporting, and implementing each of these values can provide a clear focus for one’s activities and assist in navigating where to spend scarce time and resources.

Ideally, instructors will develop courses around course learning outcomes, mapping content, activities, and assessments to course learning outcomes. Instructors exhibiting this type of teaching tend to have a sufficient if not high level of mastery over their subject and a high level of comfort with regard to teaching. A deep interest in one’s discipline and care for one’s students, relevant and reflective professional development opportunities, practicing teaching improvements, and valuing instructional practices are paths to developing efficacious instruction. Instruction which increasingly prioritizes these types of practices strikes a different balance between instructor expertise, teaching methods, and critical selection and use of course materials.

Learning and working to understand the realities of one’s campus or campuses, campus cultures, policies, practices, values, pressures, motivators and incentives are the probably the hardest part of this work and take the most time, but are well worth the investment. Understanding course material evaluation and selection process will likely require a brief review of institutional policy regarding textbook or learning resource selection. Conversations with each of the departments on your campuses can be helpful. A call to each departmental administrative assistant or advisor with the following questions is a good place to start:

• Do the majority of your instructors assign required books?
• Are course materials selected by committee or individual instructors?
• Who are the point people regarding committee-selected course materials?
• What is the course material adoption schedule look like in your department?
• How does reporting of textbook adoptions (to the bookstore or other) work?
• I’m interested in learning more about how course materials are selected. Who else do you suggest I contact?

The registrar of your institution will be able to direct you to someone who can explain how your institution handles approvals for new or updated courses and whether there are requirements to list learning resources used in the course.

Liaison librarians may be aware of department-wide curriculum initiatives and needs. If your campus has a Center for Excellence in Teaching and Learning or instructional design support embedded in another unit, you may also be able to glean useful information about course design/redesign assistance and helpful insight regarding learning resource selection processes and motivations.

Your bookstore, if cooperative, may also be a useful source of information regarding textbook and other learning resource adoptions. Some helpful ideas for building a relationship with your bookstore can be found in Bell’s 2018 article.

As you gather information and build trust, think about what open resources and open practices might contribute to resolving stated problems in current departmental resource selection processes. Some academic librarians have gone as far as to contribute to and coordinate year-long textbook evaluation processes for selected high-enrollment courses within willing campus departments. For those instructors overwhelmed with large-scale changes toward open educational resources, piloting an open resource as an alternative text is an option, as are incremental changes to incorporate open pedagogical practices which replace one reading or assignment over a period of time. The Open Pedagogy Notebook is one place to look for or share examples of open pedagogical practices (DeRosa & Jhangiani, 2018). The text A Guide to Making Open Textbooks with Students (Mays, 2017) may also be a helpful resource.

Novel and Purpose-made Course Materials

Course materials include any discrete media, format, or system deployed in support of the learning process. Cost and copyright concerns notwithstanding, course materials can now be almost anything. Some materials and processes used in courses today were not designed to be used in courses: news and academic journal articles, movie clips, equipment designed for industry, household items, 3D printers, beach balls, Twitter, Wikipedia authoring, and so on. These items and processes are used outside of classrooms and have made their way into courses. This is a refreshing trend, as the application of these materials on teaching and learning may enrich students’ lives and help them to see the world around them as having potential for learning and exploration.

Purpose-built course materials are different than materials not specifically designed for learning. Textbooks, educational videos, workbooks, digital flashcards, lab environments, problem sets, online modules, interactive quizzes, clickers, educational apps, learning management systems, and various assessment and engagement tools are created specifically for learning and are most often purpose-built for educational contexts. They are intended to be consumed or acted upon by students in specific ways with specific outcomes in mind. Purpose-built learning resources always have embedded assumptions about what constitutes learning, how people learn, what learning is for, and how the system supports or facilitates that learning. Because educational technology is built by a wide variety of people, some systems may reflect sophisticated and well-conceived pedagogical philosophies; others might not. Rather than engaging with learning as a process, some may envision students primarily as containers for content provided by instructors as seen in this public domain illustration.

Interactive systems designed by those with expertise in various pedagogical philosophies may rely heavily on pedagogical philosophies of behaviorism, constructivism, cognitivism, and any number of other pedagogical approaches.¹⁰ Learning resources and approaches are not pedagogically neutral.

Even as learning resources have changed with the times, they continue to be marketed as “teaching and learning aids” to solve problems (Wakefield, 1998). For the most part in the United States, development of course materials and educational technologies is a for-profit endeavor. This raises some ethical issues. For-profit ed tech companies serve two or more masters: student learning, generating a profit for shareholders, developing their research base, capturing market share and so on. Competition between these factors may challenge the most ethically minded ed tech company to deviate from valuing student learning above other factors. It is certainly possible to imagine the existence of a company that sells learning resources of value without being overly swayed by a profit motive. However, this is very difficult to do without powerful and built-in accountability structures. Like any business, commercial publishers are responsible to their shareholders for financial gains, so the conflict between product quality for ultimate end users (e.g., instructors and students) vs. shareholders is often difficult to navigate. (Potential authors courted by publishers also face these conflicts in deciding whether or not to sign a publication agreement. They are encouraged by potential royalties and legitimately enjoy attention, respect, and relationships with publishers. They often transfer copyrights to the degree possible, limiting access to their work, and give up control to write the book they want to write.)

Commercial approaches also have an impact on the development of learning resources. Publishing industry veteran Beverlee Jobrack’s book about the K–12 textbook industry describes how commercial incentives shape the K–12 textbook publication process. Jobrack explores how market research, competitor analysis, and focus groups lead to the development of educational materials rather than educational research, rigorous study, and effectiveness of past use of course materials. Publishers rarely fund studies to understand the development of a subject and how it has been taught in the past, strengths and weaknesses of previously used materials, nor the educational research literature. In focus group sessions “publishers confirm that teachers rarely care about program effectiveness when weighted against a perceived useful design … and when curriculum specialists are in the room, they nearly always prefer research-based materials, but realize that it would be an upward battle for their teachers to accept them” (Jobrack, 2012, p. 62). As a result, textbook development focuses mainly on features that are appealing rather than effective. While disheartening, Jobrack’s observation that the development process for commercial textbooks focuses on aspects that appeal rather than being chosen for their effectiveness is an observation for open education advocates and instructors who develop or adapt open educational resources. At the end of the day, if effectiveness is more important than appeal, openly licensed resources focused on effectiveness should be different in important ways than those developed with appeal in mind.

Some faculty feel compelled out of habit to require a textbook even if it is not used very much in the course. Other faculty explore pedagogies as far away as possible from passive, consumable resources, some using Wikipedia assignments or creating a textbook as part of their course (DeRosa & Robinson, 2017). Others are implementing practices to encourage student agency, such as giving students flex or pink time (Baird, Kniola, Lewis & Fowler, 2015). Increasingly, instructors are seeing student engagement with the course and course content as the key to improve learning (Hunt et al., 2016)

A Way Forward?

Let’s return to the question posed early in this chapter: Where can an open education advocate, or simply someone that cares about teaching and learning, start to make a contribution if they are not the course material decision-maker? For open education advocates, the keys to addressing the course material adoption issues on campus rest in working to understand the distinct realities of campus and departmental contexts and cultures, gathering information, building trust among instructors, decision-makers, and others working to address course material and teaching-related issues on campus, introducing a new paradigm of values and open educational practices (West, 2017).

Academic librarians and instructional designers already do many things to model, champion, critique, invent, improve upon, and/or support open educational practices. These may include but are not limited to: open access authoring and publication, creating and building sustainable, Creative Commons-licensed editable curriculum materials, modeling pedagogies and web development strategies dependent upon openly licensed content and open source software, contributing to open source infrastructure, developing open and sustainable models of operation through collaborative networks, and implementing student-centric pedagogies which grant increased student agency or emphasize creation of artifacts which have value beyond the classroom.

Others contribute to and aid faculty, staff and students to interact with authored content in ethical and sustainable ways by creating, remixing, and sharing research outputs with open licenses. OER are just one of many possible ways to implement the ethic or concept of openness. Librarians may be engaged or desiring to be engaged in course material or learning resource initiatives at their academic institution, including organizing evaluation or selection of openly licensed or other course materials and course material formats.

Building new expertise helpful to processes where there is no or minimal expertise may have even more potential. A few examples from this brief overview of course material production and selection where open advocates could add value include: provide more support for faculty training in course material evaluation and selection, curate tools and methods for all effort levels of course material review, and when developing openly licensed course materials, focus on effectiveness more than appeal.

And finally, build trust. Everything runs on trust and, in an ideal world, accurate information, sharing, and trustworthy processes and systems. Course material selection decisions are based on trust in people, information, and/or processes. Accurate information, reliable services that provide needed information, support, logistics or somehow add value, and a willingness to listen, learn, and respond with integrity should greatly add to creating a way forward that keeps learners at the center, values transparency, requests and accepts constructive feedback, gives credit, and promotes sharing.

References