Promoting Teaching & Learning
Teaching & Learning
- Overview of the Scholarship of Teaching and Learning
- Scholarship of Teaching and Learning: Key Teaching and Learning Concepts
- Creating Scholarship on Teaching and Learning
- Research Ethics and Institutional Review (IRB)
- Scholarship of Teaching and Learning: Assessing and Measuring Student Learning
- Research Tools for the Scholarship of Teaching and Learning
- Evidence-supported practices rather than informal “lore”
- Teaching as a scholarly activity
SoTL emphasizes improvements in teaching and building relationships with students as co-collaborators rather than passive participants (Bishop-Clark and Dietz-Uhler, 2012). According to Chick (2013), SoTL includes:
- “asking meaningful questions about student learning and about the teaching activities designed to facilitate student learning
- answering those questions by first making relevant student learning visible as evidence of thinking and learning (or mis-learning), and then systematically analyzing this evidence
- sharing the results of that analysis publicly to invite review and to contribute to the body of knowledge on student learning in a variety of contexts, and
- aiming to improve student learning by strengthening the practice of teaching (one’s own and others’)”
SoTL is a movement within higher education that focuses on studying one’s own practices (similar to “self study” used in numerous fields or “teacher action research” used in the K-12 setting). It is conducted at the level of the course, and focuses on systematic practitioner inquiry using research-based approaches. Educational research is typically discipline-based, and it focuses on research on/about/in educational settings, often more broadly than a single course. It is sometimes called “discipline-based educational research.” For a nice overview about the differences between SoTL and other kinds of educational research, we suggest you visit the DBER Group site.
“Learning Sciences” is an interdisciplinary field that focuses on the science of learning, or using scientific research to understand learning. It often employing quasi-experimental approaches and/or experimental approaches to resarch. Journals to explore for the Learning Sciences include: Journal of the Learning Sciences, Cognition and Instruction, Interdisciplinary Journal of Problem-Based Learning, Instructional Science.
Prepared by Dr. Dana Lynn Driscoll, Department of Writing and Rhetoric and CETL Faculty Fellow, firstname.lastname@example.org
This page provides an overview to some basic concepts within the teaching and learning literature that are useful for SoTL researchers. There are many more, but this is meant to introduce the reader to the basics. For those interested in learning more about educational research topics and areas of interest, please consider reading the entirety of How People Learn as well as reviewing The Learning Classroom: Theory into Practice course for more information.
- How People Learn (Full book in PDF): http://www.nap.edu/download.php?record_id=9853
- The Learning Classroom: http://www.learner.org/courses/learningclassroom/support_pages/index.html
Adult Learning Principles (Knowles, et. al). Knowles’ work on adult learning in the 1950’s-1980’s has provided educators with a conceptual understanding of how adults learn best. Knowles and colleagues argue that adult learning should be based on four principles: relevancy (connected), engaged (students aren’t just sponges for information but must be engaged in the process), active (related closely to where knowledge will be used, applied immediately), and learner-centered (focused on learning, not teaching). For more information, see: Knowles, M. (1970), The Modern Practice of Adult Education-Andragogy vs. Pedagogy. New York: Association Press.
Understanding vs. Rote Memorization of Isolated Facts
Learning researchers distinguish understanding (or “learning”) from rote memorization. Wiggins and McTighe (2001) suggest that we can teach our courses with understanding, not just knowledge, in mind in Understanding By Design. They argue that understanding is “to make connections and bind together our knowledge into something that makes sense of things (whereas without understanding we might see only clear, isolated, or unhelpful facts). But the word also implies doing, not just a mental act….to understand is to be able to wise and effectively use—transfer—what we know, in context, to apply knowledge and skill effectively, in realistic tasks and settings.” (p. 7) They posit that understanding is not one concept, but a related series of abilities and suggest that one way of designing curriculum is to use backwards design—identifying the “desired results” first, then determining “acceptable evidence”, and finally planning “learning experiences and instruction” (p. 18). For more information, see Wiggins, Grant & McTighe, Jay. (2001). Understanding by design. Upper Saddle River, NJ: Prentice Hall.
Using Pre-Existing Knowledge / Constructivist learning: While a traditional view of teaching sees students as “sponges” soaking up information (or “banking”), researchers recognize the critical value of pre-existing knowledge. This principle is called “constructivism,” and suggests that all knowledge is constructed from previous knowledge. A constructivist view fronts that we must scaffolding knowledge over time and encourage students to build on previous knowledge and deepen understanding (this also ties to theories of transfer of learning, see below). See: Piaget (1952), Vygotsky (1978) and How People Learn.
High-Impact Practices. In 2008, the Association of American Colleges and Universities released High-Impact Educational Practices: What They Are, Who has Access to Them, and Why They Matter (George Kuh). Since then, these practices have been recognized and employed at universities throughout the country—and many administrators (including those at OU, see the recent High Impact Practice Grants) have been interested in promoting said practices. These practices, all considered “active” learning, are as follows: first-year seminars and experiences; common intellectual experiences; learning communities; writing-intensive courses; collaborative assignments and projects; undergraduate research; diversity/global learning; service learning and community-based learning; internships, and capstone courses and projects. These are less about learning science, but do indicate some programmatic best practices.
More info: http://www.aacu.org/leap/hip.cfm
Metacognition: Metacognition, defined broadly as “thinking about thinking” (Desautel, 2011) or one’s ability to critically reflect upon one’s own thinking/learning processes. And understand how their learning works. Theories of metacognition have been used in learning contexts to describe a variety processes and practices linked to successful learning. While metacognition’s importance for successful learning has been articulated across a variety disciplines, including education, psychology, and mathematics, many researchers have recently suggested that defining and identifying metacognitive elements continues to problematic. As Scott and Levy have recently observed, “Metacognition is a fuzzy concept but widely used by the research community […] However, it is still unclear if there is an umbrella concept with one major factor that can be labeled metacognition or whether metacognition has clear and distinct factors upon which researchers can base their research” (121). Schraw and Dennison (1994) describe metacognition as a set of related thinking processes that lead to students understanding their learning ( “knowledge of cognition”) and adapting their behaviors based on their understanding (what “regulation of cognition”).
Scott, B. M. & Levy, M. G. (2013). Metacognition: Examining the components of a fuzzy concept. Educational Research 2(2), 120-131.
Schraw, G., & Dennison, R. S. (1994). Assessing Metacognitive Awareness. Contemporary Educational Psychology, 19, 460-475.
Motivation, or one’s willingness to exert effort “affects the amount of time that people are willing to devote to learning” (How People Learn, p 60). Researchers have found that many issues affect motivation, including the difficulty or ease of material, social opportunities, the appearance of usefulness of learning, and their overall value of the task (How People Learn, p. 60) One recent study (Murayama et. al, 2013) of 3500 math students in Germany over five years examined what factors (intelligence, motivation, study habits) contributed to short-term or long-term learning. They discovered that, regardless of intelligence, surface learning techniques (extrinsic motivation==motivation based on grades, memorization) lead students towards short-term gains, deep learning (elaboration, building connections, and intrinsic motivation) produced significantly positive gains and long-term academic achievement. Their work, suggests, then that motivation is one of the most important factors for long-term, deep learning. More information can be found in: How People Learn (pp 60-61)
Murayama, K., Pekrun, R., Lichtenfeld, S. and vom Hofe, R. (2013), Predicting Long-Term Growth in Students' Mathematics Achievement: The Unique Contributions of Motivation and Cognitive Strategies. Child Development, 84: 1475–1490. doi: 10.1111/cdev.12036
Theories of Multiple Intelligence: Multiple theories of intelligence, and the kinds of intelligences that learners have and are available to use, are another factor often considered in learning research. One theory of multiple intelligences commonly used was posited by Howard Gardner (1983) in Frames of Mind: The Theory of Multiple Intelligences. His eight kinds of intelligences and associated abilities were: musical/rhythmic, visual/spatial, verbal/linguistic, logical/mathematical, bodily/kinesthetic, interpersonal, intrapersonal, and naturalistic (he later also suggested that moral intelligence was another kind). A second set of intelligence theories is Steinberg’s (2003) triarchic theory of intelligence, which describes three kinds of intelligences that learners need. These three intelligences are: Creative intelligence: intelligence required to develop solutions to problems and formulate new ideas; Analytical intelligence: intelligence required to solve problems and to assess the quality of ideas (this is traditionally privileged in school settings); Practical intelligence: intelligence that is needed to actually implement ideas effectively in a variety of settings. More information:
- Gardner, Howard (1993), Multiple Intelligences: The Theory in Practice. New York: Basic Books.
- Gardner, Howard (2000), Intelligence Reframed: Multiple Intelligences for the 21st Century. New York: Basic Books.
- Sternberg, R. J. (1985). Beyond IQ: A Triarchic Theory of Intelligence. Cambridge: Cambridge University Press.
Student Dispositions / Thinking Dispositions. Perkins et. al. (2000) argued that a lot of what we call “intelligent behavior” depends less on abilities but more on students’ dispositional qualities (such as curiosity, open-mindedness, reasonableness, reflectiveness, metacognition, mindfulness). Their underlying premise is that “intelligent behavior involves more than ability” and that more research is needed in understanding these dispositional qualities (p. 289). Bronfenbrenner and Morris’ (2006) bioecological model (a model of human development that has recently been applied to learning) also recognizes dispositional qualities as critical to successful learning. More information:
Bronfenbrenner, U., & Morris, P. A. (2006). The bioecological model of human development. In R. M. Lerner & W. Damon (Eds.), Handbook of Child Psychology (Vol. 1, pp. 793-282). New York, NY:: Wiley.
Perkins, D., Tishman, S., Ritchart, R., Donis, K., & Andrade, A. (2000). Intelligence in the Wild: A Dispositional View of Intellectual Traits. Educational Psychology Review, 12(3), 269-293.
Transfer of Learning: Transfer of learning is using, adapting, or otherwise repurposing prior knowledge, skills, or strategies in new settings. Many learning sciences and SoTL researchers (including the author of this guide) believe that learning and transfer are synonymous. The question of how transfer occurs, how we measure transfer, and how we best teach for transfer has plagued educational researchers in a variety of fields for over 100 years (Haskell, 2001). Recently, long-term transfer researchers Perkins and Salomon (2012) suggested that thinking dispositions and motivation might be keys to understanding transfer; and that students often engage in detect, elect, connect behaviors where one must first be primed to detect a connection, elect to make it, and then proceed with adapting/using the knowledge/skill/strategy in a new place.
More information: How People Learn Ch 3
Perkins, D. N., & Salomon, G. (2012). Knowledge to Go: A Motivational and Dispositional View of Transfer. Educational Psychologist, 47(3), 248-258. doi: 10.1080/00461520.2012.693354
Salomon, G., & Perkins, D. N. (1989). Rocky Roads to Transfer: Rethinking the Mechanisms of a Neglected Phenomenon. Educational Psychologist, 24(2), 113-142.
Haskell, R. E. (2000). Transfer of Learning: Cognition and Instruction. New York: Academic Press.
Novices vs. Experts
The development of expertise is one of the major areas that learning scientists consider; in How People Learn, the authors suggest that experts differ substantially from novices in at least six ways. These ways include: the ability to notice “meaningful patterns of information not noticed by novices”; the acquisition of a “great deal of content knowledge that is organized in ways that reflect a deep understanding of their subject matter”; the fact that their knowledge “cannot be reduced to a set of isolated facts…instead is ‘conditionalized’ on a set of circumstances”; their ability to flexibly” retrieve important aspects of their knowledge with little attentional effort” (pg. 31).
More info: How People Learn, Ch 2.
Bishop-Clark, C. & Dietz-Uhler, B. (2012). Engaging in the scholarship of teaching and learning. Sterling, VA: Stylus Publishing
Cambridge, B. (2004). Campus progress: Supporting the scholarship of teaching and learning. Grand View, MO: American Association for Higher Education
Cook, C. & Kaplan, M. (2011). Advancing the culture of teaching on campus. Sterling, VA: Stylus Publishing
Huber, M. T. (2004). Balancing Acts: The scholarship of teaching and learning in academic careers. Sterling, VA: Stylus
McKinney, K. (2007). Enhancing learning through the scholarship of teaching and learning: The challenges and joys of juggling. San Francisco, CA: Jossey-Bass
McKinney, K. (2013). The scholarship of teaching and learning in and across the disciplines. Bloomington, IN: Indiana University Press
Savory, P.; Burnett, A. & Goodburn, A. (2007). Inquiry into the college classroom: A journey toward scholarly teaching. Boston, MA: John Wiley and Sons Inc.
See the full CETL Library List.
But like any research project, before you dive into a project, it is necessary to start with a literature review to review relevant research and draw upon previous scholarship. It might be that your question was asked, but in a different way, or that your question has yet to be asked—and like any other academic field, drawing upon previous work and building knowledge from previous research is necessary. A literature review will also give you insight also into the methods of inquiry used for your research question.
Here are two places you might seek out SoTL research:
- Teaching journals in your field. Many fields have journals dedicated to teaching within a particular field, such as Teaching and Learning in Nursing, Teaching Business and Economics, The Journal of Teaching Writing, Teaching Statistics,
- Broader journals on teaching and learning. There are also numerous journals that focus on education in broad ways. Here are just a few of those journals:
Journal of the Learning Sciences
Teaching and Learning Inquiry
Learning and Individual Differences
The Journal of General Education
- Books & Edited Collections. Although not as timely, books and edited collections are also quite useful.
A variety of funding opportunities exist for SoTL research. Internally at Oakland University, funding for SoTL can be found through these sources:
OU Funding for SoTL Projects:
- OU URC Faculty Fellowship Award ($10,000, summer, due in two weeks)
- OU URC Faculty Research Award ($1,200, due end of January)
- CETL Innovations in Teaching Grants (2014)
- OU Educational Development Grants (2014)
External Funding for SoTL Projects
External funding usually takes place for larger-scale projects or multi-institutional projects. The following three agencies offer grants on educational topics—for more information on external funding, we suggest contacting the OU Office of Research Administration.
- Spencer Foundation
- Department of Education
- National Science Foundation
In addition to federal and foundation funding, some professional organizations offer smaller competitive grants for research on teaching and learning. Check with your professional organizations to see if such grants are available.
Research Ethics and Institutional Review (IRB)
SoTL researchers are bound by the same ethical obligations as other researchers conducting research on human subjects. SoTL research has additional considerations because the participants of your study are often also your students—this page describes some of the basics of the Institutional Review Board (IRB) process and why these research ethics matter.
Respect for persons
- That individuals be treated as autonomous agents
- That those with diminished autonomy (persons with disabilities, prisoners, etc.) are entitled to protection
- That participants have confidentiality and/or anonymity within studies (depending on the context)
- That individuals are treated in an ethical manner by making efforts to secure their well being. This means: doing no harm and maximizing possible benefits and minimizing possible harms.
- Injustice occurs when some benefit to which a person is entitled is denied without good reason or when some burden is imposed unduly
How the Belmont Report principles are applied are as follows:
Informed consent (respect for persons)
- Comprehension (do participants understand what they are being asked to do?)
- Voluntariness (are they volunteering without coercion?)
- Assessments of risks/benefits (do they understand the risks/benefits of the study?)
Assessment of Risks and Benefits
- Can the researcher clearly articulate the risks and benefit to society? Do the benefits outweigh the risks?
Selection of subjects
- Are participants selected out of convenience (prisoners, students) or out of what is best for the study?
IRB: The IRB (institutional review board) is a federally-mandated committee that performs reviews of research to ensure ethical treatment of human subjects. All research that takes place on campus that involves human subjects must be subject to review—including SoTL research.
All of the above information is covered in the CITI training course in much more detail; anyone submitting an IRB application must take the CITI training course prior to having his or her study approved by OU’s IRB.
In addition to the broad issues outlined by the Belmont Report, additional considerations for SoTL researchers should be considered:
Informed consent in studying your own class: One of the most challenging aspects of SoTL research is participant recruitment and the protection of participants. Because your own students are your participants, you must ensure that participation in your study is voluntary and that no negative consequences can occur for students who choose not to participate. The easiest way to address this critical ethical issue is to ensure that you do not know who consented to be in the study till after the term ends.
What many SoTL researches have done in the past on campus, and what has been approved by our IRB board in previous studies, is as follows. This particular approach makes the assumption that your SoTL work is built into the course itself and that you aren’t engaging in quasi-experimental work.
1) Teach the your class as normal
2) Towards the end of the term, take 5 minutes at the end of the class to explain the research and what you are trying to accomplish. I usually do this with a study “information sheet” that students can take home and read.
3) On a separate day, I ask in a 3rd party person (such as colleague or department administrative assistant, specified in the IRB application) to pass out and collect informed consent forms while I am not present in the room. The 3rd party person keeps consent forms in a secure location until after grades have been posted and the course concludes. In my last SoTL study, a department administrative assistant did this work since she also handles course evaluations in a similar manner; the IRB found this acceptable.There are reasons that you might want to gain informed consent for a smaller portion of a course; again, a 3 rd party person (co-author, colleague, etc.) would be most helpful in ensuring that students are not coerced and that students’ right to decline to participate and have no negative consequences is protected.
Privacy. De-identification is a way to ensure confidentiality with SoTL classroom research. After I receive consent forms back at the conclusion of the term, I will spend time giving each student who agreed to participate a pseudonym (qualitative analysis) or number (quantitative analysis) and de-identifying all of the data before any other analysis takes place. I keep one password protected “linking” file that does connect the data back to individual students in case I need it later. I usually let the data sit for a month or two, which generally has me forget the pseudonyms and then I can look at it with fresh eyes.
Comparison groups in Classroom Research. Hutchings (2003) and numerous others have discussed the issue of interventions and comparison groups in educational research. If you believe that a new pedagogical approach will be more effective than the old, is it ethical to teach one class with the new approach and one with the old approach and compare them? The answer isn’t always clear. One of the strategies suggested by Bishop-Clark and Dietz-Uhler (2012) is to compare across semesters (although there are also issues with this approach, especially in establishing meaningful comparisons if semesters are inherently different (i.e. if you get different kinds of students in the fall vs. the winter terms). Another approach is to use a pre-test/post-test method, where students are compared with themselves at the beginning and end of term. With this approach, a pre-test / pre-measure would be administered, then the intervention as part of the course, then a post-test / post-measure and students are compared to themselves and their growth. All of these comparison options have drawbacks and benefits; it is up to the SoTL researcher to decide what is the most ethical and reasonable comparison.
Student work. If you plan on using examples of any student work as part of your study (such as quoting student reflections or showing student feedback) make sure that students consent to have their work used in this manner. I add this into my consent forms, making a “blanket” statement about student work that covers these things—that way, if I want to end up using some student quotes, my students have already consented. I also usually ask them if they want their name associated with the work or if they prefer to be anonymous. Some students are proud of their work and prefer to have their name attached, so keep this in mind when designing your study.
“Normal Class Procedures.” Faculty are (hopefully) constantly changing their teaching approaches and updating their curriculum on a regular basis. A lot of what SoTL teacher-researchers do can be considered “normal class procedures” if those procedures have educational value and are built into the course in a way that enhances student learning. I’ll give two examples here of how I integrated SoTL research and my teaching. In one study, I wanted to do a pre / post measure using a transfer of learning survey inventory after doing a unit focused on the transfer of learning and building metacognitive strategies in my WRT320: Peer Tutoring in Composition course. I decided to build reflective writing around this inventory, and use it not just as an inventory (which would really only have research value), but as an online activity in the course, where students would take the inventory at the beginning of the term, then again at the end of the term, and then reflect on how their own perspectives on transfer of learning have changed across the term. Pedagogically, this provided the students with a great self-assessment tool and allowed them to chart their growth. Research-wise, it allowed me to see their growth and also gain access to their thinking about this change. I still use the inventory in this way, long after the research has been concluded.
In a second study, a larger research team and I were interested in understanding how metacognitive strategies described in reflective writing correlated with the quality of student writing. We built in a series of reflective prompts into our courses; the reflective writing served many positive functions within the class of encouraging students to be more self aware and consider their learning processes, while also allowing us to collect data on our study. Because both of these activities had as much value pedagogically as they did from a research perspective, I was able to justify their use to the IRB in a way that indicated that they were part of the normal operating procedures of the course (and again, I’m still using both in my classes long after the studies have concluded).
If you are able to plan your SoTL project thoughtfully, you can develop a course that allows you to enhance your students’ learning as well as collect meaningful data on your course. Most of this kind of work, then, falls into the IRB’s exempt category.
Submitting IRB Applications at OU: Applications at OU are submitted using IRBnet: http://www.oakland.edu/?id=24457&sid=177
Basic Human Subjects Training: All investigators are required to go through CITI training prior to conducting their research studies and submitting an IRBnet application. The CITI training provides you a basic knowledge of research ethics and is quite useful for developing studies in an ethical manner. You can sign up and do the CITI training here: https://www.citiprogram.org/Applications come in three levels of review: Exempt; Expedited; and Full review. Most SoTL research falls into either exempt (normal class procedures and educational data) or expedited (if collecting recordings). Please review the IRB guidelines for full guidelines on the different categories of research.
Direct measures: When assessments are based students’ actual performances or demonstrations of knowledge attained, it is said to be a direct measure of student learning. This might include performance on portfolios, written assignments, tests, demonstrations, etc. Whether or not that measure is reliable or valid is a concern.
Eliminating Bias: Bias is a critical issue for SoTL researchers, especially when you know the students well—if you are studying your own class, and you want to know if students have learned, you need to be careful to eliminate any potential bias in your work. For example, if you want to look at student papers to find out if students have improved from their first research paper to their last research paper, if you grade all papers and compare the grades, you might inadvertently be biasing your results. An alternative is remove identifying information from the papers, give them to several colleagues with a rubric, and see how they rate them.
Formative Assessments: Formative assessments are assessments/measures gathered during a course or unit to understand where learners are at a particular point in time. The goal of formative assessments is to improve a program/course/approach as it is ongoing for the set of learners undergoing the assessment. It can also be useful for studying student learning over time.
Indirect measures: An indirect measure is when assessments or research is based on students’ reactions to and self-reported discussions of learning. Self-reports are quite useful for certain kinds of data collection (see above) but often should be combined with some direct measures (depending on the goal of your study).
Qualitative: Data that is interpretative or descriptive in nature is qualitative. Qualitative data lends itself to smaller-scale work where understanding a smaller group of students is more useful. It is also useful for pilot and exploratory work.Quantitative: Data that is numerical in nature is quantitative. This kind of data lends itself to larger-scale work, research designed to generalize beyond the immediate population.
Reliability: Reliability refers to the degree to which a study, learning assessment tool, survey, test, etc. produces consistent results over time. For example, if one were using a rubric to study the improvement in student writing ability, confusing areas on the rubric that could not be consistently applied would be a threat to reliability.
Rubric: A rubric is a tool used for scoring student work. It generally has categories based on course learning objectives and has descriptions of unacceptable and acceptable features of the student work. Rubrics can be used by SoTL scholars for a variety of research purposes.
Summative Assessments: Summative assessments are assessments/measures gathered at the conclusion of a course or unit to understand what learners have learned. They are usually used to understand the effectiveness of the course so that it can be adapted for future students.
Transferability: Transferability is a term used in qualitative research to describe the applicability of the study to other contexts (not to be confused with transfer of learning, which refers specifically to students’’ ability to apply/adapt learning to new circumstances beyond a course).Validity: External validity refers to the generalizability of the findings; in that they are valid beyond the sample/course/set of students studied. Internal validity refers to how accurate or meaningful the results are, and whether or not the results are actually measuring what the study set out to measure. If I ended up using a rubric that wasn’t actually measuring what I was hoping to find, it would be a threat to internal validity.
When conducting an SoTL study, there are a number of considerations you can make regarding the kinds of data and measurements you collect.
Existing Data. A great deal of data that universities collect on students may be available for research purposes (this data is usually reported through the Institutional Research office or though specific programs and services). This includes responses to surveys like the National Survey of Student Engagement, grade distributions, and graduation information, and so forth. With an appropriate IRB approval, you can often get access to other kinds of grade or course-taking data. Existing data can be useful for:
- Academic performance and course-taking patterns
- Graduation and attrition rates
- Comparison of before/after program changes
- Understanding of student population and demographics
Oakland University’s Office of Institutional Research and Assessment provides a wealth of data about students on their website. This includes:
- Quick facts
- OU Data book with information on enrollment, grade distribution, attrition, and graduation
- IPEDS reports
- The National Survey of Student Engagement
- The CRIP Survey for incoming freshmen students
- The Collegiate Learning Assessment (this was the test discussed in Arum and Roksa’s Academically Adrift)
Self-reported Data. Self-reported data is data that students self-report and is useful for understanding reactions to a course, beliefs, values, experiences, reflections on learning, etc. SoTL researchers also recognize that while self-reported data is useful, it can be biased or inaccurate. It's a best practice idea to combine self-reported data with other kinds of data (including direct measurements).
1) Self reported data includes:
- Reflective Writing / student feedback
- End-of-Semester Comments/Evaluations
- NSSE Surveys and other national surveys
Portfolios: Portfolios are a common way that student learning can be assessed; these collections of student work are usually accompanied by reflective writing where students discuss their learning processes. For research or assessment purposes, portfolios are typically read by a group of experts or trained raters using a rubric. Raters in larger studies or assessments are usually trained in advance to ensure reliability.
Tests or Assessments (especially pre-post). Tests of students’ knowledge is another common assessment of student learning. Standardized tests or instruments, such as the Collegiate Learning Assessment, provide very reliable measures of student learning (although there have been serious issues raised about the external and internal validity of such measures). Instructor-created tests are also a very common approach to engaging in SoTL research.
Written Work or Projects: Many SoTL researchers use a variety of written projects or work to assess student learning.
Oral Presentations: Another form of assessment that can be assessed using a rubric.
Observations and Performance: If students are engaged in performance-based learning (such as surgical operations, dance, teaching) observations of student activity is another way to measure student learning.
Research Tools for the Scholarship of Teaching and Learning
One of the things that can greatly aid a Scholarship of Teaching and Learning (SoTL) researcher are various research tools that allow for faster and more accurate analysis and interpretation of data. This section covers some of the common research tools that are useful for SoTL research. Some of these tools are freely available, while others have minimal or substantial costs. Some schools or colleges at OU provide the tools, while others do not—please inquire within your department and/or school/college for availability.
Survey programs are one of the simple ways to make SoTL research easier. You can use survey programs for classroom surveys, instructor surveys, and pre- and post-surveys in SoTL research.
- Google Forms: Google Forms is a free service (found through Google Drive - https://drive.google.com). It allows you to create a simple form, in the form of a survey or questionnaire, and distribute that form via an email link. It does not have any kind of sophisticated analysis software (so in this case, what you pay is what you get). It is best used for simple surveys, or surveys that are qualitative in nature and require additional analysis either way.
- Survey Monkey (www.surveymonkey.com). Survey monkey is probably the most well known of the survey tools. The free version will allow you up to 100 responses and 10 questions. The paid versions (starting at $24 monthly billed per month) allow for more complex surveys, more questions, and unlimited responses. The program also allows for basic calculation of responses.
- Qualtrics (www.qualtrics.com). Qualtrics is an advanced survey tool, and my preferred choice. Qualtrics allows for you to track individual responses (such as sending surveys out to a class list and seeing who responds). This is particularly useful for pre/post survey designs (otherwise, you are relying on student-generated numbers, which have their own problems) Qualtrics also has sophisticated analysis software that is quite easy to use.
The options for analyzing quantitative data are largely based on how complex your quantitative analysis is and what programs you have access to. Most of these programs (save Excel) are quite expensive, although sometimes schools/colleges provide packages to faculty.
- Microsoft Excel: Excel is a really great tool for data management, data input, data cleaning, and so forth. I usually start with my data in Excel just to look at it, play with it a bit, etc. Then if I need to do any kind of analysis, I’ll take it to SPSS.
- Statistical Package for the Social Sciences (SPSS): This is probably the most well-known of the software packages (and one they teach you in grad school quant methods classes). A few departments and schools on campus purchase licenses. It allows for all of the advanced inferential statistics calculations (depending on the version you purchase), and can calculate descriptive statistics quite quickly compared to Excel.
- SAS: SAS is similar to SPSS in terms of quantitative analysis.
Qualitative analysis can be done by hand, but a program can greatly aid you in terms of seeing big patterns, managing data, training coders (if you are doing larger-scale research) and so forth. Even for something as simple as an interview analysis, I find that using a program is superior to doing it by hand.
- Microsoft Word. For qualitative coding, MS word (particularly, the “comment” feature) can be used to code student writing, interviews, and so forth. You’ll have to do a lot of analysis by hand (e.g. how many times did X code come up?) but it is free. I wouldn’t use this for large projects, but I sometimes use it to develop qualitative codes.
- Dedoose (Deddose.com). Dedoose is my favorite program to use for one simple reason – its incredibly cheap compared to other options, and it is web-based so it can be shared across researchers (which is great for collaborative projects). Dedoose also does some mixed methods coding, and allows for mixed methods analysis, which I really like. It does have its quirks, but it is far superior to Word, doing things by hand, etc. It runs about $10-$12/month and offers a free 30 day trial.
- Nvivo. The premier program for qualitative analysis is Nvivo. I have limited experience with this program, but my experiences with it suggest that if you are serious about qualitative research and plan to do a lot of it, Nvivo is a good option.
- Odesk.com: I, along with most other researchers I know who engage in interview work, use Odesk.com. It is a site that links up independent professionals with people who need a job done (like transcription).
- Research Assistants: The other option for transcription that many people use is to have a research assistant do that work (assuming you have one, which isn’t always the case).
I only started using reference management tools a few years ago, and I really wish I had found them sooner! They organize your references, allow you to find those pesky PDFs you’ve lost, and can convert across citation styles quickly. The two most common reference management tools are Endnote and Refworks. Refworks is free through the OU library.
| Oakland University's Second Instructional Fair|
Learn from other faculty an effective instructional strategy, active learning technique, classroom activity or assessment that you use to promote student success. Each OU faculty creates a display or interactive demonstration, along with a one-page handout, of an effective strategy in order to discuss and share ideas with colleagues across the disciplines.
We had a successful Instructional Fair on Wednesday, January 21, 2015. We thank our presenters and participants for exchanging ideas. See below for handouts from each faculty presentation.
Download the 2015 Instructional Fair Packet.
Review the 2013 Instructional Fair Packet.
"Comics and Content," Greg Allar
"Problem-Based Assessment," Greg Allar
"Service Learning," Rachel Smydra
"Instructional Design for Motivation," Misa Mi
"Thesis Hunt," Melissa St. Pierre
Tech Tools and Online Learning Strategies
"Flipped Classroom," Lynda Poly-Drullard and Barb Penprase
"Panopto," Laura Dinsmoor
"Wiki Way Collaboration," Stephen Loftus
"Real Time Systems Course," Subramaniam Ganesan
"Cyber Course: Better Skill Learning," Kieran Mathieson
"Extra Credit to Save Your Life," Kathleen Spenser
"Show Me Tutorials," Kathleen Doyle
"Course Pages," Beth Kraemer
"Library Tutorials," Beth Kraemer
"Widget for Moodle," Mariela Hristova
"Copyright Course," Julia Rodriguez
"Plagiarism Course" Julia Rodriguez
Excellence in Teaching
Register to Share Your Experience in the OU Classroom
Just one hour, and lunch is on us.
Please register for the Winter 2016 sessions to join us for this very important Students for Excellence in Teaching Forum (SET). Participation is limited to 22 so please register early.
Fall 2016 dates:
|Strengths and Issues about OU Courses and Instructors||Tuesday, Oct. 25||Register|
|Student Perspective of OU's Course Evaluations||Tuesday, Nov. 22||Register|
What do professors do to facilitate your learning? What approaches, strategies, techniques work? What engages you? What would you like to see more of in your courses? What helps? What hinders? What do you see as problems in classes?
The Director of the Center, Judy Ableser, would like to meet with students monthly to get your input. Students for Excellence in Teaching Forum (SET) is a time to hear about the wonderful approaches to teaching and learning that is occurring on campus and to hear and share concerns, suggestions, and recommendations for improvements. It is not a forum to talk about individual poor professors or courses (that is important, however, that should be addressed at the department level or through Student Services).
The Center for Excellence in Teaching and Learning (CETL) provides faculty support to enhance the quality of teaching excellence across campus. Although we are here to provide support to faculty, our main goal must be to support student learning, and therefore, our students are truly our significant stakeholders. We want to hear from you. Your voice is essential to our work.
Looking forward to hearing your voice!
Judith Ableser- Director
Center for Excellence in Teaching and Learning
Each year the Oakland University Senate Teaching and Learning Committee and the University Research Committee honor faculty whose teaching and research efforts have achieved singular praise or recognition. These individuals receive the Teaching Excellence Award, Research Excellence Award, New Investigator Research Excellence Award or Excellence in Teaching Award for their work. Recipients are chosen based on nominations submitted by Oakland Students, faculty and staff, student letters of recommendation and materials provided by the nominee.
|Excellence in Teaching Award|
Helena Riha, Ph.D.
Department of Linguistics
Dr. Helena Riha graduated with an M.A. in Chinese Linguistics and a Ph.D. in Linguistics from The Ohio State University. She is bilingual in English and Slovak and fluent in Chinese, bringing an international perspective to her teaching. Dr. Riha has taught eleven undergraduate and graduate courses at OU in Linguistics and International Studies. Over 2,000 students have taken her on-campus and online courses. Dr. Riha regularly invites guest speakers to small classes to promote discussions between students and experts. Eleven OU faculty and graduate students have visited her classes to discuss a variety of language-related topics. Dr. Riha also developed interviews with experts online, a learning activity on which she published scholarly articles. She employs a personalized approach in her large campus classes as well. Dr. Riha developed daily exercise sheets for her 80-student General Education course to help students engage with the material. Her students describe her as dedicated, clear, and methodical. In the words of one student, Dr. Riha is "one of the best teachers at OU -- prompt, thorough, and dynamic."
|Teaching Excellence Award|
Anthony Tuf Francias, Ph.D.
Assistant Professor of Teacher Development and Educational Studies
Anthony Tuf Francis, Ph.D. is an Assistant Professor of Teacher Development and Educational Studies at Oakland University and is lead faculty of the history and social studies secondary program. He received his B.Ed. from the University of Toledo in 1996 and taught history and social studies in Toledo Public Schools from 1997 – 2007, during which he received awards for teaching excellence, co-authored articles on urban education, and gave speaking engagements about relating to students and addressing the needs of diverse learners. He received his M.Ed. in Curriculum and Instruction from Bowling Green State University in 2005, and his Ph.D. in Teacher Education from the University of Michigan in 2013. His contributions to teaching and learning at Oakland University have been premised on two foundational standards for his own teaching: (1) There should never be a moment in any of his classes that students feel are not worth their attention, time, and money; and (2) Students should always learn in the same environments, experiencing the same methods, that they are learning to use in their own classrooms.
Share Your Publication
Ableser, J. (2012). Exemplary teaching practices across educational contexts (P–20+): Unifying principles and an ecological model for teaching for all to learn. Journal of Teaching and Learning, 8(2). Retrieved from http://ojs.uwindsor.ca/ojs/leddy/index.php/JTL/article/view/3133/pdf.
Ableser, J. (2011). Teaching for all to learn: Exemplary practices from preschool to graduate school. The Scholarship of Teaching, 22-26. Retrieved from http://umflint.edu/tclt/SOT_FA2011_8_24FINAL.pdf#page=24
Arya, P., Christ, T., & Chiu, M. M. (2015). Links between characteristics of collaborative peer video analysis events and literacy teachers’ outcomes. Journal of Technology and Teacher Education, 23(2), 159-183.
Arya, P., Christ, T., & Chiu, M. M. (2015). TAP (Teacher learning and application to pedagogy) through digital-video mediated reflections. In H. Gillow-Wiles & M. Niess (Eds.), Handbook of Research on Teacher Education in the Digital Age (pp. 334-356). Hershey, PA: IGI Global. Link to chapter
Arya, P., Christ, T., & Chiu, M. M. (2013). Facilitation and teacher behaviors: An analysis of literacy teachers’ video-case discussions. Journal of Teacher Education, 65(2), 111-127. http://jte.sagepub.com.huaryu.kl.oakland.edu/content/early/2013/11/26/0022487113511644.full
Arya, P., & Christ, T. (2013). An exploration of how professors' facilitation is related to literacy teachers' meaning construction process during video-case discussions. Journal of Reading Education, 39(1), 15-22.
Carver, C. L. & Klein, C.S. (2013, October). Action research: A tool for promoting faculty development and continuous improvement in leadership preparation. NCPEA International Journal of Educational Leadership Preparation, 8(2). Retrieved from ww.ncpeapublications.org/
Christ, T., Arya, P., & Chiu, M. M. (2015). A three-pronged approach to video reflection: Preparing literacy teachers of the future. In E. Ortlieb, L. Shanahan, & M. McVee (Eds.), Video Reflection in Literacy Teacher Education and Development: Lessons from Research and Practice. Bingley, UK: Emerald. Link to chapter
Christ, T., Arya, P., & Chiu, M. M. (2014). Teachers’ reports of learning and application to pedagogy based on engagement in collaborative peer video analysis. Teaching Education, DOI: 10.1080/10476210.2014.920001 http://www.tandfonline.com.huaryu.kl.oakland.edu/doi/abs/10.1080/10476210.2014.920001
Christ, T., & Wang, X. C. (2013). Exploring a community of practice model for professional development to address challenges to classroom practices in early childhood. Journal of Early Childhood Teacher Education, 34(4), 350-373. http://www.tandfonline.com.huaryu.kl.oakland.edu/doi/full/10.1080/10901027.2013.845630
Christ, T., Arya, P., & Chiu, M. M. (2012). Collaborative peer video analysis: Insights about literacy assessment & instruction. Journal of Literacy Research, 44, 171-199. http://jlr.sagepub.com.huaryu.kl.oakland.edu/content/early/2012/03/28/1086296X12440429.full.pdf
Christ, T., Arya, P., and Chui, M.M. (2012, March 29). Collaborative peer video analysis: Insights about literacy assessment and instruction. Journal of Literacy Research, 44(2). doi: 10.1177/1086296X12440429.
Coon, J. L. (2013). How other nations approach reading and writing. In A. S. Horning., E.W. Kraemer (Eds.)
Reconnecting Reading & Writing. (pp. 55-68). Anderson, SC: Parlor Press. WAC
Dadashzadeh, M. (2010, Winter) Consuming web services: A Yahoo! newsfeed reader. Journal of Information Systems Education, 21(4). Retrieved from http://search.proquest.com/docview/849236603?accountid=12924
Dadashzadeh, M. (2007). Specification and enforcement of semantic integrity constraints in Microsoft access. Journal of Information Systems Education, 18(4), 6. Retrieved from http://search.proquest.com/docview/200119793/fulltextPDF/1347C0C2A4958213D81/3?accountid=12924
Daniel, D. (2012, February). Teaching students how to research the past: Historians and librarians in the digital age. The History Teacher, 45(2). 261-282. Retrieved from http://huaryu.kl.oakland.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=eft&AN=72954404&site=ehost-live
Driscoll, D. L. Connected, disconnected, or uncertain: Student attitudes about future writing contexts and perceptions of transfer from first year writing to the disciplines. Retrieved from http://wac.colostate.edu/atd/articles/driscoll2011/index.cfm
Estes, T. (2007). Constructing the syllabus: Devising a framework for helping students learn to think like historians. The History Teacher, 40(2). Retrieved from http://www.historycooperative.org/journals/ht/40.2/estes.html
Folland, S. (2011). When America’s colleges don’t teach and students don’t learn: Two recent books reviewed [Review of the books Academically adrift and Higher education? How colleges are wasting our money and failing our kids—and what we can do about it]. Oakland Journal, (21). Retrieved from http://www2.oakland.edu/oujournal/archive.cfm
Ganesan, S., Ding, X., & Rusek, A. (2016). Teaching real time system scheduling using low cost microprocessor board. Retrieved from: http://people.cst.cmich.edu/yelam1k/asee/proceedings/2016/faculty_regular_papers/2016_ASEE_NCS_paper_3.pdf
Hoag, M. (2013). Hearing “what might have been”: Using recomposition to foster music appreciation in the theory classroom. Journal of Music Theory Pedagogy, 27, 47-70. Retrieved from https://drive.google.com/file/d/0B0ZI8di-pEDvTzRLaFBDWFZwRjQ/view
Hoag, M. (2013). Seven strategies for enabling success in the first-year music theory sequence. Journal of Music Theory Pedagogy-Online. 1, 1-17. Retrieved from https://drive.google.com/file/d/0B0ZI8di-pEDvV0FqLW10cmNpckE/view
Ingram, I., & Flumerfelt, S. (2009). Ethical development and diversity training for educational leaders. School Leadership Review, 4(2), 55-78.
Kahlen, F., Flumerfelt, S., Siribang-Manalang, A., & Alves, A. (2011). Benefits of lean teaching. Proceedings of the Annual International Mechanical Engineering Congress & Exposition, IMECE2011-63150, Denver, Colorado, USA. 1-7.
Loftus, S. (2015) Understanding integration in medical education. Medical Science Educator, 25, 357–360 Special edition on integration. doi:10.1007/s40670-015-0152-4. Retrieved from http://link.springer.com/article/10.1007/s40670-015-0152-4/fulltext.html
Mathieson, K. (2016, Jan 23). Is scholarly teaching worth rewarding? Higher Education Revolution. Retrieved from https://higheredrevolution.com/is-scholarly-teaching-worth-rewarding-19c66edc5c3d#.adh8pjysk
Michel, R. M. (2014). Finding the SurPriSe: A case study of a faculty learning community. Academic Leadership Journal in Student Research, 2. Retrieved from http://www.academicleadership.
Moore, C. (2016). The future of work: What Google shows us about the present and future of online collaboration. TechTrends, 60(3): May/June 2016. 233-244. doi: 10.1007/s11528-016-0044-5 Retrieved from http://link.springer.com/article/10.1007/s11528-016-0044-5
Moore, C. (2016 June 6). Frame your feedback: Making peer review work in class. Faculty Focus. Retrieved from http://www.facultyfocus.com
Murphy, A. "Enhancing the motivation for good teaching with an improved system of evaluation.” Financial Practice and Education, 9(Fall/Winter 1999), 100-104.
Oakley, B., Felder, R. M., Brent, R., & Elhajj, I. (2004). Turning student groups into effective teams. Journal of Student Centered Learning, 2(1), 9-34. Retrieved from
Riha, H. (2011, September 27). Online student forums with experts. Educause Quarterly, 34(3). Retrieved from http://www.educause.edu/ero/article/online-student-forums-experts
Riha, H., Berven D., Joseph, B.D., and Winford, D. (2010). Forums with experts as a way to teach sociolinguistics online. American Speech, 85(2). 238-250. Retrieved from http://americanspeech.dukejournals.org/content/85/2/238.full.pdf
Rusek, A. & Ganesan, S. (2016). Teaching time domain reflectometry in EMC course. Retrieved from: http://people.cst.cmich.edu/yelam1k/asee/proceedings/2016/faculty_regular_papers/2016_ASEE_NCS_paper_2.pdf
Sidelinger, R. J., Bolen, D. M., Frisby, B. N., & McMullen, A. L. (2011). When instructors misbehave: An examination of student-to-student connectedness as a mediator in the college classroom. Communication Education, 60(3), 340-361. Retrieved from http://www.tandfonline.com/doi/abs/10.1080/03634523.2011.554991
Wendell, D. L., & Pickard, D. (2007). Teaching human genetics with mustard: Rapid cycling brassica rapa (fast plants type) as a model for human genetics in the classroom laboratory. CBE-Life Sciences Education, 6(2), 179-185. Retrieved from http://w.lifescied.org/content/6/2/179.short
Wiggins, J. (2001). Teaching for musical understanding McGraw-Hill Humanities/Social Sciences/Languages.
Wiggins, J. (2011). Vulnerability and agency in being and becoming a musician. Music Education Research, 13(4), 355-367. Retrieved from http://www.tandfonline.com/doi/pdf/10.1080/14613808.2011.632153