Inclusive Assessment

What is Assessment? 

Assessment is an iterative, methodical, and participatory process designed to strengthen teaching and learning practices (Suskie, 2009). Just as doctors deliver treatments in consultation with medical data about patients, educators deliver meaningful learning experiences in consultation with assessment data about students. 

Predominantly white institutions have traditionally assessed diversity, equity, and inclusion (DEI) through student outcomes, such as enrollment numbers and graduation rates of students of color, instead of assessing faculty outcomes, such as classroom teaching informed by promising practices (Bensimon, 2007). Moreover, student evaluations represent the most prevalent means to assess classroom teaching despite research revealing the biased and unreliable nature of such tools (Uttl, White, & Gonzalez, 2017), with disproportionately negative feedback about women and people of color (Pratt, 2015; Schmidt, 2015). Even when professors receive student evaluations after the course has ended, few structured opportunities exist to adjust pedagogical practices that ensure high-quality, equitable student learning opportunities. 

When we harness assessment as an educational tool, there are infinite possibilities for amplifying DEI content in curricula, strengthening inclusive teaching practices in the classroom, and cultivating cultures that support DEI for students, staff, and faculty. Together, we can explore ways to assess DEI possibilities that go beyond numbers and biases so we create robust and equity-oriented learning ecologies. 
 

two students in chairs with papers in their laps

Why Consider Assessment in DEI?

  • Kindles potential transformative learning opportunities for students, staff, and faculty to become agents of change and agents of curiosity (Freire, 2014).  
  • Continually strengthens pedagogical possibilities for students, staff, and faculty to interact with DEI intellectually, interpersonally, and intrapersonally.   
  • Creates curricular cohesion for academic units to articulate significant learning outcomes that builds a unified identity of their degreed program. 
  • Fosters interdisciplinary partnerships for academic units to expand the spectrum of learning experiences through a constellation of touch points across disciplines. 
  • Supports recruitment, retention, and persistence of all students, staff, and faculty, particularly people from economically and politically vulnerable communities.  
  • DEI learning outcomes help institutions create culturally engaging campus environments (Museus, Yi, & Saelua, 2016) that shape how professors design and teach curricula; staff design and lead co-curricular initiatives; students know the arc of their academic trajectories; and employers explore the repertoire of knowledge, values, and skills that students developed from their educational experiences (Melguizo & Coates, 2017)
     
intense conversation

Key Tenets

  • Assessment is a process that can help educators optimally and critically transform cultures and pedagogical practices in their classrooms, curricula, departments, units, and institutions. 
  • Curricula, teaching practices, students, staff, faculty, policies, and unwritten norms shape cultures of teaching and learning (Cole & Engeström, 2007). 
  • Trust, transparency, and appreciative inquiry are foundational for cultural change (Whitney & Trosten-Bloom, 2003) so that students, staff, and faculty can use their power and voices to transform their realities and the world around them (Freire, 2014). 
  • Re-imagining and re-creating the world as it could be (Boal, 1995) is possible when we can engage in assessment and critical reflection. 
  • Institutions can harness the transformative potential of assessment when the onus and support of DEI is distributed across governing bodies, institutional leadership, department chairs, students, staff, and faculty. 
     

Definitions

  • pencil drawing on line graph

    Assessment

    A participatory, iterative, and systematic process for the collection and analysis of evidence about the achievement of outcomes for the purposes of continuous improvement (adapted from the Higher Learning Commission and Case Western Reserve University). 

  • grad cap

    Student learning outcomes (SLOs) 

    Statements that articulate what students will know, value, and do at strategic time points:

    1. completion of a course for course-level SLOs, or
    2. completion of a degree for program-level SLOs, or
    3. completion of general education courses for institution-level SLOs. Some professional organizations may refer to these with different terms, such as objectives, indicators, abilities, or competencies 

    (adapted from Case Western Reserve University and NILOA, 2020). 

  • check list

    Direct assessment 

    Measures of learning based on student performance or demonstrations of the learning itself. Scoring performance on exams, papers, portfolios, presentations or the execution of lab skills exemplify direct assessment of learning (adapted from NILOA, 2020). 

  • person presenting venn diagram to crowd

    Indirect assessments 

    Perceptions or attitudes to make inferences about student learning. Surveys, self-reflections, interviews, and focus groups are all indirect evidence of learning (adapted from Carnegie Mellon University Eberly Center for Teaching Excellence & Educational Innovation). 

  • two people with speech bubbles

    Formative assessment 

    A process designed to give ongoing feedback over the course of an intervention. Faculty can use formative assessment to make changes midstream and in situ instead of making final decisions (Allen, 2004).

  • checklist icon

    Grammar

    While grammar traditionally describes the system and structure of using language in varied contexts, we expand grammar to describe the framework––practices, activities, and principles––of using assessment in resonance with diversity, equity, and inclusion (DEI) contexts (Paguyo & Nosaka, 2018). 

  • Venn diagram

    Summative assessment 

    A comprehensive measure of achievement at the end of an instructional unit or course of study that is used for decision-making purposes (adapted from Case Western Reserve University and NILOA, 2020). 

  • gears

    Rubrics

    Tools that describe the performance expectations for an assignment or piece of work. Teams of faculty experts from postsecondary educational institutions across the country have developed VALUE rubrics related to intercultural knowledge and competence and global learning (adapted from AAC&U). 

  • computer icon

    Online Assessment

    Online assessment is the iterative and systematic process of collecting and analyzing deliverables from students to reflect upon how faculty are teaching and how students are learning in online environments. In addition to exploring student performance, online assessments consider a multitude of dimensions, such as attitudes toward e-learning platforms and technologies (Hewson & Charlton, 2019); access to sufficient environments and materials for teaching and learning (e.g., solid internet connectivity and updated softwares); and supportive living environments that create sustained emotional, cognitive, physical, and financial stability (Paguyo & Iturbe-LaGrave, 2020). Online assessment helps faculty pivot and adapt their online pedagogies to facilitate robust student learning and engagement. 

Recommended Articles

student graduating with honors

The Impact of Culturally Engaging Campus Environments on Sense of Belonging

Low rates of student persistence and degree completion are a major concern of colleges and universities across the United States. Of all incoming students enrolled at four-year institutions in 2005, less than 60% completed their bachelor’s degree within six years.

Read the Article
student with hand raised

The Value of Assessing Higher Education Student Learning Outcomes

Improving higher education student learning plays a critical role in advancing human capital. Assessment lies at the heart of such improvement, underlining the need for research and innovation in this field. 

Read the Article
  • References

    Bensimon, E. M. (2007). The underestimated significance of practitioner knowledge in the scholarship of student success. The Review of Higher Education, 30(4), 441-469.

    Boal, A. (1995). Rainbow of desire. New York: Routledge.

    Cole, M., & Engeström, Y. (2007). Cultural-historical approaches to designing for development. In J. Valsiner & A. Rosa (Eds.), The Cambridge handbook of sociocultural psychology (pp.484-507). New York: Cambridge University Press.

    Freire, P. (2014). Pedagogy of the oppressed: 30th anniversary edition. New York: Bloomsbury Publishing.

    Hewson, C, & Charlton, J.P. (2019).  An investigation of the validity of course‐based online assessment methods: The role of computer‐related attitudes and assessment mode preferences. Journal of Computer Assisted Learning, 35, 51-60.  

    Melguizo, T., & Coates, H. (2017). The value of assessing higher education student learning outcomes. AERA Open, 3(3), 1-2.

    Museus, S.D., Yi, V., & Saelua, N. (2016). The impact of culturally engaging campus environments on sense of belonging. The Review of Higher Education, 40(2), 187-215. 

    Pratt, C.D. (2015). Teacher evaluations could be hurting faculty diversity at universities. The New York Times. 

    Schmidt, B. (2015). Is it fair to rate professors online? The New York Times. 

    Suskie, L. (2009). Assessing student learning: A common sense guide. (2nd ed.). San Francisco, CA: Jossey-Bass.

    Uttl, B., White, C.A., & Gonzalez, D.W. (2017). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teacher ratings and student learning are not related. Studies in Educational Evaluation, 54, 22-42.  

    Whitney, D., & Trosten-Bloom, A. (2003). The power of appreciative inquiry: A practical guide to positive change. (2nd ed.). San Francisco, CA: Berrett-Koehler Publishers, Inc. 

full auditorium with hands raised

Teaching Tools

student speaking in microphone at pride event

Proposed Syllabus Statement about Assessment

In this class, we will work collaboratively to create space for everybody to learn, regardless of student, teaching apprentice, or professor roles. With its commitment to quality education, the University uses assessment as a tool for professors to design and deliver inclusive learning environments. Some assessments help professors learn how to make teaching pivots in the moment so they can immediately respond to your needs. Other assessments help students learn how to engage more deeply with the course content. Some assessments feel less like completing an assignment or less like taking a test, but seem more like engaging in an activity. Some assessments are graded, other assessments are not graded, but all assessments — whether they come in the form of minute papers or exams or responses to a discussion prompt — are designed for robust learning. 

Students studying

Assessment Guidelines

 (adapted from Thomas, Iturbe-LaGrave, Paguyo, & Swanson, 2020; Ekowo and Palmer, 2016; and Weston, 2017) 

  • Develop shared understanding of purpose and outcomes of assessment at multiple strata of an institution: course, program, department, collegiate, and institution. 
  • Create transparent guidelines about how data will be collected, when data will be collected, who has access to the data, and how quality analysis will be conducted to inform assessment and continuous improvement. 
  • Raise awareness about the potentially harmful effects of suboptimal assessment processes, particularly when biases inform inappropriate assessment practices and interpretations regarding people from economically and politically vulnerable communities. 
  • Develop a supportive infrastructure that makes explicit how assessment will guide organizational change. 
  • Never use assessment to inform teaching evaluations, conduct public shaming, or monitor people through big brother-type approaches. 
  • Always use assessment as a tool and a process for continually strengthening how to create classes, programs, departments, colleges, and institutions to enact DEI. 
     

Blueprint for Assessment

The following table highlights assessment tools that faculty and staff can use to practice inclusive assessment in different DEI dimensions.

 
Dimension of DEI Assessment Tools

 

Curriculum + Strategic Plan 
  • Conduct audit of DEI topics and representation of authors covered in the curriculum
  • Alignment with institutional student learning outcomes related to DEI 
  • Alignment with accreditation guidelines related to DEI 
  • Alignment with strategic plans related to DEI at the institutional and college-specific level 
Pedagogical
  • Critical Incident Questionnaire (Brookfield, 2008) 
  • Midterm Student Feedback (Tollefson, 2005) 
  • Minute Paper (Tufts University Center for the Enhancement of Learning & Teaching) 
  • Peer Observation of Teaching (University of Toronto Centre for Teaching Support and Innovation, 2017) 
  • DU Inclusive Teaching Practices Faculty Video-Consultation Protocol 
  • Alignment with course design practices to promote DEI 
  • Conduct audit of DEI topics and representation of authors covered in the syllabus
  • Document ITP trainings, FLCs, webinars, and conferences attended for professional development 
Students
Staff and Faculty
Infrastructure 
  • Conduct audit of whether DEI counts in policies for advancement, promotion, and tenure
  • Conduct audit of whether DEI exists in faculty and staff recruitment and hiring policies 
  • Conduct audit of whether departmental support is available for faculty and staff to engage in DEI development (time, stipends, letters of support, etc.)

 

  • References

    Allen, M.J. (2004). Assessing academic programs in higher education. Bolton, MA: Anker Publishing Company, Inc. 

    Association of American Colleges and Universities (AAC&U). (2009). Intercultural knowledge and competence VALUE rubric. Retrieved from https://www.aacu.org/value/rubrics/intercultural-knowledge

    Bensimon, E.M., Dowd, A.C., & Witham, K. (2016). Five principles for enacting equity by design. Retrieved from https://www.aacu.org/value/rubrics/intercultural-knowledge. 

    Brookfield, S. (2003). Understanding classroom dynamics: The Critical Incident Questionnaire. In, S.N. Bernstein (ed.). Teaching developmental writing: Background readings. (2nd ed.). Boston, MA: Bedford/St. Martin’s Press. 

    Center for the Enhancement of Learning & Teaching. (n.d.) CELT tip sheets: The minute paper. Tufts University. Retrieved from https://provost.tufts.edu/celt/files/MinutePaper.pdf

    Depaul Teaching Commons. (2020). Teaching observations. Retrieved from https://resources.depaul.edu/teaching-commons/teaching-guides/reflective-practice/Pages/teaching-observations.aspx

    Ekowo, M., & Palmer, I. (2016). The promise and peril of predictive analytics in higher education: A landscape analysis. New America Education Policy. Retrieved from https://www.newamerica.org/education-policy/reports/predictive-analytics-in-higher-education/

    Higher Education Research Institute (HERI). (2020). Overview of sureys. Retrieved from https://heri.ucla.edu/overview-of-surveys/

    Hoadley, C. (2016). Online pedagogy from the learning sciences perspective. In C. Haythornthwaite, R. Andrews, J. Fransman & E. M. Meyers (Eds.), The SAGE Handbook of E-learning research (2nd ed., pp. 25-42). Thousand Oaks, California: SAGE. doi:10.4135/9781473955011.n2. 

    MTV. (2020). Take action to stop discrimination and bias at the resources below. Retrieved from https://www.mtvact.com/features/Look-Different      

    Paguyo, C.H., & Nosaka, T. (2018). Practitioners and researchers engaging in layers of learning: Researching and designing interventions in postsecondary education through the ballast of equity. In E. Mendoza, B. Kirshner, & K. Gutiérrez (Eds.), Designing for equity: Bridging learning and critical theories in learning ecologies for youth (149-166).   

    Project Implicit. (2020). Implicit social cognition. Retrieved from https://www.projectimplicit.net/

    Thomas, D., Iturbe-LaGrave, V., Paguyo, C.H., & Swanson, K. (2020). Teaching in a data-rich online environment: Creating a culture that enriches and empowers faculty. Retrieved from https://otl.du.edu/teaching-in-a-data-rich-online-environment-blog-series-creating-a-culture-that-enriches-and-empowers-faculty/

    Tollefson, S. (2005). Sample midterm evaluations. Office of Educational Development. University of California, Berkeley. Retrieved from https://teaching.berkeley.edu/sites/default/files/sample_midterm_evals.pdf

    University of Toronto Centre for Teaching Support and Innovation. (2017). Peer observation of teaching. Retrieved from https://teaching.utoronto.ca/wp-content/uploads/2017/01/Peer-Observation-of-Teaching-Guide.pdf.

    Weston, D. (2017). From Data Rich to Data Smart: Empowering Teaching, not Monitoring Teachers. In Lucy Rycroft-Smith, Jean-Louis Dutaut (Eds.) Flip the System UK: A Teachers’ Manifesto. United Kingdom: Taylor & Francis.

     

students on campus lawn withcomputers

Supportive Resources

Websites

Podcasts