UD Assessment Manual
Student Learning Outcomes
The University of Delaware is a recognized leader in developing and practicing innovative approaches to teaching. As a student-centered research university with a demonstrated commitment to excellence in instruction, we are in the process of developing, implementing and institutionalizing a university-wide student learning outcomes assessment program. The student outcomes assessment program has one central goal, which is to create a University of Delaware culture of continuous academic improvement which is based upon accountability and learning. However, there are several aligned goals that have been identified at other institutions and which are shared by the University of Delaware:
- Increase our confidence that we are putting our time and resources into activities that we value as an institution
- Increase our confidence that we are allocating resources to areas that are producing the outcomes we value
- Gather and use data that will enable us to make decisions that lead to improved instruction, stronger curriculums, and effective and efficient policies
- Strengthen our ability to say that our graduates are well-prepared to succeed in their future endeavors
- Have ready access to data that will satisfy the requirements of accrediting agencies and funding agencies, and will inform various accountability driven conversations
- Gather and use data that will strengthen arguments for increased funding and/or resource allocations to areas that are producing valued outcomes
- Increase the effectiveness of our communications about the value of the University of Delaware education.
Assessment is not new to the university. Many academic units already engage in assessing student learning, whether formally for accreditation purposes or informally as part of an ongoing discussion about what students know, understand, and can do with their knowledge as a result of their educational experiences. The University conducts extensive assessments of academic degree-granting programs on an on-going basis, through periodic Academic Program Reviews, and through Faculty Senate evaluations of new academic programs both at the time of proposal for provisional approval and again for the granting of permanent status after a usual period of five years. Educational enrichment programs, such as the First Year Experience, are also routinely assessed.
The university is committed to building and sustaining a campus-wide student learning outcomes assessment program in which academic units define clear, concise and measurable student learning outcomes, identify opportunities within and outside of the classroom and the curriculum for students to achieve those outcomes, apply measures to assess whether the desired outcomes are being achieved, and use the results of the assessment for decision-making that improves instruction, strengthens the curriculum, and forms the basis for policy development and resource allocations.
During the last decade, colleges and universities have been called upon by a strong and influential externally driven movement to publicly demonstrate how academic programs continuously improve. National organizations and agencies, and some state legislatures, have been among those demanding more visible accountability and concrete verification that fiscal and human resources invested in educational institutions are being used in ways that result in high quality education. As one means to require accountability, many of these organizations and agencies are requesting that institutions of higher education use assessment of student learning outcomes as a means of demonstrating valuable and/or improving academic programs.
Additionally, the Middle States Commission on Higher Education(MSCHE), UD’s institutional accrediting agency since 1921, as set a standard for assessing student learning, which must be met by the University:
Standard 14: Assessment of Student Learning
Assessment of student learning demonstrates that the institution’s students have knowledge, skills, and competencies consistent with institutional goals and that students at graduation have achieved appropriate higher education goals.
MSCHE expects that accredited institutions will implement comprehensive institutional assessment plans that employ student outcomes assessment measures in general education and in all undergraduate and graduate majors. It outlines an accredited institution as one that is characterized by:
- Articulated expectations of student learning at various levels (institution, degree, program, course) that are consonant with the institution’s mission and with the standards of higher education and of the relevant disciplines;
- A plan that describes student learning assessment activities being undertaken by the institution, including the specific methods to be used to validate articulated student learning goals/objectives;
- Evidence that student learning assessment information is used to improve teaching and learning; and
- Documented use of student learning assessment information as part of institutional assessment.
The growth of the assessment movement during the last decade has demonstrated that assessment is becoming an important tool for better understanding and responding to the needs of an increasingly diverse student population. Colleges and universities are increasingly turning to both nationally developed and locally designed assessment methods and instruments as a means of improving teaching and learning practices. The rationale for the increased focus on the development of assessment programs in academic majors is grounded in the belief that collecting systematic data improves awareness of how well students can integrate content, skills, and attitudes. Assessment research has provided useful information and insight on how students learn and what students learn, going beyond traditional measures that provide useful but limited student and programmatic data.
III. University Of Delaware’s Assessment Approach
The University recognizes that academic units and faculty differ in their abilities to engage in student outcomes assessment, and that substantial assistance – as well as direct communications about the value of assessment – are required to fully involve the faculty in meaningful assessment activities. In September 2005 a number of strategic initiatives were undertaken to communicate the University’s commitment to assessment, offer direct resources to jump-start pilot assessment activities, and provide training, tools, personalized assistance and educational opportunities to faculty and staff. Chief among these initiatives was the creation of the Office of Educational Assessment, which provides assistance to academic units engaged in assessing student learning and development. More specifically, it assists academic programs in formulating, collecting and analyzing information about student learning, and maintains a website that collects best practices, tools, and resources that are available to faculty (http://ctal.udel.edu/). The website also serves as the main depository of unit assessment plans and assessment results
It is recognized that the prime responsibility for academic matters resides within the faculty. Thus, the University’s approach to assessing student learning relies upon the following principles:
- The program, to be successful, requires full faculty and departmental engagement in the conversation about, and the design and practice of, student learning outcomes assessment
- Faculty determine the desired learning outcomes for students in their department/program
- Faculty devise and implement the assessment methodologies that are most appropriate for their stated outcomes
- Academic units are best suited to determine how to use the assessment results for internal programmatic improvements
- The assessment process is iterative within academic units, is manageable within resource bases, is objective, and is meaningful to both faculty and students
- Assessment is not an exercise, but a means of gathering and using information that faculty has determined to be important and integral to future decision-making about programmatic quality and capacities
IV. Developing a Departmental/Program Assessment Plan
When developing and implementing outcomes assessment strategies, academic units should keep the following three purposes in mind: to improve, to inform, and/or to prove. The results from an assessment process should provide information which can be used to determine whether or not intended outcomes are being achieved and how the programs can be improved. An assessment process should also be designed to inform departmental faculty and other decision-makers about relevant issues that can impact the project and student learning.
When developing assessment programs that measure student learning to determine programmatic strengths and weaknesses, faculty often ask, “Aren’t course grades a satisfactory measure of student performance?” Course grades are one source of information about student achievement. But there are significant short-comings for basing assessment of student learning solely on course grades. A traditional letter grade may suggest how much, and perhaps how well, individual students have learned the prescribed information being tested on that particular exam, but the grades, either singly or in combination, do not necessarily reflect the role of that test in the context of the overall departmental objectives for the major. A different view, such as one or more of the suggested assessment methods, will help to focus on the overall objectives.
Developing a program-specific plan to meet assessment objectives is not an easy process. The following four-step approach has enabled many academic units to develop effective plans for assessing student learning in their programs.
STEP 1: Define key learning outcomes for students in the program.
A department’s key learning outcome statements (sometimes referred to as student learning goals) answer this question: What should our students be able to do AFTER they have completed our program. These statements should be results oriented, clearly understood, and measurable. The following links offer many examples of learning outcome statements that are specific to a variety of disciplines, as well as suggestions for organizing faculty to develop these statements: Writing Meaningful Student Learning Outcomes
Georgia State-Gen Ed goals student learning outcomes statements could be more specific to enhance the ability to measure
STEP 2: Identify those courses, assignments, or experiences that ensure that your students have the opportunity to achieve the learning goals you’ve established.
It is unfair to assess whether students are achieving your learning goals, if they have not been offered the opportunity to practice the skills and knowledge that will lead to their success. One method that is often used to determine “where” students have such opportunities is a curriculum map. To read more about curriculum mapping see this site- http://manoa.hawaii.edu/assessment/howto/mapping.htm
The following illustrates the process of curriculum mapping.
- Establish a grid also called a curriculum map
- Faculty should place an X under the learning goal that in their class they have a graded activity that addresses that goal.
- They should then write down how they measure that specific goal and share this information with other program faculty, especially those teaching the same course. They should be sure to state whether this has little emphasis, some emphasis, moderate emphasis, or great emphasis in this course.
- After reviewing the grid and exchanging information, faculty should look for redundancies and gaps. They should question whether there is enough or too much emphasis in the courses and in the different course sections.
- Next the faculty should expand their grid and include the assessment strategies.
P = Paper E = Exam PO = Portfolio L = Lab
S = Standardized Test O = Oral Presentation I = Internship
- Some Assessment may be embedded such as instructor developed course exams, students’ presentations, and evaluations of assignments. Non-embedded assessments would include items such as portfolios, Institutional writing exam, Standardized tests such as the ETS program exams.
Francis.P.(2005) Presentation to Middle States Commission on Higher Education Student Learning Assessment Conference.
STEP 3: Identify and describe instruments, methods, timetable for assessing student achievement at important stages in the program (conduct the assessment).
Once the key learning outcomes have been identified, assessment methods for collecting student data can be chosen. In this step, you are answering the following questions: 1) How will you assess how well your students are achieving the learning goal (measures? Instruments? Methods of collecting data? 2) When do you expect to begin collecting the assessment data/information? 3) Who will collect and interpret the results? 4) How often will you collect the assessment data/information? Because departments often define a variety of educational goals and objectives, comprehensive assessment strategies frequently require the use of more than one assessment instrument to determine program effectiveness. The section
“Assessment Methods for Measuring “
will help define and explain various assessment techniques, and the following links offer many examples that you will be able to adopt or adapt:
Logic Models can help you create a systematic approach to programmatic assessment. There are many different types of logic models. Click the link below to see more information about them. http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html
Evaluation Matrix, Logic Model Sample, and Template are provided here.
Rubrics that have been developed and applied for measuring general education
You may also find the following suggestions/advice useful as you think about how to assess student learning:
- Design your measurement tool
- What can you measure? Class behaviors, co-curricular behaviors, engagement behaviors, attitude changes, application or transfer of information, longitudinal measures.
- Locally developed measurement tools tend to have tremendous content validity. Don’t be afraid to work with your colleagues to develop your own tool. Or, consult with the Office of Educational Assessment for assistance.
- Simple, course embedded tools are best because they contribute to lasting and more meaningful learning.
- Ask yourselves, what will we do with good or bad results? If you won’t do anything with the information, then you shouldn’t be collecting it. Only collect the data you need that will lead to effective decision-making.
- Take measurements and collect the data.
- Assessment Techniques-range from surveys, focus groups, portfolios, tests, quizzes, senior thesis.
Direct Measures-allow you to see your students demonstrate that they have achieved the student learning outcome. These measures have greater validity for assess learning outcomes. Direct Measures can be: portfolio assessments using a rubric of student projects,Papers, Theses/Dissertations, Exhibitions, Performances, Case studies, Clinical Evaluations, etc.
Indirect Measures- Learners report whether or not they met the learning goal. E.g. Alumni Surveys asking how well did the program prepare them for the job market.
- You don’t need to test everyone. You can sample students if you like. Please make sure that you obtain enough data to be sure of your results.
- Keep enough data to prove to outsiders that you did what you said you did. Keep samples across the range of performance. If you found out that over 80% of your students could not do a skill, document it accordingly. Reviewers do not want or expect perfection.
Gonsalves, S.V. (2005) Presentation to Middle States Commission on Higher Education Student Learning Assessment Conference.
STEP 4: Summarize the results of your assessment, and use the results for program improvement.
In this step, the following questions are answered: 1) What do the results tell you about how well students are meeting the learning goals you’ve set for them? 2) How have you used this information to help support your students, and to make decisions about curriculum, policies, resources allocations, etc.? 3) As a result of this assessment, will you be making changes to your goals, your assessment schedule? 4) What will be your next “round” of assessment goals? 5) Who needs to know the results and how will you share them? Internal communication of results is essential for faculty to be able to use the results in ways that they see valuable. If the assessment sits on a shelf, the faculty have failed in their responsibility to engage in a useful process that results in data they want and need to evaluate those learning outcomes they have identified as being vital to their educational mission. The following links describe the assessment plans and results of various departments in a number of universities.
This website from Kent State contains information from the assessment process of every department at Kent State. You will be able to view goals, objectives, approaches, measures, and the assessment process timeline developed by just about every discipline.
In summary, only through widespread faculty and departmental involvement can an institution as complex as ours devise effective and efficient program-based assessment plans that will produce results beneficial for all academic units. With assessment planning located primarily in the departments, faculty exercise their responsibility to devise appropriate methods to measure student academic achievement and program effectiveness. This process gives widespread ownership of assessment planning to faculty and enables them to determine the methods and instruments that are most applicable to their educational objectives and missions.
Summary of Actions to Develop an Assessment Plan in the Major *
- Agree on your mission
- Create goals for student outcomes and processes
- Identify related activities for each goal
- Brainstorm appropriate measures
- Evaluate and select measures
- Identify appropriate assessment methods
- Develop a plan for collecting data
- Prioritize goals
- Set timeline, milestones
- Implement assessment plan
- Use data to improve processes
- Communicate results
* From Hatfield, Susan, “Assessment in the Major – Tools and Tips for Getting Started.” Paper presented at the 1997 Assessment Conference in Indianapolis. Professor Hatfield is the Assessment Coordinator at Winona State University
V. Department/Program Reporting Responsibilities
A function of the Office of Educational Assessment (OEA) is to assist units in collecting and maintaining their assessment reports in a format that is easily retrievable by the unit (to facilitate the use of the assessment) and easily accessible to our accrediting agencies. To that end, the OEA has established a simple, web-based information collection system on its website. As units develop their learning goals and their assessment plans, they will enter the information on the reporting forms posted on the CTAL website at ctal. Annually, faculty will also report the results of their assessments using the template on the same website. Faculty and staff of the University will be able to view the plans and reports, but only the submitting department will be able to edit or make any changes/updates to their postings.
VI. Last Thing To Remember
As you follow these steps, keep in mind that there is no need to reinvent the wheel. Many other programs in your discipline have already engaged in the student learning outcomes process, and there is much that you can adopt or adapt from their work. Be sure to periodically look at the Office of Educational Assessment website at ctal to find many materials that will give you ideas as you proceed; you will find that the Tools and Resource Library pages contain information that is particularly useful and regularly updated. And the very last thing to remember is that the Office of Educational Assessment stands ready to help you. Contact us at firstname.lastname@example.org
Critical Thinking Rubric
This rubric is designed to evaluate the extent to which undergraduate students evaluate claims, arguments, evidence, and hypotheses.
Results will be used for program improvement purposes only.
Download the Critical Thinking Rubric (PDF version)
|Component||Component Fully Met|
(rating = 3)
(rating = 2)
(rating = 1)
|Component Not Met|
(rating = 0)
|Accurately interpret evidence and thoughtfully evaluate alternative points of view||Information is taken from source(s) with enough interpretation/evaluation to develop a comprehensive analysis or synthesis. Viewpoints of experts are questioned thoroughly.||Information is taken from source(s) with enough interpretation/evaluation to develop a coherent analysis or synthesis.|
Viewpoints of experts are subject to questioning.
|Information is taken from source(s) with some interpretation/evaluation, but not enough to develop a coherent analysis or synthesis. Viewpoints of experts are taken as mostly fact, with little questioning.||Information is taken from source(s) without any interpretation/ evaluation.|
Viewpoints of experts are taken as fact, without question.
|Draw judicious conclusions, justify results, and explain reasoning||Not only develops a logical, consistent plan to solve problem, but recognizes consequences of solution and can articulate reason for choosing solution.|
Conclusions and related outcomes (consequences and implications) are logical and reflect student’s informed evaluation and ability to place evidence and perspectives discussed in priority order.
|Having selected from among alternatives, develops a logical, consistent plan to solve the problem.|
Conclusion is logically tied to a range of information, including opposing viewpoints; related outcomes (consequences and implications) are identified clearly.
|Considers and rejects less acceptable approaches to solving problem.|
Conclusion is logically tied to information (because information is chosen to fit the desired conclusion); some related outcomes (consequences and implications) are identified clearly.
|Only a single approach is considered and is used to solve the problem.|
Conclusion is inconsistently tied to some of the information discussed; related outcomes (consequences and implications) are oversimplified.
|Engage in skepticism, judgment, and free thinking||Extends a novel or unique idea, question, format, or product to create new knowledge or knowledge that crosses boundaries.||Creates a novel or unique idea, question, format, or product.||Experiments with creating a novel or unique idea, question, format, or product.||Reformulates a collection of available ideas.|
|Engage in abstract reasoning, questioning and understanding||Actively seeks out and follows through on untested and potentially risky directions or approaches to the assignment in the final product.|
Integrates alternate, divergent, or contradictory perspectives or ideas fully.
|Incorporates new directions or approaches to the assignment in the final product.|
Incorporates alternate, divergent, or contradictory perspectives or ideas in a exploratory way.
|Considers new directions or approaches without going beyond the guidelines of the assignment.|
Includes (recognizes the value of) alternate, divergent, or contradictory perspectives or ideas in a small way.
|Stays strictly within the guidelines of the assignment.|
Acknowledges (mentions in passing) alternate, divergent, or contradictory perspectives or ideas.