Research notes

Rethinking Assessment for 21st Century in Teacher Education: From Process to Product, from Self-assessment to Teacher Work Samples From E-portfolios to Digital Stories



Abstract

In a project based learning curriculum, participants will be encouraged to learn through a process where they write questions and to integrate new media and technologies into their media projects. When it comes to assessment, usually the instructor’s role is to ask the questions, give the assignments, and judge the products whereas the role of students is to produce successful outcomes. From a constructivist point of view, this is inappropriate in relation to students' learning. Therefore, alternative approaches to project assessment are critical especially in teacher education programs. This study is based on participatory action research. We will design a model and test it in different types of courses. The goal is to explore and design new assessment tools, templates, and strategies so that the participants will be more motivated to reflect their own learning process and to take responsibility for their own learning.

Participants will be invited to showcase their stories, share their process papers, and assessment projects that they developed over the internet. Research will be shared in faculty forums in the College of Education, University Research day, and regional and national conferences. The study will be presented in at least one regional i.e. NERA (Northeastern Educational Research Association) conference and one national conference i.e. SITE (Society for Information Technology and Teacher Education) conference.


Charts

Assessment in Teacher Education

Assessmentv1.jpg




Assessment for Learning vs Assessment of Learning



AfL.png
http://educon21.wikispaces.com/204-5



Assessment for 21st Century

Traditional Assessment - Web-based assessment (E-portfolios)

Chart is adopted by Bobby Elliot- http://www.scribd.com/doc/461041/Assessment-20



Assessment 1.0
Assessment 1.5
Assessment 2.0

Traditional
Computer based
21st Century
Emphasis on
Product
Product
Process and Product
Outcome
Text
Text, audio, video
in addition to text, audio, video
Games, animations, simulations
Assessed by
Teacher
Teacher, Peer, Self
Teacher, Peer, Self, anybody
Meaning
Given

Negotiated
Completed by
done alone
done in groups, peers
done collaboratively
Completed
descriptive

researched/deep
Done
closed book
computer assisted
open web
Location
mostly classroom-based
computer lab
everywhere
Format
mostly paper-based
(exam, quiz, paper)

blogs, wiki, podcasts, secondlife projects
Focuses on
passive learning

active learning
Evaluates
memorized information

authentic learning experiences
Information is distributed
highly synchronised (in terms of time and place)and controlled.

highly asynchronised.










Assessment tools

Tools.jpg
http://www.informationliteracy.org/builder/view/1622

Assessment is the central part of teaching and learning.
Teachers use a variety of assessment strategies: journaling, concept maps, e-portfolios, interactive virtual gaming, simulations, authentic problem solving, problem based projects, wiki, blogs, podcasting (interviews).




Evolution of Assessment

evalution_of_assessment.jpg
Original at http://www.slideshare.net/lastkaled/assessment-20




E-portfolios


Balancing.jpg
Original at http://electronicportfolios.org/balance/index.html



E-PORTFOLIOS
http://electronicportfolios.org/balance/index.html
Workspace/Process==
The Collection or Digital Archive
Repository of Artifacts
Personal Information
Reflective Journal (eDOL)*

  • Portfolio as Process
    • Organization: Chronological – *eDOL (Electronic Documentation of Learning – Crichton & Kopp (2008) U. of Calgary) Documenting growth over time for both internal and external audiences
    • Primary Purpose: Learning or Reflection
    • Reflection: Immediate (focus on artifact or learning experience) - Reflection in the present tense
Showcase/Product==
The “Story” or Narrative
Multiple Views (public/private)
Varied Audiences (varied permissions)
Varied Purposes

  • Portfolio as Product
    • Organization: Thematic – Electronic Portfolio documenting achievement of Standards, Goals or Learning Outcomes for primarily external audiences
    • Primary Purpose: Accountability or Showcase
    • Reflection: Retrospective (focus on achievement or thematic organization) - Reflection in the past tense
    • Reflection: Prospective (Direction) - Set goals for future learning - Reflection in the future tense

Procedure: on a daily/weekly basis Levels 1 & 2

  • Integrate technology across the curriculum (generate digital artifacts to store in online repository/ digital archive)
  • Maintain a reflective journal in the form of a blog (organized in reverse-chronological order)
    • When saving items in digital archive, link to a blog entry that contains an immediate reflection on the document and/or the learning associated with an experience (such as service learning) - reflection in the present tense
    • Create a label or tag that categorizes the entry (with attached artifact, where appropriate) by learning standard/goal/outcome
    • Provide feedback to learner in the form of comments in the blog or collaborative editing (by teachers and/or peers)
  • Periodically review the collection for evidence of growth/change over time
===Procedure: on a periodic basis (end of class/term/year) Level 3===

  • Review the blog entries and collected evidence and Select specific entries to demonstrate the achievement of the selected standards/goals/outcomes
    • Use the tags/labels in blog entries to review all entries that match selected criteria (generate pages within the blog)
  • Reflect on why the selected artifacts (with associated reflections) constitute evidence of achieving specific standards/goals/outcomes (retrospective reflection... in the past tense)
  • Write future learning goals related to specific standards/goals/outcomes (prospective reflection/direction... in the future tense)
  • Organize a hyperlinked presentation of evidence (with reflections)
  • Present portfolio to an audience (either real or virtual)
  • Evaluate the learner's self-assessment of the achievement of the standards/goals/outcomes that are presented (by teachers and others)

Consistency

What is consistency in assessment?
Consistency among teachers are able to make judgements about student learning that are not dependent on the individual teacher, student, location or time and are based on a shared understanding of syllabus standards of learning.
Consistency in making assessment judgements needs to develop:
  • Across assessment tasks
  • Across teachers of different classes
  • Across key learning areas (such as Creative Arts, Mathematics, Science)
  • Across time (within one key learning area over a period of years)
  • Across schools
(From http://www.curriculumsupport.education.nsw.gov.au/consistent_teacher/consistency.htm)

How to design assessments


SMART.jpg

Summary

Study will be completed in two semesters, (Fall 2011, Spring 2012) and total of 6 credits. The study will focus on 1) Collaborating with teacher education faculty to develop assessment model for 21st century teacher education program integrating new technologies; 2) Researching innovative assessment models and best practices among P16 educators and assessment software for P16 education; 3) Conducting the research using new technologies (i.e. Wink (http://www.debugmode.com/wink/) and Web2.0 technologies) to collect data among teacher candidates; 4) Developing assessment models that may provide self and peer assessment strategies teacher education programs; 5) Providing tools and strategies for research participants to develop their own assessment models, strategies and templates.
In a project based learning curriculum, participants will be encouraged to learn through a process where they generate their own questions and integrate new media and technologies into their projects. When it comes to assessment, usually the instructor’s role is to ask the questions, give the assignments, and judge the products whereas the role of students is to produce successful outcomes. From a constructivist point of view, this is inappropriate in relation to students' learning. Therefore, alternative approaches to project assessment are critical especially in teacher education programs. This study is based on participatory intervention research. We will research current innovative assessment tools, strategies for 21st Century teaching and learning and design a model and pilot it in different types of teacher education courses. The goal is to explore and design new assessment tools, templates, and strategies so that the students will be more motivated to reflect their own learning process and to take responsibility for their own learning.
This study focuses on assessment for school improvement that can lead to gains in student achievement. View that the relationship between assessment practices and effective schools should be re-evaluated; arguments for day-to-day classroom assessment, which has an advantage over standardized tests; how the demand for accountability, accreditation is helping educators clarify their achievement goals. In recent years, there are major breakthroughs in using technology in assessment and our understanding of the effective use of assessment to benefit student learning. We have gained new insights into cognitive processes and have succeeded in connecting them to new assessment strategies that promise unprecedented achievement gains for students. Yet in districts, schools, and classrooms across the nation, educators still assess student learning the way their predecessors did 60 years ago because they have not been given the opportunity to learn about these new insights and practices.
This study examines to role of assessment strategies, tools, and rubrics in media education. It explores role of assessment in the learning and teaching. It brings new understanding of the power of assessment in learning by aligning local, national, and international media and technology standards. The study will use Chris Worsnop’s (1997) model of authentic media assessment which consists of content, environment, process, performance of students. The study is about the work that students produce for teachers to mark and educators concern about being consistent in grading and having appropriate standards in their assessment.
The study emphasizes the importance of studying assessment from process to product and outlining the elements of assessment in teacher education programs in order to gain different perspectives in our teaching and learning styles and strategies. It is about rethinking the relationship between assessment practices and effective teacher education curriculum.
The study outlines the difficulties and unique characteristics of assessing new media projects such as Web 2.0, videos and multimedia presentations. The study explores three key topics in order to understand the educational experiences of the participants: the wide range of meanings participants associate with assessment in teacher education; the impact of developing alternative assessment tools and strategies on participants’ reaction and understanding of evaluating non-print projects; and the ways in which the participants respond to assessment of learning versus assessment for learning.
The main outcome of the study is to design a research based and innovative assessment tool that can be used in the school for Global Education and Innovation at College of Education. As accreditation for NCATE and Middle States becomes an important means of self-regulation and self study, teacher education programs intend to strengthen and sustain the quality and integrity programs, courses, student learning outcomes.The proposed study will inform the current assessment issues, challenges and accomplishments in our teacher education program and looks at the asse


Teacher candidates who are enrolled in Certification Programs at College of Education are directly affected by the issues being investigated. Currently, we are using teacher work samples (TWS) as a critical assessment piece for teacher candidates. In addition to increasing demand for culturally relevant pedagogy, there is a great need for developing assessment tools, rubrics, and strategies for 21st Century education. This study will not only provide authentic assessment tools, templates, and rubrics but also develops new tools to integrate international education. The study outlines the difficulties and unique characteristics of assessing new media projects such as web based project, videos and multimedia presentations as well as print projects.

The contemporary trend in assessment has been shifted from summative to formative assessment for the last decades. One reason for this shift is the movement from assessment of learning to assessment for learning whereas the other one is, to able to provide a substantial, future-directed, and process-based feedback. For example, Keppell and Carless (2006) suggest using feedback in a way that feeds forward into the future. The process-based feedback and therefore, the process-oriented assessment in teacher education are rather important and challenging. Teacher educators have emphasized the importance of focusing on and assessing students’ problem-solving processes rather than only product of the process. Similarly, it is challenging because observing every students’ problem-solving processes is time consuming even if in researches. However, by sharing responsibilities of process-
oriented assessment, teacher may help students extend their limits.

Practical implications

Despite the use of these tools for research purposes, teachers can easily integrate this system into their classroom activities either by engaging students to create their own portfolios (McDougall & Karadag, 2008) or to collaborate with their peers (Karadag & McDougall, 2008). Teachers may ask them to explore problems, record their own work, and prepare digital portfolios based on their selective work. Students’ selection of their work as well as to explore and work in this proposed environment may improve their conceptual understanding of the topics being taught. Similarly, collaborating with their peers on their work may positively affect their learning.

To date, few scholarly studies have investigated the impact of new media (Web 2.0) on assessment in education. This study attempts to fill the gap by outlining the natural links between role of assessment in teacher education and educational assessment.

Rationale

A number of years ago, I attended NJEdge (www.njedge.net) meeting focusing on assessment in higher education. One faculty had a dilemma how to grade a student who created a wikipage and put his/her draft paper after it was assigned and submitted his/her paper that was edited by many peers and scholars at the end of the term. It was a highly debated discussion focusing on how to assess online work, adding check-points in the grading system, revising academic integrity policy, etc. Since then, I posed this question to some of my colleagues and students. As I am getting ready to teach EMSE4900- Senior Seminar course in Spring 2011(Teacher candidates in this course develop a Teacher Work Sample (TWS) portfolio that complements their student teaching.), I expect the challenges of motivating students to develop such a massive work, of providing timely feedback, of assessing their work to support their student teaching experience more enriching and stimulating.

The contemporary trend in assessment has been shifted from summative to formative assessment for the last decades. One reason for this shift is the movement from assessment of learning to assessment for learning whereas the other one is, to able to provide a substantial, future-directed, and process-based feedback. For example, Keppell and Carless (2006) suggest using feedback in a way that feeds forward into the future. The process-based feedback and therefore, the process-oriented assessment in teacher education are rather important and challenging. Teacher educators have emphasized the importance of focusing on assessing student work; problem-solving processes rather than only product of the process.

This study will benefit teacher candidates, teacher educators, K-12 educators, parents, media specialists, and administrators who seek alternative strategies and tools for assessing students' work and for integrating new media and technologies in assessment for learning.



Resources

Educational Software/ Tools to use:
Wink.org
Polleverywhere.com
http://en.wikipedia.org/wiki/Assessment_for_Learning


References
Anderson, P (2007). What is Web 2.0? JISC Technology and Standards Watch. pp. 14-26.
Adamy & Milman (2009) Evaluating Electronic Portfolios in Teacher Education. Charlotte: Information Age Publishers.
Barrett, H. (2010). Balancing the Two Faces of ePortfolios. Educação, Formação & Tecnologias, 3(1), 6-14. Retrieved Jan 10, 2011 http://eft.educom.pt
Barrett, H. (2004-2008) "My Online Portfolio Adventure." [Retrieved January 10, 2011 from http://electronicportfolios.org/myportfolio/versions.html
Beck, R. & Bear, S. (2009) "Teacher's Self-Assessment of Reflection Skills as an Outcome of E-Folios"
Black, P., and Wiliam, D. (1998). "Inside the Black Box: Raising Standards Through Classroom Assessment." Phi Delta Kappan, October 1998.
Cooper, D. (2006). Talk About Assessment: Strategies and Tools to Improve Learning. Toronto, ON: Thomson Nelson.
Government of British Columbia.
Cousin, G (2003). Learning from Cyberspace. JISC.
Crisp, G. (2007). The e-assessment handbook. London: Continuum.
JISC (2008). Effective Practice with e-Portfolios. [Retrieved January 10, 2011 from http://www.jisc.ac.uk/whatwedo/themes/elearning/eportfolios/effectivepracticeeportfolios.aspx]
Jonassen, D. H. (1992). Evaluating Constructivist Learning. In T. M. Duffy and D. H.
Jonassen (Eds). Constructivism and the Technology of Instruction. New Jersey:
Karadag, Z. & McDougall, D. (2008). E-contests in mathematics: Technological challenges versus technological innovations. Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2008 (pp. 6331-6336). VA: AACE.
Keppell, M. and Carless, D. (2006). Learning-oriented assessment: a technology-based case study. Assessment in Education: Principles, Policy & Practice, 13:2, 179 –191. Lawrence Erlbaum Associates, Publishers.
O’Connor, K (2002). How to Grade for Learning. Arlington Heights, IL: Skylight.
McDougall, D. & Karadag, Z. (2008). Tracking students’ mathematical thinking online: Frame analysis method. 11th International Congress on Manitoba Education, Citizenship, and Youth. (2006) Rethinking Assessment with Purpose in Mind: assessment for learning, assessment as learning, assessment of learning Winnipeg, Manitoba, Canada: MECY
Mathematical Education. Monterrey, Nuevo Leon, Mexico.
Owen, Grant, Sayers and Facer (2006). Social Software and Social Learning. Futurelab, Opening Education Series.
Pellegrino, JW (1999). The Evolution of Educational Assessment.William Angoff Memorial Lecture Series (ETS).
Petersen, N., Center for Teaching, Learning, & Technology, Washington State University (2008). "Case Studies of Electronic Portfolios for Learning" [Retrieved January 10, 2011 from: http://wsuctlt.wordpress.com/2008/03/14/case-studies-of-electronic-portfolios-for-learning/]
QCA. Evaluation of the 2006 pilot of the key stage 3 ICT test. September 2006. QCA Assessment Research Team.
Wiggins, G. (1998). Educative Assessment. San Francisco. CA: Jossey Bass.
Windham, C (2005). The Student’s Perspective: Educating the Net Generation. Chapter 5. Educause E-book
Worsnop, C. (1997). Assessing Media Work. Mississauga, Canada: Wright Communications.