But... Thou Shall NOT… Use Formative Assessments...

... as Part of the Summative Grade

A few weeks ago I was working with a group of secondary teachers on ideas connected to our latest book, A Fresh Look at Grading and Reporting in High Schools. At one point there was a murmur in the group. When I inquired about the murmur some of the teachers said, “But, we were told, 'Thou shall NOT use formative assessments as part of the summative grade.’”

The notion of evidence of learning – and what makes good evidence – is key when it comes to summative assessment. After all, everything a student does, says, or creates is potentially evidence of learning. What counts? It is all about purpose. Are you considering the evidence of learning in a formative way – to inform instruction? Or, are you considering the evidence of learning to determine how well and how much a student has learned? It is about purpose.

The evidence – observations, conversations, and products – that is used to determine the summative grade depends on the teacher. It is a professional decision. In A Fresh Look at Grading and Reporting in High Schools, Sandra Herbst and I write about the entire grading process but in this post I want to focus on one aspect of that decision-making process.

Should evidence of learning be excluded simply because it has been used to inform instruction during the learning? That is, should formative assessment information only be used for formative purposes? And, should summative assessment information be used only for summative purposes? Or, as the secondary teachers put it, “We understood that we were not to use formative assessments as part of the summative grade.”

Information is information. And, what we do with information depends on purpose. The Assessment Reform Group (one member, Gordon Stobart, is one of our International delegates) in a 2009 document titled, "Fit for Purpose” put it this way, “It should be noted that assessments can often be used for both formative and summative purposes. “Formative” and “summative” are not labels for different types or forms of assessment but describe how assessments are used.” (2009, 9)

One of the pre-reading documents by Ministry of New Zealand, submitted by the New Zealand team members for the International Symposium, puts it this way, “Sometimes assessment is referred to as being either “formative” or “summative.” The formative use of assessment information is an important part of everyday practice. It is a diagnostic process concerned with identifying achievement and progress and strengths and weaknesses in order to decide what action is needed to improve learning on a day-by-day basis. The summative use of assessment is concerned with “summing up” achievement at a specific point of time. However, these summations can be used not only to ascertain the level of achievement at a specific point of time but also to look back and consider what progress has been made over a period of time compared with expected progress.” (Ministry of Education, 2011, 14)

In summary, one needs to consider all the evidence of learning – the student’s entire learning journey – in order to better understand what and how much has been learned. When teachers make a summative assessment of a student’s learning, we engage in making an “informed professional judgment.” And, to do so, we use all the information available to determine how well students have learned what they needed to learn, can do, what they need to be able to do, and can articulate what they need to articulate in relation to the standards or outcomes for the course or subject area.

Statements such as, “Thou shall not use formative assessment as part of the summative grade” are not helpful when ‘informed professional judgment’ is at work.

References:

Mansell, W., James, M. & the Assessment Reform Group (2009) Assessment in schools. Fit for purpose? A Commentary by the Teachingand Learning Research Programme. London: Economic and Social Research Council, Teaching and Learning Research Programme.

Ministry of Education (2011). Ministry of Education Position Paper: KO TE WHĀRANGI TAKOTORANGA ĀRUNGA, Ā TE TĀHUHU O TE MĀTAURANGA, TE MATEKITENGA. Retrieved March 17, 2014 Ministry of Education Position Paper: Assessment [Schooling Sector] http://www.minedu.govt.nz/~/media/MinEdu/Files/TheMinistry/MOEAssessmentPositionPaper_October11.pdf

Tagboard: https://tagboard.com/AforLConversation/160953

The Same Beast in an Updated Outfit? Distorted and Subverted AforL

Sue Swaffield (2011), in this well-written and interesting paper, reviews the emergence of assessment for learning in terms of its research roots and two major implementation projects. She then goes on to document how this work was represented differently when used as the basis for policy changes in England which, similar to those in other jurisdictions, appear to be a significant change yet are really - in my words -  ‘the same beast in an updated outfit’ - or as Sue points out, "distorted and subverted."

Those of us who are interested in how implementation initiatives can go ‘off track’ will find this paper informative both in terms of assessment for learning and how assessment for learning can be co-opted for other purposes.

Sue Swaffield is one of the International Delegates who will be joining the Assessment for Learning: Canada in Conversation with the World conference in April 2014. You can review the agenda here and register here.

To cite this article: Swaffield, Sue. (2011). Getting to the heart of authentic Assessment for Learning. Assessment in Education: Principles, Policy & Practice, 18:4, 433-449, DOI: 10.1080/0969594X.2011.582838.

To link to this article: http://dx.doi.org/10.1080/0969594X.2011.582838

Co-constructing Criteria - Research Supports Practice!

Co-constructing Criteria - Assessment Research Supports Powerful Classroom Assessment Practices 

Since I co-authored my first assessment book, Together is Better: Collaborative Assessment, Evaluation and Reporting (1992) based on our school-based research, I’ve been writing about the power of co-constructing criteria in support of student learning and achievement.

Now, as I am reading the more than 100 papers submitted as pre-reading by delegates to the International Symposium, I am enjoying the number of papers that provide the research support for high-quality assessment practices such as the deep involvement of students in the assessment process.

One paper in particular, by Andrade(2013), has gathered together the research in support of co-constructing criteria. For example, when students are engaged in examining samples, using work samples, or “co-creating success criteria, and monitoring their own progress through self- and/or peer assessment” they learn more (Andrade, Du, & Mycek, 2010; Andrade, Du, & Wang, 2009; Ross and Starling, 2008).

Andrade(2013) goes on to write, “The quality of the success criteria makes a difference, of course. Citing Moreland and Jones (2000), Brookhart (2007) notes that formative assessment and the instructional decisions based on it can actually thwart the learning process if the success criteria are trivial (e.g., focused on social and managerial issues such as taking turns at the computer) rather than substantive (e.g., focused on procedural and conceptual matters related to learning about computers).”

I’m going to continue reading – as a quick scan forward shows that Andrade is addressing self-assessment, goal setting, feedback, and so on. You can hear Heidi Andradeyourself at Assessment for Learning: Canada in Conversation with the World in Fredericton, April 11 & 12, 2014. You can register here.

References

Andrade, H. (2013). Classroom assessment in the context of learning theory and research. In J. H. McMillan (Ed.), SAGE handbook of research on classroom assessment (pp. 17-34). New York. SAGE.

Andrade, H. (2010). Students as the definitive source of formative assessment: Academic self-assessment and the self-regulation of learning. In H. Andrade & G. Cizek, Handbook of formative assessment. (pp. 90-105). New York: Routledge.

Andrade, H., Du, Y. & Mycek, K. (2010). Rubric-referenced self-assessment and middle school students’ writing. Assessment in Education, 17(2), 199-214.

Andrade, H., Du, Y. & Wang, X. (2009). Putting rubrics to the test: The effect of a model, criteria generation, and rubric-referenced self-assessment on elementary school students’ writing. Educational Measurement: Issues and Practices, 27(2), 3-13.

Andrade, H. G. (2001). The effects of instructional rubrics on learning to write. Current Issues in Education, 4(4). Retrieved from http://cie.ed.asu.edu/volume4/number4

Brookhart, S. (2007). Expanding views about formative classroom assessment: A review of the literature. In J. H. McMillan (Ed.), Formative classroom assessment: Theory into practice. New York: Teachers College Press.

Davies, A, Cameron, C., Politano, C. and Gregory, K. (1992).  Together Is Better: Collaborative Assessment, Evaluation, and Reporting. Winnipeg, MB: Portage and Main Press. 

Gregory, K., Cameron, C. and Davies, A. (1997/2011). Knowing What Counts: Setting and Using Criteria, 2^nd Ed. Courtenay, BC: Connections Publishing.

Gregory, K., Cameron, C. and Davies, A. (2013). Établir et utiliser des critères, 2^nd Ed. Courtenay, BC: Connections Publishing.

Ross, J. A. & Starling, M. (2008). Self-assessment in a technology supported environment: The case of grade 9 geography. Assessment in Education, 15(2), 183-199.

Tagboard: https://tagboard.com/AforLConversation/160953

Separate the ‘chaff from the wheat’ ?! Assessment and Social Justice

I'm enjoying reading this report from Futurelab.org.uk about assessment and social justice - thoughtful and challenging ideas combined with wonderful writing. Here is an excerpt:

"We must never discount the persistent rump of disingenuous educationalists and assessors who consider that the primary role of assessment is to separate the ‘chaff from the wheat,’ the weak from the able, the malingerers from the stalwarts of tomorrow’s society. For them, assessment can categorise and organise; it is simply inevitable that those who have will get more, and those who have not – tough luck.

As we have indicated earlier, we see assessment as being a process that should be used primarily in support of learning. In all that assessment does it must be fair and seen to be fair, as expressed by the AERA Standards (AERA et al. 1999). Camara and Lane (2006) express fairness as a lack of bias in items and tests, examinees having comparable opportunities to demonstrate their standing on a construct, there being comparable outcomes across groups, and individuals and groups having adequate opportunities to learn.

Social justice aims to create circumstances in which society’s current inequities are eradicated, or at least their worst disadvantages are mitigated. If a high-stakes assessment is being used, for entry to university for example, social justice demands that students should not be disadvantaged by circumstances and influences that have nothing to do with the constructs under examination. Plummer (2003) would see these circumstances as the ‘Big 8’ dimensions of diversity: race, gender, ethnicity/nationality, organizational role, age, sexual orientation, mental/physical ability, and religion. At the risk of discriminating against some of these, we will look briefly at disability; the
combination of language, culture, race, ethnicity, and poverty; and gender."

Excerpt from Assessment and Social Justice - Report 16 by John Gardner, Bryn Holmes, and Ruth Leitch, prepared for FutureLab.org.uk

Tagboard: https://tagboard.com/AforLConversation/160953

Failure is NOT good, great or particularly inspiring

I’ve been reading a lot of tweets this week celebrating failure. I even posted one myself. And yet, I actually don't like failure.

Consider -

Example: You are running along a city rooftop and there, in front of you, is the gap between two buildings. You leap across because you believe in the power of failure. You fall. You fail because you didn’t have what it takes to be successful in that situation.

Example: A trusted friend is persistent. "You can do this!" Reluctantly you agree to try. You fail. You fail because you were right. You weren't ready.

Example: You take your new car on a road trip. You are thrilled to be out on the road testing its limits. You fail to notice the slight pulling to the right. Suddenly you are off the road and into the ditch. You succeed in destroying your new car. The nuances of success eluded you. Next time you'll pay attention to the signals. You didn't know. You didn't know enough to be successful.

We usually take risks when we are expecting to succeed. Or we do something that we don’t realize is a risk; it is simply the next step. Malcolm Gladwell of the New Yorker, quoting economist Albert O. Hirschman’s analysis writes, “They [people] are ‘apt to take on and plunge into new tasks because of the erroneously presumed absence of a challenge—because the task looks easier and more manageable than it will turn out to be.’” Gladwell goes on to say, “The entrepreneur takes risks but does not see himself as a risk-taker, because he operates under the useful delusion that what he’s attempting is not risky. Then...people discover the truth—and, because it is too late to turn back, they’re forced to finish the job.”

Perhaps the people who love failure actually love something else. Maybe they love…

The power of vision

The power of personal goals

The power of innate talent

The power of persistence

The power of specific feedback

The power of good teaching and coaching

The power of knowing enough to be within reach of success

The power of choice

The power of preparation

The power of audacity

  

Or perhaps it is simply success that they love.

Failure that occurs when one is within reach of success is totally different than other kinds of failure. 

Failure – and risk taking – is so difficult in a classroom because it is IMPOSSIBLE to manage it for someone else. Setting someone else up to “take a risk” on our terms – our assigned task, test, or challenge – is dangerous for learners and our relationship with them.

So I’m left wondering… Do we actually celebrate failure? Or do we celebrate the success that arises after failure? If so, are we actually celebrating something else… persistence, practice, vision, talent, and guts?

Tagboard: https://tagboard.com/AforLConversation/160953

Plausibility? Accuracy? Summative Grades and Teachers' Professional Judgement

This study reported by Linda Allal, a delegate to the Assessment for Learning International Symposium in April 2014 is based on interviews with 10 sixth-grade teachers. The main findings highlight the layers involved in teachers' professional judgement and the implications for ongoing professional learning.

This paper begins by describing the range of professional judgements teachers make day-by-day and then focuses on the way teachers determine summative grades. After a brief historical review Linda Allal shares some of the research relating to social moderation - that is, "...meetings where teachers discuss, confront, and negotiate grades assigned to student work...."

I appreciated the author's comments that summative assessment may be more about "sense-making" and "plausibility" than "accuracy." "Plausibility," Allal (2012) writes, "implies that a judgement or decision is socially constructed in a manner that is persuasive for all concerned parties." She goes on, "... the message conveyed by a grade may be more significant than the accuracy of the grade."

Allal (2012) is well worth reading by anyone interested in supporting teachers in making informed professional judgement. In the research paper, Linda Allal describes the context for the study, methodology used, data collection, data analysis and major findings. I found myself highlighting these sections:

" The data we collected showed that teachers’ judgements about end-of-term grades involved two main operations: (a) gathering information from a variety of sources; and (b) combining information in an interpretative synthesis that sometimes included but was not limited to an arithmetic algorithm."

"When a student’s results on these formal assessments fluctuated and/or when teachers were not convinced that they provided a complete and valid picture of a student’s understanding of the subject matter, all teachers examined other, more qualitative sources of information."

"This process of combining quantitative and qualitative information entailed an interpretative synthesis in which the teacher decided how much importance would be given to various types of information and which pieces of information appeared to be most relevant in the case of a given student."

"Given the long tradition, in Geneva as in other school systems, of basing report card grades on averages of several test grades, it is not always easy to convince parents that a focus on learning progression is more important than calculating averages, but the teachers believe that the chances of doing so are increased when they adopt shared collegial responsibility for their assessment policy."

"The grading procedures developed by each teacher reflected personal choices about sources of information, about how to combine information and arrive at a judgement, and about ways of adapting their assessment procedures to take into account student specificities."

"From the teacher’s viewpoint, you have to use all the information you have about a student to formulate the judgement that is the most appropriate for that individual. Although you may lack equivalent information about other students, it does not diminish your responsibility to use judiciously the information you have."

Here is the reference for the complete report: Linda Allal (2012): Teachers’ professional judgement in assessment: a cognitive act and a socially situated practice, Assessment in Education: Principles, Policy & Practice, DOI:10.1080/0969594X.2012.736364. To link to this article, go here.

Linda Allal reports much of her research and work in French. You can find a listing of her French publications on her CV here. There is a powerpoint presentation about Assessment for Learning by Linda Allal available here.

The Expert Learner: Challenging the Myth of Ability

Delegates, in preparation for the International Symposium on Assessment for Learning in Fredericton, NB April 2014, have a lot of reading to do. More than 90 pre-readings have been submitted. It is a daunting and yet enjoyable task.

This evening, I enjoyed reading a chapter from a recent book by Gordon Stobart titled, The Expert Learner: Challenging the Myth of Ability.  

If you've ever had a chance to listen to Gordon Stobart, you'll have already experienced his wry sense of humour. It is always a treat to read a well-written, research-based text where the author has the ability to make a reader smile in delight. And, where such care is taken to help the reader understand that our students can become 'expert learners' if we but change the way we teach and the way we support student learning.

John Hattie is right in his review when he says this book is full of optimism. In this easy-to-read book, Gordon Stobart helps readers understand the "multiplier effect" and see powerful ways make a difference for all learners. 

Gordon Stobart clearly helps us see that none of us are prisoners of our genes or of our past - all learners can be taught to be 'expert learners.' You can find it as an ebook or order a print copy here.

Gordon Stobart is coming to Fredericton, NB in April 2014 and will be part of Assessment for Learning: Canada in Conversation with the World. Join us!

Tagboard: https://tagboard.com/AforLConversation/160953