Feedback on my feedback

While feedback is one of the most effective interventions, not all feedback is good. Many studies do show a negative effect of some feedback. An analysis by Kluger and DeNisi (1996) of 3000 research reports- and excluding a number of poorly designed reports- showed that effect sizes were ‘highly variable’ and 38% were negative. (Source: Dylan Wiliam at the Festival of Education)

Giving feedback isn’t enough. The quality of feedback matters and we spend so much time writing feedback and marking books that it is ridiculous not to ensure that the feedback is good enough. With that in mind, in this post I am casting a hyper-critical eye on a piece of feedback I gave which didn’t work in the way I wanted.

Following an essay on war poetry, a number of my year 9 students received this target: ‘Try to analyse the effect of specific language techniques’. They tended to spot a technique and use a quotation but the analysis was simple and along the lines of ‘makes the reader want to read on’ and ‘makes it more effective.’ They were given the following activity to support, based on The Woman in Black:

“…gazing first up at the house, so handsome, so utterly right for the position it occupied, a modest house and yet sure of itself, and then looking across at the country beyond. I had no sense of having been here before, but an absolute conviction that I would come here again, that the house was already mine, bound to me invisibly.”

Personification is a language technique that makes something which is not alive seem as if it is. Can you find an example in the passage above? Write the quotation.

Why do you think the writer might want the house to seem alive?

 Problem 1: The task doesn’t really offer any additional support

Example 1The first issue I spotted is illustrated by this example where the student has performed at the same level that they were performing at in the first place. They answered the question, but nothing in the activity, or my advice, gave them any support in answering the question better. The feedback itself might be a useful reminder for the next essay but if the student couldn’t do it before then a poorly framed task like this will not help them. The task was supposed to make students think about the effect of language. However, with no support to do it better, then nothing changed!

Problem 2: Poorly chosen example and poorly worded questions

The technique of personification was chosen because we were studying it in the following lesson. I think I picked the wrong example to illustrate this. Although the complexity of the text is not beyond the class, there are a number of aspects which needed explaining: handsome; modest; absolute conviction. Too much time was taken simply trying to work out what the passage was saying. This quotation is harder to analyse in the context of the whole text and I could have used an example from the start of chapter two which personified fog as evil. Students had just studied foreshadowing and pathetic fallacy so the analysis of the language would have been more straightforward and actually more precise if I had chosen that extract.

Even if the quotation had been straightforward, there is an imprecision in my design of tasks and questions. Technically, personification is a language technique that makes something which is not alive seem as if it is but this is not precise enough and certainly doesn’t get to the heart of how personification works in writing. In asking the question ‘Why do you think the writer…’, I felt that I was allowing the students scope to offer wide-ranging interpretations and I did get some thoughtful responses from students. However, the question didn’t have an explicit link with the previous one so students didn’t link the evidence they had chosen to the explanation of the effect. In the example below, I am happy that the student is exploring the effect but the response isn’t really linked to the particular quotation they have chosen.

 Example 2

 

 

 

 

 

Problem 3: The task doesn’t make them think solely about the thing I want them to think about

Doug Lemov writes in Practice Perfect,‘When teaching a technique or skill, practise the skill in isolation until the learner has mastered it.’

In this instance, students should have been thinking about the analysis part. I instead tried to make them learn a new piece of terminology (or at least refresh their understanding), find evidence and then analyse it. They had to do a lot of work before even getting to the key part. The final question could have been attempted without any of that part. The student below found the first part so difficult that they spent very little time on the analysis part. One student wrote nothing.

Example 4As I said earlier, I am being hyper-critical, but feedback is too important and frankly too time consuming for there to be any room for poorly designed feedback. Feedback needs to be designed so that they unavoidably think about the feedback and not other aspects. Explanations need to be crystal clear and precise to allow this to happen.

 

Evaluating the impact of written feedback

A couple of weeks ago I was putting together my slides for #TMSBradford and I took some photographs of mail merge feedback sheets completed by students. (This is one method I use for reducing time spent on marking while increasing impact.)

The problem was, I had to reject the first 3 or 4 examples because they hadn’t actually met the targets in the feedback exercise. Some had written very little and some had completed the feedback incorrectly e.g. used the wrong ‘there’ in a homophone activity.

feedback
This student has attempted the task but has not added all of the full stops.

This matters because it illustrated that the written feedback I had taken time to produce in these cases had zero impact! In some ways you could argue that there was a negative impact as a misconception was allowed to continue.

As a result of this, I then decided to look at a range of my books across all of my classes and look at the impact of my written feedback.

In some books, there was feedback that had no impact on future pieces of work- identified errors continued into the next task. Sometimes the errors were fixed immediately afterwards but appeared further down the line. In both instances, the time spent marking appeared to be wasted and the issues needed to be addressed again.

Of course, most of the time, students did act on the feedback and it is important to track those instances of success and do more of the things that work.

Dylan Wiliam, speaking at the Festival of Education, explained that written feedback is effective but that some studies actually show a negative effect. It is therefore not ok just to do written feedback, it must be of sufficient quality and have a worthwhile impact for it to be worth the time it takes. This is why this extra layer of evaluating the impact of written feedback becomes necessary.

I have already written about the entire process of making written feedback work. I would add the ideas that follow to that sequence so it becomes: before, during, after, evaluate.

Here are my recommendations for ensuring that written feedback does actually make a difference:

Ensure that any written feedback comes with advice on how to improve.

If you say ‘You need to organise writing into clear paragraphs’ then the few students who just forgot about paragraphs will possibly remember next time but the ones who don’t actually know when to take a new paragraph are not going to be able to understand just from that. It needs to come with some guidance on how to do that. Then students need an opportunity to put that guidance into practice. This is clearly the most important stage of the written feedback process so students need to be given time to do this. They also need to develop the mindset of relishing their feedback and wanting to use it to improve. Teachers need to assure students that their feedback is simply the most important thing for them to improve.

After giving feedback, read the responses to the feedback and ensure that any misconceptions are addressed.

Read the response to feedback immediately (or as soon as you can). If you don’t check that they have understood and improved as a result of the feedback then you risk the possibility that the feedback didn’t work. If students don’t improve from your feedback, refine it. Was the wording of your feedback helpful? Were students given sufficient time to read and respond? If a student just didn’t try then you can discuss it with them. This is obviously much more effective soon after the feedback.

If you only write a comment with no opportunity to act on the feedback then it is going to be difficult for you to ascertain whether they have even understood the feedback. Asking students to comment on your feedback or phrasing the feedback as a question might be one way to do that. Even then, you need to evaluate the quality of student responses. A comment of ‘Thanks Sir. I have read my feedback and understand it’ isn’t really that illuminating!

Tackle wide misconceptions in class too.

Take the opportunity to teach things that are coming up repeatedly in your marking. Do this in addition to students acting on the feedback. The feedback is much more likely to stick if it is accompanied by this.

The taxonomy of errors is a good approach to this. Andy Sammons has a couple of blog posts on the subject too: DIY LEARNING: Taxonomy of Errors and Using Taxonomy of Errors for feeding forward. This is my example of common errors in descriptive writing for use with a class.

Ensure that students revisit their targets repeatedly.

There are a number of ways to do this:

Make sure most recent targets are displayed on the front of exercise books or folders. Make them easy to refer to.

Another idea, which I will experiment with next year, is to create feedback bookmarks to be kept in exercise books.

Use the targets to feed forward into the next piece of work. Make this the first thing that you focus on when you mark.

Meticulously record targets for students and monitor them. I RAG them on a spreadsheet- If I don’t I lose track of the targets I have given in the past.

Have routines embedded to ensure students’ targets are memorised. This could be that they answer with their target in response to the register. It could be that you repeatedly ask individuals about their targets. It could also be part of a call and response.

(I’m not interested in students being able to recite these simply so they can say it when someone observing asks them, I want them memorised because I want them to be conscious of what they need to do and then do it in their work.)

Don’t give too much feedback to act on.

Sometimes you will identify a number of things that need addressing. You should be precise with your feedback to ensure that progress is made as a result of the feedback. Write too many targets and there is a danger that they will not be acted on.

Like any area of professional practice, we need to continually refine our approach to written feedback so that we maximise the impact. While I still rate the quality of my written feedback highly, there is still much more room for improvement.