The ‘Why’ of Testing

March is almost over. When it ends, so officially ends the school year here in Japan as well. Which means that tests have been marked, desks wiped down, and grades entered into each student’s report card. Again this year I am left with the nagging feeling that those grades, those indelible marks of ‘achievement’, fail to capture the important story of what learning is all about. But at least this year there were moments where I did feel that assessment was doing its job. That students were taking the kinds of tests which highlighted their own development and which fostered, as opposed to inhibited, learning. I would like to share two of the alternative assessment techniques I used in my classroom this year. While neither one was perfect, they both helped my learners, and myself, come to a better understanding of how and why we take tests in my school.

Student Designed Testing

This year, I taught one ‘standard’ English class built around a lexis and grammar focused syllabus. Each unit consisted of 3 target grammar points and a 1500 word article containing a high number of set phrases and multi-word verbs. Instead of focusing on memorisation of vocabulary and grammar rules, I spent a majority of class time helping students develop the underlying skills they need to be more effective language learners. We worked on summarising skills, creating comprehension questions as a means of checking understanding against a classmate’s, and techniques for identifying multi-word verbs and phrases. At the end of each unit, one group of four students was designated to design the unit-quiz which would then be administered to the other students. While some of the skills students learned in class made it onto each quiz, most of the questions required nothing more than memorisation of large chunks of text and the application of discrete grammar rules. When I followed up with students, one group explained that for the standard tests they would be taking in the future, whether it be a university entrance exam or the TOEIC test, knowing lots of vocabulary and grammar rules would help them get the best possible grade. It was a classic example of negative washback, or the tendency for teachers to teach to a test. Only in this case, it was the students themselves who had chosen to learn specifically for a high stakes test they would be taking sometime in the future.

Collaborative Learning Assessment & Social Testing

In a recent article on ‘social testing’, Tim Murphey (2013, p. 30) points out that, “our minds are no longer, if ever they were, isolated, independent, and individual entities, but rather our minds and our brains are interconnected and networked and work best with other minds in collaboration.” The way we test students on their learning is becoming more and more disconnected from the way students utilise that learning in the real world. By expecting students to merely study in order to fill in blanks on a piece of paper, or answer questions put to them by a teacher during an oral interview, we are arbitrarily cutting them off from the collaborative process of meaning making which is crucial to most learning outside of the classroom. With that in mind the final exam for my intermediate 4 skills English class this year consisted of the following stages:

  1. Students, in small self-selected study group of 3 or 4, had two 50-minute class periods (as well as any time they wanted to use after school) to work through their class notes to create a 3×5 inch study-card which contained all the information they thought might be useful when taking the final exam.
  2. Students took the test with the aid of their study-card.
  3. At the beginning of the next class, I gave the tests back to the students unmarked. They were then given 15 minutes to make changes or additions to their exam using a red pen. They were not allowed to use their study-cards at this point.
  4. Students were then given 10 more minutes to consult with members of their own study group and make further changes.
  5. Students were given 10 minutes to consult with members of other groups and make any final changes or additions to their answers.
  6. I collected and graded the exams giving an original score, a score after revision, and a final score which was an average of the two.

I realize that, on the face of it, this certainly doesn’t seem like much of an evaluation. Students who had not, in any traditional sense, ‘learned’ the class content still had the potential to receive a passing grade. But in the end, the students’ test grades—a test which was extremely long and covered almost all the material for 8 months of coursework—did not significantly differ from their scores in class up to that time. I would also argue that students, through this process, ended up gaining something vital to their future as language learners. They learned to see studying for exams as a dynamic process and to recognise the importance of social capital. As Murphy points out, there is real value in helping students develop, “their ability to learn more socially rather than testing an isolated brain unconnected to others. (ibid. p. 29)” In feedback interviews with students at the end of the school year, many students highlighted how much more information they retained from this final exam than from other tests. One student in particular said, “I always study just so I can pass the test. Then I forget everything as fast as I can because I really hate studying. But this time, I remember most of what was on the test. Especially the things I learned from the other students.”

A final word (which is hardly the final word)

Just as our students rarely focus on and learn what it is we thinking we are teaching during our regular class time, my students’ ideas of the value and importance of a test is often very different from my own. Some take tests purely to get a good grade. Some to prepare themselves for a future, even more difficult test. Some for the pleasure of learning. Perhaps an important side-effect of thoughtful assessment is to allow students to recognise the various and sometimes conflicting reasons for taking a test. Before students put pencil to paper or begin to review their notes, they should be encouraged to consciously evaluate for themselves why they are taking an exam, to ask, “Why and how am I preparing for this test?” If they can find a meaningful answer to that question, they might see the evaluation process as a valuable opportunity to grow as a language learner, regardless of the grade handed back to them at the end of the year.

References:

Murphey, T. (2013). Social Testing: Turning Testing into Healthy Helping and the Creation of Social CapitalPeerSpectives, 10, 27-31

Join the conversation on assessment, the heartbreak and hope, by letting us add a link to your assessment blog post here on the iTDi ‘Assessment is…” issue:

Divya Madhavan, Matt Shannon, and Rose Bard in our original ‘Assessment Issue.’

Chuck Sandy, Luke Meddings, Hengameh Ghandehari and David Dodgson from the ‘More on Assessment’ Issue

Assessment is Sometimes Heartbreak

Naomi EpsteinIt’s that time of year again at the high school. The twelfth graders are about to take a series of final exams before graduating. Every year there are a few students who break my heart. But this year one student seems to stand out in particular.

naomi290315-3

We’ll call him P. Just like the other heartbreaking students before him, he “bought” the school system’s slogan “hard work = success”, worked hard, did his homework, missed very few classes and reviewed the material. Unlike those other students, he remembers vocabulary items better than most of the students in all my classes. He’s curious about words, and brings in brings in words he encounters online. Even more remarkable, he demonstrates a more extensive world knowledge than many of the other Deaf & hard-of-hearing students I teach.
This week we had another Mock Exam in preparation for the finals. Unseen reading passages are the main and most important section of our final exams. Students at his level are required to answer questions that summarize the main idea of a paragraph and demonstrate a command of vocabulary and grammar.

naomi290315-2

The topic of the unseen reading passage was changes at NASA. P. knew what NASA was — not something to take for granted in my classes. He remembered to use the highlight-marker the way we practiced. P told me proudly that he had remembered some of the words without using the dictionary.
Once again P. got the lowest grade in his class: a barely passing grade. Lower than students who, to put it politely, are not model students at all.
Despite all the ways we work on reading comprehension, he can’t seem to integrate the information in the text well. Some things (in every test) baffle him even after we discuss them in mother tongue. In the aforementioned text there was a paragraph explaining how in the past only NASA employees could work on space projects (today the situation is different, that is one of the changes presented). P. simply could not understand the answer to the question related to who used to pay the people who worked on space projects (P. was able to translate the word employee correctly into his mother tongue). We discussed it for 10 minutes afterwards and he still did not see the connection between the word “employee” and how it implied the source of the payment. I tried to give examples closer to his reality, (in mother tongue!) such as the fact that I teach him at school (I’m not his employee) vs. a private tutor who could come to his home (he is then the employer). P. still didn’t understand it. Other students did not have a problem with this question!

naomi290315-1

After every test P. looks so disappointed to see his peers get higher grades, while he barely gets a passing grade. He knows he works harder than they do. He looks at me and what can I say?!!
We just continue practicing…

Reassessing Assessment

Dave DodgsonThink of assessment and many teachers think of tests – that moment of fixed time when all the learning of the past few weeks, months or years comes down to a series of multiple choice questions about grammar, vocabulary, course content, with perhaps a writing task or reading comprehension thrown in.

Tests are one enduring feature of education. In every language learning setting I have worked in, tests have been a regular and dominant feature. They are used to determine success or failure, report grades, and determine how ‘good’ someone is judged to be in English.

There are alternatives of course: project work, portfolios, oral assessments, self-assessment and more, some of which I have discussed in a previous iTDi post. I previously worked with young learners and tried to give them as much of a say in their assessment as possible, even getting them to compose their own exam questions, but this has had limited success. I did this for my classes in the year group I was teaching in but it was not adopted elsewhere in the school. Prescribed questions on discrete language items still held sway and I expect my old area of influence has now reverted back to the same format.

However, my new job has offered the chance to start afresh in many ways. I am now the coordinator of a new language school for adults in Gabon. I have set up the school, designed the learning programme, and implemented an assessment programme from scratch. This post will detail how I have approached this task.

In essence, we have three kinds of ‘tests’: we have ‘placement tests’ to put new students in the right level; there is also an ‘exit test’ which takes place at the end of each 6-week course; and we have external exams such as TOEIC and TOEFL. The external exams have been included because of market demand. Even here in central Africa, such exams are much sought after by employers and universities so they are a necessary component of our programme. Like them or not, they are a demand we have to cater to, and there is little we can do to change the style or content of those tests.

However, I have had the chance to affect the style and content of those other tests. The quick turnaround of courses means there is little time to get students to produce their own questions but there are other ways I can get these tests to be more meaningful, purposeful, and therefore more indicative of the students’ language level.

The placement tests

  • They begin with the speaking component. There are a few ‘starting point’ questions but our teachers are trained to let the conversation flow depending on the input of the student.
  • There then follows some short reading and writing tasks (graded according to the performance in the speaking test) but not with multi-choice questions or strict instructions. The tasks are designed to provoke thought and the questions are open-ended to give the student a chance to show what they can produce rather than what grammar they know (or can guess!)
  • This gives a much more accurate impression of where the student will fit into our learning programme than a multi-choice test would. Students can freeze when faced with a grammar based test. Conversely they can get lucky with their choices in an A/B/C/D test. Our speaking test gives them the chance to loosen up and our written test gives them the chance to express themselves and fits much better with the philosophy of our school.

End of course assessment

  • It actually begins at the start of the course! In the first week of a new 6-week cycle, the students are given a written task to complete and a speaking activity to engage in. The writing task provides a wealth of information about where the student is in terms of their English ability and is used to inform the focus of the course (see this recent post by Willy Cardoso for another explanation of this).
  • The same tasks are then repeated in the final week of the course. The students and teacher then compare the earlier effort with the more current one to identify areas of progress and things that still need to be worked on. This is great for showing students how far they have come and in making learning plans for future courses.
  • The final week also includes a self-assessment task asking the student to identify their progress and what they are struggling with. This is then discussed with the teacher in an end-of-course feedback meeting.
  • There is ‘pen and paper’ assessment as well but it contains no grammar or vocabulary questions. Instead, it features an ‘input text’ (a reading passage, an audio recording, or a video). Students are then asked to respond to it through a series of open-ended questions or a short written response.
  • All of this tests not what points of grammar they have memorised but what language they can use and how they can express themselves. Surely that is what language learning is all about!

I have been lucky in this job. I was asked to design a learning and assessment programme and I was trusted to implement my own ideas. Not everyone is in this position. You may work in a school where there is a designated ‘test writer’ or tests that come with the course book are used as a standard.

But what I would say to that is ask ‘why?’ Why is a test prepared and imposed by someone from outside the class? Why is an isolated test of the content of recently taught units used? Ask these questions. Suggest alternatives. Start a conversation about assessment. Decide with your colleagues and with your students what approach to testing will work best to measure language ability and to inform future learning.

That is where the importance of testing lies, not in the summative or the formative but in the informative. Start the conversation, get the information, and use it to move forward.

 

Outside Influences Issue

Teachers learn about teaching from teachers. But teachers can be found in so many places, many of them far away from a traditional classroom. In this issue of the iTDi blog, the ‘Outside Influences’ issue, we learn about the people outside of ELT who have helped iTDi members on their journeys to become the teachers they are now.
Kevin SteinHow to Change a Life by Kevin Stein
Josette LeBlancAn Outside Influence from Within My Family by Josette LeBlanc
Syke A.K. portraitWalking with K.T. Mohandas by Syke A.K.
Chuck SandyUnleash the Superpowers by Chuck Sandy

How to Change a Life

Kevin SteinWhen I was 16 years old, I applied for a job as a summer camp counsellor. For 11 years I had spent each and every one of my summers at Camp Tamarack. I couldn’t think of anything I wanted to do more than work at the camp that had played such an important part in my growing up. Before camp started, there was a two week intensive training session which included role plays with a licensed social worker, lessons on child development, workshops on drama activities, and first aid training. We were working from morning revelry at 6 AM, our breath white in the cold Michigan July mornings, to 10 PM at night. On the last night of the pre-camp training, the assistant director of the camp, gave the following speech:

In three weeks, the counsellors of the oldest kids at camp are going to be taking their bunks out on a canoe trip for 6 nights and 7 days. On the last mile of that trip, the river reverses course because of a dam. That last mile of canoeing is nearly impossible. 17 years ago, there was a young camper on the very same canoe trip. He was the smallest member of his bunk. He spent the whole summer trying just to keep up with the other kids who seemed so much bigger, so much stronger than him. During the canoe trip, he felt pretty useless. He paddled, but it felt like he was just dipping his paddle into the water. But when they got to the point where the river reversed its current, everything changed. He could feel the water pushing against his paddle. He could see the other campers struggling. So he paddled as hard as he could. And the canoe barely pushed its way through the water. When they finally reached the small stretch of beach where they landed the canoes, that small boy couldn’t feel his arms. He couldn’t even lift the paddle he was holding high enough to put it in the truck waiting to carry the canoes and paddles back to camp. At that moment, his counsellor came up to him and said, “You know, there was a moment when I thought we weren’t going to make it. I was just about to pull the canoe to the side of the river and have us carry the canoe the rest of the way. But then I looked at you, at how hard you were paddling, and I knew we would make it. And we did. We made it because of you. Now you might be wondering how I know so much about just what that small camper did and heard that day. I know because that camper was me. And it’s because of what my counsellor said to me 17 years ago that I’m standing up here in front of you today. It’s the reason I work at this camp. It changed my life. Each and every one of you can do the very same thing. Each one of you can change a child’s life. Not by empty praise, but by watching your campers carefully. By noticing when they do something that perhaps they never thought they could do. And at that moment, letting them know that you noticed it, too. That yes, they really did do something extraordinary. That’s what it means to be a camp counsellor.

When I heard that speech I realised that yes, that is what I wanted to do as well. I wanted to be there for campers and students and friends at just those moments, to have the chance to change a life. That was over 27 years ago. I still haven’t figured out how to always notice those moments. And I certainly haven’t found the right words to say each and every time I am lucky enough to notice them. But, on a warm summer night, the day before my campers came to put their trust in me for a summer, I took the first step to understanding what being a teacher might be all about.

The iTDi community is the richest and most diverse group of teachers I’ve ever had the good fortune to work with. I feel lucky to have had the chance to read the interviews in this issue, to learn more about the people who have helped the iTDi-ers become the gifted and unique teachers that I have come to know. And with this issue, we are also taking the conversation to the wider community. A number of iTDi bloggers have written ‘Outside Influences’ posts on their own blogs. I hope you will take a few moments to read and enjoy them as I have. And if you decide to join in the conversation and write a post for your own blog, let me know and I will added to the growing list of teachers sharing the stories of the people who have helped them help make iTDi the place where, ‘Together, we can change the future.’

‘Outside Influences’ blog posts from around the web:

Sandy Millin on the (heroic) women of her family

James Taylor on Carl Sagan

Vicky Loras on her first real teacher

Anne Hendler on why sometimes you need a turtle

Josette LeBlanc continues the conversation with her father

Theodora Pap on her personal champion, Kostas Michalakis

Me (again) on learning laughter