Principles of testing for the classroom

This was the title of a Cambridge English Teacher webinar presented by Shakeh Manassian and what follows is a summary of what she had to say.


What is a test?

Talking mainly about summative assessment, a test is/has:

• a tool or device
• an activity which helps to elicit certain types of performance
• a measure of learner performance - evidence of what a learner knows, understands and can do
• a defined duration
• a clear purpose
• a standard delivery format
• tasks which relate back to what was taught and learned
• a variety of task types to ensure fairness
• an evaluation of the evidence with reference to a set of criteria or a standard (this could be our own or an internationally recognised one, such as CEFR)
• marks which are indicative of the learners' underlying ability
• results which are used to make inferences about the ability of the learners
• these inferences must link back to the purpose for testing the learner, and the skills and abilities we thought we were eliciting

Purposes of testing

A clear purpose helps to identify:

• the kind of evidence we need
• the task types

The purpose also links to the way we:

• mark the learners' work
• interpret learners' performances
• make inferences about learners' abilities
• report our findings
• make decisions

The purposes of testing include:

• giving feedback
• checking progress
• analysing learning needs
• deciding what you're going to teach next
• selecting for a particular course
• assessing suitability for the next level

The purposes of international language tests will be different from those you use in the classroom.  In most cases, the purpose is generally to report on the performance of candidates in such a way that organisations that use the results can make decisions, for example, for selection.

Qualities of good coursebook learning tasks

• Focused
• May come before or after a presentation of new language
• Are part of a series of learning tasks
• Fit in with the curriculum
• May lead to freer activities
• Often have an example at the beginning
• Are repetitive
• Help learners understand what has been taught
• Give learners an opportunity to practise

All of these things make learning tasks very different to testing tasks.  Such activities are not suitable in a testing context, not least because they repeat the same piece of language many times.

What makes a good testing task?

• it has a clear purpose which is stated in writing (for example, in a syllabus or handout)
• it's linked to a given model of language teaching and learning
• it makes the best use of the time available
• it tries to be authentic
• it isn't focused on a single element of language
• it has a clear marking scheme

What makes a good reading learning task?

• it exploits the reading text
• it develops a variety of reading skills
• it takes account of the classroom context
• it tries to develop other learning skills
• it tries to develop other language skills

What makes a good reading testing task?

• it tests a variety of reading skills.
• questions are ordered in the same order as the information appears in the text - we are testing comprehension, not information location.
• questions are clearly worded and are appropriate to the level.  All students should be able to access the questions - we are testing their ability to find the answers.
• questions should be unbiased.
• questions shouldn't contain the same wording as the text - we should paraphrase so that students have to demonstrate their understanding of the language.
• each question should test only one reading skill.
• all options in multiple choice questions must relate back to something in the text.
• there needs to be a clear indication of the marks being allocated to each response.
• questions must have the right level of difficulty.
• texts should be interesting to our students and as authentic as possible.
• texts must allow us to test the kind of skills we're interested in.
• the selection of the topic and the sources should reflect the purposes of testing.
• the length of the texts should be appropriate for our purposes and for the time available.
• a variety of texts and question types should be used to be fairer to the students and to enable us to make broader inferences about what a learner can do.
• the test must allow the comparison of performance across huge cohorts of candidates.

Testing speaking

• there needs to be a clear purpose.
• we need to have a clear idea of the performance we want to elicit.
• there needs to be a clearly defined format.
• the testing tasks and questions must be clearly designed.

Conclusions - some principles of testing

We need to have:

• a clear purpose
• a clear understanding of how the results will be used
• a clear identification of the performance to be elicited
• designed tasks that elicit enough of this performance
• tasks that are fair to learners
• a set delivery format
• clear marking criteria and marking schemes
• a clear approach to the interpretation of the results

We also need to:

• know what each of the questions we've designed is testing
• make sure that we're testing key skills more than once, but that we're not overtaxing the learner
• be able to relate the results back to the skills we've tested
• be consistent in our approach each time we test our learners
• document decisions and actions

 

Helping your students to become effective writers

Julie Moore

This was the title of a recent webinar hosted by Oxford University Press and presented by Julie Moore.  What follows is a summary of what she had to say.

Writing in a digital age

What do we write?

• shopping lists
• greetings cards
• e-mails
• text messages
• Facebook updates and comments
• online reviews
• Tweets
• blogposts and comments
• personal profiles
• etc., etc.

Of this list, perhaps only the first two are handwritten.  Everything else is typed.  We are probably doing more writing than ever before, but not by hand.  Writing content isn't just by professionals anymore.  Everyone writes and publishes online.  Writing skills, therefore, should be an increasingly important part of what we teach, but they are often neglected.  Far too often, writing is moved to a homework task and then taken in and marked, but without any meaningful teacher input.

 Noticing features of texts

The first step in improving students' writing is getting them to understand how writers convey information and ideas effectively to their readers.

When speaking, you're usually face to face.  You're in the same place, you have a shared context.  You have gestures and facial expressions to help with communication.  You can ask for clarification.  Also, listeners tend to be sympathetic and forgive grammatical errors.  In writing, the situation is totally different.  You often don't know who is reading your work.  You need to be much clearer and pay much more attention to accuracy when you write.

How should we notice?

We should start with a reading text and work with it in the usual manner - lead-in, vocabulary analysis, comprehension questions, etc. - so that students become very familiar with it.  Then we can go back and analyse the features of the writing in order to understand what makes it readable.  It's important to integrate reading and writing skills.

What features should we notice?

1. Genre - When writing, it's very useful to start with a model to learn about the genre; the appropriate style of writing, the type of vocabulary to use, etc. 
2. Structure - We can focus on a particular aspect of structure, topic sentences for example, and get students to recognise it and understand what makes it effective.
3. Coherence and cohesion - Get students to look for the language and mechanisms that link ideas together.

Follow the noticing with a writing activity.  Ask students to transfer the principles of what they have seen to their own writing.  You must set them an easier task to write than the text they've just read.  For example, you could get students just to write effective topic sentences.  When they've written them, swap with a partner who has to guess what's to come in the rest of the paragraph.  This will tell them if they have written a good topic sentence.  Alternatively, when a complete writing task is done, ask students to highlight or underline their topic sentences and check that they're clear and that all of the supporting sentences are relevant.

The important thing is to get students thinking about how they write and how they convey their meaning to the reader.

Stretching students' writing

The key here is to focus on a specific skill.  For example:

Summarising

• this is a useful skill in many contexts (profile writing, review writing, etc.)
• it's about expressing key information concisely
• it's paraphrasing and reusing language from the text - very important to prevent plagiarising, especially in the 'cut and paste' age.
• it's ideal for pairwork and groupwork

If you've been working with a text for a while and the students are very familiar with it, it's a valid exercise to ask them to turn it over and summarise it from memory.  For longer and more complex texts, get students to extract some keywords from which they then have to write a summary.

Single sentence summaries are fun and quick to do.  Give students news stories and ask them to summarise them in just one sentence.  For example, a newspaper article might elicit:

Gareth Bale moved to Real Madrid for £85 million.

 

You then ask students to elaborate, for example, by asking, 'Who's Gareth Bale?'  This could lead to:

 

Welsh footballer, Gareth Bale moved to Real Madrid for £85 million.

 

'What is Real Madrid?'  'Where did he move from?'

 

Welsh footballer, Gareth Bale moved from Tottenham to top Spanish team, Real Madrid for £85 million.

 

'When did he move?'  'Is £85 million a lot for a footballer?'

 



In September, Welsh footballer, Gareth Bale broke the world transfer record when he moved from Tottenham to top Spanish team, Real Madrid for £85 million.

 

Now, we have a very good summary sentence!  We have stretched the students so they have written more, but still limited them to one sentence.  This encourages students to use prepositions, reduced relative clauses, etc., and moves their writing from the simple sentence to a much more complex one without ever having to talk about grammar!  We are gently pushing our students to do better.

 

Stretching students' writing - a summary

• Focus on specific micro-skills: summarising, describing a sequence or process, persuading, making suggestions, etc.
• Give short, classroom activities which involve pair and group work.
• Gradually build and stretch language skills
• Reading/listening/speaking into writing - you could get students to listen and then summarise what they heard - the summary doesn't have to be based on a reading.  You could even get students to summarise a group discussion.

Drafts and feedback

 

How do we give feedback on students' writing?

 

Some traditional ways:

• Correct everything.
• Underline or highlight where mistakes are and ask students to correct their own work.
• Use a writing correction code.
• Write comments (both positive and negative) on students' work.

 

Some alternatives:

• Give focused feedback - don't try to give feedback on everything at the same time because students are overwhelmed, put the work away and forget about it.  It's better to focus on one area - structure, content, grammar or language.
• Write questions and prompts that students have to respond to.
• Get students to rewrite parts of their work, based on the feedback you've given, rather than all of it.
• Get students to evaluate, check and edit before they hand in their work.  Ask students to bring their work to class and give them clear guidelines on how to check it - perhaps in groups.
• Teach editing an proofreading skills.

Develop a personal proofreading checklist

 

1. Get students to make their own proofreading checklist of common language issues, based on feedback on previous writing assignments.  They could include:

• general language areas - spelling, punctuation
• areas of grammar - subject-verb agreement, articles, etc.
• areas of vocabulary - uncountable nouns, collocations, etc.

If you are going to do this in class, get students to bring in lots of examples of their work from past lessons.

 

2.  It can be very difficult to spot errors in your own writing, so get students to work in groups and discuss how they can use the tools and techniques below to systematically check their writing.  Get them to come up with other ideas of their own.

• computer spell-check
• computer 'find' facility
• dictionary
• peer feedback
• reading aloud

Teaching writing skills - a summary

• Noticing features of texts: reading into writing.
• Understanding and applying key principles.
• Focusing on specific writing skills.
• Stretching students' writing skills.
• Varying feedback.
• Teaching editing and proofreading skills.



Switching on the lightbulb - getting students to come up with ideas

This was the title of a recent webinar hosted by Macmillan Education and presented by Lindsay Warwick.  Having just completed two years in Vietnam, where generally students had few thinking skills, this was a topic of great interest to me.  What follows is a summary of what Lindsay had to say.

Sometimes the problem isn't how to say something in English, but, rather, what to say.  For this reason, it's extremely important to scaffold activities properly.

Assessing knowledge

Sometimes students don't have ideas because:

• they don't have a background in thinking at all (if they have been in an education system where all the learning was by rote, for example)
• the topic isn't relevant to them and they have no knowledge of it

To assess knowledge of a new topic, put pictures pertinent to the topic around the room and pin a blank piece of paper under each one.  Ask students to walk around the room and write questions about the pictures on the sheets of paper.  They're not allowed to repeat any questions so they are pushed to come up with less obvious ones.  Then put the students into groups and give each group one of the pictures and the accompanying list of questions.  Ask them to discuss the questions and speculate on what the answers might be.  Give them some useful language to help them with this:

• Perhaps/probably/possibly
• It might/could be ...
• I guess ...
• It's likely/unlikely
• I doubt it's ... because ...
• I think ...
• I'm pretty sure ...
• I'm not sure but ...

As the students are doing this, monitor them to assess their knowledge of the topic.  Things they don't know could make a good homework task - students have to go away and do the research.  Students are much more likely to be able to discuss things if they have the appropriate background knowledge.

We can use this version of Bloom's Taxonomy to ask the right questions and help our students to be better thinkers:

 

 

This helps us to plan better lessons.

 

A practical example

 

Consider the question:

 

What are the pros and cons of a formal national identification system?

 

First, give students a quiz to test their knowledge of the subject.

 

 



This encourages students to speculate and opens their minds to the issues involved in the topic.  (Answers - 1. F - not the USA, 2. T, 3, T, 4. F - 11 million, 5. T)

Next, give students a list of points and ask them whether or not they are relevant to the question under discussion:

 

In this case, B, D and J are irrelevant to the discussion - they are not pros or cons.  This is a challenging activity to be used with higher level students.

Next, get the students to sort the points into pros and cons.  Give half the class the pros and the other half the cons and ask them to rank them in order of importance.  Get them to do this individually first, then with a partner, then with a small group, and so on up to half the class.  After doing this, students will be better able to discuss the question and better able to come up with their own ideas when the next discussion question comes along.

Debatabase

This is a great website for finding out about debates and discussions.  It lists points for and points against for discussion topics and is a really useful resource  for teachers to help us prepare for in-class debates.

Scaffolding

It's extremely important to give our students guidelines.  For example:

How has our relationship with the environment changed?
Give at least five examples.

We need to push our students and encourage collaboration.  For example:

What do you consider the main causes of disease?
Work in pairs and consider economics, lifestyle, emotional well-being, as well as medical reasons.  Think of at least two examples of each.

Good scaffolding is the means by which we support our students and guide them in the right direction.

What's the question?

Give students five answers to the same discussion question and ask them to come up with the question.  For example, these answers:

should lead to the question:

What would the world be like without energy?

Five caps

This activity encourages different ways of thinking by asking students to consider things from different perspectives.  We do this with role-plays in the classroom and we can do the same thing with discussions.  Give the students a question.  For example:

How has social networking changed the lives of young people today?

Then give each student a 'cap' - teenager, parent, teacher, employer, government think tank member - and ask them to debate the question from their character's point of view.  Invariably, the conversation will last much longer than it would otherwise have done.

Five minute Guardian debates

These are five-minute videos, created by The Guardian newspaper in the UK and made available on You Tube, which can be used as the basis for in-class discussions for B2 level students and above.

Problem solving: reverse brainstorming

Here, we take a problem and reverse it.  By looking at an issue from the other direction, we can often see it more clearly and so generate more ideas.  For example, if the original question is:

How do you think the problem of over-consumption can be addressed?

Consider instead:

What do you think the causes of over-consumption are?

Decision tree

This is a good way to push students to think harder about the solutions they propose - what would the impact/results be?

In the example above, for instance, the consequences of an education campaign might be a) that it's expensive and b) that it only reaches young people.

Mindmeister.com is a good website for creating all kinds of mind maps.

Prove it!

We're constantly making assumptions and sometimes these are wrong.  Give students a list of generally held truths and ask them to prove them to be true or false.  They have to go away and research, leading to a whole new discussion on where to find good sources (digital literacy).

In the examples above, only 3 and 5 can be proved to be true, and then only in a limited way.  These are great discussion topics!

Connections

Give students some pictures and ask them what the link is between them.  For example, here there could be a connection in each column:

 

1. Dutch, 2. the future, 3. water, 4. man-made inventions.

 

Then you could mix the photos, or ask students to make connections across the rows or diagonally.  The idea is to get students to think creatively and come up with whatever connections they can think of, however tenuous, as long as they can explain their reasoning.

 

I wonder

 

Students have to think of a question based on a theme represented by a picture.  Here, for example, the theme is 'old and new':

 

 

Questions could be things like:

 

I wonder what the builders of the old tower would think if they saw the modern office block?

 

I wonder how much taller the new building is than the old one?

 

I wonder if these two buildings are in the same town?

 

Thinking time

 

Use a site like tricider.com to get students thinking outside of class.  Post a question on the site.  Students answer it and then comment on each others' ideas.  This is great because it gives students time to formulate their ideas, something which is not always possible in class.



 

 





 

Warmers, fillers and other quick activities

This was the title of a recent webinar hosted by Oxford University Press and presented by John Hird.  What follows is a summary of what he had to say.

Some examples of warmers & fillers

1. Pairs - show students a collage of pictures and ask them to put them into pairs.  For each pair, students should say what they like and what they don't like.  For example, 'I like apples, but I don't like oranges.'

 



As a variation for higher level groups, you could give them a series of sentences about the pictures, ask them to work out the code or 'rule', and then get them to add sentences of their own.

For example,

• I drink coffee, but I don't drink tea.
• I like sheep, but I don't like cows.
• I read books, but I don't read magazines.

Here, the code is the double letter:

• I drink coffee, but I don't drink tea.
• I like sheep, but I don't like cows.
• I read books, but I don't read magazines.

2. Sick sheep - Tell your students, 'I have a friend who has a farm.  He has 45 cows, 32 pigs and 20 sick sheep.  How many sheep does he have?'  They will probably all say 26!!  :-)

3. I-Spy - a good way to revise vocabulary, particularly at beginner and elementary level when studying classroom language.

4. Word (dis)association - students have to say a word which has some association with the word given by the previous student.  For higher level students, word disassociation is a good variation.  Here, students have to come up with a word which has no link with the previous word.  If a link can be argued by others in the class, then no point is given to the student who came up with the word.

5. Categories - students have to come up with things in a category - shapes, colours, sports, animals, etc.  At higher levels, you can encourage students to think laterally - six things that are round, for example.  Students work in teams and get extra points if they come up with something which no other team has thought of.  As a variation, you could play 'match the teacher' where teams get extra points if they think of the same items which you had written down beforehand.

Why do warmers?

• to wake students up
• to energise and stimulate
• to focus minds
• to get them into 'English mode'
• to get them talking
• because they're fun
• because they're preparation free

Why use fillers?

• to fill time
• to change the pace and dynamics of the lesson
• because they're flexible
• to help students relax
• because they're fun
• for speaking practice
• because they're motivating
• because they're preparation free!

Why use lead-in activities?

• to introduce the topic
• to activate the schema - both knowledge of the topic and the language within the topic
• to focus minds
• to generate interest
• to pre-teach vocabulary

Examples of lead-in activities

1. Brainstorm - this is the classic lead-in using some kind of mind map or spidergram.

2. Guess the topic - use realia or pictures related to the topic.  Introduce them one at a time and get students to speculate on the theme.

3. What's in my pocket/bag? - students ask yes/no questions to find out what it is.  Try to make the item personal, so that you have a story to tell the students - a concert ticket or football programme, for example.  If they can't guess, slowly reveal the item.

4. Top 3s - ask students to list their top three in a category (cities, food, movies, music, etc.), compare their lists with their classmates, and explain the reasons for their choices.  You could start by modelling it yourself and explaining your own choices.

5. Class vocabulary list - each student has a piece of paper.  They have to write a word on it - an adjective to describe a city, for example.  The papers get passed around the class and everyone adds their word to everyone else's piece of paper.  If their word is already on a piece of paper, they have to think of another one.  When students get their original paper back, everyone has a list containing the same words.  Students are producing their own record and generating their own language.

Consolidation activities

• review
• recycle
• revisit

1. One and only - ask students to think of words to do with a topic covered in class.  They only get a point if they are the only one to write that word.

2. Alphabet race - students have to think of a word pertaining to the topic studied which begins with each letter of the alphabet.

3. Help the teacher - the teacher tells a story and pretends to forget key words.  Students have to supply these words.  For example, around town:

I went to town on Saturday.  I left my car in the ____________.  It's next to the _________ - oh, what's it called? It's where they put on plays and musicals.  First, I went to the ___________ because I needed some flowers for my Mum. etc. etc.

 

Students have to shout out the answers.  They generally find this a very engaging activity.

 

4. Mime it - the teacher mimes some vocabulary and students have to guess what it is.  The student who gets it right repeats your mime and adds one of their own, and so on.

 

Getting the most out of coursebook activities

 

1. Picture descriptions

• Students can look at a picture in a coursebook for a while, cover it, and then describe it, or answer questions about it, from memory.
• Students can speculate about the lives of the people in a picture.
• Teachers can identify and teach extra vocabulary from pictures.
• Students can speculate about what's happening in the picture, what happened just before it was taken, and what happened afterwards.
• Students can invent a back story.
• They can guess who took the photo and why.

2. Extra questions - students write extra questions about a text for classmates to answer.  This is a good exercise for early finishers.  Alternatively, students write the answers and their classmates have to come up with what the question would be.

 

3. Act it out - students act out tapescripts, adding sound effects, etc.  This is a great way to practise pronunciation and intonation.

 

4. Book quiz - ask questions at the end of a lesson, a unit, or the entire book, to see how much students can remember.