Principles of testing for the classroom

This was the title of a Cambridge English Teacher webinar presented by Shakeh Manassian and what follows is a summary of what she had to say.


What is a test?

Talking mainly about summative assessment, a test is/has:

• a tool or device
• an activity which helps to elicit certain types of performance
• a measure of learner performance - evidence of what a learner knows, understands and can do
• a defined duration
• a clear purpose
• a standard delivery format
• tasks which relate back to what was taught and learned
• a variety of task types to ensure fairness
• an evaluation of the evidence with reference to a set of criteria or a standard (this could be our own or an internationally recognised one, such as CEFR)
• marks which are indicative of the learners' underlying ability
• results which are used to make inferences about the ability of the learners
• these inferences must link back to the purpose for testing the learner, and the skills and abilities we thought we were eliciting

Purposes of testing

A clear purpose helps to identify:

• the kind of evidence we need
• the task types

The purpose also links to the way we:

• mark the learners' work
• interpret learners' performances
• make inferences about learners' abilities
• report our findings
• make decisions

The purposes of testing include:

• giving feedback
• checking progress
• analysing learning needs
• deciding what you're going to teach next
• selecting for a particular course
• assessing suitability for the next level

The purposes of international language tests will be different from those you use in the classroom.  In most cases, the purpose is generally to report on the performance of candidates in such a way that organisations that use the results can make decisions, for example, for selection.

Qualities of good coursebook learning tasks

• Focused
• May come before or after a presentation of new language
• Are part of a series of learning tasks
• Fit in with the curriculum
• May lead to freer activities
• Often have an example at the beginning
• Are repetitive
• Help learners understand what has been taught
• Give learners an opportunity to practise

All of these things make learning tasks very different to testing tasks.  Such activities are not suitable in a testing context, not least because they repeat the same piece of language many times.

What makes a good testing task?

• it has a clear purpose which is stated in writing (for example, in a syllabus or handout)
• it's linked to a given model of language teaching and learning
• it makes the best use of the time available
• it tries to be authentic
• it isn't focused on a single element of language
• it has a clear marking scheme

What makes a good reading learning task?

• it exploits the reading text
• it develops a variety of reading skills
• it takes account of the classroom context
• it tries to develop other learning skills
• it tries to develop other language skills

What makes a good reading testing task?

• it tests a variety of reading skills.
• questions are ordered in the same order as the information appears in the text - we are testing comprehension, not information location.
• questions are clearly worded and are appropriate to the level.  All students should be able to access the questions - we are testing their ability to find the answers.
• questions should be unbiased.
• questions shouldn't contain the same wording as the text - we should paraphrase so that students have to demonstrate their understanding of the language.
• each question should test only one reading skill.
• all options in multiple choice questions must relate back to something in the text.
• there needs to be a clear indication of the marks being allocated to each response.
• questions must have the right level of difficulty.
• texts should be interesting to our students and as authentic as possible.
• texts must allow us to test the kind of skills we're interested in.
• the selection of the topic and the sources should reflect the purposes of testing.
• the length of the texts should be appropriate for our purposes and for the time available.
• a variety of texts and question types should be used to be fairer to the students and to enable us to make broader inferences about what a learner can do.
• the test must allow the comparison of performance across huge cohorts of candidates.

Testing speaking

• there needs to be a clear purpose.
• we need to have a clear idea of the performance we want to elicit.
• there needs to be a clearly defined format.
• the testing tasks and questions must be clearly designed.

Conclusions - some principles of testing

We need to have:

• a clear purpose
• a clear understanding of how the results will be used
• a clear identification of the performance to be elicited
• designed tasks that elicit enough of this performance
• tasks that are fair to learners
• a set delivery format
• clear marking criteria and marking schemes
• a clear approach to the interpretation of the results

We also need to:

• know what each of the questions we've designed is testing
• make sure that we're testing key skills more than once, but that we're not overtaxing the learner
• be able to relate the results back to the skills we've tested
• be consistent in our approach each time we test our learners
• document decisions and actions

 

Helping Students Improve their Speaking Abilities for Proficiency Tests - an #eltchat summary

Smith & Jones Talking Heads

This is a summary of the #eltchat which took place at 12noon BST on Wednesday, 11th July, 2012.  It was an informative discussion with most of the ideas coming from the participants themselves, rather than through links to external websites and articles.  The chat was expertly moderated as always, this time by @Shaunwilden, @rliberni and @BrunoELT.

We began by establishing that we were including all types of speaking tests in our chat, not just CPE.  We also agreed that the main worry for students taking a proficiency test (apart from the speaking itself!) is the thought of the exam room and wondering what the examiner expects.

Should we be aiming for accuracy or fluency?

Whilst in an ideal world we want our learners to be both fluent and accurate, the consensus seemed to be that fluency is the key for proficiency tests.  'Drying up' is the worst case scenario for most students and, if it happens, it knocks their confidence and it's very difficult for them to get going again.  Accuracy is more difficult to achieve in the heat of the moment and the effective communication of ideas is usually the most important thing.  If we focus on accuracy or grammar over fluency, the danger is that students think about words and structures and don't actually speak!

What role does personalisation play in a speaking test?

@teacherphili told us that, in his experience, some institutions 'help' students by making the test easier with familiar pictures.  This is probably not a problem if the tests are internal, but could mean that students are ill-equipped for external tests.

Of course, we all like to talk about ourselves, so it's helpful to have personal anecdotes to tell.  Students are given the opportunity to do this in all internationally recognised proficiency tests.  When students talk about something that matters to them, they sound naturally enthusiastic.


Practical ideas to help learners improve their speaking for tests:

• Start with lots of general fluency practice to build confidence and overcome the fear factor, before moving on to more exam-type activities.
• Encourage students to read about varied topics so that they have ideas.  If they do not have ideas, there test is over!
• Use speaking board games to encourage fluency.
• Play 'Just a Minute', based on the long-running Radio 4 programme - great fun and really engaging for the students, especially if you can supply them with bells, buzzers or whistles!
• Play the old favourite 'Chinese whispers' - good for listening practice and also for highlighting pronunciation issues.
• Play 'Impromptu Speeches' - someone plays the MC who invites students to speak for a minute on a topic drawn at random from a hat.
• Use storytelling where each student gives one line and the next must follow on in his or her own way.
• Try 'shadowing' - a technique where you begin by repeating what your partner says and then move on to paraphrasing, like a translator, but in the same language.  I attended a workshop on this recently and am currently 'road-testing' it in class.  Early results are good and I plan to blog about it soon.
• Record students as they practise for their speaking tests and encourage them to critique each other.  When I started doing this here in Vietnam, my students were initially reluctant to peer correct, but, over the weeks, I think I've unleashed a monster and I now have to remind students to give some positive feedback, too!!
• Use Web 2.0 tools to facilitate the recording process.  Start with mobile phones, which students find less intimidating, and then progress to sites like Audioboo, Voxopop or Vocaroo which can be used for students to build up portfolios.  These can then be used to show students their improvement over time.
• Use correction sheets to give feedback after speaking practice and review/revise at the start of the next lesson.  (Leave room on the sheet for some positive comments.)
• Use pictures and photos to spark conversation, especially from eltpics.
• Practise the long turn with silly topics - this reduces the stress and students can concentrate on sequencing their talk (via @andyscott55). This activity is a great precursor to the 'real' tasks.
• Get students to perform a live news programme as if they were on air.  They have to speak about exam-type topics and have performance pressure (via @Sharonzspace).
• Practice speaking via Skype (student to student, rather than student to teacher!).
• Don't forget to give advice on appropriate body language - a good speaking test score is not all about utterances.
• You should also remind students about the role of listening in dialogic speaking, linking this to appropriate (physical) responses.
• Get students to listen to/watch real candidates doing real tests to show them what is good or bad.  I use IELTS 5.0 from Garnet Publishing as a core text for some of my students.  This is a great book with lots of listening to real answers from candidates, which invites critique and reflective practice.
• Make sure that students know the format of the test inside out - there should be no surprises on the day.

Practical ideas to help students overcome their nerves:

• Encourage students to think silently for a minute about the picture or question before starting to speak. @timjulian60 believes that this can help them to gain confidence, though @Shaunwilden worries that examiners might misunderstand the silence and think that the candidate doesn't know what to say!
• We need to remember that even native speakers stumble when faced with high-level proficiency tests (CAE, CPE, IELTS, TOEFL, etc.) so lots of practice is needed in gathering and organising your thoughts.
• Nerves are a huge problem for my students, so I like to use roleplays with someone being their 'worst nightmare' examiner!
• If you are teaching YLs, funny accents and voices can help.
• Remind students that examiners are not monsters - that, in fact, they want the candidates to do well, but that they must demonstrate their ability.

(A few) Links:

• Some speaking activities from British Council Esol Nexus which provide learners with models to analyse and copy - via @pysproblem81.
• How we can use minimal pairs to help L2 listeners decode what they hear via @teflerinha.
• A review of @mybrainshark including the importance of practising for speaking tests - via @teacherphili.
• Spoken performances illustrating the 6 levels of the CEF via @louisealix68.
• Personalisation example from T>S for speaking practice via @teacherphili.
• Constructive alignment via @louisealix68.