Back in 2000-2002 during a research project called MIRA Multimedia Interface for Research and Authoring I created a tool for reflective practitioners in school called Multimedia Clipboard. Later on, most of the features of this tool have been integrated into the OLEFA IMS-CMS http://www.olefa.com which offers a powerful web based environment including a database module that can be designed and personalized for multiple uses.Multimedia Clipboard permitted the collecting and grouping of multiple types of data in order to create layered and multi-perspective descriptions and interpretations of learning situations.
With this kind of tools teachers can evaluate students learning products and processes by creating chunks of information, and include background information of situations and concepts in which these products, processes and their interpretation are embedded.
Once an item is created by importing for example a video clip, the interpretation can be broadened by adding descriptions from different perspectives (comments by students, peers, teachers, parents etc.) and deepened or "thickened" by adding layers of information and interpretation (see the concept of "thick descriptions" by Clifford Geertz and Gilbert Ryle).
This approach is a lot different from a quantitative evaluation, but also from a so called evaluation by competencies in which learning is split up into modules, key stages and key competencies and in which students learning outcomes are, in the end, always compared to normative clusters of knowledge.
The greatest difference however lies in the fact that evaluation by competencies is proclaimed to be formative but neither sets the learner nor the teacher in the focus of an inquiry but the implicit norms.
In opposition to this, a descriptive and interpretive (ethnographic) approach puts the learner and the teacher(s) in the focus because their judgments, ethical considerations and theories become part of the evaluation process. Here, teachers are not observers, but actors in a learning process, and an evaluation is created or co-constructed through collaboration and negotiation. And, students are not objects of evaluation and inquiry but co-evolving subjects. A tool like Multimedia Clipboard is as much a instrument for evaluation as a means to gain insight in the functioning of evaluation and the nature and power of the hidden theories involved - including those which gave birth to the tool itself. Furthermore, and because of the non-linearity of the tool, a small change in the tool has the potential to open a high variety of new possibilities to interconnect information. A small change in a check list of competences - which I consider to be a linear system - will never have such an impact.
By adopting open instruments and ethnographic approaches, evaluation will take the shape of a narrative based on self-organization and reflection instead of a diagnosis conforming to a given structure. We will also reach a high level of unpredictability, variety, redundancy and even messiness, while simultaneously reducing centralized control and normative comparability between individual learning outcomes. On the other hand we will open to all actors involved a space for self-reflection, creativity and double-loop learning.
As there is a growing demand for alternative forms to evaluate students learning, I think that evaluation by normative competencies cannot be but a transitional stage before adopting open tools like Multimedia Clipboard which will extend the possibilities and concepts of so called learning records and portfolios and which will at the same time force teachers in a position of self-reflection.
When we talk about evaluation, it is very important to put the student and the teacher in the focus, because it will give them a tool to become reflective praticioners.
ReplyDeleteThe problem of all quantitative evaluation is that teachers believe that it comes after the fact. Items are not real problems. Current tests focus on aptitudes. They yield results that are analyzed on the national level rather than used to improve the performance of students. They want students to plug-in what was “learned” out of context. If you learn to drive, you won’t do it by written tests.
Evaluation by competencies will still be decontextualized and output-measuring, say it is measuring the past. Those tests aren't enabling, they aren't forward looking. Current assessment practices place too much emphasis on the assessing content (various skills who had been put under drill instruction) and aren't linked to learning, neither in time nor in space.
Above all those tests allow us to make placement decisions. Don’t forget that student scores can be used to evaluate schools and teachers.
By using comparative test scores to assess the relative “quality” of teachers and schools, it is no wonder that “teaching to the test” is a common practice.
Tests are built by and for educational systems to produce “the needs for its own services and products, i.e. the labour and instruments of correction” (Bourdieu).
Complex problem solving, divergent thinking and collaborative efforts of learners aren't taken into account. As people don't increase their competency (notice the word) by building a tower of new facts and skills, but by transforming their knowledge structures, this reconfiguration should be observed and taped, filmed, noticed,...
New strategies and models can't be measured by tests or by evaluating by competences (this learner has got 7 out of 10 - so does he know or not, that is here the question).
But as evaluation isn't quite so romantic, nor should it be so dramatic or let's say shakespearian, we should focus on process recording in real learning situations, a more “authentic” assessment, or challenging performance assessment.
A tool like the one who was set up by Pino would be a good one to record ongoing processes.
So if we have to assess, (and I don’t feel the need to assess everything that is happening in classroom, because I don’t think that a quantitative evaluation provides us with an oppurtunity to focus on learning processes), we have to link assessment, curriculum and instruction.
As we have adapted the curriculum to the needs of the students (at least we try so), and our “instruction”, we have to adapt assessment.
We should plan assessment (if it has to be) together with instruction, with the learning process, so that assessment can give us an opportunity to push instruction forward.
There we will have to take risks and to learn by mistakes, just as we expect our students to do.
As we try to not disturb the community of practice that the classroom makes up, we should involve peer assessment, which will relieve us of some grading responsibilities.
Students will be responsable by constructing their responses when we use a more challenging assessment.
When we talk about evaluation, it is very important to put the student and the teacher in the focus, because it will give them a tool to become reflective praticioners.
ReplyDeleteThe problem of all quantitative evaluation is that teachers believe that it comes after the fact. Items are not real problems. Current tests focus on aptitudes. They yield results that are analyzed on the national level rather than used to improve the performance of students. They want students to plug-in what was “learned” out of context. If you learn to drive, you won’t do it by written tests.
Evaluation by competencies will still be decontextualized and output-measuring, say it is measuring the past. Those tests aren't enabling, they aren't forward looking. Current assessment practices place too much emphasis on the assessing content (various skills who had been put under drill instruction) and aren't linked to learning, neither in time nor in space.
Above all those tests allow us to make placement decisions. Don’t forget that student scores can be used to evaluate schools and teachers.
By using comparative test scores to assess the relative “quality” of teachers and schools, it is no wonder that “teaching to the test” is a common practice.
Tests are built by and for educational systems to produce “the needs for its own services and products, i.e. the labour and instruments of correction” (Bourdieu).
Complex problem solving, divergent thinking and collaborative efforts of learners aren't taken into account. As people don't increase their competency (notice the word) by building a tower of new facts and skills, but by transforming their knowledge structures, this reconfiguration should be observed and taped, filmed, noticed,...
New strategies and models can't be measured by tests or by evaluating by competences (this learner has got 7 out of 10 - so does he know or not, that is here the question).
But as evaluation isn't quite so romantic, nor should it be so dramatic or let's say shakespearian, we should focus on process recording in real learning situations, a more “authentic” assessment, or challenging performance assessment.
A tool like the one who was set up by Pino would be a good one to record ongoing processes. Isn’t evaluation a certain search for proof that tends to set the focus on the visual aspect of learning? A classroom is a complex environment and common evaluation standards aren’t the best way towards understanding. Complex systems tend to be very unstable and unpredictable. This level of unpredictability is very important. Students come to class every day with their developmental stages but the trajectories in which development occurs can be partly unpredictable. This unpredictability gives us a good opportunity for self-reflection (for students and teachers). I think we have to focus on it.
When you flip a coin, what can you predict?
Well you can predict with 50% that it will land heads up. But what about how many times it will flip and roll? Where will it land? Will we find it or will it land in the grass and it will be out of sight for a moment?
So if we have to assess, (and I don’t feel the need to assess everything that is happening in classroom, because I don’t think that a quantitative evaluation provides us with an oppurtunity to focus on learning processes), we have to link assessment, curriculum and instruction.
As we have adapted the curriculum to the needs of the students (at least we try so), and our “instruction”, we have to adapt assessment.
We should plan assessment (if it has to be) together with instruction, with the learning process, so that assessment can give us an opportunity to push instruction forward.
There we will have to take risks and to learn by mistakes, just as we expect our students to do.
As we try to not disturb the community of practice that the classroom makes up, we should involve peer assessment, which will relieve us of some grading responsibilities.
Students will be responsable by constructing their responses when we use a more challenging assessment.
I remember the Multimedia Clipboard Pino presented us during the MIRA project, and I wondered some times ago, if it still exists. During the project I was really excited about this tool, and the first “try outs” one of the project members showed to us were really convincing. If you consider now the ongoing portfolio discussion, I would say that this tool could be used as an “electronic portfolio”. One of the advantages would be the linking-up of videos, texts, background information, products, interviews, and different points of viewing (student, parents, teacher and others…). Thus the information about the student and his/her learning will be “thickened”. Moreover, the evaluation will not be “second hand”, as the reader will be able to consider the product and the background information him – or herself. On the other hand the traditional evaluation leaves the interested observer with an interpretation of an achievement expressed through points and notes.
ReplyDeleteAs Pino and Joël Mischaux already pointed out, the traditional evaluation is more or less summative. And changing to an evaluation by competencies will not have much of an affect on this tradition, if we stick to the tests or the standardised tests, the ministries all around Europe are keen to present as the answer to the educational “misery”.
Anyhow I consider this change of focus on competencies as a step into the right direction. It is true that the evaluation by competencies includes implicit norms to which the students are compared. (Until now they were compared to norms the single teacher put up in his tests.) On the other hand, any teacher acting seriously according to an approach by competencies will have to reflect on his concept of evaluation. Every student will have “a different level” of attainment for any competence. It would not be credible if the teacher would put all the students through the same test or to any test at all! Thus she/he will have to reflect on her/his evaluation methods. This will be the time to introduce new ways of assessment including cooperative reflections between students, teachers and the other actors in school-life. And may be the Multimedia Clipboard could be one tool enabling all the actors to participate in evaluation.