Sunday, June 1, 2008

Designing Tools for Evaluation

Back in 2000-2002 during a research project called MIRA Multimedia Interface for Research and Authoring I created a tool for reflective practitioners in school called Multimedia Clipboard. Later on, most of the features of this tool have been integrated into the OLEFA IMS-CMS http://www.olefa.com which offers a powerful web based environment including a database module that can be designed and personalized for multiple uses.

Multimedia Clipboard permitted the collecting and grouping of multiple types of data in order to create layered and multi-perspective descriptions and interpretations of learning situations.
With this kind of tools teachers can evaluate students learning products and processes by creating chunks of information, and include background information of situations and concepts in which these products, processes and their interpretation are embedded.

Once an item is created by importing for example a video clip, the interpretation can be broadened by adding descriptions from different perspectives (comments by students, peers, teachers, parents etc.) and deepened or "thickened" by adding layers of information and interpretation (see the concept of "thick descriptions" by Clifford Geertz and Gilbert Ryle).
This approach is a lot different from a quantitative evaluation, but also from a so called evaluation by competencies in which learning is split up into modules, key stages and key competencies and in which students learning outcomes are, in the end, always compared to normative clusters of knowledge.

The greatest difference however lies in the fact that evaluation by competencies is proclaimed to be formative but neither sets the learner nor the teacher in the focus of an inquiry but the implicit norms.

In opposition to this, a descriptive and interpretive (ethnographic) approach puts the learner and the teacher(s) in the focus because their judgments, ethical considerations and theories become part of the evaluation process. Here, teachers are not observers, but actors in a learning process, and an evaluation is created or co-constructed through collaboration and negotiation. And, students are not objects of evaluation and inquiry but co-evolving subjects. A tool like Multimedia Clipboard is as much a instrument for evaluation as a means to gain insight in the functioning of evaluation and the nature and power of the hidden theories involved - including those which gave birth to the tool itself. Furthermore, and because of the non-linearity of the tool, a small change in the tool has the potential to open a high variety of new possibilities to interconnect information. A small change in a check list of competences - which I consider to be a linear system - will never have such an impact.

By adopting open instruments and ethnographic approaches, evaluation will take the shape of a narrative based on self-organization and reflection instead of a diagnosis conforming to a given structure. We will also reach a high level of unpredictability, variety, redundancy and even messiness, while simultaneously reducing centralized control and normative comparability between individual learning outcomes. On the other hand we will open to all actors involved a space for self-reflection, creativity and double-loop learning.

As there is a growing demand for alternative forms to evaluate students learning, I think that evaluation by normative competencies cannot be but a transitional stage before adopting open tools like Multimedia Clipboard which will extend the possibilities and concepts of so called learning records and portfolios and which will at the same time force teachers in a position of self-reflection.

Some Reading Milestones

  • Towards reflexive method in archaeology : the example at Çatalhöyük (edited by Ian Hodder) 2000

  • The Book of Learning and Forgetting (Frank Smith) 1998

  • Points of Viewing Children's Thinking: A Digital Ethnographer's Journey (Ricki Goldman-Segall) 1997

  • Verstehen lehren (Martin Wagenschein) 1997

  • Computer im Schreibatelier (Gérard Gretsch) 1992

  • The Boy Who Would Be a Helicopter. Uses of Storytelling in the Classroom (Vivian Gussin Paley) 1991

  • La cause des adolescents (Françoise Dolto) 1988

  • Scuola di Barbiana. Die Schülerschule. Brief an eine Lehrerin. (Edition of 1980) / read in German 1982
    Letter to Teacher by the Schoolboys of Barbiana (1970)
    Lettre à une maîtresse de'école, par les enfants de Barbiana (1968)
    Lettera à una professoressa (Original Edition) 1967


  • Vers une pédagogie institutionnelle (Aïda Vasquez, Fernand Oury) 1967



Documentary Films on Education

  • Eine Schule, die gelingt (by Reinhard Kahl) 2008

  • Les temps des enfants (Jacques Duez) 2007

  • Klassenleben (by Bernd Friedmann und Hubertus Siegert) 2006

  • Lernen - Die Entdeckung des Selbstverständlichen
    (Ein Vortrag von Manfred Spitzer) 2006

  • Die Entdeckung der frühen Jahre
    Die Initiative "McKinsey bildet" zur frühkindlichen Bildung (by Reinhard Kahl) 2006

  • Treibhäuser der Zukunft - Wie in Deutschland Schulen gelingen (by Reinhard Kahl) 2004

  • Treibhäuser der Zukunft / Incubators of the future / Les serres de l'avenir; International Edition (by Reinhard Kahl) 2004

  • Journal de classe, 1ères audaces (1), Les échappés (2), Sexe, amour et vidéo (3), L'enfant nomade (4), Remue-méninges (5) (by Wilbur Leguebe, Jacques Duez, Agnès Lejeune) 2004

  • Spitze - Schulen am Wendekreis der Pädagogik (by Reinhard Kahl) 2003

  • Journal de classe, (by Wilbur Leguebe and Agnès Lejeune; Jacques Duez) 2002

  • Etre et Avoir (by Nicolas Philibert) 2002

  • The Stolen Eye (by Jane Elliott) 2002

  • The Angry Eye (by Jane Elliott) 2001

  • A l'école de la providence (by Gérard Preszow) 2000

  • Blue-Eyed (by Jane Elliott) 1996

  • A Class Divided (by Jane Elliott) 1984

  • Eye of The Storm (with Jane Elliott) 1970

Past quotes of the day

For every problem, there is one solution which is simple, neat and wrong. Henry Louis Mencken

Traveler, there is no path. Paths are made by walking.
Antonio Machado

The best way to predict the future is to invent it. Immanuel Kant

The significant problems we face cannot be solved at the same level of thinking we were at when we created them. Albert Einstein

To paraphrase a famous quotation, all that is necessary for the triumph of damaging educational policies is that good educators keep silent. Alfie Kohn

We used to have lots of questions to which there were no answers. Now, with the computer, there are lots of answers to which we haven't thought up the questions. Peter Ustinov

I had a terrible education. I attended a school for emotionally disturbed teachers. Woody Allen

A professor is someone who talks in someone else's sleep. W. H. Auden

When I was an inspector of schools I visited one classroom and looked at a boys book. He'd written, 'Yesterday, Yesterday, Yesterday, Sorrow, Sorrow, Sorrow, Tomorrow, Tomorrow, Tomorrow, Love, Love, Love.' I said, 'That's a lovely poem.' He said, 'Those are my spelling corrections.' Gervase Phinn

Real thinking never starts until the learner fails. Roger Schank

If what is wanted is a reexamination of schooling in terms of purpose, structure and process, then testing programmes are the wrong vehicle (...) Caroline V. Gipps

Not everything that counts can be counted, and not everything that can be counted counts. Albert Einstein

Act always so as to increase the number of choices. Heinz von Foerster

Another way of avoiding teaching is by relying exclusively on a textbook, workbooks, and other commercially packaged learning materials. Teaching is reduced to administering a set curriculum without giving any thought to the substance of what the students area learning or to their particular needs. H. Kohl

The right to ignore anything that doesn't make sense is a crucial element of any child's learning - and the first right children are likely to lose when they get to the controlled learning environment of school. F. Smith

Learning is the human activity which least needs manipulation by others. Most learning is not the result of instruction. It is rather the result of unhampered participation in a meaningful activity. - Ivan Illich

Too often we give our children answers to remember rather than problems to solve. - Roger Lewin

I have never let my schooling interfere with my education. - Mark Twain