The notschool students are all
individuals; some were keen to be in school, but unable to be
there, some irreparably damaged by experiences of school and
delighted to not be there.
Clearly there is no common base of
experience only the common consequence that they are not in school
and have not been there for some time. In this context many simple
research strategies are not open to us: direct interview,
questionnaire completion and standardised tests could all revive
echoes of past difficulties and damage the entire project's
contribution. Similarly there are parents who will be delighted
and willing to comment but also parents who are unwittingly or
consciously at the root of the student's difficulties, and any of
the standard interview techniques might trigger a hositility which
is further damaging. The problem is of course that we are not
clear, indeed cannot be clear, which student and which parent is
in which category. This means that whilst we have good
ethnographic evidence - from critical event logs, from letters and
notes, from recollection of direct contact, from actual activity
on-line... we cannot ever get any uniform meter without placing
progress at risk and clearly that would be unacceptable. What can
emerge though, is consensus from the aggregation of this
ethnographic evidence. An external evaluator will, by and large,
be evaluating the methodology rather than collecting
data.
It is also clear that when learning
has gone as spectacularly wrong as it has for these students, with
all their capabilities and strengths, many have a vested interest
in a particular interpretation of that failure. Threading a way
around the misinformation such as lack of esteem for the student,
blame and indeed the many personal agendas, can only be achieved
by those who are daily engaged in the project, and to that extent
the project personnel are key witnesses for the
research.
In short we need to protect the
students above all else and this will inevitably affect the
quality of the research evidence, but conversely it will vouchsafe
their progress and hopefully offer clearer evidence of what does
and does not work. It will always be better to have patchy
evidence of clear progress rather than robust quantification of
failure.
All these students differ.
Indeed their diversity is a part of the reason that school has not
fitted them well. There is no common base line in aptitude,
engagement, fragility, creativity... anything. It is clear that
some have retained an enthusiasm for one key area of learning:
fishing, art, arithmetic whilst others have a broad interest in
learning but no "major"; again diversity is the key
characteristic. If we might generalise at this point the students
are all more able as learners than the school system had
predicted. Falling out of school was a loss of opportunity both
for the students and for the schools, but this is a
generalisation. In this context there is no generic baseline to
measure project success against.
Harder still for research design as
we cannot establish that baseline for each student individually
either. In the two diagrams below a student at A in serious
decline might be viewed as a success if two terms later they were
still at A and the decline was arrested, but for a student at B,
already moving forward, the project might be disappointed if they
remained at B for even one term. We don't, of course, have the
luxury of retrospective analysis and are unable to plot the lines
illustrated, so we cannot apply a vector of change to any baseline
for each student. The exercise is pointless.
What we can do however is to be
confident about expectations.