As is normally the case the world isn’t as tidy and simple as we would like it.    Hattie’s study is a good example of research in that it gathered data from across a number of different studies and contexts, plus over a period of time.   It therefore presented findings which could be more easily generalised across educational settings and contexts.   The issue here is the generalisable nature of the findings.    It means that the findings “generally” hold true.   In specific contexts or situations it is therefore possible that the findings may not hold true.   Looking at education in general this is all well in good but teachers are dealing with individual students in their classrooms and therefore should be seeking to find what works for each child.    Holding too strong a view in relation to research findings may lead to practices that don’t work with certain students being applied because the research shows they “generally” work.   Worse still it could lead to practices that do work in a given situation and/or context being labelled as “generally” inappropriate and not being tried.     We need to see educational research as a guide but be careful to understand that in some situations, doing the opposite may equally be effective.

Hattie’s study is based on a thorough and large data set meaning its statistical reliability is reasonable high.   One problem with educational research is that most studies are not based on such a large data set.   They are often based on a very small sample of schools and students.   Studies are often conducted within a specific context such as a certain geographical area, national or region culture, certain age range or curriculum subject.    The validity of the findings when generalised outside the context of the study is often questionable.   I remember my own masters level study when we were guided on the need to state that the findings “suggested” or “pointed towards” as opposed to “demonstrating” or “showing” something to be true.   You will find in most good education research a similar language in the conclusions.    Without a large amount of data gathered from different contexts across a period of time it is highly unlikely any research findings can be generally applied across all or even most educational contexts.    Even where findings are generalizable this doesn’t mean they are replicable in an individual context.

I need to be clear, I am not saying we shouldn’t use educational research in directing practice in individual schools and classrooms.   What I am saying is we should do so with an awareness of the limitations, and bear these in mind.

The step I am currently undertaking as part of my bid to continually improve is to seek some anonymous feedback from colleagues with regards leadership where I myself am one of the leaders to which those invited will provide feedback.

Sticking your head above the parapet so to speak is never easy and never without some worry or concern with regards the feedback you may receive.

From a research perspective the responses received will be based on the interpretation of the questions being asked and then the perception of the individuals providing the feedback.    Their perception may be coloured by recent events, which due to ease of recall will appear more important than more frequently occurring events which may have resulted in an inverse response.    An individuals state of mind and emotional state on the day they provide their feedback may have an impact on the feedback they provide.    Where a person is having a good day and therefore feeling positive, they are more likely to respond in a positive fashion however where they are having a bad day, where the world is against them the opposite is also true.    If they have recently received bad news the response is also likely to be less positive.

From a statistical point of view I know there are various ways I can interpret the data with each approach potentially resulting in different findings.    A simple look at the highest and lowest average scores may seem to suggest the strengths and areas for development however a look at standard deviations may indicate a high average resulting from some widely fluctuating scores.    This initially apparent strength may therefore turn out to be either inconclusive or even an area for development.

Given all these variables it may be easier to decide to avoid asking the questions.   My choice however is to ask the questions as I would prefer to have data which may upset me rather than having no data at all.   At least if I have upsetting data I have a position to work from and to improve from as opposed to existing in blissful ignorance and therefore having no clue that things need improving.   I also have a baseline to work from in terms of checking if any actions taken have made any difference.

I await the results of the feedback with an element of trepidation and an element of anticipation.