Tag: data analysis

Microsoft PowerBI

Microsoft PowerBI is an excellent tool for use in presenting and analysing school data, allowing staff to explore and interact with data which traditionally may be locked away in complex and very flat spreadsheets.

Schools have access to a massive amount of data.   This includes information about each student, academic data from assessment and testing, or from professional judgments made by teachers.    Secondary schools will also have baseline data such as the Centre for evaluation and monitoring (CEM) MIDYIS or ALIS data.   You will have data on attendance and on where students have been acknowledged for their efforts, or where they have had to be warned regarding poor effort or behaviour.  The above only scratches the surface of the available data.   For me this has long been a challenge in that all of this data is usually in difficult to read spreadsheets, where without well developed skills in using excel for example, trends and patterns will not be easy to identify. Even with well developed spreadsheet skills, attempts to analyse and interpret will be time consuming.  In addition it is often extremely difficult to bring together data sets such as looking for possible links between academic data, behaviour, attendance, etc.

PowerBI allows you to take all of this data and start exploring it.    You can create reports which present the data in simple graphical form however allow for the data to be explored.    For example you might display the count of behaviour issues by gender.   Clicking on a given gender would then filter to this gender, thereby allowing you to see other graphs such as academic performance or attendance by the selected gender, but also still showing the full cohort average, thereby allowing you to see where a particular subset of students vary from the average.

The above example shows how PowerBI displays focus on a given subset of data within graphs.  The dark pink bars relate to the selected focus whereas the light pink show dark for the whole data set.

Clicking other graphs would then allow you to easily explore other subsets of the data.   You can create reports allowing filtering by SEN status, native language, gender, subject, year and any other fields for which you have data.

PowerBI also comes with its own analytics engine which will analyse your data and identify where subsets of your data deviate from the average.     It is clear Microsoft are continuing to develop this functionality as when I first used this it identified correlations which were obvious and therefore of little use however more recently when I have used the analytics it has identified some more useful correlations.   I suspect this area will be further developed over time bringing greater potential for how it could be used.

The one drawback with PowerBI at this point is licensing.    For free you can create your PowerBI reports for individual use or can share these as files for viewing in the Desktop application complete with full editing rights however the main potential as I see it is to centrally create PowerBI reports and share them via Sharepoint so that staff can access as and when required but without the ability to change the report and without the complexity of the desktop applications interface.   You basically present them with a web page of the data for staff to interact with and explore using the graphs and other visuals and filtering provided by the person who creates the report.   For this Microsoft are currently charging a per user per month cost.      Given the potential power which PowerBI could provide to schools my hope is that Microsoft will eventually reconsider this and make PowerBI more affordable for use by schools.

PowerBI for me is about putting school data in the hands of staff in a way that is quick and easily to interpret plus usable.   It is about being able to explore data by simply clicking on individual elements and about using the data we already capture more efficiently.    With carefully crafted reports, generated through discussion with staff, the time taken to manage and analyse school data can be reduced, yet staff can be empowered to know and use the available school data appropriately.   If you haven’t tried PowerBI with you school data I would recommend you give it a try.

 

Thoughts from the Bryanston Education Summit

I attended the 2nd Bryanston Education Summit during the week just past, on 6th June.   I had gone to in the inaugural event last year and I must admit to having found both years to be interesting and useful.   The weather both years has been glorious which also helps to add to the event and the beautiful surroundings of the school.   Here’s hoping Bryanston keep it up, and run another event next year.

During the day I attended a number of different presentations on different topics so I thought I would share some of my thoughts from these sessions.

The first presentation of the day was from Daisy Christodoulou who was discussing assessment.    She drew a really useful analogy in comparing preparing students for their exams with preparing to run a marathon.    It isn’t something where you can jump straight into a marathon distance on day 1 of training.  You need to slowly build up your preparations, focusing on developing certain skills and approaches.   You need to have a plan and then work to this plan, although amending it as needed as you progress, should injury arise or due to weather conditions, etc.    I found myself wondering about how often we actually spend with our students in discussing this plan, the proposed goal of the subject or year and how we will all, teachers, students, support staff and others, work towards those goals.

Daisy also spent some time discussing summative versus formative assessment suggesting that the use of grades should be kept to a minimum of only once or twice per year.   My first reaction to this was concern as it seemed to disregard the potential benefits of spaced retrieval testing which ultimately would result in a score representing the number of correct answers.   Following further thought my conclusion was that spaced retrieval is very focussed on knowledge plus just indicates where an answer is right or wrong as opposed to grading which is more a judgement of students ability.   As such it may be possible to reduce overall summative assessment grading while still making regular use of testing of student knowledge.   I think this also highlights the fact that assessment and testing are actually different things even although they are often generally used as two interchangeable terms referring to the same thing.

Mary Myatt was the second presenter who discussed how we might make learning high challenge but low threat.    As she discussed Sudoku I couldn’t help but draw parallels with computer gaming.  In both case we engage, of our own free will, in a form of testing.   In both cases the key is the low threat nature of the testing.    For me the question is therefore how do we make classroom learning and assessment low threat.    Mary suggested a path towards this in discussing with students our expectations such as setting reading outside their current ability level, which is therefore challenging, but telling them this and then promising to work through it with them in future lessons.   I think this links to building an appropriate classroom culture and climate such that students feel able to share the difficulties they face and work through them with the class.  It is very much about developing an open culture and positive or warm climate in which mistakes and difficulties are not seen as something to be feared or embarrassed by, but to be embraced, shared and worked through together.   Another thing I took away from Marys session was a list of books to read;  My bookshelf will be added to with some of her recommended books shortly.

The third of the sessions which I found most useful was the session by Andy Buck.    He discussed leadership drawing a number of concepts from the book Thinking Fast and Slow by Daniel Kahneman, a book which is one of my favourites.     I particularly enjoyed the practical demonstrations where he evidenced how we all show bias in our decision making.  This is a fact of being human and the way the brain works, we bring to decision making processes assumptions and viewpoints based on previous experiences, upbringing, etc.   He also, linked to this, demonstrated anchoring, managing to influence a whole room of educational professionals to get a question in relation to the number of Year 11 students in the UK wrong.   Statistics suggest that a percentage of the audience should have got this question correct based on a normal distribution of responses however using anchoring Andy influenced the audience away from the correct answer.   I have since used a very similar approach in a lesson with Lower 6 students to show how easily I can influence their answer and to suggest that Google, Amazon, Facebook, etc. with their huge amounts of data on individuals may therefore be able to influence individuals to a far greater extent.

There was also a presentation on VR in education which has opened my mind up a little to the possible applications of VR.   This might therefore be something we experiment with at school in the year ahead.

Microsoft’s Ian Fordham presented on the various things Microsoft are currently working on.   I continue to find the areas Microsoft are looking at such as using AI to help individuals with accessibility and in addressing SEN to be very interesting indeed.   I also was very interested by his mention of PowerBI as I see significant opportunities in using PowerBI within schools to build dashboards of data which are easy to interrogate and explore.    This removes the need for complex spreadsheets of data allowing teachers and school leaders to do more with the data available however with less effort or time required.    I believe this hits two key needs in relation to the data use in schools, being the need to do more with the vast amounts of data held with schools however the need to do it in a more efficient way such that teachers workload in relation to data can be reduced.

I also say a presentation by Crispin Weston on data use in school.    His suggestion that we need to use technology more to allow us to more easily analyse and use data is one I very much agree with.   This partly got me thinking about the Insights functionality in PowerBI as a possible way to make progress in this area.   He also talked about causation and correlation suggesting his belief that there is a link between the two and that the traditional call that “correlation is not causation” is in fact incorrect.   At first I was sceptical as to this however the key here lies in the type of data.    Where the data is simple and results in a simple linear trend line the resulting reliability of an argument that correlation equal causation is likely to be very low.   The world is seldom simple enough to present us with linear trends.    If, however the data over a period of time varies significantly and randomly and the second data element follows this however the reliability that correlation equals causation is likely to be significantly higher.     I think the main message I took away from Crispins session was to take data and findings with a pinch of salt and to ensure that context is taken into account.  If it looks simple and clear then there is something which hasn’t been considered.

Overall the day was a very useful one and the above is a summary of just some of the things I took away.   I must admit to taking 5 or 6 pages of tightly written notes, hastily scribbled on an iPad during the course of the day.

I hope that Bryanston decide to repeat the conference next year and is the quality of presenters and their sessions continues, that it becomes a reliable yearly event.   Here’s hoping the trend of good weather also continues should they decide to run the summit again next year.

 

 

 

PowerBI and School Data

Ever since I started playing around with PowerBI I have found it to be very useful indeed and I must admit that I am most likely only scratching the surface.

I came to experiment with PowerBI to try and address some issues I see with data management.    School data is often presented in colour coded spreadsheets showing student performance against baselines for example.   Different sheets are used to present different views on the data such as showing the performance by subject, by gender or the performance of students by SEN status or by EAL status.   Each additional view on the data, of which there are very many, presents us with another sheet of data.  The data is often presented as flat tables of figures however in some cases may involve pages upon pages of different graphs and charts each showing different views on the available of data.   The logic here being that each additional view on the data gives us more data that we can interpret and therefore a greater opportunity to draw insightful conclusions and from there develop actions.   I believe the reality is the reverse of this.

My belief is that teachers and heads of department don’t have a lot of time to analyse and interpret data, and therefore presenting them with so much data is counterproductive.  Having so many different views on the data presented at once also is difficult to process and to understand.   This in turn leads to either ignoring the data altogether or to giving it only a very cursory glance.   For those that love data it may lead to excessive amounts of time spent poring of the data, to data overload, where time spent planning actions, as opposed to analysing data, would be more productive.    As such I subscribe to the belief that “less is more”.

This is where PowerBI comes in.    PowerBI allows me to take my mountains of spreadsheet data and present it in a very easy to digest graphical format where each of these graphs and charts are interactive.    In PowerBI rather than one sheet by subject and another sheet for gender based data, you have just one set of graphs and charts.   You would just click on a gender or select a gender and all the graphs will change to show the results for that gender.   You might then click an SEN status to see how students who are male with SEN needs are doing compared to students on average.    This means we can combine all our different views which are normally represented by different sheets on a spreadsheet into a single set of graphs and charts.   The user then accesses the various views of the data by clicking on and through these graphs and charts.

The benefit of PowerBI is the ability to dynamically manipulate and explore the data by clicking through various graphs and filters.   You develop an almost tangible feeling for the data as you explore through it.   This is something that flat spreadsheets, even if graphs are included, lack.   Also, as you have less to look at, in one set of graphs rather than pages and pages of them, you have more time to explore and engage with the data.

The one current drawback to PowerBI is simply cost.   It is free to use as an individual both web based or via a desktop application, and you can share via sharing desktop app developed BI files however if you want to share via the web platform or if you wish to publish internally via SharePoint you will need a Pro license for each user.    Where you are sharing with a large number of users, even at educational pricing, this can become expensive.   Hopefully this is something Microsoft will be looking at and can resolve in the near future.

Schools continue to be sat on mountains of data.    PowerBI is a tool which allows us to present this data in a more user-friendly form which then allows it to be easily explored and manipulated, allowing more time to plan actions and bring about continuous improvement.  If you haven’t already done so I definitely recommend putting some of your school data in PowerBI and having a play with its capabilities.

School Data: The tip of an iceberg

Schools gather a wealth of data in their everyday operation, everything from attendance information, academic achievement, library book loans, free school meals and a wide range of other data.    We use this data regularly however I think we are missing out on many opportunities which this wealth of data might provide.

The key for me lies in statistical analysis of the data looking for correlations.     Is there a link between the amount of reading a student does as measured by the number of library loans and their academic performance for example?     Are there any indicators which might help is in identifying students who are more likely to under perform?

The issue here is how the data is stored.   A large amount of the data is stored in tables within our school management system however no easy way exists in order to pull different data together in order to search for correlations.    I can pull out data showing which students have done well, which subjects students perform well in, etc. however I can’t easily cross link this with other information such as the distance the student travels to school or their month of birth.    Some of the data may exist in separate systems such as a separate library management system, print management system and catering system.    This makes it even more difficult to pull data together.

A further issue is that the data in its raw format may not make it easy for correlations to be identified.    Their postcode for example is not that useful in identifying correlations however if we convert this to a distance from the school we have a better chance of identifying a correlation.

In schools we continue to be sat on an iceberg worth of data although all we can perceive is that which lies above the water.   We perceive a limited set of possibilities in terms of what we can do with the data.    Analysing it in terms of pupil performance against baselines with filtering possible my gender, SEN status and a few other flags however given the wealth of data we have this is just the start of what is possible.    We just need to be able to look below the water as the potential to use the data better and more frequently is there, and in doing so we may be able to identify better approaches and more effective early interventions to assure the students in our care achieve the best possible outcomes.

Data: Making better use?

One of my areas which I want to work on over the next year will be that of Management Information.   In my school as in almost all schools we have a Management Information System (MIS), sometimes referred to as a SIS (School or Student Information System).    This systems stores a large amount of student data including info on their performance as measured by assessments or by teacher professional judgement.    We also have data either coming from or stored in other data sources such as GL or CEM in relation to baseline data.   These represent the tip of the iceberg in terms of the data stored or at least available to schools and their staff.

Using the data we then generate reports which do basic summaries or analysis based on identified factors such as the gender of students, whether they are second language learners of English, etc.  Generally these reports are limited in that they consider only a single factor at a time as opposed to allowing for analysis of compound factors.   So gender might be considered in one report and then age in another, but not gender and age simultaneously.   In addition the reports are generally reported in a tabular format, with rows and columns of numeric values which therefore require some effort in their interpretation.    You cant just look at a tabular report and make a quick judgement, instead you need to exercise some mental effort in examining the various figures, considering and then drawing a conclusion.

My focus is on how we can make all the data we have useful and more usable.    Can we allow staff to explore the data in an easier way, allowing for compound factors to be examined?    Can we create reports which present data in a form from which a hypothesis can be quickly drawn?    Can the data be made to by live and dynamic as opposed to fixed into the form of predetermined “analysis” reports?   Can we adopt a more broad view of what data we have and therefore gather and make greater use of a broader dataset?

I do at this point raise a note of caution.   We aren’t talking about doing more work in terms of gathering more data to do more analysis.  No, we are talking about allowing for the data we already have to be better used and therefore better inform decision making.

I look forward to discussing data on Saturday as part of #EdChatMeda.    It may be the after this I may be able to better answer the above questions.

A-Level results and football: Another enlightening analysis

Now the A-Level and GCSE results are out the usual sets of analysis and observations based on the data have started making an appearance.    As usual causal explanations have been developed to explain the data, using what Naseem Taleb described as the backwards process.   The resulting judgments have been established to fit the available data without any consideration for the data which is not available.

The perfect example is an article in the guardian (Wales A-Level results raise concerns pupils falling behind rest of UK, Richard Adams, Aug 2016)  discussing the A-level results in Wales as compared with the results in England.   The overall drop in the percentage entries achieving A* and A dropped in England “only slightly to 25.8%” while in Wales I “fell more steeply to 22.7%”.     The causal explanation apparently arrived at by one “expert” was that boys had been “possibly distracted by the national football team’s success at Euro 2016”.    This fails to consider the total number of entries in England when compared with Wales;   I suspect Wales would have less entries therefore resulting in increased variability in Welsh results versus English results.      The data also fails to include any information in relation to the students GCSE results.   Had the Welsh students achieved lower GCSEs results than their English counterparts it may be that their overall lower level of achievement could amount to “better” results given their lower starting point as measured by GCSEs.

Another possible conclusion, which is easy for me to draw as a Scotsman and most likely more difficult for an Englishman, is that the data shows something which wasn’t related to the Welsh football performance at all.    The English A-Level results could be better due to English students throwing themselves into their work following England’s poor showing during Euro 2016.  It’s the same data but a different conclusion which has been generated and made to fit the data available without any consideration for the data which isn’t available.

Having considered further this issue I think I am now more inclined than ever to agree with Talebs comments regarding the importance of the unread books in a library rather than the read ones.    Talebs discusses how a home library filled with read books gives a person the illusion of knowledge; the person has read it all.    A library filled largely with unread books however makes clear all that we do not yet know and have not read.    Reading each of these commentaries and analysis in relation to the A-Level data isn’t making me more informed or more educated, in fact it may be blinding me to the “true” facts or to other possibilities.    I think, therefore, that this will be my last post moaning about “expert” analysis or results as from now on I need to stop reading the analysis in the first place!

 

Some thoughts on GCSE and A-Level results

Having read various articles following the recent A-Level and GCSE results I cant help but think that schools and more importantly education in general needs to make a decision as to what we are seeking to achieve, and stop acting re-actively to limited data which has been used to draw generalized conclusions.

Take for example the shortage of STEM graduates and students.    This was and still is billed as a big issue which has resulted in a focus on STEM subjects in schools.   More recently there has been a specific focus on computer programming and coding within schools.     In a recent article it was acknowledged that the number of students taking A-Level Computing had “increased by 56% since 2011” (The STEM skills gap on the road to closing, Nichola Ismail, Aug 2016).     This appears to suggest some positive movement however in another article poor A-Level ICT results were cited as a cause for concern for the UK Tech industry (A Level Results raise concern for UK tech industry, Eleanor Burns, Aug 2016).  Now I acknowledge this data is limited as ideally I need to know whether ICT uptake has been increasing and also whether A-Level Computing results declined, however it starts to paint a picture.

Adding to this picture is an article from the guardian discussing entries:

Arts subjects such as drama and music tumbled in terms of entries, and English was down 5%. But it was the steep decline in entries for French, down by 6.5% on the year, as well as German and Spanish, that set off alarm bells over the poor state of language teaching and take-up in Britain’s schools.

( Pupils shun English and physics A-Levels as numbers with highest grades falls, Richard Adams, Aug 2016)

So we want STEM subjects to increase and they seem to be for computing, however we don’t want modern languages entries to fall.   Will this mean that next year there will be a focus on encouraging students to take modern foreign languages?    And if so, and this results in the STEM numbers going down will we then re-focus once more on STEM subjects until another subject shows signs of suffering.

It gets even more complex when a third article raises the issue of Music A level Entries which “dropped by 8.8% in a single year from 2015 and 2016”.  (We stand back and allow the decline of Music and the Arts at our peril. Alun Jones, Aug 2016).    Drama entries are also shown to have seen a decrease this year (Dont tell people with A-Levels and BTecs they have lots of options, Jonathan Simons, Aug 2016).  So where should our focus lie?   Should it be on STEM subject, foreign languages, drama or Music?

I suspect that further research would result in further articles raising concerns about still further subjects, either in the entries or the results.   Can we divide our focus across all areas or is there a particular area, such as STEM subjects, which are more worthy of focus?  Do the areas for focus change from year to year?

As I write this my mind drifts to the book I am currently reading, Naseem Talebs, The Black Swan, and to Talebs snooker analogy as to variability.     We may be able to predict with a reasonable level of accuracy, a single snooker shot however as we try to predict further ahead we need more data.    As we predict five shots ahead the quality of the surface of the table, the balls, the cue, the environmental conditions in the room, etc. all start to matter more and more, and therefore our ability to predict becomes less and less accurate.      Taking this analogy and looking at schools what chance do we have of predicting of the future and what the UK or world will need from our young adults?    How can we predict the future requirements which will be needed from the hundreds of thousands of students across thousands of schools, studying a variety of subjects from a number of different examining bodies, in geographical locations across the UK and beyond.

These generalisations of data are subject to too much variability to be useful.    We should all focus on our own schools as by reducing the scope we reduce the variability and increase the accuracy.   We also allow for the context to be considered as individual school leaders may know the significant events which may impact on the result of their cohort, individual classes or even individual students.  These wide scale general statements as to the issues, as I have mentioned in a number of previous postings, are of little use to anyone.   Well, anyone other than editors wishing to fill a space in a newspaper or news website.

 

 

 

 

 

Schools and Big Data

As Director of IT I am often directly involved with our School Management Information System (MIS, sometimes referred to as a Student Information System, SIS).   Throughout my career I have encountered and worked with a number of different MIS vendors.     My general opinion is that they are all “much of a muchness” as although they have different features, strengths and weaknesses, when you average them out the benefits and drawbacks are equal in terms of their magnitude.

These systems contain and allow us to collect a variety of data including both formative and summative student performance data.    We then design reports which allow us to interrogate the data and display it in different data.    This addresses the functionality side of an MIS however is rather weak in terms of the usability.    Users need to know which report displays which information so they can select and use the correct report at the correct time.

Within my school we are currently working on making our system more usable by developing a dashboard system to present important information directly to teachers without them have to seek it out.   This would represent an improvement however I feel still falls some way short.

One way improvement could be brought about on the above is to put more power in the hands of the users, allowing them to easily create their own reports using the data which is available.    The issue with this is it relies both on staff having the skills in data analysis to be able to design effective reports, plus it relies on them having the motivation to undertake this task.   Personally I believe this approach would be very beneficial for a small number of staff within a school, with the majority being unable to access it, even where the schools culture is very much around the use of data.   It would also potentially add another job to teaching staffs role in the need for them to design reports to analyse their data, which would represent an issue given the current situation in relation to workloads.

I think the solution lies with Big Data.   Within the IT world there is a lot of discussion with regards Big Data where large data sets are analysed to reveal trends or patterns, with this info then presented to users.   I see this as being of benefit in education.   As opposed to having to check different reports showing different sub-sets of our data such as the performance of male students vs female students, the system would identify the trends that exist for us.   The system would identify where there are correlations, without users needing to be aware of a potential correlation, therefore removing the potential for a correlation to be missed as we weren’t aware of it.    The system would also be able to look at data at a micro and macro level, either down to individual teachers groups assessment results this year, our out to patterns which may exist across a number of years.

Almost all schools have an MIS these days however they are still very much based on their origins, that of very structured data being analysed by reports.     It is about time we looked at the potential for data warehousing, data mining and Big Data to have an impact on how data is used in schools.