Postscript

Once again my thanks to the Organising Committee:

Lori Lockyer from Macquarie University
Margaret Hicks and Shane Dawson from University of South Australia
Phil Long from The University of Queensland
Ron Oliver from Edith Cowan University (unfortunately called overseas and couldn't participate live)
Peter Tregloan University of Melbourne
Gabrielle Gardiner UTS

Now the conference has closed I thought it might be worth saying a little more about where we might go to from here.

It was very pleasing to see the number of people who were interested in collaborating on further developments, and on the OLT grants to be announced in particular. Participants will receive an email with a link to a Google Doc - please complete that and indicate your area of interest in collaborating. we will put people together and go from there.

Finally, here is a list of other organisations around the world that I know of who are working in this area

Society for Learning Analytics Research

Solar Flare UK and Doug Clow's blog from the event

Solar Flare Purdue

The next international SoLAR event is in Leuven, Belgium 8-12 April 2013.

Afternoon of Day 2 of SoLAR

Second paper session

First steps in learning analytics with LearnTrak
Susan Tull
Unfortunately I couldn’t watch this talk –  Prezi which always makes me dizzy so instead I will refer to an external summary of the project from
http://www.educause.edu/ero/article/trak-first-steps-learning-analytics

which includes the following summary of major points:

• A current focus at the University of Canterbury is to develop a culture of institution-wide accountability for the attraction, admission, support, retention, and success of students.
• Lecturers are encouraged to become part of the early intervention process and develop skills using this first step of the system so that they can effectively engage with learning analytics.
• LearnTrak software lets lecturers view data on student accesses to resources and activities in a user-friendly graphical form.
• An organization using Moodle can adopt this tool and Canterbury's method of implementation as a first step toward implementing learning analytics.

There were lots of very positive comments on this paper so it was disappointing to miss it.

 Moving beyond a fashion: likely paths and pitfalls for learning analytics
David Jones, Colin Beer

This was a great presentation on paths and pitfalls – admitted he had intended to be more negative, but changed his mind as a result of Susan's presentation. He also admitted to being a glass half-empty person so said we should bear that in mind when he says we are into the gold rush days of learning analytics.

After mentioning his previous Indicators Project, he posed his big (and refreshing) question How can we stop Learning Analytics from becoming a fad? He said there is some value in Learning Analytics but how can we prevent the fad behaviour which consists of big unrealistic expectations, poor implementation, then abandonment and moving on to the next fad.

He showed Birnbaum's 2000 model as an alternative to Gartner’s Hype cycle 

One of the terrific images in his presentation was of a swamp - but he had some suggested paths through the swamp which he referred to as changing the game of education.

Do it to – (academics) dominant model – managing from the dark side. Data warehouses likely to fail or not used very much
Do it for – (academics and students) – change thinking and planning
Do it with – evolutionary development – failures of rationality - Have to embed learning analytics in the design of courses

Link to his paper on this
Predicts no-one will be taking about Learning Analytics in 5 years time
David has posted his slides and references.

Exploring data from existing systems that are useful in assisting student learning: An Indonesian Perspective
Yasmin Erika Faridhan
PhD student – also using Prezi
Great to see a doctoral student here and sorry I can't provide more detail of her Prezi talk. Instead here are exerts from her abstract:

In Indonesia, extracting data for learning analytics purposes is not an easy job.  Although internet and information technology usage in Indonesian higher education has been growing fast in this twenty-first century, their integration in the form of student learning systems has only been surfacing recently. Compared to other countries in the Southeast Asia region, Indonesia is lagging behind Singapore and Malaysia in adopting educational technology to support student learning.

In this presentation I am going to introduce my PhD study on learning analytics in the Indonesian higher education environment.  I would be very interested in starting conversations with other researchers who have experiences with learning analytics, especially in an international context.  My work has a statistical focus and I would also be interested in finding out who else is working in this area.  This research is presented in order to obtain feedback that may provide insight into the next step of this project.

The final session was essentially about where to from here?


Rob Phillips gave an interesting report back from Australasian Council for Open and Distance Education (ACODE) in terms of the interesting work being done by members, and the projects they have decided to undertake including completing an annotated bibliography briefing papers on LA.

There was a panel session involving 4 of us either talking about what we are doing in our institutions or what we should be doing. Rather than what often happens at a gathering or early adopters, there was much discussion about ensuring we don't repeat the mistakes of the past - learning design and learning objects were mentioned as two areas that there was great enthusiasm about but haven't lived up to their earlier promise.

One aspect of this was ethics and there was discussion about whether students should be able to opt out of having their data included. A brilliant suggestion was made about the need to have a teaching and learning ethics committee which would mirror the research ethics committee 

The final talk was from Siobhan Lenihan who is Director of Grants and Fellowships for the Office for Learning and Teaching (OLT). She ended on the day on a good note by announcing that one of the first topics to be commissioned in 2013 by  OLT will be in the general area of Learning Analytics. There is an indicative budget of $1.5M for 4 projects, one of which will be on the topic of Learning Analytics. There seemed to be a general feeling in the audience that rather than these being competitive grants, we should try to find ways for everyone to collaborate.

Morning of Day 2 of SoLAR

The great crane incident meant we had to change venue - our original space was right next to the crane and it decided to close the adjacent spaces in preparation for the weekend attempt to remove the damaged crane.

The opening keynote today was Alfred Essa whose presentation was:

Innovation at the intersection of MOOCs and Learning Analytics

He started out by saying he’s not sure why he was invited to do a keynote .... well, that's why Alfred - my experience is that it is the humble folk who often have the most interesting things to say :)

He showed a movie trailer from Moneyball. The main point of showing the video would have been to show the way the use of data analytics transformed a poorly performing baseball team to a leader

One of the interesting lines from the trailer was “we’ve got to think differently” I was thinking at the time how many people say that when things are not going well, but how difficult it can be to make the changes necessary to actually think differently – most end up doing things in much the same way.

His challenge was How quickly can we mobilise the research agenda around analytics?

In another candid moment Alf said "I'm not going to say much about MOOCs so it's a misleading title … but it’s really about scale – how can we bring cost down?"

In talking about our challenges he mentioned:

Student success – students are taking longer and longer to complete their courses, and attrition is higher

Institutional effectiveness - we have to be able to provide evidence that our students are learning

How can we do this at scale? He talked about 3 levels of analytics

Learning Analytics – need to focus on the learner as individual (mentioned Ken Robinson's work highlighting our existing factory model – everyone presumed to start the same and has same experience – how to meet needs of individual learner)

Academic Analytics – retention rates, focus in institutions, course, department

Enterprise analytics – looks at wider data –  all systems, all data at the enterprise level

This short list can appear overwhelming to most on where to start imho, but he then described three levels of maturity

1 data access + reporting  - data about the past – most of us are here

2 predicted future – forecasting and predictive modelling (techniques such as predictive model)

3 desired future – holy grail – how do I want things to be in the future – key is finding the optimal path – not about more data – I have more data than I need

Everyone seemed a little taken when Alf put up the words of the famous Sting song – I realised for the first time, the true meaning of the lyrics

every step you take, 

every move you make, 

every bond you break, 

every step you take 

.. I'll be watching you 

We are capturing a lot of (surveillance) data

He used a great metaphor for learning analytics - Student as pilot, teacher as air traffic control. Student needs data!

He noted something I have also often reflected on -  why do people (such as physicists) abandon data for anecdotes when talking about education? Loved his reference to Lee Schulman's quote “the plural of  anecdote is not data” 

He referred to Eric Mazure's work in trying to inprove eduaction using data

We know very little about learning (not sure I completely agree with this ...)

The importance of predictive modelling – very interesting comparison:

Weak human + good machine + better process can beat 

strong human + good machine + inferior process

Predicted a big area for the future is behavioural economics – mentioned 2 books - 

Thinking, fast and slow 

and 

Predictably irrational

We behave in irrational ways all the time and it is predictable. This gives us some basis for framing choices in a different way – eg when enrolled in a program, students make choices of the sequence of subject – make bad choices all the time – can we use optimisation to help students make smarter choices to provide better pathways?

Alfred finished up by demonstrating some wonderful D2L tools to help teachers visualise individual progress of students and their social networking activity. Access to this should be that we can design a personalised intervention to get the students back on track

Closed with "You and I have a once in a lifetime opportunity to change the vector of learning"

Alfred said he would put his slides up - watch his Slideshare account  

Paper sessions

Flip with care

Abelardo Pardo, Mar Pérez-Sanagustin, Hugo A. Parada G., Derick Leony

Started out by posing the questions

Flip Learning? Or change the balance?

His view – its about time BUT Flip with care

How can we gather meaningful data on what users do

I really liked his next slide on "Privacy as currency. Users pay with their (loss of) privacy" 

The discussion was on what is affordable in terms of privacy? Showed a browser add-in – embedded in the LMS. A Moodle instance. For each individual student there was a graph including – attempted question, was correct/ incorrect/ and looked at responses. This is very useful for not only the academic teaching this class but also for looking at the level of engagement of other classes – is my class more or less engaged

Presentation available at www.slideshare.net/abelardo_pardo


Characterising student exploration strategies using inquiry-based learning resources


Barney Dalgarno, Gregor Kennedy, Sue Bennett

Project is mainly about inquiry-based learning project but is situating it today within LA

Showed some great simulations on blood alohol and global warming – they did a ‘scientific study’ with randomisaiton and pre- and post-test testing. Found students didn't learn a great deal in either mode. But then looked at the degree to which they were systematic in their approaches and found significant differences.

Examining the use of in-situ audio annotations to provide feedback to students
Andrew Dekker, Kirsten Zimbardi, Phil Long et al

Age-old problems of feedback “lack of critical anaysis’ is not useful for a student

Showed new project called UQMarkup

iPad interface so markers could get documents onto their device so they can mark anywhere

audio annotations. Course coordinator dashboard looks great

There was comment on twitter about a similar project:

http://www.comp.rgu.ac.uk/docs/info/project/isg-openmentor-sw.htm

Showed lots of graphs on how long students look at feedback etc, length of audio annotation

how students move through document, how many times they listen to the audio feedback 

Student feedback – some prefer written feedback, others felt it was more personal

Will post slides of presentation and also software up as open-source  +1