Photo: ‘Valentine Lightfulness’. h.koppdelaney CC-BY-ND

Do OER prompt critical reflection by educators who use them, resulting in improved practice? Part 3 of my blog series on reflection moves from abstract theorisation to concrete evidence around OER Research Hub (OERRH) hypothesis E, which raises this very question.

JISC (2013) suggest that the use of OER ‘encourages open educational practice, allowing teachers to share, reflect upon and get feedback on their work’.  OERRH Hypothesis E focuses on the reflection component of this assertion. The first step in collecting evidence related to this hypothesis has been to find a definition for ‘critical reflection’. This has been something of a challenge, largely due to the myriad of definitions available and the nebulous nature of the reflection process itself, which exists only in the abstract until made concrete through external representation of the process and the consequences of that process (e.g. through reflection diaries and discussion with peers).

In a recent blog post I explored various conceptions of reflection, highlighting Mezirow’s (1990) theory of transformative learning as particularly germane to the OERRH research, transferrable from the formal learning addressed by Mezirow to educators’ formal and/or informal encounters with OER either in the context of professional development or actual teaching practice. In particular, I point to the process of cognitive dissonance that might occur ‘when an educator encounters a resource featuring a very different teaching approach from their own typical practice and is prompted to reflect on the different outcomes that might be achieved by such an approach, in comparison with their habitual methods’.  I also highlight the relevance of Brookfield’s critical incident technique and Serafini’s (2002) argument that busy practitioners need time if they are to critically reflect on their practice in addition to Serafini’s point about the importance of dialogue with other educators as part of the reflection process.

Our research to date has involved field work and surveys with our collaborators, in addition to literature review-focused desk research. Covering the latter first, we have thus far identified a small but respectable body of literature on the pedagogical benefits of using OER (see, for example, Bliss et al., 2013), but this does not explicitly refer to improvement in educators’ critical reflection. The OER Research Hub sought the views of the UK OER community around this topic when premiering their ‘OER-chery’ activity at OER13 in order to gather attitudinal data from participants. The findings showed that higher education educators involved with OER are overwhelmingly convinced that Hypothesis E is both important and can be proven as true. Spurred on by this wave of support for the hypothesis we set about gathering empirical evidence…

OER-chery in action at OER 13

Measuring perceived, and indeed actual improvement in a teacher’s practice poses a not inconsiderable challenge and, to date, we have been been working solely on gathering self-declarations about educator’s beliefs around the relationship between their use of OER, changes in their critical reflection and subsequently improved practice. For example, OER Research Hub researcher Beck Pitt has been working with educators connected with the Bridge to Success project and reports that ‘educators who are aware of using OER reflect on their own practice as they think about how they integrate these resources with existing course materials, and when they respond to student feedback and their own personal experience when it comes to facilitating subsequent presentations of the course’.

Interestingly, OERRH researcher Bea de los Arcos reports a counter view derived from her research with K12 teachers in the Vital Signs citizen science program, based at the Gulf of Maine Research Institute: ‘We have gained an indication that teachers who are not particularly mindful of the open aspect of the resources engage in a similar process of reflection to those teachers who are aware they are using OER: a process of reflection instigated by the fact that the OER in hand is a new tool and thus requires consideration of how it fits in with the standing curriculum and the changes that changes need to be made as the program is implemented in different classes/years’.

Bea de los Arcos also reports some Hypothesis E-related evidence from her survey of educators (n=285) in the Flipped Learning Network, revealing that ‘the survey data shows 89% of K12 teachers agree or strongly agree that using free online resources helps them reflect more on the way they teach, and 70% agree or strongly agree that they more frequently compare their own teaching with others’. Interestingly, my own OpenLearn survey (n=988) revealed that the use of OER by OpenLearn-using educators from a range of educational sectors appears less connected with reflection that does the Flipped Learning Network educators’ use of free online resources, with 56%% of OpenLearn-using educators agreeing that using OpenLearn OER helps them to reflect more on the way they teach, and just 34% indicating that they more frequently compare their own teaching with others as a result of using OpenLearn OER. The disparity could be connected with the more explicit pedagogical experimentation that is involved in ‘flipping’ a classroom, which itself may put educators in a reflective and pedagogically experimental frame of mind.

Flipped Learning and OpenLearn-using educators were also asked the reasons why they used OER, as were educators using the Open University’s ITunes U and YouTube platforms, surveyed by OERRH Fellow Patrina Law together with the Institute of Educational Technology’s Anna Page and Graham Healing. Figure 1 compares the four groups of educators’ responses to five question options connected to some extent with reflective practice.

Figure 1: OpenLearn, Flipped Learning Network, YouTube and iTunes U-using educators’ reasons for using OER/free online resources

The results indicate that some educators are using OER for reflection-related purposes, including experimenting with new teaching methods, staying up to date with their subject and getting new ideas. Again, the Flipped Learning Network K12 educators emerge as particularly reflective. However, it is not possible to conclude, based on the survey results, that resources other than OER would not have been used in the same way. Additionally, the low scores across all four groups for comparing teaching materials with those of other educators suggests that the reflective practice taking place is quite focused on specific curriculum development needs (staying up to date or getting new ideas) rather than more broad-ranging meta-improvements.

The OpenLearn and Flipped Learning Network surveys also asked educators about further interaction with OER that could be seen as a measure of critical reflection: adding comments to a repository regarding the quality of a resource; adding comments to a repository regarding ways in which a resource might be used; and adapting a resource to meet your own needs. Figure 2 compares the OpenLearn and Flipped Learning Network educators’ responses, showing that the latter are slightly more active in adding comments to a repository and both groups of educators adapt the resources that they find.

Figure 2: Flipped Learning Network and OpenLearn-using educators’ responses to further critical reflection-related questions

Interviews with some of the OpenLearn and Flipped Learning Network educators will allow us to gain a fuller understanding of the meaning of the survey results. It may also be possible to get access to lesson observation reports in order to make more concrete ‘before and after OER use’ comparisons. In addition, we are expectant that forthcoming data from surveys (for example OERRH researcher Rob Farrow’s work with CCCOER), interviews and the interpretation of qualitative data regarding the use and reuse of educational materials from OER repositories will help us to build on our existing findings and paint a richer picture of educator reflective practice.

[Reblogged from http://artofoer.wordpress.com/2013/08/09/through-a-glass-darkly-part-3-oer-critical-reflection-and-educators-improved-practice-some-early-evidence/]