Between analysis and transformation: technology, methodology\ud and evaluation on the SPLICE project

Part of book or chapter of book English OPEN
Johnson, Mark ; Lager, Peter ; Pollard, Bill ; Hall, Graham ; Edwards, Miranda ; Whitfield, Pete ; Ward, Rupert (2009)
  • Publisher: Association for Learning Technology
  • Subject: LB2300 | LC5201

This paper concerns the ways in which technological change may entail methodological development in e-learning research. The focus of our argument centres on the subject of evaluation in e-learning and how technology can contribute\ud to consensus-building on the value of project outcomes, and the identification of mechanisms behind those outcomes.\ud \ud \ud We argue that a critical approach to the methodology of evaluation which harnesses technology in this way is vital to agile and effective policy and strategymaking in institutions as the challenges of transformation in a rapidly changing educational and technological environment are grappled with.\ud \ud \ud With its focus on mechanisms and multiple stakeholder perspectives, we identify Pawson and Tilley’s ‘Realistic Evaluation’ as an appropriate methodological approach for this purpose, and we report on its use within a JISC-funded project on\ud social software, SPLICE (Social Practices, Learning and Interoperability in Connected Environments). The project created new tools to assist the identification of mechanisms responsible for change to personal and institutional technological practice. These tools included collaborative mind-mapping and focused questioning, and tools for the animated modelling of complex mechanisms. By using these tools, large numbers of project stakeholders could engage in a process where they were encouraged to articulate and share their theories and ideas as to why project outcomes occurred. Using the technology, this process led towards the identification and agreement of common mechanisms\ud which had explanatory power for all stakeholders.\ud \ud \ud In conclusion, we argue that SPLICE has shown the potential of technologicallymediated Realistic Evaluation. Given the technologies we now have, a methodology based on the mass cumulation of stakeholder theories and ideas about mechanisms is feasible. Furthermore, the summative outcomes of such a process are rich in explanatory and predictive power, and therefore useful to the immediate and strategic problems of the sector. Finally, we argue that as well as generating better explanations for phenomena, the evaluation\ud process can itself become transformative for\ud stakeholders.
  • References (8)

    Austria, J. (2007) Developing Evaluation Criteria for Podcasts. In Libri, 57, no. 4: 179-272. [accessed February 2009] Avgerinou, M., Salwach, J. and Tarkowski, D. (2007) Information Design for Podcasts. In C. Montgomerie & J. Seale (Eds.), Proceedings of ED-MEDIA, 754-756. Chesapeake, VA: AACE Bell, A. (2007) A+ Podcast Rubric. University of Wisconsin, Stout. Available at www.uwstout.

    edu/soe/profdev/podcastrubric.html [accessed January 2009] Bongey, S., Cizadlo, G. and Kalnbach, L. (2008) Explorations in course-casting: podcasts in higher education. Campus-Wide Information Systems 23, no. 5: 350-367.

    Campbell, G. (2005) There's Something in the Air: Podcasting in Education. Educause, 33-46.

    Carvalho, A.A. and Aguiar, C. (2009) Impact of Podcasts in teacher Education: from Consumers to Producers. Proceedings of SITE, 2473-2480. Chesapeake: AACE.

    Carvalho, A.A., Aguiar, C., Santos, H., Oliveira, L. and Marques, A. (2009) Podcasts in Higher Education: Students' and Lectures' Perspectives. IFIP - 9th WCCE (accepted).

    Carvalho, A.A., Aguiar, C., Carvalho, C.J. and Cabecinhas, R. (2008) Influence of Podcasts Characteristics on Higher Students' Acceptance. In C. J. Bonk, M. M. Lee & T.H.

    Reynolds (eds), Proceedings of E-Learn, 3625-3633. Chesapeake: AACE ROLE MAKES LEARNING PLANS (PERSONAL DEVELOPMENT)

  • Metrics
    views in OpenAIRE
    views in local repository
    downloads in local repository

    The information is available from the following content providers:

    From Number Of Views Number Of Downloads
    University of Huddersfield Repository - IRUS-UK 0 236
Share - Bookmark