The Research Competition is now closed. Click here to view the winners.

Winning Proposals

Who’s Achievement? — Evaluating Self-Constructed and Peer-Evaluated Badge Systems in Online Classrooms

Digital Media + Learning Competition: Badges for Lifelong Learning: Dissertation Grant

“Who’s Achievement? — Evaluating Self-Constructed and Peer-Evaluated Badge Systems in Online Classrooms”

Reginold A. Royston and Ashley Ferro-Murray, UC Berkeley

“The principal goal of education in the schools should be creating men and
women who are capable of doing new things, not simply repeating what other
generations have done” (Piaget 1953).

In online classrooms, educators are starting to use badges as achievement incentives to actively engage students. Like points, levels or titles in video gameplay, badges may signal status or success in the context of virtual classrooms. But how many educational experiences and learning strategies are missed if forms of achievement are solely predetermined by instructors?

In our online course, “23 Principles of Internet Citizenship,” we introduce an achievement system that allows students to construct their own badges and evaluate each other’s work towards two course goals: Group Collaboration and Creativity. A Digital Media and Learning grant will help us analyze online classroom interactions and students’ interpretations of success amid this open-source achievement structure. We will also pay particular attention to how participants succeed in unanticipated ways— how students innovate on learning, rather than simply achieving standardized goals.

We address these key questions: What forms of social capital do students develop when they are allowed to define achievement for themselves? How can learning strategies that deviate from classroom structure and goals be recognized as innovations or alternative forms of engagement? How then is achievement to be measured?

PROJECT DESIGN

While online badge and training systems like the Khan Academy (www.khanacademy.org) and BadgeStack (www.badgestack.com) provide a structure for participatory engagement and individualized learning approaches, it is important to consider the ways in which these new systems of achievement exclude some classroom engagements: When the marker for having learned something is a ‘badge,’ what learning is left out? Whose learning strategies are marginalized?

“23 Principles of Internet Citizenship” utilizes an open-source badge creation system, that attempts to de-hierarchialize notions of achievement. Upon completion of introductory units, students earn the privilege to create their own achievement badges in response to course material. Students can claim badges upon completion of each online learning unit. As the class progresses, students modify and mold badges to critically respond to course material. The open creation, exchange and collaboration of students’ badges allows for peer-review and the development of an emergent achievement system.

In this research, we also look at how experiences of failure, exclusion, and marginality often inform the process of innovation and creativity. “Low-achievement” or social exclusion may inspires alternative forms of cultural production, including aesthetic and symbolic resistance to established norms (Halberstam 2011). From our interdisciplinary perspective, we draw upon similar New Media interventions in art history and diaspora studies. Digital art and performance art grew out of experiments that privileged artistic process and innovation (Bishop 2001, 2010; Fernandez 2006; Neumark 2006). Innovation here comes not from the object-oriented art market, but from ephemeral acts of experimentation, failure, and creativity. We apply these artistic principles to the ephemeral processes of learning — student interaction and engagement. The field of diaspora studies also informs the conceptual nature of this work, allowing us to examine the unique forms of social capital that develop via virtual communities. Digital diasporas have been essential facilitators of intellectual exchange in our current moment (Laguerre 2006; Alonso & Oiarzabal 2010). We will ask our students to work beyond a normative boundaries of citizenship and culture, to address the ways in which online learning has the potential for global reach.

However innovative, standardized badge systems may promote normative models of "success" and "achievement.” In response to research that uncovers exclusivity and bias in standardized testing models (Sacks 1999; Koretz 2008), we focus on the creation and acquisition of badges that support personal growth and transformation instead of treating the badge as a standardized goal.


IMPACT

Online classrooms that deploy badge systems often focus on younger secondary school students, rather than lifelong learners. Such badge systems tend to emphasize game-like, play-oriented systems of achievement in the vein of Boy/Girl Scout merit badges, or video-game ‘leveling’ (Galloway 2006; McGonigal 2011). Social gaming, however, often appeals to a diverse age-set, especially adult learners (McGonigal 2011). Foursquare, Gowalla, and Habbo Hotel, for example, are three badge systems that emphasize mature as well as playful themes, through online sociality and learning projects attached to real-world experiences. “23 Principles of Internet Citizenship” similarly incorporates physical world projects with experiences in the online classroom for adult learners. Thus, our research attempts to provide an analysis of how such “social” badges translate to educational and/or training achievement systems relevant to lifelong learners — those seeking educational opportunities outside of traditional, institutional classroom experiences.

The HASTAC Digital Media and Learning Competition Dissertation Research Grant will support the analysis of our open-source badge system. This will be a revelatory case study (Yin 2003) that allows us to more fully develop theory about the role of collaboration in achievement, and critical forms of learning, previously unexplored in the online classroom. The development and application of “23 Principles of Internet Citizenship” is sponsored by the University of California Office of the President, as a course in Summer 2012 and continuing semesters. Beyond this application, support from the Digital Media and Learning Competition will enable us to analyse of the impact of our course on open-source achievement and lifelong learning.

During the funded year, we will also collaborate with the Berkeley Center for New Media (where we are both research fellows) on its Fall 2012 symposium on online education, and we plan to publish a comprehensive article about our findings. We are impressed by the rich conversation about badges and online learning that is already taking place via HASTAC’s web site and conferences, and are eager to bring these discussions to our own disciplines - African American Studies, Globalization Studies, Performance Studies, Art History, Art Practice, and New Media.

Citations

Alonso, A., & Oiarzabal, P. J. (2010) Diasporas in the New Media Age: Identity, Politics, and Community. Reno: University of Nevada Press

Bartle, R. (1996), "Hearts, Clubs, Diamonds, Spades: Players Who suit MUDs," accessed at, http://www.mud.co.uk/richard/hcds.htm, Jan. 1, 2012

Bishop, C. (2006), Participation. Cambridge, MA: The MIT Press.

Chandler, A. & Neumark, N. (2005) At a Distance: precursors to art and activism on the Internet. Cambridge, MA: The MIT Press.

________. (2005), Installation Art: A Critical History. New York: Routledge.

Cohen, R. (1997) Global Diasporas: An Introduction. University of Washington Press: Seattle.

Fernandez, M. (2006). “Historicizing Process and Responsiveness in Digital Art.” ed. Amelia Jones. A Companion to Contemporary Art since 1945. Malden, MA: Wiley-Blackwell Publishing.

Galloway, A. R. (2006). Gaming: Essays on Algorithmic Culture. Minneapolis: University of Minnesota Press.

Gilroy, P. (1993) The Black Atlantic: Modernity and Double Consciousness. Cambridge, MA: Harvard University Press.

Halberstam, J. (2011). The Queer Art of Failure. Durham: Duke University Press.

Koretz, D. M. (2008). Measuring Up: What Educational Testing Really Tells Us. Cambridge, Mass: Harvard University Press.

Laguerre, M. (2005) Laguerre, M. S. (2005). The Digital City: The American Metropolis And Information Technology. Houndmills, Basingstoke, Hampshire: Palgrave Macmillan.

McGonigal, J. (2011). Reality is Broken: Why games make us better and how they can change the world. New York: Penguin Press.


Piaget, J. ([1953]1997). The Origin of Intelligence in the Child. London: Routledge.

Sacks, P. (1999). Standardized Minds: The high price of America's testing culture and what we can do to change it. Cambridge, Mass: Perseus Books.

Yin, R. K. (2003). Case Study Research: Design and Methods. London: Sage.