Note: The information provided here refers to past Digital Media and Learning Competitions, and is provided for archival and reference purposes only. If you are seeking the current Digital Media and Learning Competition, please navigate to dmlcompetition.net
PAST
COMPETITIONS

This project was submitted by an applicant to the Digital Media and Learning Competition.

Leveraging Lifelong Learning The Leverageā„¢ System as a Framework for Implementing & Managing Badges for Lifelong Learning

Overview

For users of web-driven learning environments, having accessible tools for understanding data and socializing knowledge is crucial. Seeing scores, experiences, and achievements over time as they relate to others within communities helps uses contextualize learning and goal-set based on performance records. The Leverage™ Data Analysis and Content Delivery Framework has a seven-year history of fulfilling this analysis and data visualization need for large user groups in commercial gaming and military simulations. Across and within disparate environments and systems Leverage has delivered dynamic leaderboards, user profiles, and adapted content based on performance metrics within user populations numbering in the millions. As a natural extension we will use the same framework for delivering Badges for Lifelong Learning. a visual representation of a learner’s breadth and depth of aptitude and experience on competencies presented within defined constructs.

 

A. Content Alignment

As our Leverage™ system serves as a backend infrastructure for collecting and utilizing data across and within both content and online environments, we hope to partner with one or several Stage One winners or collaborators. Corporation for Public Broadcasting: American Graduate Initiative Badges, Digital On-Ramps, and Innovative Communication Design Badges are the projects we feel we could most expertly serve by demonstrating existing systems we have deployed using Leverage that echo many of the goals each project team has outlined.

 

B. Badges

If we were to implement Leverage as a tracking and delivery infrastructure of any of the above-named projects, our architecture would allow us to monitor all content, media, and functions within a web and mobile environment. Our system allows for subject matter experts – the Urban Affairs Coalition, for example – to provide a catalogue of all text, media, etc., that a user to Digital On-Ramps might peruse and to outline an “expert path” or competency model by which performance can be gauged. This might be a simple as “the user watched this video” or as complex as monitoring time on task, sequences of interactions, and ongoing performance on periodic assessments within the website. Leverage can simultaneously monitor thousands of data points for millions of users, allowing individuals and cohorts to be tracked through their own individual learning paths through the Digital On-Ramps ecosystem. Badges that track progress may represent discretely defined achievements or skills and knowledge obtained, but they will continually and dynamically reflect a user’s growth and experiences across all environments utilizing the Open Badge Infrastructure. Because Leverage is a platform-agnostic, highly scalable infrastructure with proven capacity for highly granular data collection and responsiveness, there is no need to constrain the parameters of badges in ways other than those the issuer defines as valid.

 

For example, in our efforts supporting the America’s Army Game Platform, Leverage routinely cached 2 terabytes of data daily cataloging 10 million users’ performance in one of 40 available game scenarios, within approximately 40 training and general missions, across 60 achievable badges, ribbons, and ranks. Then, all of this data was used to provide the user with a summary visual of their overall progress on seven overarching proficiency categories. In total for America’s Army, Leverage tracked close to 1,000 individual data points for each user continually. Working with Army SMEs, we created a multi-dimensional “cube” that allowed Leverage, while doing the bulk of processing server-side so as not to impact the user’s experience, to process dependencies between user actions and available content, to consider historical trends, and to reflect back to users real-time, to the moment, achievements and feedback.

 

In America’s Army, users achieve recognition for completing tasks within scenarios (i.e. accomplishing a proficient level of marksmanship in a training experience), for completing bundles of tasks across scenarios (i.e. having expert level marksmanship within non-contiguous engagements of the game, regardless of time between engagements), for achieving a certain defined rank (as measured by the accomplishment of multiple outlined tasks and missions with minimum defined levels of expertise), and for having varying levels of leadership, duty, respect, selfless service, honor, integrity, and personal courage (as monitored through all interactions, experiences, and historical performance). Because the definitions for these achievements are available through the public portal, members of the community and visitors to the site are able to clearly identify what value each award has and even approximate time invested to earn. Additionally, the receipt of certain achievement awards is a pre-requisite for unlocking additional rounds and features and may impact actual performance in future engagements. For example, performing in a less than exemplary way in demonstrations of respect in the game-based scenarios defined as basic training impact the resources cache that the user is able to tap in later game engagements. The only way to increase resource access is to return to basic training and replay scenarios, striving to improve performance on measures that were lackluster prior.

 

Extrapolated for use in the American Graduate Initiative, Leverage would, for example maintain “hooks” into both the CPB main American Graduate website, as well as each participating local public station. Using a proficiency modeled defined in collaboration with CPB, Leverage’s authentication component would validate each user upon login (requiring email addresses) to each station, cue the user’s profile from the Leverage database and the user’s badge packpack from the OBI, monitor if and when the user has accomplished certain achievements as defined by SMEs through Leverage, and issue a new badge utilizing the Mozilla baking service. All this is accomplished simultaneously with login and dynamically as the user’s experiences progress. All feedback is real time all the time and all the same features can be replicated: monitoring engagement with content, monitoring engagement with peers, assessing performance where applicable, tracking historical tendencies and usage, aggregating information from data sources external to the main site and centralizing them in Leverage for use in conferring badges, providing detailed definitions of badges and related metrics, and more.

 

While we can provide art design for new badges for each environment, we recognize that certain issuers may have their own branded visual aesthetic we would be required to match. As well, we are able to easily take any existing badges or awards that any Stage 1 winner or collaborator has and ensure that they are implemented properly so that Leverage may deploy them in communication with the OBI.

 

Because there are no practical limits to the amount of data that can be collected on a single user over time, Leverage would summarize and cache information for users after badge receipt, but never discard the information. Users should always have a historical record of their own achievement and the option to discard, hide, or re-categorize badges for display at their own discretion, but Leverage would maintain them all so that they can always be revisited, recalled, and even revised, if applicable. For example, I might have achieved an Innovative Communication Design Badge last year for engaging a series of technology resources, but now the University of Main has added new criteria or additional suggested supplemental content for that same badge. Now mine has become dated, but not invalidated. By being able to call up detail on the achievements I amassed in order to receive the badge initially, I can easily learn what I need to do “re-up” or “revitalize” my award. Leverage can manage all these functions.

 

C. Technology

Leverage infrastructure provides administrators and developers interfaces for defining rules, tagging content, choosing events to monitor, and organizing achievements outside the framework of a designated website or discrete content object. Leverage also allows for the input of external data sources to be factored into overall rule execution. This allows us to monitor a user’s activity in forums (including peer ratings of posts and performance), in games, on websites, with web-hosted media, and even on mobile platforms simultaneously and to both deliver targeted content (i.e. badges, individualized messaging, etc.) and to communicate with other systems such as the OBI. 

 

Utilizing this technology, we envision creating widgets that allow badge issuers to easily communicate with the Leverage system through an admin panel on their on website – and – a desktop widget for individual users to manage badges received from multiple sites. In this way users will be able to more clearly define their place in the larger, global badge ecosystem and shape how they are viewed publically through badges collected.  We would do a “straw poll” of final Badge awardees to determine what site protocols and systems each are using, then take this initial population of badge issuers to determine how best to architect the badge management widget for maximum utility.

 

D. Team

Our development team, Pragmatic Solutions, is comprised of software engineers, database architects, web designers, interface designers, learning scientists, and artists. Our seven year history as a company has given us experience with military simulations, commercial games, learning management systems, online commerce systems, assessment design, and interactive learning technology. Questions of how to monitor user performance and engagement across technology platforms, asynchronous environments, in, around, and outside of web-driven situations, and how to take data and use it for analysis and feedback are central to our motivation and expertise.

 

Leverage_badgecategories_AmericasArmy.pdf

populatedpersonneljacket.pdf

Leverage_badgeart_dml.pdf

blog comments powered by Disqus