The last of my January reviews of the main projects that I am working on is about Learning Resource Metadata, which is the theme that I have been working on for the longest. One of my first jobs in learning technology was maintaining and publishing a catalogue of computer-based resources that were useful for teaching Physics at university. That was back in 1994, and some of my work was still around when the Internet Archive started indexing. Since then I have been through several more online learning resource guides, and got involved in standards for learning resource metadata through the CETIS Metadata Special Interest Group, IEEE LOM / IMS Learning Resource Metadata, DCMI Education Community, and LRMI. LRMI is still going and that is what this blog post is about.
One of the most exciting recent developments has been that through the IEEE P2881 Learning Metadata Working Group LRMI—and more importantly the approach it embodies, as described below—is being incorporated in formal standards.
About the work
LRMI exists as an RDF vocabulary of terms (properties, classes and concept schemes) that can be used with terms from other vocabularies to describe learning resources. The LRMI vocabulary is declared as equivalent properties and classes in Dublin Core namespace and in Schema.org (mostly under Learning Resource). There are classes, properties and, in the Dublin Core namespace, concept schemes providing broad, top-level controlled vocabularies / value enumerations for some of the properties.
While we have done some technical work over the last year or so (notably creating a concept scheme for Learning Resource Type, and adding German translations to the published concept schemes) the main thrust of the current work is focused on promotion and dissemination. A major part of our effort currently underway is setting up to gather information about who is using LRMI, how they are using it, so that we can create documentation based on current real life examples.
Application Profiles are key to understanding how the LRMI vocabulary may be used. Application Profiles allow the “mixing and matching” of terms from different base vocabularies, and the overlay of additional rules or other information for how the terms are to be used in a specific application—see my previous posts on DCMI Tabular Application Profiles for more about how we do that. The LRMI terms focus on the educational aspects of any form of learning resource (text book, instructional video, web-based or face to face course, V/AR, podcast…), and are designed to supplement other vocabularies designed to describe any general resource of a similar form. So the idea is that you would select the terms from whatever vocabularies best describe the form that your learning resource takes, be it book, video, web site, event etc., and then use the LRMI properties to describe the educational characteristics.
This approach is at the root of that taken by the IEEE P2881 Working Group (described here). That work is not aiming to replace IEEE 1484.12.1 Standard for Learning Object Metadata (LOM), but to provide an RDF-based alternative for uses where the flexibility of RDF is beneficial. The current draft in progress is an application profile based mostly on LRMI terms (from the DCMI namespace) and Dublin Core Terms.
About my role
About twelve years ago I was mostly done with metadata. I was more interested in Open Education Resources (OERs) and how discoverability was supported by having the full text on open view. But then, along came schema.org. Andy Powell asked what would it be necessary to add to the metadata understood by Google in order to adequately describe an OER. That approach was very much in line with the DCMI Education an UKOER metadata line of thinking, so I wrote a blog post in reply. Then, when the LRMI project got funding and was asking for input, I used that blog post to lever my way into the advisory group. And my involvement kind of grew from there. I now chair the Dublin Core LRMI Working Group, which is tasked with maintaining and advocating for the use of the LRMI vocabulary, and I represent the work in other fora, such as IEEE P2881.
It took a while, but we now have in LRMI a Learning Resource Type concept scheme that defines a controlled vocabulary of terms that you might use to describe the type or nature of a learning resource.
Why it took a while: what is a learning resource, and what is a type?
Over the last year or so I have been contributing to a Dublin Core Metadata Initiative Interest Group, lead by Karen Coyle, that has been drafting recommendations on how to create and document application profiles in simple a tabular format. At the end of last year we had a webinar presenting the work so far, which now available on YouTube. For me the really useful thing about this approach, which we are calling DC TAP, is that it is a human-friendly approach that also lends itself to machine processing—at the end of the webinar recording Tom Baker demos a python script that will take a tabular application profile exported in CSV format and turn it into ShEx that can be used to validate data.
IEEE are standing up a working group for Learning Metadata, to build on IEEE Std 1484.12.1 Standard for Learning Object Metadata while exploring new paradigms and technology practices in education. I submitted the following as a suggestion for a starting point. It’s heavily based on the presentaion I did at the start of the LRMI Metadata in Use panel session for DCMI Virtual. I think it is a reasonable introduction to, and reflection on the strengths of LRMI (we can do the weaknesses some other day). Many thanks to colleagues in the DCMI LRMI Task Group who commented on it, and especially Staurt Sutton for the 10 principles enumerated at the end.
I am somewhat late in writing this, but back in May some new properties developed by LRMI were added to schema.org that simplify and expand how schema.org can be used to describe learning resources and educational events.
The new properties are:
teaches: The item being described is intended to help a person learn the competency or learning outcome defined by the referenced term.
assesses: The item being described is intended to assess the competency or learning outcome defined by the referenced term.
These are added as properties of CreativeWork and EducationEvent, and can both be used with either a DefinedTerm or text as the value (or URL, which is an allowed value for any schema.org property).
In a related change, the domain of educationalLevel, (“the level in terms of progression through an educational or training context”), which was added last year for EducationalOccupationalCredentials was expanded so that it can also be used with CreativeWork and EducationEvent. It also can have DefinedTerms, text or URL as a value.
I’ve been experimenting with ways of putting JSON-LD schema.org metadata into HTML created by MkDocs. The result is a python-markdown plugin that will (hopefully) find blocks of YAML in markdown and insert then into the HTML that is generated. You can find the plugin on github, and you can read more about the development of it in some pages generated by MkDocs (that incidentally use the plugin).
XCRI-CAP (eXchanging Course Related Information, Course Advertising Profile) is the UK standard for course marketing information in Higher Education. It is compatible with the European Standard Metadata for Learning Opportunities. The W3C schema course extension community group has developed terms for describing educational courses that are now part of schema.org. Here I look at translating the data from an XCRI-CAP xml feed to schema.org json-ld. Continue reading →
I have a new publication: “Analysing and Improving Embedded Markup of Learning Resources on the Web,” which Stefan Dietze and Davide Taibi have presented at the 2017 International World Wide Web Conference in Perth Australia. I played a minor role in the “analysing” part of this work, the heavy lifting was done by my co-authors. They analysed data from the Common Crawl to identify sites that were using LRMI terms in their schema.org markup. The analysis provides answers to important questions such as: who is using LRMI metadata and which terms are they using? How many resources have been marked up with LRMI metadata? Are the numbers of users growing? What mistakes are being made in implementing LRMI? Continue reading →
A balloon debate is a debate in which a number of speakers attempt to win the approval of an audience. The audience is invited to imagine that the speakers are flying in a hot-air balloon which is sinking and that someone must be thrown out if everyone is not to die. Each speaker has to make the case why they should not be thrown out of the balloon to save the remainder.
Our “balloon” is a metaphorical one, it stands for the volunteer effort that maintains LRMI (LRMI seeming somewhat like a balloon in that it is kept going by hot air), and instead of carrying people it is trying to carry out work that will help people describe learning resources. While it is by no means sinking, like a real balloon LRMI can only sustain a certain (work-)load, and so we need to be selective in the work which is taken on. This exercise was intended to provide LRMI with ideas about which work to prioritize, while providing those who take part with information about the future directions that are open to LRMI.
Debate format
The debate proceeded through three rounds, the first round was about choosing candidate ideas for future work, the second eliminated those which seemed least feasible, and the third was to select the highest priority ideas. On the day we didn’t really get to the third round, but the results were nonetheless interesting.
The Ideas
We started with the following ideas, seeded by myself with help from other members of the LRMI task group (especially Stuart Sutton who provided several of the descriptions).
1. Structured, controlled vocabularies for LRMI properties.
Several key LRMI properties take text for their expected value type. The use of free text for properties such as learningResourceType makes it difficult to compare data from different providers. Where there are suggested values mentioned in the LRMI spec for these, the LRMI Task Group is already working to provide definitions of terms in RDF (as SKOS Concepts). This suggestion is a continuation of this work, to try as far as possible to provide controlled vocabularies for relevant LRMI properties.
2. Drop the Alignment Object for the most common alignment types
The mechanism of indicating how a resource relates to an educational framework involves a level of indirection which is both arcane and potentially powerful, however it’s full power is not fully developed. The suggestion here is that the indirection be removed by creating properties for those alignment types which are clearly important. So, for example, it is often important to state the educational level of a resource (e.g. in terms of Grade Level). Currently this would be an educationalAlignment with an AlignmentObject having alignmentType of educationalLevel. The indirection of the AlignmentObject could be removed if there were a property of a learning resource called educationalLevel referencing directly a point in a grade level framework.
Figure 1a: representing an educational alignment with an alignment object.
The alignment type property of the alignment object can set to specify the nature of the relationship, e.g. that this represents the educational level of the resource.
Figure 1b: a possible way of representing an alignment such as educational level of the resource more simply. Note: the value provided could be text or a URI; representing the relationship of the node to an educational framework is not yet solved in Schema.org.
3. Develop the Alignment Object to be more expressive
The mechanism of indicating how a resource relates to an educational framework involves a level of indirection which is both arcane and potentially powerful, however it’s full power is not fully developed. The suggestion here is that properties be added to the AlignmentObject to allow additional information about the alignment between a resource and a point in an educational framework. This additional information may include factors such as: who asserts that the alignment holds, what evidence they have for this alignment, how good is the alignment.
4. Recommendations for referring to educational frameworks
From the point of view of facilitating resource discovery the relationship between a resource and an educational framework is key. Showing how a resource relates to a framework which is understood by educators and learners allows them to find resource suitable for their needs. This may be manifest in discovery interfaces as faceted search or browse categories. One problem is identifying the relevant frameworks for different types of educational alignment in different educational contexts (e.g. what is the Scottish equivalent of K-12 for educational level?). Another problem is that variation in how these frameworks are expressed in LRMI metadata makes creation of such services more difficult than it need be (what is the framework name for K-12? What URIs are best to use?). We could help by creating an inventory of frequently used educational frameworks and recommendations on how to refer to them.
5. Declare a “Learning Resource” type
Currently, a Learning Resource is not formally declared in the schema.org schema as a subtype of CreativeWork. Instead, it is inferred that a Learning Resource is a kind of CreativeWork since the LRMI properties were included as part of the CreativeWork type. The lack of an explicit LearningResource subtype to CreativeWork makes it difficult for some doing markup or implementing systems using schema.org to recognize that schema.org in fact supports description of learning resources. They see EducationEvent as a subtype of Event, but no LearningResource as a subtype of CreativeWork.
6. Define a minimal subset of schema.org for describing learning materials
The schema.org schema is quite large and can be intimidating for some wanting to define minimally viable learning resource descriptions–e.g., where to begin, what properties are most important, how should they be used. Publishing a suggested profile of schema.org that defines a minimaly viable learning resource description while leaving open the addition of other properties needed with a particular use, might assist implementers needing a means to jumpstart development of their own profile based in schema.org.
7. Create support materials explaining LRMI properties
Recently, examples of how to use the AlignmentObject as well as the LRMI properties of CreativeWork have been added as ‘footers’ to the relevant schema pages at schema.org. While an excellent beginning, these additions are not enough. Other types of support materials for describing learning resources using the LRMI properties and classes need to be defined, developed, and published. Such materials might include a one-stop “primer” that combines narrative with examples and covers the inevitable points in usage where subjective decisions must be made where there are alternative legitimate paths forward.
8. Collate information about existing LRMI implementations
Specifications always need to be interpreted in order to be used in specific contexts, and however much normative and informative material is provided, we cannot hope to cover all contexts. One way that people implementing a specification can make choices that do not lead to unnecessary divergence is by referring to other implementations from similar contexts. LRMI could facilitate this by collating information about where these implementations can be found. This may also be useful in monitoring the uptake of the specification, identifying problems commonly encountered and providing examples of good practice.
9. Create an editor for LRMI metadata
Several tools exist that can create LRMI metadata within the scope of a single project or service’s needs. What is suggested here is that LRMI create, assist or promote the creation of an editor that can serve as a reference implementation: independent of the choices that would need to be made for any use in practice but flexible enough to be tailored practical use and, importantly, illustrating good practice in the implementation of LRMI.
The voting
In the end we discussed and voted on the issues of which ideas seemed most appealing and then, after eliminating those ideas which were not popular, we discussed and voted on the ideas that seemed least feasible, then had a short discussion about the results. We kept a tally of the votes on a flip chart: green ticks for ideas we thought attractive, red ticks for those we thought least feasible.
The ideas most liked were:
Structured, controlled vocabularies for LRMI properties,
Define a minimal subset of schema.org for describing learning materials, and
Collate information about existing LRMI implementations.
As was pointed out in the discussion, there are relationships between the ideas which mean that some other ideas become easier if these are achieved first (e.g. a general purpose editor becomes easier if you have agreed a general purpose subset of schema for it to edit).
Those ideas which we thought most difficult to achieve were
Structured, controlled vocabularies for LRMI properties and
Declare a “Learning Resource” type.
The specific issues with both of these centred around achieving agreement or consensus on terms and definitions.
I’m pretty sure that everyone involved in LRMI will see something in these results that they think a sensible outcome and something which raises an eyebrow or two. I’m also fairly sure that it won’t be the same things for everyone.. which is to say, the discussions will continue.
So what now?
Some of the ideas discussed are based on work that individuals or groups involved in LRMI are already scoping out. As a group we have made a start on formally describing some values for use with LRMI terms as SKOS concepts (see the DCMI/LRMI Github repository). I have had a masters student work on creating a subset of schema.org for learning resources. There is an editor for LRMI in the works, which looks like it could meet the sort of requirements described above. And as I described in the presentation I gave at the workshop, there is work describing how LRMI is used. So hopefully we will be able to make progress on some of these ideas.
As I said during the workshop, LRMI is a volunteer effort. People work on whichever aspects of it interests them. If you are interested in any of the ideas described above, or have some other ideas that you think would be valuable, please join us and I hope will be able to achieve more together.
