Category Archives: LRMI

New properties for learning resources in schema.org

LRMI, , ,

I am somewhat late in writing this, but back in May some new properties developed by LRMI were added to schema.org that simplify and expand how schema.org can be used to describe learning resources and educational events.

The new properties are:

teaches: The item being described is intended to help a person learn the competency or learning outcome defined by the referenced term.

assesses: The item being described is intended to assess the competency or learning outcome defined by the referenced term.

These are added as properties of CreativeWork and EducationEvent, and can both be used with either a DefinedTerm or text as the value (or URL, which is an allowed value for any schema.org property).

In a related change, the domain of educationalLevel, (“the level in terms of progression through an educational or training context”), which was added last year for EducationalOccupationalCredentials was expanded so that it can also be used with CreativeWork and EducationEvent. It also can have DefinedTerms, text or URL as a value.

As well as providing a way of describing the educational characteristics of EducationEvents, which was previously not possible, these changes simplify how learning resources can be described. Previously in order to describe these characteristice one had to use the educationalAlignment property and an AlignmentObject with alignmentType property of “teaches”, “assesses” or “educationalLevel”. Not only was this a somewhat complex indirect way of saying something simple, but we think that the use the AlignmentObject had the effect of hiding the availability of these important properties.

Having DefinedTerm as a value means that one can still describe the value as being drawn from an educational framework, the framework being modeled as a DefinedTermSet, just as one could when using various properties of the AlignmantObject. In fact there is the improvement that you can now provide a URL for the framework being used, not just the url of the target term and the name of the framework. Here are a couple of examples:

{
    "@context": "http://schema.org/",
    "@type": "CreativeWork",
    "name": "Designing a treasure map",
    "educationalLevel": {
        "@type": "DefinedTerm",
        "inDefinedTermSet": "US Grade Levels",
        "name": "2",
        "url": "http://purl.org/ASN/scheme/ASNEducationLevel/2"
    },
    "url": "http://example.org/lessonplan"
}
{
  "@context":  "http://schema.org/",
  "@type": "WebPage",
  "name": "The Declaration of Arbroath",
  "about": "Wars of Scottish independence",
  "learningResourceType": "lesson plan",
  "educationalLevel": [
    {
      "@type": "DefinedTerm",
      "name": "Level 2",
      "inDefinedTermSet": {
           "@type": "DefinedTermSet",
           "name": "SCQF",
           "url": "https://scqf.org.uk/"
       }
    },
    {
      "@type": "DefinedTerm",
      "name": "KS 3",
      "inDefinedTermSet": {
          "@type": "DefinedTermSet",
          "name": "The National Curriculum for England",
          "url": "https://www.gov.uk/government/collections/national-curriculum"
      }
    }
  ],
  "url" : "http://example.org/lessonplan",
  "isBasedOn": "http://www.bbc.co.uk/education/clips/z3sjtfr"
}

The educationalAlignment and AlignmentObject are still valid terms, and indeed are still the only means of describing some other educational characteristics of learning resources. However, going forward, we suggest you use the new teaches, assesses, educationalLevel properties in preference.

We will add these new terms to the DCMI LRMI namespace shortly, I hope, giving them a stable existence independent of schema.org.

Next steps?

If you’re interested in what may be coming next, here’s some of what has been discussed.

A property to express “the skill, knowledge or competence a learner needs before using a learning resource” is on our list of simple properties, which would complete the trio of requires, teaches, assesses. However the name “requires” has lead to confusion over what is required (a pen and paper?) and by whom/what.

Also, take a look at the schema.org gitub issue 1401 and you will see plans and discussion around creating a class for learning resources. I hope this will be limited to CreativeWorks and will sit alongside EducationEvent acting as a convenient place to find the properties relevant to describing educational characteristics, and also allowing such properties to  be used when describing Videos, Books, Products, etc as learning resources (i.e. by declaring LearningResource as an additional type) without needing to add education-specific properties higher up the schema.org hierarchy.

Translating course descriptions from XCRI-CAP to schema.org

cetis, Course description, LRMI, , ,

XCRI-CAP (eXchanging Course Related Information, Course Advertising Profile) is the UK standard for course marketing information in Higher Education. It is compatible with the European Standard Metadata for Learning Opportunities. The W3C schema course extension community group has developed terms for describing educational courses that are now part of schema.org. Here I look at translating the data from an XCRI-CAP xml feed to schema.org json-ld. Continue reading →

New publication analysing LRMI metadata on the web

LRMI, , ,

I have a new publication: “Analysing and Improving Embedded Markup of Learning Resources on the Web,” which Stefan Dietze and Davide Taibi have presented at the 2017 International World Wide Web Conference in Perth Australia. I played a minor role in the “analysing” part of this work, the heavy lifting was done by my co-authors. They analysed data from the Common Crawl to identify sites that were using LRMI terms in their schema.org markup.  The analysis provides answers to important questions such as: who is using LRMI metadata and which terms are they using? How many resources have been marked up with LRMI metadata? Are the numbers of users growing? What mistakes are being made in implementing LRMI? Continue reading →

LRMI balloon debate at #DCMI16 Metadata Summit, Copenhagen

LRMI, ,

I summarised the presentations given at the LRMI workshop “Building on Schema.org to describe learning resources” at the Dublin Core conference in my previous post, this post is about the discussion part of the workshop. The discussion was organised along the lines of a balloon debate:

A balloon debate is a debate in which a number of speakers attempt to win the approval of an audience. The audience is invited to imagine that the speakers are flying in a hot-air balloon which is sinking and that someone must be thrown out if everyone is not to die. Each speaker has to make the case why they should not be thrown out of the balloon to save the remainder.

https://en.wikipedia.org/wiki/Balloon_debate

Our “balloon” is a metaphorical one, it stands for the volunteer effort that maintains LRMI (LRMI seeming somewhat like a balloon in that it is kept going by hot air), and instead of carrying people it is trying to carry out work that will help people describe learning resources. While it is by no means sinking, like a real balloon LRMI can only sustain a certain (work-)load, and so we need to be selective in the work which is taken on. This exercise was intended to provide LRMI with ideas about which work to prioritize, while providing those who take part with information about the future directions that are open to LRMI.

Debate format

The debate proceeded through three rounds, the first round was about choosing candidate ideas for future work, the second eliminated those which seemed least feasible, and the third was to select the highest priority ideas. On the day we didn’t really get to the third round, but the results were nonetheless interesting.

The Ideas

We started with the following ideas, seeded by myself with help from other members of the LRMI task group (especially Stuart Sutton who provided several of the descriptions).

1. Structured, controlled vocabularies for LRMI properties.
Several key LRMI properties take text for their expected value type. The use of free text for properties such as learningResourceType makes it difficult to compare data from different providers. Where there are suggested values mentioned in the LRMI spec for these, the LRMI Task Group is already working to provide definitions of terms in RDF (as SKOS Concepts). This suggestion is a continuation of this work, to try as far as possible to provide controlled vocabularies for relevant LRMI properties.
2. Drop the Alignment Object for the most common alignment types
The mechanism of indicating how a resource relates to an educational framework involves a level of indirection which is both arcane and potentially powerful, however it’s full power is not fully developed. The suggestion here is that the indirection be removed by creating properties for those alignment types which are clearly important. So, for example, it is often important to state the educational level of a resource (e.g. in terms of Grade Level). Currently this would be an educationalAlignment with an AlignmentObject having alignmentType of educationalLevel. The indirection of the AlignmentObject could be removed if there were a property of a learning resource called educationalLevel referencing directly a point in a grade level framework.

Figure 1a: representing an educational alignment with an alignment object. The alignment type property of the alignment object can set to specify the nature of the relationship, e.g. that this represents the educational level of the resource.Figure 1a: representing an educational alignment with an alignment object.
The alignment type property of the alignment object can set to specify the nature of the relationship, e.g. that this represents the educational level of the resource.

simplifiedalignment-simpleschemagraphFigure 1b:  a possible way of representing an alignment such as educational level of the resource more simply. Note: the value provided could be text or a URI; representing the relationship of the node to an educational framework is not yet solved in Schema.org.

 

3. Develop the Alignment Object to be more expressive
The mechanism of indicating how a resource relates to an educational framework involves a level of indirection which is both arcane and potentially powerful, however it’s full power is not fully developed. The suggestion here is that properties be added to the AlignmentObject to allow additional information about the alignment between a resource and a point in an educational framework. This additional information may include factors such as: who asserts that the alignment holds, what evidence they have for this alignment, how good is the alignment.
4. Recommendations for referring to educational frameworks
From the point of view of facilitating resource discovery the relationship between a resource and an educational framework is key. Showing how a resource relates to a framework which is understood by educators and learners allows them to find resource suitable for their needs. This may be manifest in discovery interfaces as faceted search or browse categories. One problem is identifying the relevant frameworks for different types of educational alignment in different educational contexts (e.g. what is the Scottish equivalent of K-12 for educational level?). Another problem is that variation in how these frameworks are expressed in LRMI metadata makes creation of such services more difficult than it need be (what is the framework name for K-12? What URIs are best to use?). We could help by creating an inventory of frequently used educational frameworks and recommendations on how to refer to them.
5. Declare a “Learning Resource” type
Currently, a Learning Resource is not formally declared in the schema.org schema as a subtype of CreativeWork. Instead, it is inferred that a Learning Resource is a kind of CreativeWork since the LRMI properties were included as part of the CreativeWork type. The lack of an explicit LearningResource subtype to CreativeWork makes it difficult for some doing markup or implementing systems using schema.org to recognize that schema.org in fact supports description of learning resources. They see EducationEvent as a subtype of Event, but no LearningResource as a subtype of CreativeWork.
6. Define a minimal subset of schema.org for describing learning materials
The schema.org schema is quite large and can be intimidating for some wanting to define minimally viable learning resource descriptions–e.g., where to begin, what properties are most important, how should they be used. Publishing a suggested profile of schema.org that defines a minimaly viable learning resource description while leaving open the addition of other properties needed with a particular use, might assist implementers needing a means to jumpstart development of their own profile based in schema.org.
7. Create support materials explaining LRMI properties
Recently, examples of how to use the AlignmentObject as well as the LRMI properties of CreativeWork have been added as ‘footers’ to the relevant schema pages at schema.org. While an excellent beginning, these additions are not enough. Other types of support materials for describing learning resources using the LRMI properties and classes need to be defined, developed, and published. Such materials might include a one-stop “primer” that combines narrative with examples and covers the inevitable points in usage where subjective decisions must be made where there are alternative legitimate paths forward.
8. Collate information about existing LRMI implementations
Specifications always need to be interpreted in order to be used in specific contexts, and however much normative and informative material is provided, we cannot hope to cover all contexts. One way that people implementing a specification can make choices that do not lead to unnecessary divergence is by referring to other implementations from similar contexts. LRMI could facilitate this by collating information about where these implementations can be found. This may also be useful in monitoring the uptake of the specification, identifying problems commonly encountered and providing examples of good practice.
9. Create an editor for LRMI metadata
Several tools exist that can create LRMI metadata within the scope of a single project or service’s needs. What is suggested here is that LRMI create, assist or promote the creation of an editor that can serve as a reference implementation: independent of the choices that would need to be made for any use in practice but flexible enough to be tailored practical use and, importantly, illustrating good practice in the implementation of LRMI.

The voting

voting from LRMI workshop DCMI CopenhageIn the end we discussed and voted on the issues of which ideas seemed most appealing and then, after eliminating those ideas which were not popular, we discussed and voted on the ideas that seemed least feasible, then had a short discussion about the results. We kept a tally of the votes on a flip chart: green ticks for ideas we thought attractive, red ticks for those we thought least feasible.

The ideas most liked were:

  • Structured, controlled vocabularies for LRMI properties,
  • Define a minimal subset of schema.org for describing learning materials, and
  • Collate information about existing LRMI implementations.

As was pointed out in the discussion, there are relationships between the ideas which mean that some other ideas become easier if these are achieved first (e.g. a general purpose editor becomes easier if you have agreed a general purpose subset of schema for it to edit).

Those ideas which we thought most difficult to achieve were

  • Structured, controlled vocabularies for LRMI properties and
  • Declare a “Learning Resource” type.

The specific issues with both of these centred around achieving agreement or consensus on terms and definitions.

I’m pretty sure that everyone involved in LRMI will see something in these results that they think a sensible outcome and something which raises an eyebrow or two. I’m also fairly sure that it won’t be the same things for everyone.. which is to say, the discussions will continue.

So what now?

Some of the ideas discussed are based on work that individuals or groups involved in LRMI are already scoping out. As a group we have made a start on formally describing some values for use with LRMI terms as SKOS concepts (see the DCMI/LRMI Github repository).  I have had a masters student work on creating a subset of schema.org for learning resources. There is an editor for LRMI in the works, which looks like it could meet the sort of requirements described above. And as I described in the presentation I gave at the workshop, there is work describing how LRMI is used. So hopefully we will be able to make progress on some of these ideas.

As I said during the workshop, LRMI is a volunteer effort. People work on whichever aspects of it interests them. If you are interested in any of the ideas described above, or have some other ideas that you think would be valuable, please join us and I hope will be able to achieve more together.

LRMI at #DCMI16 Metadata Summit, Copenhagen

LRMI, , ,

I was in Copenhagen last week, at the Dublin Core Metadata Initiative 2016 conference, where I ran a workshop about LRMI entitled “Building on Schema.org to describe learning resources” (as one of my colleagues pointed out, thinking of the snappy title never quite happened). Here’s a quick overview of it.

There were three broad parts to the workshop: presentations on the background organisations and technology; presentations on how LRMI is being used; and a debate where attendees got to think about what could be next for LRMI.
Continue reading →

Schema course extension update

Course description, LRMI, , ,

This progress update on the work to extend schema.org to support the discovery of any type of educational course is cross-posted from the Schema Course Extension W3C Community Group. If you are interested in this work please head over there.

What aspects of a course can we now describe?
As a result of work so far addressing the use cases that we outlined, we now have answers to the following questions about how to describe courses using schema.org:

As with anything in schema.org, many of the answers proposed are not the final word on all the detail required in every case, but they form a solid basis that I think will be adequate in many instances.

What new properties are we proposing?
In short, remarkably few. Many of the aspects of a course can be described in the same way as for other creative works or events. However we did find that we needed to create two new types Course and CourseInstance to identify whether the description related to a course that could be offered at various times or a specific offering or section of that course. We also found the need for three new properties for Course: courseCode, coursePrerequisites and hasCourseInstance; and two new properties for CourseInstance: courseMode and instructor.

There are others under discussion, but I highlight these as proposed because they are being put forward for inclusion in the next release of the schema.org core vocabulary.

showing how Google will display information about courses in a search galleryMore good news:  the Google search gallery documentation for developers already includes information on how to provide the most basic info about Courses. This is where we are going 🙂

Schema course extension progress update

cetis, Course description, LRMI, ,

I am chair of the Schema Course Extension W3C Community Group, which aims to develop an extension for schema.org concerning the discovery of any type of educational course. This progress update is cross-posted from there.

If the forming-storming-norming-performing model of group development still has any currency, then I am pretty sure that February was the “storming” phase. There was a lot of discussion, much of it around the modelling of the basic entities for describing courses and how they relate to core types in schema (the Modelling Course and CourseOffering & Course, a new dawn? threads). Pleased to say that the discussion did its job, and we achieved some sort of consensus (norming) around modelling courses in two parts

Course, a subtype of CreativeWork: A description of an educational course which may be offered as distinct instances at different times and places, or through different media or modes of study. An educational course is a sequence of one or more educational events and/or creative works which aims to build knowledge, competence or ability of learners.

CourseInstance, a subtype of Event: An instance of a Course offered at a specific time and place or through specific media or mode of study or to a specific section of students.

hasCourseInstance, a property of Course with expected range CourseInstance: An offering of the course at a specific time and place or through specific media or mode of study or to a specific section of students.

(see Modelling Course and CourseInstance on the group wiki)

This modelling, especially the subtyping from existing schema.org types allows us to meet many of the requirements arising from the use cases quite simply. For example, the cost of a course instance can be provided using the offers property of schema.org/Event.

The wiki is working to a reasonable extent as a place to record the outcomes of the discussion. Working from the outline use cases page you can see which requirements have pages, and those pages that exist point to the relevant discussion threads in the mail list and, where we have got this far, describe the current solution.  The wiki is also the place to find examples for testing whether the proposed solution can be used to mark up real course information.

As well as the wiki, we have the proposal on github, which can be used to build working test instances on appspot showing the proposed changes to the schema.org site.

The next phase of the work should see us performing, working through the requirements from the use cases and showing how they can be me. I think we should focus first on those that look easy to do with existing properties of schema.org/Event and schema.org/CreativeWork.

schema for courses

Course description, LRMI,

UPDATE: there is a new W3C community group schema course extend set up to progress these ideas. Please join if you are interested.

This is essentially an invite to get involved with building a schema extension for educational courses, by way of a description of work so far. If you want to reply it’s sent as an email schema.org mail list.

About a year ago there was a flurry of discussion about wanting to markup descriptions of courses in schema. Vicky Tardiff-Holland produced a proposal which we discussed in LRMI and elsewhere as a result of which various suggestions were and comments were added to that proposal.

I also led some work in LRMI around scope, use cases, requirements, existing data; which I hope will lead to validating/refining the proposal by some example data that could be used to demonstrate that it met the use cases.

I am up for another push on courses. I share the doc I was working on in the hope that it is good starting point. It’s a bit long, so here is an overview of what it contains:

  • scope: concerning discovery of any type of educational course (online/offline, long/short, scheduled/on-demand) Educational course defined as “some sequence of events and/or creative works which aims to build knowledge, competence or ability of learners”. (out of scope: information about students and their progression etc; information needed internally for course management rather than discovery)
  • comparators: a review of some established ways of sharing similar data
  • use cases
  • requirements arising from the use cases
  • mapping to some existing examples. I used hypothes.is to annotate existing web pages that describe different types of course, e.g. from Coursera or a University, tagging the requirement that the data was relevant to. Here’s an example of a page as tagged (click on a yellow highlight to show the relevant requirement as a comment with a tag)
    hypothes.is aggregates the selected information for each tag, to give a list of the information relevant to each use case, for example cost

I think the next step would be to review the use cases and requirements in light of some of the observations from the mapping, and to look again at the proposal to see how it reflects the data available/required. But first I want to try to get more people involved, see whether anyone has a better idea for how to progress, or if anyone wants to check the work so far and help move it forward.

Finally, I’m aware the docs and discussions so far around schema for courses are a scattered set of scraps and drafts. If there is enough interest it would be really useful to have it in one place.

Presentation: LRMI – using schema.org to facilitate educational resource discovery on the web and beyond

cetis, LRMI, ,

Today I am in London for the ISKO Knowledge Organisation in Learning and Teaching meeting, where I am presenting on LRMI and schema.org to facilitate educational resource discovery on the web and beyond. My slides are here, mostly they cover similar ground to presentations I’ve given before which have been captured on video or which I have written up in more detail. So here I’ll just point to my slides for today (& below) and summarise the new stuff.

LRMI uptake

People always want to know how much LRMI exists in the wild, and now schema.org reports this infomation. Go to the schema.org page for any class or property and at the top it says in how many domains they find markup for it. Obviously this misses that not all domains are equal in extent or importance: finding LRMI on pjjk,net should not count as equal to finding it on bbc.co.uk, but as a broad indicator it’s OK: finding a property on 10 domains or 10,000 domains is a valid comarison. LRMI properties are mostly reported as found on 100-1000 domains (e.g. learning resource type) or 10-100 domains (e.g. educational alignment). A couple of LRMI properties have greater usage, e.g. typical age range and is based on URL (10-50,00 and 1-10,000 domains respectively), but I guess that reflects their generic usefulness beyond learning resources. We know that in some cases LRMI is used for internal systems but not exposed on web pages, but still the level of usage is not as high as we would like.

I also often get asked about support for creating LRMI metadata, this time I’m including a mention of how it is possible to write WordPress plugins and themes with schema / LRMI support, and the drupal schema.org plugin. I’m also aware of “tagging tools” associated with various repositories, e.g. the learning registry and the Illinois Shared Learning Environment. I think it’s always going to be difficult to answer this one as the best support will always come from customising whatever CMS an organisation uses to manage their content or metadata and will be tailored to their workflow and the types of resources and educational contexts they work in.

As far implementation for search I still cover google custom search, as in the previous presentations.

Current LRMI activities

The DCMI LRMI task group is active, one of our priorities is to improve the support for people who want to use LRMI. Two activities are nearing fruitition: firstly, we are hoping to provide examples for relevant properties and type on the schema.org web site. Secondly, we want to provide better support for the vocabularies used for properties such as alignment type (in the Alignment Object), learning resource type etc, by way of clear definitions and machine readable vocabulary encodings (using SKOS). We are asking for public review and comment on LRMI vocabularies, so please take a look and get in touch.

Other work in progress is around schema for courses and extending some of the vocabularies mentioned above. We have monthly calls, if you would like to lend a hand please do get in touch.

Checking schema.org data with the Yandex structured data validator API

LRMI, ,

I have been writing examples of LRMI metadata for schema.org. Of course I want these to be valid, so I have been hitting various online validators quite frequently. This was getting tedious. Fortunately, the Yandex structured data validator has an API, so I could write a python script to automate the testing.

Here it is 

#!/usr/bin/python
import httplib, urllib, json, sys 
from html2text import html2text
from sys import argv

noerror = False

def errhunt(key, responses):                 # a key and a dictionary,  
    print "Checking %s object" % key         # we're going on an err hunt
    if (responses[key]):
        for s in responses[key]:             
            for object_key in s.keys(): 
                if (object_key == "@error"):              
                    print "Errors in ", key
                    for error in s['@error']:
                        print "\tError code:    ", error['error_code'][0]
                        print "\tError message: ", html2text(error['message'][0]).replace('\n',' ')
                        noerror = False
                elif (s[object_key] != ''):
                    errhunt(object_key, s)
                else:
                    print "No errors in %s object" % key
    else:
        print "No %s objects" % key

try:
    script, file_name = argv 
except:
    print "\tError: Missing argument, name of file to check.\n\tUsage: yandexvalidator.py filename"
    sys.exit(0)

try:
    file = open( file_name, 'r' )
except:
    print "\tError: Could not open file ", file_name, " to read"
    sys.exit(0)

content = file.read()

try:
    validator_url = "validator-api.semweb.yandex.ru"
    key = "12345-1234-1234-1234-123456789abc"
    params = urllib.urlencode({'apikey': key, 
                               'lang': 'en', 
                               'pretty': 'true', 
                               'only_errors': 'true' 
                             })
    validator_path = "/v1.0/document_parser?"+params
    headers = {"Content-type": "application/x-www-form-urlencoded",
               "Accept": "*/*"}
    validator_connection = httplib.HTTPSConnection( validator_url )
except:
    print "\tError: something went wrong connecting to the Yandex validator." 

try:
    validator_connection.request("POST", validator_path, content, headers)
    response = validator_connection.getresponse()
    if (response.status == 204):
        noerror= True
        response_data = response.read()   # to clear for next connection
    else:
        response_data = json.load(response)
    validator_connection.close()
except:
    print "\tError: something went wrong getting data from the Yandex validator by API." 
    print "\tcontent:\n", content
    print "\tresponse: ", response.read()
    print "\tstatus: ", response.status
    print "\tmessage: ", response.msg 
    print "\treason: ", response.reason 
    print "\n"

    raise
    sys.exit(0)

if noerror :
    print "No errors found."
else:
    for k in response_data.keys():
        errhunt(k, response_data)

Usage:

$ ./yandexvalidator.py test.html
No errors found.
$ ./yandexvalidator.py test2.html
Checking json-ld object
No json-ld objects
Checking rdfa object
No rdfa objects
Checking id object
No id objects
Checking microformat object
No microformat objects
Checking microdata object
Checking http://schema.org/audience object
Checking http://schema.org/educationalAlignment object
Checking http://schema.org/video object
Errors in  http://schema.org/video
	Error code:     missing_empty
	Error message:  WARNING: Не выполнено обязательное условие для передачи данных в Яндекс.Видео: **isFamilyFriendly** field missing or empty  
	Error code:     missing_empty
	Error message:  WARNING: Не выполнено обязательное условие для передачи данных в Яндекс.Видео: **thumbnail** field missing or empty  
$

Points to note:

  • I’m no software engineer. I’ve tested this against valid and invalid files. You’re welcome to use this, but it might not work for you. (You’ll need your own API key). If you see something needs fixing, drop me a line.
  • Line 51: has to be an HTTPS connection.
  • Line 58: we ask for errors only (at line 46) so no news is good news.
  • The function errhunt does most of the work, recursively.

The response from the API is a json object (and json objects are converted into python dictionary objects by line 62), the keys of which are the “id” you sent and each of the structured data formats that are checked. For each of these there is an array/list of objects, and these objects are either simple key-value pairs or the value may be an array/list of objects. If there is an error in any of the objects, the value for the key “@error” gives the details, as a list of Error_code and Error_message key-value pairs. errhunt iterates and recurses through these lists of objects with embedded lists of objects.