I sometimes get asked for asked for advice by people who are thinking of setting up national infrastructure for OER based on institutional open access research repositories or similar, often with the rationale that doing such would mirror what has worked for open access research papers and cultural heritage. My advice is to think hard about whether it is appropriate to treat OER in the same way as these other types of resource. This week I read a paper, “Towards a Devolved Model of Management of OER? The Case of the Irish Higher Education Sector” by Angelica Risquez, Claire McAvinia, Yvonne Desmond, Catherine Bruen, Deirdre Ryan, and Ann Coughlan which provides important evidence and analysis on this topic.
Tag Archives: repositories
Publications on WordPress
I have posted details of my publications on this site. More interestingly, I did so with a plugin and theme extension which builds on previous work on presenting publications in WordPress and using WordPress as a semantic platform. These are useful steps towards WordPress as a lightweight repository. Continue reading
Book chapter: Technology Strategies for Open Educational Resource Dissemination
A book with a chapter by Lorna M Campbell and me has just been published. The book is Open Education: International Perspectives in Higher Education edited by Patrick Blessinger and TJ Bliss, published by Open Book Publishers.
There are contributions by people I know and look up to in the OER world, and some equally good chapters by folk I had not come across before. It seems to live up to its billing of offering an international perspective by not being US-centric (though it would be nice to see more from S America, Asia and Africa), and it provides a wide view of Open Education, not limited to Open Education Resources. There is a foreword by David Wiley, a chapter on a human rights theory for open education by the editors, one on whether emancipation through open education is theory or rhetoric by Andy Lane. Other people from the Open University’s Open Education team (Martin Weller, Beatriz de los Arcos, Rob Farrow, Rebecca Pitt and Patrick McAndrew) have written about identifying categories of OER users. There are chapters on aspects such as open science, open text books, open assessment and credentials for open learning; and several case studies and reflections on open education in practice.
Open Education: International Perspectives in Higher Education is available under a CC:BY licence as a free PDF, as very cheap mobi or ePub, or reasonably priced soft and hard back editions. You should get a copy from the publishers.
Technology Strategies for OER
The chapter that Lorna and I wrote is an overview drawing on our experiences through the UKOER programme and our work on LRMI looking at managing the dissemination and discovery of open education resources. Here’s the abstract in full, and a link to the final submitted version of our chapter.
This chapter addresses issues around the discovery and use of Open Educational Resources (OER) by presenting a state of the art overview of technology strategies for the description and dissemination of content as OER. These technology strategies include institutional repositories and websites, subject specific repositories, sites for sharing specific types of content (such as video, images, ebooks) and general global repositories. There are also services that aggregate content from a range of collections, these may specialize by subject, region or resource type. A number of examples of these services are analyzed in terms of their scope, how they present resources, the technologies they use and how they promote and support a community of users. The variety of strategies for resource description taken by these platforms is also discussed. These range from formal machine-readable metadata to human readable text. It is argued that resource description should not be seen as a purely technical activity. Library and information professionals have much to contribute, however academics could also make a valuable contribution to open educational resource (OER) description if the established good practice of identifying the provenance and aims of scholarly works is applied to learning resources. The current rate of change among repositories is quite startling with several repositories and applications having either shut down or having changed radically in the year or so that the work on which this contribution is based took. With this in mind, the chapter concludes with a few words on sustainability.
Sustainability and Open Education
Last week I was on a panel at Edinburgh University’s Repository Fringe event discussing sustainability and OER. As part of this I was asked to talk for ten minutes on some aspect of the subject. I don’t think I said anything of startling originality, but I must start posting to this blog again, so here are the notes I spoke from. The idea that I wanted to get over is that projects should be careful about what services they tried to set up, they (the services) should be suitable and sustainable, and in fact it might be best if they did the minimum that was necessary (which might mean not setting up a repository).
Between 2009 and 2012 Jisc and the HE Academy ran the UK Open Education Resources programme (UKOER), spending approximately £15M of Hefce funding in three phases. There were 65 projects, some with personal, institutional or discipline scope releasing resources openly, some with a remit of promoting dissemination or discoverability, and there were some related activities and services providing technical, legal, policy support, & there was Jorum: there was a mandate that OERs released through the project should be deposited in the Jorum repository. This was a time when open education was booming, as well as UKOER, funding from foundations in the US, notably Hewlett and Gates, was quite well established and EU funding was beginning. UKOER also, of course, built on previous Jisc programmes such as X4L, ReProduce, and the Repositories & preservation programme.
In many ways UKOER was a great success: a great number of resources were created or released, but also it established open education as a thing that people in UK HE talked about. It showed how to remove some of the blockers to the reuse and sharing of content for teaching and learning in HE (–especially in the use of standard CC licences with global scope rather than the vague, restrictive and expensive custom variations on “available to other UK HEIs” of previous programmes). Helped by UKOER, many UK HEIs were well placed to explore the possibilities of MOOCs. And in general showed the potential to change how HEIs engage with the wider world and to help make best use of online learning–but it’s not just about opening exciting but vague possibilities: being a means to avoid problems such as restrictive licensing, and being in position to explore new possibilities, means avoiding unnecessary costs in the future and helps to make OER financially attractive (and that’s important to sustainability). Evidence of this success: even though UKOER was largely based on HEFCE funding, there are direct connections from UKOER to the University of Edinburgh’s Open Ed initiative and (less directly) to their engagement with MOOCs.
But I am here to talk sustainability. You probably know that Jorum, the repository in to which UKOER projects were required to deposit their OERs, is closing. Also, many of the discipline-based and discovery projects were based at HE Academy subject centres, which are now gone. At the recent OER16 here, Pat Lockley suggested that OER were no longer being created. He did this based on what he sees coming in to the Solvonauts aggregator that he develops and runs. Martin Poulter showed the graph, there is a fairly dramatic drop in the number of new deposits he sees. That suggests something is not being sustained.
But what?
Let’s distinguish between sustainability and persistence: sustainability suggests to me a manageable on-going effort. The content as released may be persistent, it may still be available as released (though without some sort of sustainable effort of editing, updating, preservation it may not be much use). What else needs sustained effort? I would suggest: 1, the release of new content; 2, interest and community; 3, the services around the content (that includes repositories). I would say that UKOER did create a community interested in OER which is still pretty active. It could be larger, and less inward looking at times but for an academic community it doing quite well. New content is being released. But the services created by UKOER (and other OER initiatives) are dying. That, I think , is why Pat Lockley isn’t seeing new resources being published.
What is the lesson we should learn? Don’t create services to manage and disseminate your OERs that that require “project” level funding. Create the right services, don’t assume that what works for research outputs will work for educational resources, make sure that there is that “edit” button (or at least a make-your-own-editable-copy button). Make the best use of what is available. Use everything that is available. Use wikimedia services, but also use flickr, wordpress, youtube, itunes, vimeo,—and you may well want to create your own service to act as a “junction” between all the different places you’re putting your OERs, linking with them via their APIs for deposit and discovery. This is the basic idea behind POSSE: Publish (on your) Own Site, Syndicate Elsewhere.
Book now available. Into the Wild – Technology for Open Educational Resources
With great pleasure and more relief I can now announce the availability of Into the wild – technology for open educational resources, a book of our reflections on the technology involved in three years of the UK OER Programmes.
From the blurb:
Between 2009 and 2012 the Higher Education Funding Council funded a series of programmes to encourage higher education institutions in the UK to release existing educational content as Open Educational Resources. The HEFCE-funded UK OER Programme was run and managed by the JISC and the Higher Education Academy. The JISC CETIS “OER Technology Support Project” provided support for technical innovation across this programme. This book synthesises and reflects on the approaches taken and lessons learnt across the Programme and by the Support Project.
This book is not intended as a beginners guide or a technical manual, instead it is an expert synthesis of the key technical issues arising from a national publicly-funded programme. It is intended for people working with technology to support the creation, management, dissemination and tracking of open educational resources, and particularly those who design digital infrastructure and services at institutional and national level.
You may remember Lorna writing back in August that Amber Thomas, Martin Hawksey, Lorna and I had written 90% of this book together in a Book Sprint. Well, the last 10% and the publication turned in to a bit of a marathon-relay, something about which I might write some time, but now the book is available in a variety of formats:
- If you want glossy-covered paperback, then you can order it print-on-demand from Lulu (£3.36); if you’re not so fussed about the glossy cover and binding then there is a print-quality pdf you can print yourself.
- If you have an ePub reader you can download, there is a free download of an epub2 file.
- If you have a Kindle, you can download the .mobi file and transfer it, or if you prefer the convenience of Amazon’s distribution over whisper-net you can buy it from them (77p, they don’t seem to distribute for free unless you agree to give them exclusive rights for all electronic formats).
- finally, if you prefer your ebook reading as PDFs, there is one of those too.
All varieties are free or at minimum cost for the distribution channel used; the content is cc-by licensed and editable versions are available if you wish to remix and fix what we’ve done.
CETIS publications, now on WordPress
We have recently changed how we present our publications to the world. Where once we put a file on the web somewhere, anywhere, and entered the details into a home-spun publication database, now we use WordPress. We’re quite pleased with how that has worked out, so we’re sharing the information that might help others use WordPress as a means of presenting publications to the world (a repository, if you like).
Why WordPress?
First, what were we trying to achieve? The overall aims were to make sure that our publications had good exposure online, to have a more coherent approach to managing them (for example to collect all the files into one place in case we ever need to migrate them), and to move away from the bespoke system we were using to a system that someone else maintains. There were a few other requirements, we wanted something that was easy for us to adapt to fit the look and feel of the rest of our website, that was easy to maintain (familiarity is an important factor in how easy something is–it’s easy to use something if you know how to use it), and we wanted something that would present our publications in HTML and RSS sliced and diced by topic, author, and publication type: a URL for each publication and for each type of publication and feeds for everything. We’re not talking about a huge number of publications, maybe 100 or so, so we didn’t want a huge amount of up-front effort.
We thought about Open Journal Systems, but there seemed to be a whole load of workflow stuff that was relevant to Journals but not our publications. Likewise we thought about ePrints and Dspace, but they didn’t quite look like we wanted, and we are far more familiar with WordPress. As a wildly successful open source project, WordPress also fits the requirement of being maintained by other people, not just the core programme, but all those lovely plugins and themes. So the basic plan was to represent each publication in a WordPress post and to use a suitable theme and plugins to present them as we wanted.
The choice of theme
Having settled on WordPress the first decision was which theme to use. In order to get the look and feel to be similar to the rest of the CETIS website (and, to be honest, to make sure our publications pages didn’t look like a blog) we needed a very flexible theme. The most flexible theme I know of is Atahualpa, with over 200 options, including custom CSS snippets, parameters and HTML snippets it’s close to being a template for producing you own custom themes. So, for example, the theme options I have set include a byline of By %meta('By')%. %date('F Y')%
which automatically inserts the additional metadata field ‘By’ and the date in the format of my choice, all of which can be styled any way I want. I’ll come back to the “byline” metadata later.
One observation here: there is clearly a trade-off between this level of customisation and ease of maintenance. On the one hand these are options set within the Atahualpa theme that can be saved between theme upgrades, which is better than would have been the case had we decided to fork the theme or add a few lines of custom code to the theme’s PHP files. On the other hand, it is not always immediately obvious which setting in the several pages of Atahualpa theme options has been used to change some aspect of the site’s appearance.
A post for each publication
As I mentioned above we can represent each publication by creating a WordPress post, but what information do we want to provide about each publication and how does it fit into a WordPress post? Starting with the simple stuff:
- Title of the publication -> title of WordPress post.
- Abstract / summary -> body of post.
- Publication file -> uploaded as attached media.
- Type of publication -> category.
- Topic of publication -> tag.
Slightly less simple:
- The date of the publication is represented as the date of the post. This is possible because WordPress lets you choose when to publish post. The default is for posts to be published immediately when you press the Publish button, however you can edit this to have them published in the past 🙂
- The author of the publication becomes the author of the post, but there are some complications. It’s simple enough when the publication has a single author who works for CETIS, I just added everyone as an “author” user of WordPress and a WordPress admin user can attribute any given post to the author of the publication it represents. Where there are two or more authors a nifty little plugin called Co-Authors Plus allows them all to be assigned to the post. But we have some publications that we have commissioned from external authors, so I created an user called “Other” for these “external to CETIS” authors. This saves having a great long list of authors to maintain and present, but creates a problem of how to attribute these external authors, a problem that was solved using WordPress’s “additional metadata” feature to enter a “by-line” for all posts. This also provides a nicely formatted by-line for multi-author papers with out worrying about how to add PHP to put in commas and “and”s.
- The only other additional metadata added was an identifier for each publication, e.g. the latest QTI briefing paper is No. 2011:B02.
Presenting it all
As well as customisation for the look and feel, the Atahualpa theme allows for menus and widgets to added to the user interface. Atahualpa has an option to insert a menu into the page header which we used for the links to the other parts of the CETIS website. On the left hand side bar we’ve used the custom menu widget to list the tags and categories to provide access to the publications divided by topic and publication type as HTML and as a feed (just add /feed to the end of the URL). Also on the left, the List Authors plugin gives us links to publications by author.
In order to provide a preview of the publication in the post I used the TGN embed everything plugin. The only problem is that the “preview” is too good: it’s readable but not the highest quality, so it might lead some people to think that we’re disseminating low quality versions of the papers, whereas we do include links to high quality downloadable files.
The built-in WordPress search is rubbish. For example, it doesn’t include the author field in the search (not that the first thing we tested was vanity searching), and the results returned are sorted by date not relevance. Happily the relevanssi plugin provides all the search we need.
Finally a few tweaks. We chose URL patterns that avoid unnecessary cruft, and closed comments to avoid spam. We installed the Google analytics plugin, so we know what you’re doing on our site, and the login lock plugin for a bit of security. The only customisation that we want that couldn’t be done with a theme option or plugin was providing some context to the multi-post pages. These are pages like the list of all the publications, or all the briefing papers, and we wanted a heading and some text to explain what that particular cut of our collection was. Some themes do this by default, based on information entered about the tag/catergory/author on which the cut is made, but not Atahualpa. I put a few lines of PHP into the theme’s index.php template to deal with publication types, but we’ve yet to do it properly for all possible multipost pages.
And in the end…
As I said at the top, we’re happy with this approach; if you have any comment on it, do please leave them below.
One last thing. Using a popular platform like WordPress means that there is a lot of support, and I don’t just mean a well supported code base and directory of plugins and themes. One of the most useful sources of support has been the WordPress community, especially the local group of WPUK, at whose meet-ups I get burritos and advice on themes, plugins, security and all things wordpressy.
WordPress for hosting and describing learning resources
On 5 August I gave a presentation about Delores Selections with the above title to the CETIS Advances in Open Systems for Learning Resources workshop at the Edinburgh Repository Fringe meeting. Below is the powerpoint presentation I used and the (lightly editted) notes taken by Nicola Osborne’s in her live blog of the event.
Slide 1
Delores is: Delivering Open Educational Resources for Engineering Design.
Slide 2
We have static and dynamic collections of university level OERs and other openly available resources relevant to Engineering Design. A static collection may include dynamic resources but the collection itself is static, once set up it stays as it is. Dynamic collections can have new materials added or taken away or developed.
ICBL, School of mathematical and computer sciences, Heriot-Watt University and the University of Bath worked together on this project, funded by HEA and JISC under OER Phase 2.
Slide 3
We used WordPress to gather resources selected by experts in design engineering as being of high quality and usefulness for the collection. We aimed for about 100 objects in that collection of materials. The dynamic collection is everything underneath that. We use a tool called Sux0r which does Bayesian filtering of content – this is how Spam filtering works. We are using that idea the other way around – filtering to detect likely design engineering materials. Then we put material through a tool designed by Bath called Waypoint which enables faceted searching by automatic classification. Because Sux0r pulls RSS feeds from collections we know of, those feeds are continually updated and the collection presented by Waypoint continues to grow. I am going to focus on WordPress but I mention this context to point out that the technically difficult stuff, the effort, the hard thinking wasn’t really in the bit I am talking about.
Slide 4
So, starting off: what do we think we need in order to have this static collection? What are the needs for describing these OERs? First up you may not want to hold an actual copy of the resources. We decided we didn’t want to hold a copy of the resource, these are pre-existing resources hosted elsewhere. What metadata do we need? Title, description, authors, origin, date, subject, classification of some sort, licence, and probably something about the resource type. Users want to see that information, not necessarily locked up in an xml file. We want to embed a preview. We may or may not want to allow comments – but we don’t want to have to manage and spam filter those for the long term. We want something with a good web presence (and findable by Google) and something that has good participation (links in many direction, embedded material, widgets etc. We want it to take part in the web). We want RSS feeds – great for pushing metadata around, we want embedded metadata (thinking RDFa, microformats etc), we want flexibility, want something easy to use and maintain (perhaps familiar), and possibly the option to export metadata?
Slide 5
The idea that we had was to use WordPress. One blog post per resource – if required you can attach resources that are single files to the post. This gives you a basic description and good web presences. WP handles author, date, tite, and you have tags and topics for classification. Also extensions for metadata and additional functionality (a big developer community there).
Slide 6
We weren’t the first people with this idea…
Slide 7 & 8
Oxford’s Triton Project are running the Politics InSpires blog. They are creating OERs within WordPress – describe and comment on current affairs and other items. They have focused on add ons around that blog.
Slide 9 & 10
Edinburgh University have an initiative called OpenMed
Slide 11 & 12
CETIS has been exploring the use of WordPress to disseminate our publications. We see a sneak preview and should note that resources are attached to posts and it looks nothing like a blog
Slide 13
Scriblio (formally WPopac) – WordPress theme to create an OPAC using WordPress
slide 14 & 15
How were our goals met? Well most of what we wanted was possible.
All those question marks are where WordPress gives you information about the post not the resource resource described in the post, which matters for us because we are describing third party resources produced and hosted elsewhere. That is you get the date, author etc of the wordpress post you wrote to describe the resource, which isn’t really what you wanted. You get RSS feeds which link to and are about the descriptions in WordPress, not the resources.
But you do get a good website that is easy to use and maintain and familiar – though the more flexibility you use, the harder it is to maintain.
An Aside
One thing I like about WordPress is Trackbacks – you can see when you’ve been blogged or linked to – people can write about you and you can then aggregate those comments on your post.
Slide 16
So some customisation…
We used WordPress’s custom fields and we adapted a theme so that these are displayed. And we will have either a Plugin or theme extension written so that the right metadata goes into the RSS feed.
slide 17 and demo of Selections site
So lets have a look in the system for bridges
We can find a description and preview of the resource, links to it etc. Looking at the admin screen you can see we are using custom fields to include metadata about the object and we have set up categories that fit the curriculum. Lesa in the audience here wrote all of the resource description – she is a trained librarian and that has really been helpful here.
FINIT
The hunting of the OER
“As internet resources are being moved, they can no longer be traced.” I read in a press release from Knowledge Exchange. This struck me as important for OERs since part of their “openness” is the licence to copy them, and I have recently been on something of an OER hunt, which highlights the importance of using identifiers correctly and of “curatorial responsibility”.
The OER I was hunting was an “Interactive timeline on Anglo-Dutch relations (50 BC to 1830)” from the UKOER Open Dutch project. It was recommended at a year or so ago as great output which pretty much anyone could see the utility of that used the MIT SIMILE timeline software to create a really engaging interface. I liked it, but more importantly for what I’m considering now I used it as an example when investigating whether putting resources into a repository enhanced their visibility on Google (in this case it did).
Well, that was a year+ ago. The other week I wanted to find it again. So I went to Google and searched for “anglo dutch timeline” (without the quotes). Sure enough, I got three results for the one I am looking for on the first page (of course, your results my vary; Google’s like that now-a-days). These were, from the bottom up:
- A link to a record in the NDLR (the Irish National Digital Learning Resources Repository) which gave the link URL as http://open.jorum.ac.uk:80/xmlui/handle/123456789/517 (see below)
- A link to a resource page in HumBox, which turned out to be a manifest-only content package (i.e. metadata in a zip file). Looking into it, there’s no resource location given in the metadata, and the pointer to the content (which should be the resource being described) actually points to the Open Dutch home page.
- Finally, a link to a resource page in JORUM. This also describes the resource I was looking for but actually points to Open Dutch project page. The URL for Jorum page describing the resource is given as the persistent link–I believe that the NDLR harvests metadata from Jorum, so my guess is that that is why NDLR list this as the location of the resource.
Finding descriptions of a resource isn’t really helpful to many people. OK, I now know the full name and the author of the resource, which might help me track down the resource, but at this point I couldn’t. Furthermore, nobody wants to find a description of a resource that links to a description of the resource. I think one lesson concerns the importance of identifiers: “describe the thing you identify; identify the thing you describe.”
This story (and I very much suspect it is not an isolated case) has significance for debates about whether repositories should accept metadata-only “representations” of resources. Whether or not it is a good idea to deposit resources you are releasing as OERs in a third-party repository will depend on what you want to achieve by releasing them; whether or not it is a good idea for a repository to take and store resources from third parties will depend on what the repository’s sponsors wish to facilitate. Either way, someone needs to take some curatorial responsibility for the resource and for the metadata about it. That means on the one hand making sure that the resource stays on the web and on the other hand making sure that the metadata record continues to point to the right resource (automatic link checking for HTTP 404 responses etc. helps but, as this post on link rot notes, it’s not always that simple).
By the way, thanks to the incomparable David Kernohan, I now know that the timeline is currently at http://www.ucl.ac.uk/alternative-languages/OER/timeline/.
UKOER Sources
I have been compiling a directory of how people can get at the resources released by the UKOER pilot phase projects: that is the websites for human users and the “interoperability end points” for machines–ie the RSS and ATOM feed URLs, SRU targets, OAI-PMH base URLs and API documentation. This wasn’t nearly as easy as it should have been: I would have hoped that just listing the main URL for each project would have been enough for anyone to get at the resources they wanted or the interoperability end point in a click or two, but that often wasn’t the case.
So here are some questions I would like OER providers to answer by way of self assessment, which will hopefully simplify this in the future.
Does your project website have a very prominent link to where the OERs you have released may be found?
The technical requirements for phase 1 for delivery platforms said:
Projects are free to use any system or application as long as it is capable of delivering content freely on the open web. … In addition projects should use platforms that are capable of generating RSS/Atom feeds, particularly for collections of resources
So: what RSS feeds do you provide for collections of resources and where do you describe these? Have you thought about how many items you have in each feed and how well described they are?
Are your RSS feed URLs and other interoperability endpoints easy to find?
Do your interoperability end points work? I mean, have you tested them? Have you spoken to people who might use them?
While you’re thinking about interoperability end points: have you ever thought of your URI scheme as one? If for example you have a coherent scheme that puts all your OERs under a base URI, and better, provides URIs with some easily identifiable pattern for those OERs that form some coherent collection, then building simple applications such as Google Custom Search Engines becomes a whole lot easier. A good example is how MIT OCW is arranged: all most of the URIs have a pattern http://ocw.mit.edu/courses/[department]/[courseName]/[resourceType]/[filename].[ext] (the exceptions are things like video recordings where the actual media file is held elsewhere).
EdReNe: Building successful educational repositories
EdReNe, the EU funded Educational Repositories Network, have just published what looks like a useful a useful set of recommendations on Building successful educational repositories [pdf]. Many of the recommendations seem motherhood-and-apple-pie stuff: engage users, have policies, etc. though some of these, e.g. “support the needs of existing communities” have interesting implications when thought through in more depth (in this case don’t base your repository strategy entirely on creating a new community).
Others that caught my eye:
- Take advantage of generally used, open standards to allow for the broadest range of partnerships, future adaptability and innovation.
With the comment “Use standards that are ‘as broad as possible'”–a reference to “web-wide” standards rather than those that are specific to the repository world? - Acknowledge that integration with a range of tools and services will greatly benefit uptake and use of digital learning resources.
“What is useful, is the ability to integrate with the tools/service that the user selects.” So you’ll need an API. - Support the development of ‘sharing as a culture’ by providing user friendly mechanisms for depositing and repurposing
- Open up information silos by a strong focus on web services, APIs and other ways of allowing seamless integration across services.
“Repositories have to interface with many systems.” - Make it easy to participate – for all members
“Barriers to participation are the single biggest problem.” - Present clear and easy-to-understand information on usage rights
and - Clearly express usage rights to users when depositing or accessing resources
The “when accessing” aspect of this is something that has been exercising me recently, it’s hard to believe how many OERs don’t have any cc-licence information displayed on the resource itself. (For extra added irony, this report bears no licence information within it.) - Support open licensing to increase impact of funding and maximize possibilities for reuse and re-purposing,
- Encourage use of CC-BY licenses when publishing own work
and - Encourage institutions to engage in sharing and production of open content (institution management)
The OER effort is clearly having an impact on repository thinking, though there are comments to these and other recs that reflect that not all resources in repositories will be open. - When content standards are encouraged, this should be done with central guidance
“A top?down strategy for fixing standards does not work anymore.” (well, it only ever worked in some limited scenarios).
The recommendations are the output of EdReNe’s 5th seminar which took place in Copenhagen, 6-7 October 2010. Thanks to LTSO for the info about this.