I was fortunate to instigate a workshop last week as part of the BC Educational Technology Users Group spring workshops in Merritt, B.C. The workshop was on “Practical Tips for Reusability and Interoperability.” In keeping with themes I laid out earlier on this weblog, I began the session with a formal execution of the term “learning object” which you can see at the link above (feel free to reuse this – maybe if it’s played enough times the term will finally die off). (more…)

As I go on to explain, it’s not the concepts the term was supposed to foster that I object to so much as the term itself, as it has left many an instructor panicked and struggling to understand what it means, as if it were something radically different from the learning content they are already producing. (Though I do actually disagree with Alan that LOs are simply links and all we need is referatories, but that’s likely another post.)

The goal of the workshop was to brainstorm with practitioners in the field some tips on facilitating reuse and interoperability that were in fact practical in terms of their current models of content production and reuse. The fact that I’ve seen more documents endorsing Content Packaging or XML-based authoring approaches than I have actual implementations of these approaches has led me to believe there to be a fair disconnect between the theory and actual practice, and this was a very small first foray into agreeing on some practical guidelines we can implement here in B.C.

As a way to help people consider re-use, I offered up this very small grid on the ‘Contexts of Reuse.’ We then looked at 10 or so actual examples of learning content and brainstormed some of the problems to reusing these, and some possible solutions. We ran out of time before we got a chance to finish the final exercise, which was to go through the list of proposed solutions and sort out the practical from the difficult or impractical, though I did have a stab at it myself beforehand, and was gratified with the extent to which it matched up with the instructor’s actual expectations.

This was the first time I had run such a workshop and I was really pleased with how it turned out. There were 15 or so folks in the room and nearly everyone was participating in the discussion and brainstorming. Another part of the experiment you may have noticed was the use of a wiki for the presentation – not only did I build any of the material for the presentation within the wiki, but was actually using it in real time as a way to document the brainstorming and resulting practical tips. The feedback so far was that they really enjoyed this interactive aspect and that for the most part the technology was an aid, not a distraction (though my boat-anchor of a laptop did it’s best to subvert this!) Brian was kind enough to let me use the UBC wiki for this presentation; part of the reason for doing it in a wiki was the hope that it would become a ‘living’ document; the reality is that it needs some housekeeping before it can serve as a structure to capture actual practical tips for a wider audience, and it may well not be on this server, but it feels like the theory has been borne out in practice. For me this was a pretty big experiment (I am not an instructional developer or designer by trade, and so feel pretty uncomfortable in those waters) but I’d like to think it was reasonably successful and hope to either do another such session soon or garden the wiki outputs to where they can be useful as a scaffold for further unmediated brainstorming and documentation of more practical tips. – SWL

<rant> Over on on the Flosse Posse weblog, Teemu Leinonen has posted a bit of a rant on the term “learning object.” I’m glad someone stepped up and said it. I agree, let’s kill off the word “learning object” and while we are at it, let’s throw “learning object repository” on the funeral pyre too. Both of these terms have led us thoroughly astray. “Learning Object” for their implication of some magical plug-and-play learnability that we’re discovering is mostly folly, and “learning object repositories” for the mistaken emphasis of the word “repository” on the container at the sake of the users and re-users and re-use, ultimately what I thought the motivation behind the whole idea was.

But my small fear is that in throwing out these terms, we’ll also throw out many of the problems they were supposed to be trying to solve – namely enabling learning content to be shared and found through means that were otherwise unavailable (e.g. searching on pedagogically useful terms that were either not directly part of the resources themselves, or else for resources that weren’t served well by conventional web search engines), and having formats for learning content that allowed it to be reused by as many systems as possible without major alterations (there are many more problems they were supposed to address, I know, but let’s leave it at that for now). I absolutely agree that the terms have gotten in the way, and have led us to propose solutions which seem to have forgotten some of the initial problems they were supposed to be solving. Actually, in the case of both “learning objects” and “LORs,” part of the issue for higher ed has been IMO in higher ed’s appropriation of the terms; we’ve assumed the terminology, but we’ve tried to change the underlying problems they were originally intended to address to suit the needs and culture of higher ed, and we haven’t done enough critical examination of the baggage underpinning the terms and original ideas to understand why this isn’t working).

So, firing squad, guillotine, maybe lethal injection as we’re now so civilized; I don’t care, but let’s move on from these terms and the 5 years (at least) of false starts that are associated with them. We likely couldn’t be moving on without having made these mistakes, but once made, repeating them over and over doesn’t suddenly make them right. </rant> – SWL

I’ve been searching for this paper without being sure it existed – “Two course developers … investigated and described from their personal points of view the complex and immediate challenges they faced as they designed an online university course based on learning objects.”

Somewhat disappointingly, they ultimately “acquired a ‘ready-made’ commercial website featuring learning objects and electronic material embedded in a comprehensive course website.” Still, this paper lays out the details of what appears to be quite a straightforward attempt to create a new online course by finding and assembling resources. The authors are clearly not evangelists for the LO approach, and make clear a number of its current shortcomings and difficulties.

We desperately need more stories like these, as well as far more serious work on what a real ‘learning object design approach’ to create new curriculum out of existing materials might look like (instead of more pie in the sky ‘automated assembly of instructional materials’ scenarios, please!). One of a number of interesting papers from Athabasca University that resulted from their participation in the SchoolNet Project ‘Learning Objects in a Box’ – SWL

Based on a reference in a recent intro to learning objects I went back to a site I thought I new, but instead found this newly developed resource that the Eduworks folks produced for the NSDL. It is really worth spending some time on, for both newbies and old hands alike. The section on “Fostering Reusability in the NSDL” is very helpful, and the Reusability Framework is, I think, top notch and I would be surprised if I didn’t start to see it show up more as a canonical reference. – SWL

This NSF-funded project in the States just makes so much sense – have the univeristy students currently studying to be educational designers and developers work on actual learning content for the K-12 system. As the site says, “Through these courses TRAILS intends to have three major effects: to better prepare tomorrow’s designers of educational tools, to better prepare the teachers who will use such tools, and—by publishing select course projects— to generate new tools for K-12 education.” Obvious? Maybe. Innovative. Seems to me, definitely! – SWL

As I was about to enter a handy-dandy Google search, I thought – “no wait! Why don’t you see if there are any ‘learning objects’ out there that could teach you what a Z-score is, and how to calculate it.” So I set out in search of my closest learning object repository to see what I could find.

Off I went, partly with a real goal of learning about Z-scores, and partly as a bit of an experiment to see what existing repositories had in store for me.

My first stop was the granddaddy (grandma?) of them all – MERLOT. Maybe not a ‘repository’ per se, but it’s been around for a while and would certainly have some good pointers. And sure enough, a search for the term ‘z-score’ brought back one result which seemed promising. The resource that was pointed to was certainly about much more than simple Z-scores, but sure enough it did have a page dedicated to that topic. So while there was a bit of deft navigation involved after the fact to locate the actual material concerning Z-scores, at least something relevant came back, which was encouraging. Still, it’s a good thing I didn’t search MERLOT for ‘z-scores‘(zero results) or ‘z score (without the dash)’ (26 results, only one of which appears relevant). And better yet that I didn’t try the federated search – which wouldn’t have found the one relevant MERLOT entry, but would have returned 10 entries from SMETE seemingly all related to astrophysics!

But in the spirit of furthering this line of enquiry, I thought I’d try out a few other repositories. Next was one closer to home – CAREO. CAREO’s also been around a while, and has learnt a lot about how repositories should work.

Sure enough, a search on ‘z-score’ does get a hit, in fact 5 hits. Problem being that they are all the same resource!

So that it’s clear, I do not mean to pick on these particular repositories. Direct searches on almost all of the repositories listed here fared about as well (one exception was this example, that provided 2 flash applets with far to much info on the topic).

As a final thought (I still needed to actually calculate some z-scores) I did the Google search I had originally meant to do. There in the first result was the specific page in the online course that MERLOT had pointed to, along with 10 other pertinent resources (some relating to Z-scores, some to ‘Z score’ and others to ‘Z scores’). My favourite turned out to be the last one on the first page of Google results – all I really wanted to know was that I needed to subtract the mean of the values from my value and divide it by the standard deviation, and that z-scores “measure the distance from the mean of a distribution normalized by the standard deviation.”

What’s to be learnt from all of this (besides the fact that I’m a lousy statistician and verbose to boot!) I’ll let you draw your own conclusions. Was my ‘study’ a fair and accurate one? Not at all. Did it mimic how real people might set about finding a learning resource? It doesn’t seem like an unreasonable scenario. Should I have expected more resources, or an easier time locating what I actually wanted? Well, if your standard is Google, which may not be entirely fair, I’d say that we have a fair ways to go. – SWL

This paper, from Dan Rehak and others at the renowned Learning Systems Architecture Laboratory at Carnegie Mellon University (which now has a new RSS feed), asks how authors actually create learning objects, and whether the current tools are supporting these actual processes or instead getting in the way. To investigate this they take the sensible step back from the technology and look at a number of low or no tech paper-based techniques for developing learning objects, with the “objective [is] to help create learning, and hide technology and standards” and thus “understand how learning technology standards can be applied in the creation of learning objects and content.”

You can draw your own conclusions – I think the process they outline is a useful one for tool builders to go through if they want to build tools that support the way people actually work. But my cursory reading didn’t reveal any huge lessons learnt from the paper-based modelling and many of the criticisms levelled at the one example tool (ReLoad) they cite could seemingly be levelled at the paper-based model as well (e.g. use of jargon for one). – SWL

For people either building new repository software or even figuring out what they need in implementing existing software, this might be of interest. Not quite a set of use cases, but maybe close – a set of scenarios which “attempt … to map the process that an educator might walk through to implement digital resources in classroom or online environments….
While the exemplars focus on K-12 learning, the Model and Workshop could apply to post-secondary instructors and designers as well. Developers may find the Case Examples informative in the development of flexible content tools.” – SWL

From Gerry Paille and his team (a partnership of BC School District #60, Open School BC and the Open Learning Agency/BC Open Univeristy to develop a ‘CANCORE-compliant’ resource network that will house materials from some of their older resource collections) comes this useful 3 module course on “using a structured language such as Standard Generalized Markup Language (SGML) or eXtensible Markup Language (XML) as a basis for producing a learning design and describing course content, activities, and assignments.”

Gerry has also gone to the effort of packaging the course as an IMS 1.1.3 Content Package. Gerry notes on the project blog that the course has yet to receive a ‘technical review’ but is still quite worthwhile. One neat feature of how they implemented this is the ‘Module Resources’ links in each of the modules, which seem to be keyword searches to the backend CAREO database to provide related supplementary resources for each module. – SWL

I don’t know if I could tell you exactly *why* MERLOT published all of this – possibly for the greater public good or possibly to maintain a fairly high degree of transparency and formality given so many stakeholders in their development process. In any case, they have shared the policies shaping their development framework as well as a number of other technical background documents, all of which should prove of some use to people developing their own repositories or even just deciding on what strategy to take in implementing one. – SWL