OK, so what is this Xanadu project? First, if you want to explore for yourself, the project is at
http://xanadu.net/. Many of the ideas behind the project are expressed its founder Ted Nelson's
Computer Lib, of which I have only read small excerpts. My source for the history of the project is Gary Wolf's
Wired article
The Curse of Xanadu. As the title suggests, the article does not paint a rosy picture, and Nelson has objected strenuously to it in the
letters column of Wired itself.
Over its lifetime Xanadu has been a lot of things to a lot of people, but I'll focus on two aspects here: Xanadu as an architecture (in this post) and Xanadu as a business model (in the next), before going on to try to make some sort of overall sense of everything. Along the way I'll also touch on Xanadu as a software engineering project (or not).
So ... those first two paragraphs don't really belong in this post. They belong in the previous one. Now, I could go back and quietly edit Part I to include them. I
explicitly asserted the right to make quiet editorial changes quite a while back when trying to deal with a mistake I'd made. But the upshot of that experience was that it's generally better to leave anything more than minor mistakes uncorrected and supply further material on the subject if needed (this theme will recur in a moment). The principle I settled on was: a blog is not a wiki. In particular, it has no visible edit history, so the blog itself must fill that role.
That's actually not a digression. Any hypertext system has to deal with exactly the questions my little editorial decision raises, particularly: How do you handle a dynamically changing interlinked set of documents? If I edit something that someone has a link to, what should they see?
In a blog (or at least in Blogger blogs), a link to a post is a link to the latest revision. Exactly what you see may depend on when you chase the link. With a wiki you also have the option of linking to a particular version of an article, which will never change, however many later edits may come along. The W3C standards for HTTP and company
recommend that links be to immutable data, but they do not
require it. The basic machinery of the web works the same either way. Having results change over time is tolerated.
Xanadu takes a different approach to this and other issues. In Xanadu, every object has a unique, secure identity. This doesn't preclude keeping multiple copies for the usual reasons of performance and reliability, but these physical copies share the same identity and so represent a single logical object. Objects in Xanadu are immutable. If I revise a post, the revised post is a separate object from the original, which still remains.
Objects are addressable via lists of numbers called
tumblers. Tumblers are ordered, and given a tumbler it is always possible to find a tumbler after it but before any other existing tumbler. This makes it easy to add new revisions. Since tumblers are hierarchic by nature, it is possible to address parts within objects -- to address, say, the third paragraph of an article or a sentence in that paragraph, or a word in that sentence.
Links between objects are two-way, and they are visible objects in their own right. Links are non-intrusive, meaning that you can add a new link to or from an object without changing that object. The endpoints of a link are just tumblers
[if I've got it right]. Since tumblers can address arbitrary parts of an object, you can define, say, a link from the word "Xanadu" in some article to the Wikipedia article on Xanadu without changing either the source article or the Wikipedia article.
Since the system retains every version of every object, and links are addresses pointing to (or into) existing objects, links are never broken (the referent is no longer there) or dangling (the referent was never created at all).
Xanadu takes this one step further by positing that all edits point back to an immutable record of the unedited original. For example, if I boldface a word in a text, the boldfacing is separate from the original text. Documents become collections of editing commands, which may reference other such collections, and so forth.
Finally (for the purposes of this discussion at least), Xanadu includes a notion of
transclusion. Nelson defines transclusion as "the same content knowably in more than one place". Transclusion isn't the same as quoting. I just quoted Nelson, but there's no way to navigate from that quote to its source. Even if I make the quote a link, as "
the same content knowably in more than one place", that's still not transclusion, first because the link points to the whole document, not the quote, but more importantly because there's no way to navigate back, or to other uses of that quote.
[From illustrations of transclusion, it's easy to interpret it as "showing quoted text inline", but that's a matter of presentation. Whether the front end chooses to show a link or the full quoted text is its business. It's the navigability that matters from an architectural point of view, because that two-way navigability requires cooperation with the outside world.]This ability to slide back and forth between (or among) different uses of the same text is fundamental to Xanadu. Other features, such as immutability and tumbler addressing, exist to enable it.
This architecture has several key differences from The Web as We Know It:
- Links in Xanadu are never broken. Web links are routinely broken.
- Both endpoints of a link are fixed. If I edit a post, I've constructed a new collection of edits pointing into the original post. Your link points to the original, unchanged post. In the web, there is only one object, which has changed out from under a link.
- Links in Xanadu are bidirectional. If you link to my post, I (or you, or anyone else) can follow that link back to whatever you linked from. I can easily and automatically navigate from a post to the comments on the post. If you've commented on a particular sentence, I can see that because my end of the link refers to the sentence specifically.
- Links in Xanadu never go away, because nothing ever goes away.
- Markup lives outside a document. If I want to boldface every example of the word "orange" in a document and you want to boldface every example of the word "banana", we can do this independently and without editing the original.
- Xanadu doesn't exist. The web does.
That last may come across as snide, but unfortunately it's true. Xanadu as a concept has been around in one form or another since the 1960s -- coming up on half a century. In all that time, there has not been one commercial implementation of anything more than superficially like it. Not from Nelson, not from the dozens of programmers who have worked with Nelson, not from anyone inspired by Nelson to make it a reality, not by someone working in isolation and coming up with the same approach independently. This wants explanation.
From a technical point of view, it's tempting to look for scalability issues and other architectural weaknesses. For example
- If I decide, say, to link every word of every Field Notes post to its dictionary definition (applying some hack for words already occurring in links), that's my business. In Xanadu, the dictionary, at least, has to know about thousands of new links [More precisely, if not the dictionary, then whatever's keeping track of the links, and anything accessing the dictionary needs access to them.]
- Since Xanadu is meant to use redundant copies for performance and fault-tolerance, the keepers of every copy will have to know (or be able to find out about) all those links as well.
- And it all has to be kept in sync as changes come along. Cache coherence is one of the hard problems, though it certainly helps that objects are immutable and the set of objects only grows. Web protocols allow for caching, but stale cache entries can and do happen. This is just a fact of web.life, and web.life goes on.
- Suppose I really did want to link to the latest revision of something, whatever that may be at the moment. If you edit that something, then the link needs to be updated as well. My document doesn't need to be, since the link lives outside it, but anything referencing that link, or more likely the combination of that link and my document, also needs to be updated, assuming it also wants the latest version of everything. Updating means creating a new copy and ensuring that whatever wants to be pointing at the latest is pointing to it. The resulting pile of corner cases and gotchas is probably resolvable, but the upshot is that the simple act of editing may have arbitrarily wide-ranging consequences. On the web, no one but you has to know you edited a page. That can cut both ways, but from experience it appears to be the right default.
- Keeping every version of everything may be expensive in some cases, though in the case of, say, this blog it wouldn't be.
These are all valid concerns, and I'm sure there are more. The various developers must have run across them, and it would be interesting to read over the resulting discussions, if they're still out there. However, I think there are two more fundamental issues.
First, is Xanadu trying to solve the right problem? It's very clear that transclusion would solve problems that Nelson finds pressing, but it's far from clear there's any general demand for it, and by now that's not because no one in the field has heard of it, or even that no one in the general populace has. Nelson explains transclusion clearly enough in the site I linked to, and "the same content knowably in more than one place" is fairly clear all by itself. But no one seems to be asking for it. Nelson himself says that people rarely grasp the power and importance of transclusion. Fair enough, but such cases generally present a barrier to widespread adoption (not always -- some things you just have to try for a while before you decide they're actually cool, and some of those catch on anyway).
But more than that, Xanadu is fundamentally a closed system. Yes, it's possible to pull in, say, a web page and treat it as Xanadu content -- pull out a quote here, reformat there and create a Xanadu-style mash-up. But that's not transclusion. There is no way for the owner of the web page to know that its content is also elsewhere. To do that, the web page itself would have to be part of Xanadu.
The converse is only slightly better. Xanadu could present a web face allowing people to view it on a web browser and create documents with links to it. The Xanadu server could track referring sites in URLs and track who's visiting it via what page. But that doesn't provide any assurance that any particular quote appears on any particular page, even in the absence of spoofing. I might later delete a link, or I might simply cut and paste text in without making a link. There may well be additional difficulties with, say, a Xanadu object pointing to a web page that links back to something else in Xanadu. I can't be bothered to think that through at the moment.
In short, to actually realize the idea of transclusion, everyone has to cooperate. That could work for a purely local application that never accesses the net, or for a collection of servers that all run Xanadu and speak whatever protocol it would use to maintain links coherently. At this point, though, there is a lot of non-Xanadu information out there, and you'd have to persuade a huge number of existing systems to switch over or at least adopt Xanadu as an add-on. Any new source of information would also have to be Xanadu-aware. Not gonna happen. The web, for its part, also requires computers to cooperate in using protocols, principally HTTP, but this is a much, much lower bar to clear.
The web, with its organically grown patchwork of standards and near-standards, its tolerance for missing pieces and other imperfections, and its lack of overarching authority necessarily lacks the coherence and uniformity of something like Xanadu. But these traits are exactly what allows it to thrive.
There's a moral to be drawn there, for those who wish information to be universally accessible to all.
