Problem

The problem is that there are few entry-level metadata consumers that can take metadata feeds, aggregate and (re) publish them. There are a lot of channels talking but not that many listening. The human agent is still doing the filtering. TBL vision of the semantic web is still a long way off. We need some smarter agents doing the work for us.

Solution

The solution is to do something practical about it and to do it in a way that will spread around the web. The solution is one cross platform metadata consumer and several platform specific metadata producers. The consumer can take registered feeds and publish aggregated and filtered channels. The producers are modules that sit on top of established CMS systems.

The use of public subjects (PSIs) such as those found in DMOZ can be used to merge channels from different sources. This is the different proposition with this system – sources can be merged. This relies on content producers marking up their content correctly.

Inspiration

I have been interested in CMS systems for quite a while. I was thinking rolling a generic version of my own from code cobbled together from various projects. The main feature of my CSS would be that it supported standard Dublin Core (DC) metadata attributes and that it could output RSS and Topic Maps. I quickly realised that this would be wasted effort because there were so many CMSs out there and that it would be better to concentrate on the interoperability aspects (metadata exchange). This lead to the realisation that a number of Metadata Producers could be written on top of existing CMS packages. The corollary was that the core of the system needed to be a cross platform metadata consumer.

Implementation

Most of the hard work has been done by the open source movement. There is an open source topic map engine, TM4J, which could be used to drive the topic map aspects of the application. The most difficult aspect is merging and an engine such as this one would save a lot of work.

The basic application would allow users to log on and customise their channels and how they would want them served. For example, UserX wants channel x,y,z and wants it delivered to their inbox every morning. Admins would also be able to go on and register new metadata sources and define merging rules. This is the real value add of the system.

The system could support a very basic metadata management system that allowed users to blog and to maintain CMS. Users could define their own subject areas and blog till their heart is content. They could then share their ideas. Users could be encouraged to reuse DMOZ PSIs.

Strengths and Weaknesses

There are a lot of nerds out there working hard on content. They belong to big companies with a lot of programming resources. However, we must remember that it is often the simple things that work on the web, e.g. HTML, RSS. Simple concepts tend to take off as well, e.g. Blogging. The right balance needs to be struck.

Business Model

The metadata consumer is the smart end of the system and we need to be able to get people to pay for the use of it. The users, those using the channels, will not pay. They can just use Google or some other source. However, content providers wanting to integrate their content might pay. E.g. those with a “topic brain” (associated subjects) publishing time sensitive resources. Content providers just get screwed from every angle, don’t they?

It might also be possible to do consulting work for the CMS providers to write the modules to output the metadata. Ie. Topic Map output routines which could dump XTM at a file location.

Outcome

I am yet to do any work on this project. Topic Map engine vendors have been stretched to come up with their basic engines. It is up to the next generation of developers to start building applications with these engines.

Problem

There are too many documents in the world: Too many resources. Search engines can only do so much – not enough. Humans think in concepts and subjects, not keywords and phrases. Navigating the resource layer is hit and miss and a time consuming process. Users need to be able to jump to the subject layer from the resource layer.

Solution
A central repository of subject keywords needs to be held to provide lookup services to clients. Clients need to be able to analyse documents and mark-up the appropriate keywords. Links could then be made from resources to subjects. The repository would have to be quality controlled to stop malicious users from corrupting it with junk entries.

Inspiration

This idea came to me at the end of a particularly bad holiday to New York in 1998. I was staying with one friend, had a falling out with her for no good reason and found myself out on the street at 11pm. I then hooked up with another friend who said he could get me into his hotel room. Well, there were no spare rooms in the hotel and he just disappeared. I found myself on the streets at 2am. I didn’t really know what to do so I wandered around, drank some coffee, hung out in a park full of male prostitutes parading in front of me, visited the emergency room of a hospital for the toilet and then shivered my way through to day break. It wasn’t the best of nights. At daybreak I walked up to Central Station and then walked downtown and caught the Statton Island ferry. I had the ideas on the return journey and feverishly wrote them up in Central Park. I recommend that ferry trip to anyone.

Implementation

You would need a big server somewhere registering the names and some client software to mark up the documents. Alternatively, the mark-up could be done on the server before being sent to the browser.

Strengths and Weaknesses

The main weakness was that the system was open to abuse. You only get out what you put into such a system. That is probably its strength as well. This means that only trusted parties should be able to enter data into the system.

Business Model

The failure of Real Names proves that even a well-funded venture in this field could fail. I never thought that there would be much money in extorting cash from businesses in name placement. I thought that money could be made by adding value to content, especially where there was a vertical market in data/information. For example, a publisher of primary materials could benefit greatly by providing links from that content to concepts which link to articles discussing the concepts at a secondary or tertiary level.

Outcome

Real Names had the “keywords” idea and had the money to implement it. Interestingly enough it was announced in 2002 that Microsoft would no longer partner with Real Names, leading to the likely demise of the organisation and system. Other companies have implemented rating systems for web pages and suggest related links. Yet more companies have made browser toolbars to mark up documents. So the field was pretty much covered. I don’t think that any of them made money.

I was to have another crack at a similar idea in Metadata Producer/Consumer where the focus was on channels rather than on the whole Internet.