Image source: Pipes.yahoo.com

Text source: BBC

January 19, 2007

Pipes allows a mash-up of web 2.0 sites

The way we use the web is changing and the future lies in mixing, mash-ups and pipes, says columnist Bill Thompson.

When the web was young we were happy just to see words and pictures on the screen in front of us.

All backgrounds were grey, all fonts were Times and anything other than a static image required a “helper application” to be loaded and run, so that video clips and sounds played in separate windows on screen.

Compared to the text-based internet of the 1980’s it was heaven, but it was only the beginning.

Since 1994 we have seen the web turn into an all-singing, all-dancing multimedia experience, with the simple page layouts we once delighted in replaced by interactive services and web-based tools, while embedded video is everywhere.

Anyone with an internet connection can have their own web site, whether a blog or a profile on MySpace, and photo and video sharing is becoming the standard way to share holiday snaps or family events.

And the quality of the experience has been enhanced by the move from a page-oriented model, where each site is collection of separate pages, to the services approach that underpins web 2.0.

Sites like Flickr and Google Maps don’t load or reload pages, they use the browser to provide interaction with online data sources.

Move from static

This is sometimes described as a move from the static, read-only web to a “read-write” model, but this doesn’t quite capture what is going on.

Bill Thompson
Yahoo! has given us a glimpse of the networked future,

Editable pages like wikis certainly work, but they are not the real breakthrough in how we are using the net, and while Wikipedia and other collaborative tools have a place, they are not the transformational technology that will drive the next generation of start-ups or challenge the dominance of the big players.

The real transformation comes from having the ability to take other people’s content and then filter, refine, recombine and reuse it in interesting and innovative ways.

This remix model brings us closer to the original vision of a hypertext, put forward by Vannevar Bush in the 1940s and realised by Tim Berners-Lee at Cern in the 1980’s.

Bush’s “Memex”, an electro-mechanical system for providing easy access to information stored on microfilm, relied on cross-references and user annotation, allowing people to add new documents but not directly to edit those they have.

Now Yahoo! has launched a new service that could have a massive impact on the way we think about our online activity.

While Google concentrates on challenging Microsoft Office with its online word processors and spreadsheets, Yahoo! has looked much more deeply into the way the net works and given us the building blocks for a brand new way of dealing with online content.

Data feed

Their new offering, Pipes, lets you take a data feed such as the result of a web search, or an RSS feed from a blog or news site, or a set of tagged photos on Flickr, and transform it to produce the outcome you want. You can then make it available for other people to see.

It’s web-based, no more complicated than creating programs for Lego MindStorms, and already stirring up a lot of interest.

Publisher Tim O’Reilly, a web zealot not noted for his reticence, calls it “a milestone in the history of the internet” and while he may be slightly over the top he is certainly right to draw attention to it as a major innovation.

Naming it Pipes is a very shrewd move, because it brings to mind the simple tools-based philosophy that underpins the Unix operating system on which Linux is based.

You can use the Unix shell to take the output of any command, like a list of files in a directory, and send – or “pipe” it directly into another command, allowing you to build complex operations from simple parts.

Yahoo!’s Pipes do the same with a simple graphical tool that lets you define and connect data feeds, filters and user prompts, so that you can quickly build the service you want. You still need some technical ability, but you don’t need to be a programmer.

Two sources

We have had mashups for a while now, like the projects coming out of the BBC’s own Backstage project, but they generally require some programming ability and usually combine two sources at a time, like the BBC News and Google Maps mashup in Ben O’Neill’s excellent news map.

Pipes take things much further.

This isn’t user-generated content, it’s user-controlled content. And unlike personalised pages or simple feed subscriptions it really does put control into the hands of the user.

Pipes mark the point at which remixing online content and creating mashups becomes something that anyone can do. If you can describe what you want then you can build it.

It is also, of course, a collaborative environment, at least for now. You can take someone else’s pipe and “clone” it to make your own, with no hint that there could be copyright or intellectual property issues here.

That may change, as it so often does, and at the moment the Yahoo! site says nothing specific about Pipes in its terms and conditions so we can’t be sure how it will evolve.

But Yahoo! has given us a glimpse of the networked future, where the world’s information is not only at our fingertips, but available to be mixed, mashed and filtered on demand, giving us what we want, when we want it – and from wherever we can get it. There will be no going back.

Bill Thompson is an independent journalist and regular commentator on the BBC World Service programme Digital Planet.