Blog – Page 37 – Niall Kennedy

Google Reader platform

Google built a feed platform that is freely available for any user with a Google account. The first implementation of the Google feed platform is the Google Reader lens, already used by thousands of Google users. The Google feed platform is web-based and therefore works with any operating system. It can be used as a pass-through for one feed at a time for small implementations or a full aggregator backend. It’s running on the Google’s expansive (and expensive) back-end hardware and network, creating fast response times and high reliability.

Atom 1.0 output

The Google Reader platform outputs all data in the Atom 1.0 format. This one output makes data extraction as easy as a traversal of one XML markup set for fast presentation using browser-based parsing in JavaScript or more advanced processing using your favorite programming language.

Enclosures are passed-through directly into the final markup. Any enclosure presented by Google Reader may or may not exist and has not undergone any pre-processing by Google.

Read and unread status is one of many different pieces of metadata surrounding every exposed item.

Subscription lists

The Google Reader platform display’s a user’s current, past, and inactive subscriptions. You may view details such as when the user subscribed to the feed and when it was last recognized as modified.

Each user can apply his or her own tags to any feed. A feed about gadgets may be tagged “computers,” “electronics,” “gadgets,” and “toys” for example. Tags allow loose organization of feed data and all Google Reader data is retrievable at the tag-level.

Relevance

Google applies relevance to any group of items you would like to retrieve. Relevance is Google’s own black box magic that takes into account link activity, word count, and other metrics to help users focus their attention on a smaller subset of items.

Synchronization

The Google Reader platform allows the synchronization of all of your feed aggregator information across machines and applications. You can synchronize data such as tags, read status, and even flag status. Developers may authenticate users using stored cookie data, name and password directly in the URL, or via SASL.

More information

The Google Reader platform is still in final development and therefore not an officially announced Google developer option. You can track the latest news on the Google Reader blog and read my unofficial Google Reader API documentation for development details.

Windows RSS platform

The Windows RSS platform is included in Internet Explorer 7 and Windows Vista, both due out later this year. The platform exposes feed and list data to all Windows applications via a common API, manages enclosure downloads, and applies common security settings to feed data and downloads.

Common Feed List

Windows XP and Windows Vista users will soon be using Internet Explorer 7 as their default browser. They may notice a glowing orange button on their toolbar, click it, and start subscribing to feeds exposed on the page. A subscribed feed is added to the Common Feed List, a centralized list of subscriptions available to any application. A user may initiate a subscription inside Internet Explorer but the new subscription is immediately added to other feed aggregators integrated with the Common Feed List back-end such as RSS Bandit or FeedDemon.

Internet Explorer 7 supports the import and export of feed lists using OPML. Online aggregators may wish to write small applications for Windows to synchronize subscriptions and item status between machines. In the world of Internet Explorer 7 actions such as “Add to My Yahoo!” or “Add to Google” may become a one-time download and localized sync instead of buttons located on multiple site and blogs.

Download Engine

Microsoft’s feed engine downloads enclosure files using the system-level Background Intelligent Transfer Service (BITS). The service adjusts its bandwidth usage based on the activity of other applications and handles network dropouts and computer restarts without interrupting the file download.

All downloads are integrated with the system’s Attachment Execute Services, a list of allowed and blocked file types. Third-party applications such as antivirus programs and other security programs sometimes adjust these settings to prevent the download of executable files with potentially malicious code execution or files that could potentially carry malicious macros such as Microsoft Office documents.

By default all feeds have enclosure download turned off. The user can choose to automatically download enclosures and have the technology above kick in.

Feed Store

The Windows RSS platform downloads fresh feed content in the background on a regular schedule. Individual items are exposed at the object-level to any application that would like to leverage the data. It is possible to build your own feed aggregator on top of the locally stored folder, feed, item, and enclosure data exposed by the Windows RSS platform. You also have access to the raw XML if you would like to dig directly into the original data.

New and updated items are marked as unread within the feed store. Applications integrated with the common feed store can toggle this read or unread status for a shared item view across multiple applications.

More information

This brief summary will likely be out of date within days of publication. Check out Microsoft’s Team RSS blog or the Vista RSS developer center for the latest announcements and updates regarding the Windows RSS platform.

Tags:

Feeds as a platform

Feed aggregator developers currently struggle with multiple data formats, unclean sources, HTTP status handling, and many other complex problems that take time away from actually developing their application. New feed platforms are emerging that will allow developers to leverage the existing work of larger teams that already sweat the small stuff every day, allowing new aggregator developers to focus on creating new experiences on top of feed data to thrill users and deliver requested features quicker and easier than ever before. Microsoft, Google, and NewsGator are three examples of established companies exposing their feed architecture to the outside world to encourage new applications.

The feed aggregator ecosystem currently faces issues that reduce the adoption of feed reading technologies for new and expert users alike:

Many agents hitting the same feeds on regular intervals. As the popularity of feed aggregators grows, will sites be able to handle every subscriber hitting a feed every half hour?
Payloads are getting bigger. Audio, photos, and video are becoming increasingly popular data formats accompanying a feed. The number of automatic downloads initiated by aggregators could discourage rich content sharing due to high bandwidth bills for each publisher.
Repopulate subscription lists and item status across multiple applications and devices. We interact with feeds on out home computer, work computer, cell phone, and possibly an online aggregator. We don’t want to enter each feed for every location or waste time with items we have already read.
Too much data, not enough time. Every new feed we add to a subscription list increases the fear that we might not be able to handle the additional attention load.
Feed subscription and delivery to the appropriate application are complicated. Modern web browsers currently display feeds using an XSLT skin on top of the data, a good first step, but users want to be able to easily track an information source over time and know they have successfully completed their task.
Feeds are created in multiple XML formats (RDF, RSS, Atom, etc) and often contain invalid XML markup and do not conform to their respective feed specifications.

But wait, there’s more! Developers also have to worry about expected best practices for the benefit of users and feed producers.

Proper handling of namespaced elements such as Dublin Core, iTunes, Media RSS, and many others.
HTTP best practices such as conditional GETs based on Last-Modified and etag values, permanent and temporary redirect handling, and permanently deleted feeds.
Secure feed handling. Some feeds are specific to one user or a group of users and employ HTTP authentication and other techniques to restrict access.
Import and export of feed lists using OPML or other common formats.
Data backup and selective sharing with others.
Search the content of your feeds.
Search and/or browse a list of feeds to help populate your aggregator with new content.

Developers of new aggregator applications need to take all of the above into account and either build it themselves or hope there are a few existing code libraries to help them out along the way. Luckily the whole process is becoming easier as feed platforms enter the market to handle complex issues and expose the underlying feed data as local objects.

I will introduce feed platforms from Microsoft, Google, and NewsGator over the next few posts to illustrate how developers have a new toolset available for their feed applications that will help connect any new application with millions of existing users on multiple platforms.

State of the aggregator

In the beginning, there was text. Early feed applications such as Netscape’s In-Box Direct, PointCast, and Microsoft Active Desktop transformed existing data formats such as a newspaper and its articles, summaries, and full-text and applied traditional publishing thoughts to a new transport medium. Almost 10 years after then introduction of early web feed formats we are just starting to tap into the full potential of feeds as a descriptive data format with rich payloads and more easily discoverable content.

Desktop aggregation

The desktop has been the traditional home of feed experiences. Our desktop aggregators are full of text-heavy content (preferably full-text feeds) with a three-pane browsing experience. Heavy payloads — full item content, audio, video, documents, etc. — are handed off to default handlers and not rendered in the client. Communication from feed publishers is one-way with few opportunities to participate in a conversation around an item such as comments, sending a private note to the author or editor, or following related items and threads exposed in the feed or through third-parties. We also still struggle with offline access, relying on the operating system’s HTML rendering engine to connect to the web when an item is viewed to pull down associated content such as images.

Aggregators of the future will help bridge these technology gaps by employing and possibly extending new platform technologies both offline and online to create a richer user experience. E-mail, IM, and feed reading are the three applications I use the most every day and I’m therefore motivated to seek out a better experience wherever it may exist.

There is room for more than one aggregator on every desktop. I currently read general feeds in NewsFire, subscribe to podcasts in iTunes, receive updated photo streams in iPhoto, and watch video episodes in FireAnt. In the future I might have a specialized application or widget showing information from the latest feed data related to earthquakes, weather, or surf conditions complete with rich media such as live camera snapshots or video streams.

Online aggregation

Online aggregators provide a consistent cross-platform experience available from any location with access to the Internet. Portals or segmented pages are more likely to provide title-only or summary views of content to save space and direct a user to the original content location for the full article. Destination sites such as Bloglines and Google Reader provide a full-text view in a bare-bones render experience to keep things simple.

Media centers

Feeds and their rich content payloads are now an integrated part of the living room experience. Your media center takes advantage of always-on broadband and a typically large screen to display high-resolution images, quality audio, and video episodes of your favorite content. Given a media center’s built-in ability to browse time shifted content the latest downloads available from your feeds simply become another line in a navigation interface, placing the latest content from the Internet alongside content recorded from locally gathered media such as TV or radio.

A media center such as a TiVo learns your preferences and recommends new content based on existing subscriptions. It is not too far of a stretch to imagine a media center that recommends content based not only on what is currently available on your 500 satellite channels but all of the available content in a directory of rich media feeds as well. Imagine getting a preview of The Simpsons before a new episode or additional materials on the show the day after it airs.

Feed delivery on a media center enables distribution of high-definition content for niche markets directly to interested consumers. I could subscribe to a Real Madrid channel and receive all the video I want on the soccer club even though they are 5000 miles away and not covered on my cable plan.

Mobile browsers

Mobile phones are becoming ubiquitous consumption devices of on-demand content. Phones are used to deliver highly targeted content to a 2″ screen while you want for the bus, commute to work, or have a few spare minutes before the next meeting. Current mobile feed readers shipped with a phone such as Motorola’s Screen3 present timely content in a small ticker-like display on the bottom of the screen. Motorola has chosen specific feed sources for its users delivering up-to-date information about weather, earthquakes, sports teams, and popular news stories every few minutes.

Later this month Sony Ericsson will ship new phones with customizable feed readers built-in. Any user can now control their own mobile information flow receiving the latest news and alerts while they are on the go.

Mobile phones are a perfect environment for rich feed payloads such as audio or video. Modern cell networks have average advertised download speeds of 300-500kbps and actual even faster speeds in real-world testing. Download speeds available on an always-connected mobile phone will continue to improve over time even if the user is unaware of the power at their fingertips.

Conclusion

We have only just begun to explore the full possibilities of current feed technologies. Rich media enclosures, related content definitions, and well-defined author data open up new possibilities for user interaction and content discoverability. I believe most future uses of syndication technology will occur behind the scenes as a transport layer opening up a common XML parsing format to multiple applications and specialized uses. We’ve only just begun to change the world of publishing, customization, and personal empowerment.

Windows Live Search updates

Microsoft introduced a new front-end for MSN Search data last night on Live.com, the new default homepage for Internet Explorer users. The new features improve the search experience for novice users with a more familiar interaction and customization.

Scrollbars, no next button

Microsoft found about 60% of their users request more than one page of results for services such as image search. They redesigned the experience, replacing the “next” and “previous” buttons common on other search sites with a with a draggable scroll up or scroll down slider that requests new result sets in the background.

Choose your thumbnail size

Thumbnails for any image search can be resized on-the fly using a special slider in the results page. The slider and resized images allow you to adjust the number of results you can see in each page view. The new image search also uses a slider to access next and previous data sets.

Hovering over any image reveals more information such as the dimensions of the image, file size, file name, and location.

Feed preview

Every feed search result can now be previewed in a special pane before subscribing to the feed on Live.com.

Search macros

The average Windows Live Search user would probably never add search operators but with search macros all they need to do is add a macro and build on top of someone else’s complex search.

More advanced users can setup their own macro for preferred review sites, conservative bloggers, or whatever else is of interest. It’s part Rollyo, part OpenSearch, and opens up many advanced features for geeks to help more novice searchers.

The first set of macros were created by Windows Live Search team members but Brady Forrest of the Search team helped some ETech attendees create their own for inclusion on the site when it opens to the public in a few weeks.

Conference WiFi is a marketing play

I’m at the Emerging Technology conference in San Diego with some cool new applications and services being announced on stage and in the hallways. The problem is, no one is able to blog about what’s happening in real-time due to an almost non-existant network connection (even for presenters wanting to live demo).

A thought for conference organizers: wireless connectivity is part of your marketing budget. It allows the audience to research products and sponsors, connect to other attendees to arrange meetings and get-togethers, and enables more people to attend knowing they will be able to simply connect back to the office if anything requires immediate attention.

Great connectivity leads to a bigger worldwide mind share for everyone involved in the conference. Assign a specific sponsor to the WiFi connectivity costs (WiFi by Cisco, Intel Centrino, etc.) and let freedom ring.

Feed platform presentation: Microsoft, Google, and NewsGator

As previously mentioned, I will lead a session on Feeds as a Platform at the Emerging Technology conference tomorrow at 1:45 p.m. I will talk about the feed ecosystem, peer into the future a bit, and introduce attendees to three real-world examples of feed platforms.

Windows RSS platform. Jane Kim of Microsoft will present Microsoft’s new technologies exposing feeds at the operating system level. You will see feed handling in action on the latest builds of Windows Vista and Internet Explorer 7 and learn why they matter to both users and developers.
Google Reader API. I will introduce the Google Reader API and how it may be used across platforms to display feeds and store personalized information in a central location.
NewsGator. Brent Simmons will discuss feed synchronization and centralized parsing agents from a developer point-of-view. Brent has implemented the Bloglines Sync API, .Mac synchronization, and most recently the NewsGator API for the millions of NetNewsWire users. Brent will show off the latest build of NetNewsWire and discuss his first-hand experiences with feed platforms and synchronization.

The session may be recorded by the conference organizers and I will post slides and pointers in a separate post after the presentation.

AIM opens to developers

AOL Instant Messenger is now officially available to external developers for custom presence, add-ons, and full integration of AIM into existing applications. I signed up for a developer key and I’ve implemented my AIM presence to my contact page as a first step. The new AIM developer features open up new methods of integration for everyone from the independent publisher all the way up to big corporations.

AIM Presence

You can now integrate your AIM status into web pages and other applications to let people know when you are online. You can also communicate your idle status, away message, and if you are currently logged on from a mobile device.

AIM Plugins

It is now possible to add your own plugin or widget to the AIM application. Think of it as a sidebar widget but within the AIM application and its defined social network.

AIM Client

The AIM Custom client is the most exciting. Create your own AIM bot, integrate buddy lists into an application, and request an entire social network using the AIMcc libraries. Starting with just one screenname you could potentially request every buddy and associated buddy profile, including things like location and URL to create social search applications defined by your predefined list of contacts.

Feeds as a platform talk at ETech

I will be at O’Reilly’s sold-out Emerging Technology conference next week learning about the latest developments in the online technology world and talking to a lot of smart people. If you are at the conference you should come to my Feeds as a Platform presentation on Wednesday, March 8, at 1:45 p.m.

I will talk about new trends in the syndication space and the technologies that make it all possible. What will the syndication space look like at the end of 2006? What are the underutilized applications of the technology primed for the most growth? What new products and services are emerging that will connect and organize more information and focus your attention?

If I don’t blog for a few days I’m probably busy putting together a 45-minute presentation, leaving room for a few surprise guests and announcements!

Automatic favorites import using browser history

TailRank launched a bunch of new features today focused on improving the meme tracker’s ability to recommend new sources and hot posts. Users can now have their favorite weblogs automatically recognized without the need for relatively confusing terms such as OPML, RSS, or blogrolls. Favorite sites are imported using a user’s browser history. I think this technology will be common across many sites in the next 6 months, and here’s how you do it.

The user clicks on an “Auto Configure” button that initiates the magic. Clicking the button loads a list of links that are potentially sites visited by the user. In TailRank’s case the top X blogs are loaded into an iFrame hidden from view using CSS height and width values of zero.

You can then cycle through the list of links using JavaScript, identifying the links with an anchor pseudo-class of visited. Pass each visited link back to the server and you now have a list of URLs stored in a user’s browser history.

It’s quick and easy for the user with a low barrier to entry.