Some syndication feeds are not meant to be displayed for the world to see. Our everyday lives contain private and confidential data we wouldn’t want anyone else to see, and especially not search. There are a few options for trying to keep things private in your feed aggregator but the implementations require proper coding and privacy from all implementors.

Examples of private feeds intended for 1:1 communication include bank balances, e-mail notifications, project status, and the latest bids on that big contract. Data in the wrong hands could be dangerous, and many companies will stay away from the feed syndication space until they feel their users’ personal data is secure.

A private feed’s data could be exposed in a variety of ways. A desktop aggregator’s feed content might be available to other users on the same computer, either through directory access or desktop search. An online aggregator might expose a feed and its content in search results or a preview mode.

Security through obscurity

Sites such as Flickr hide private photos from navigational view, but do not restrict access to photo data if shared, or if someone were to guess the semi-scrambled URI. Private things are kept private because no map or directions are available for public navigation.

Permission-based exclusion

A feed might specify a desire not to be available to other users through means such as search but it’s up to each to obey a publisher’s preferences. I proposed feed exclusion using category last February and Bloglines recently introduced a feed access control namespace specific to their application.

Publishers cannot rely on an application to nicely obey access control specified in their feeds of sensitive data.

HTTP Authentication

Feeds can be accessed using HTTP/1.1 and access authentication. A few feeds online currently use this method to deliver personalized information to their users.

• 37signals Basecamp – Track your project status.
• Measure Map – a daily update of your blog stats.
• Daring Fireball – paying members receive a user-name and password to access a full-content version of the site’s feed.

HTTP Authentication works with most desktop aggregators but runs into trouble with most online aggregators which rely on a common feed store based on feed and/or link URIs. Bloglines and Google Reader fail to load authenticated feeds, do not request credentials, and do not provide a meaningful error message. NewsGator Online supports secured feeds.

Feeds with authentication might be exposed to a broader audience than the original provider of proper credentials. A search for “Basecamp” on My Yahoo! exposes the private project management data of over 25 customers of 37signals’ Basecamp.

Summary

A feed publisher could whitelist the user-agents it knows comply with its access policies. SSL encryption might not be a bad idea either as shared aggregation spaces might not store content requested over HTTPS. It would place extra load on the server as each request requires extra processing, but if the alternative is placing your customer’s data in the Yahoo! search index then that’s not such a bad thing.

I believe large publishers such as Salesforce.com or eBay would produce more feed content if they knew their customers’ data was kept private and secure. There’s a definite demand for more content transmitted over feed syndication formats but it will take the cooperation and collaboration of security formats and consistent aggregation practices to really move the needle in the right direction.

A feed entry is not a silo, but often connected to other entries in the feed world. In this post you will learn how to express relationships between data using RSS, Atom, and common namespaces.

Comment count

Reading a feed entry can sometimes evoke action in the form of commentary. Readers often become writers, leaving comments attached to a post or article. Popular technology news site Slashdot has had an active commenting community for years, and created a namespaced element to express the total number of comments associated with a specific item.

<slash:comments>42</slash:comments>

The above example uses the slash module to define the number of comments associated with a specific item or entry. This additional data helps a user determine the popularity (or existence) of conversations around the individual entry. Aggregators may choose to sort on comment count, or simply display the total number of comments before linking to the entry’s comments area (defined in RSS 2.0).

Comment feeds

Each entry might output a separate feed for comments. Aggregators can count the total number of individual entries contained in this feed to display a total comment count, or display the comments within the post view.

<wfw:commentRss>http://example.com/post1/comments/rss</wfw:commentRss>

The above example uses the Well-Formed Web Comment namespace to define a RSS feed containing comments for the given individual entry. An aggregator can pass this URI to its parser to better understand the conversation happening in the individual entry’s comments.

Citing the source

A source element may be specified to provide more information about the origination of described data. An entry might be a copy of another resource or inspired by another piece of work.

The Flickr explore feature contains one picture sourced from an individual member’s photo stream for example. A user might want to explore more photographs from that Flickr member or immediately subscribe to his or her photo feed. Another example might be crediting an original source, such as a blogger commenting on a story reported by The New York Times.

RSS 2.0

RSS source elements exist to give credit for links and provide a method for tracking story origination. You may specify the feed URI and title in this item element.

<source url=”http://example.com/rss.xml”>Example Title</source>

If a reader notices Bob tends to find really good stories from Sue he might add Sue to a list of subscriptions. A machine might notice the relationships between stories, and follow multiple connections until it locates a node of origin.

Atom 1.0

Atom’s source element (IETF 4287, 4.2.11) credits another feed when the current entry is a copy of an original source. This applies to the Flickr example given above, a search result within a feed search engine, or a spliced feed. Elements such as feed author, copyright, and links are maintained, allowing a user to subscribe directly to a source feed, visit the corresponding web page, or view more information about the author.

Atom Threading Extensions

Atom Threading Extensions combines comment counts, comment feeds, and giving credit for the source of a story.

<link rel="replies"
      type="application/atom+xml"
      href="http://example.com/post1/comments/atom"
      thr:count="42"
      thr:updated="2006-06-28T12:11:10Z" />

The above example defines a feed containing comments about the individual entry and a count of comments received including the time of last comment. Each attribute may be interesting enough to spark an action.

Summary

There are multiple ways to express relationships between resources and spawn related actions for individual feed entries. Implementations vary, but exposing these connections and actionable data leads to a richer and more intertwined web.

Implementations vary by each piece of feed parser but here are some “in the wild” numbers taken from a recent snapshot of Google Reader subscriptions:

1. Well-Formed Web: 8.39%
2. Slash: 3.66%
3. Atom Threading: 0.54%

Many people in the technology world view the world of RSS, RDF, and Atom as a way of outputting blog content and reading the latest information in a feed aggregator. While that simplified view is true, the full world of feed syndication is a bit more complex.

Syndication feeds are extremely popular expressions of structured XML. The popularity of content production and consumption using feeds has resulted in widespread deployment of parsers able to turn something simple such as a title, publication date, and a body of text into an easily displayed message, communicating recent updates and atomic changes in our online world.

Power users might subscribe to hundreds of blogs within an aggregator such as NewsFire or Bloglines but feeds are also used to transfer specific data related to specific applications such as e-mail or a supply chain status.

Google

Google Mail produces an Atom feed for every user. The feed contains information about the number of unread e-mail messages in the account as well as entries detailing each unread message. An authenticated user can view the message title, body, sender, and send date through data contained in the Atom feed. Google uses Atom 0.3, a deprecated version of the standard, and doesn’t properly namespace their added elements, but that doesn’t stop third party sites such as Netvibes or Gmail desktop notifiers from parsing the feed to display timely information for their users.

Most of Google’s APIs are expressed using Atom or RSS syndication formats. You can access your Google search history or access your Google Calendar data expressed as a feed.

Netflix

Netflix users can keep track of the latest movie releases, their movie queue, or recommendations using Netflix data expressed as RSS. Netflix members can access this data anywhere and at any time, or other services might build upon the Netflix data.

Earthquakes

The United States Geological Survey collects earthquake data from around the world and outputs the data in multiple formats for use by enthusiasts and researchers. The RSS feed includes the time of the earthquake, the epicenter expressed as latitude and longitude, the depth of rupture, and other data. Standard RSS parsers can understand basic data contained in the feed while more advanced readers can comprehend and display detailed information present in namespaced elements.

Summary

What commonly updated data do you produce that can be expressed to the world in the form of feed standards? Widely deployed feed parsers are ready to listen, delivering your latest data to interested people and/or devices around the world.