Authenticated and private feeds

Some syndication feeds are not meant to be displayed for the world to see. Our everyday lives contain private and confidential data we wouldn’t want anyone else to see, and especially not search. There are a few options for trying to keep things private in your feed aggregator but the implementations require proper coding and privacy from all implementors.

Examples of private feeds intended for 1:1 communication include bank balances, e-mail notifications, project status, and the latest bids on that big contract. Data in the wrong hands could be dangerous, and many companies will stay away from the feed syndication space until they feel their users’ personal data is secure.

A private feed’s data could be exposed in a variety of ways. A desktop aggregator’s feed content might be available to other users on the same computer, either through directory access or desktop search. An online aggregator might expose a feed and its content in search results or a preview mode.

Security through obscurity

Sites such as Flickr hide private photos from navigational view, but do not restrict access to photo data if shared, or if someone were to guess the semi-scrambled URI. Private things are kept private because no map or directions are available for public navigation.

Permission-based exclusion

A feed might specify a desire not to be available to other users through means such as search but it’s up to each to obey a publisher’s preferences. I proposed feed exclusion using category last February and Bloglines recently introduced a feed access control namespace specific to their application.

Publishers cannot rely on an application to nicely obey access control specified in their feeds of sensitive data.

HTTP Authentication

Feeds can be accessed using HTTP/1.1 and access authentication. A few feeds online currently use this method to deliver personalized information to their users.

  • 37signals Basecamp – Track your project status.
  • Measure Map – a daily update of your blog stats.
  • Daring Fireball – paying members receive a user-name and password to access a full-content version of the site’s feed.

HTTP Authentication works with most desktop aggregators but runs into trouble with most online aggregators which rely on a common feed store based on feed and/or link URIs. Bloglines and Google Reader fail to load authenticated feeds, do not request credentials, and do not provide a meaningful error message. NewsGator Online supports secured feeds.

Feeds with authentication might be exposed to a broader audience than the original provider of proper credentials. A search for “Basecamp” on My Yahoo! exposes the private project management data of over 25 customers of 37signals’ Basecamp.

Summary

A feed publisher could whitelist the user-agents it knows comply with its access policies. SSL encryption might not be a bad idea either as shared aggregation spaces might not store content requested over HTTPS. It would place extra load on the server as each request requires extra processing, but if the alternative is placing your customer’s data in the Yahoo! search index then that’s not such a bad thing.

I believe large publishers such as Salesforce.com or eBay would produce more feed content if they knew their customers’ data was kept private and secure. There’s a definite demand for more content transmitted over feed syndication formats but it will take the cooperation and collaboration of security formats and consistent aggregation practices to really move the needle in the right direction.

16 comments

Commentary on "Authenticated and private feeds":

  1. PXLated on wrote:

    Expression Engine just added http authentication that’s tied into it’s membership module. But everything is sent plain text. So, as you pointed out, more is needed. It’s a first step though.

  2. Charlie Wood on wrote:

    Spanning Salesforce is my service that adds secure feeds to salesforce.com. It uses HTTP Authentication and SSL encryption.

    The problem I’ve run into is support on the client side. As you point out, most of the hosted readers (with the notable exception of NewsGator Online) don’t support secure feeds. Disappointingly, neither does the Windows RSS Platform. (It supports NTLM/Kerberos, but not Basic HTTP Auth. Microsoft says such support was planned, but was the victim of time constraints. Uh, ok.)

    -Charlie

  3. kellan on wrote:

    HTTP Authentication works with most desktop aggregators but runs into trouble with most online aggregators

    Its not quite as bad as all that. The mass online aggregators have some problems, but there are a number of personal online aggregators that don’t. (Including most built on Magpie)

    Bloglines and Google Reader fail to load authenticated feeds

    I don’t use Google Reader, but Bloglines has had support for HTTP Auth for quite a while. Of course HTTP Basic Auth is worse then usless if it isn’t over SSL, which Bloglines used to have trouble with. (but I haven’t checked recently)

  4. Kevin Burton on wrote:

    When I wrote the first iteration of Rojo blog search it assumed everything was public. This was really before any use of authenticated feeds. Even today authenticated feeds are rare.

    That said… I would NOT use authenticated feeds with an online RSS aggregator. The developers just aren’t comfortable with secure RSS and you might become the victim of a data leak.

    Using a product like FeedCrier might be interesting.

    I’m now using this for FeedBurner’s error message stream. This way if Tailrank has a problem integrating with Feedburner I will get a nice IM popup….

  5. Paul Querna on wrote:

    Niall,

    Bloglines doesn’t currently prompt the user for credentials, but you can encode them in the URL as such:
    https://user:password@example.com/feed/

    And it will work. Its a user-interface problem that the user isn’t prompted but the ‘feature’ is there.

  6. Nik Cubrilovic on wrote:

    Yikes! Usernames and passwords in URI’s are bad bad bad (storing my passwords in the clear with a third party, no thanks)

    Good post Niall, aggregator developers need to support HTTP auth (not just basic), SSL, client certs and a proper keychain for storing passwords and certs (most operating systems have API’s for this). Without at least that, RSS will never make it in the enterprise.

  7. Al on wrote:

    There is also good threadage of this over at Dare’s Carnage4Life
    Particularly the further security issues such as caching

    Dave winer has also noticed the thread and and scott at Computer Zen is covering it as well.

    Great post Niall
    We at Folknology need to answer these questions for a number of projects.

    regards
    Al

  8. Mark Carlson on wrote:

    Great post. SimpleFeed is working with several companies to deliver secure feeds. At this point we only deliver headlines to online readers. The risk is just too great.

    But we are confident more feed readers will support secure feeds – the benefits are huge.

  9. Phil Wolff on wrote:

    This comes down to roles and responsibilities. Do you really expect to DRM RSS payloads? A feed server’s job is to deliver the right content to the right entity. Beyond that it’s up to the consumer to keep their end of the bargain, or not. When salesforce.com feeds me an updated customer profile, it’s up to me to decide if it’s ok to email a copy to my boss with a question, to stash it on a thumb drive for my trip, or to post it on my public blog as part of a case study. Once I’ve been authorized to use a payload, and the server’s authenticated my agent (browser, feedreader, or webservice), then all that’s left is to deliver it securely. Delivered, the payload is now the recipient’s responsibility.

    I think.

    No, no, I’m sure.

    Well…

    I suppose you could DRM payloads, vs. feeds. A spreadsheet that requires biometric authentication and updated confirmation of authorization from a central service, checking each time it opens.

    But isn’t that getting a bit far from moving bits, the central thrust of syndication? Aren’t payloads a separate issue entirely?

  10. Thijs van der Vossen on wrote:

    Nik said ‘storing my passwords in the clear with a third party, no thanks’

    Err… You _have_ to store your password with a third party if you’re using a web-based aggregator anyway and you’re also storing the content of your ‘private’ feed over there…

  11. julian bond on wrote:

    Yup. Hit this problem. A private club had an rss feed with a user+password in the URL and it ended up in a public aggregator and was searchable. We’ve had to remove all private RSS feeds as a result.

    Key to the problem is that RSS is machine readable. This seems to make it more likely that you end up with data leaks. No matter what security method you use, you end up relying on 3rd party systems to honour that security and maintain the same access control. I don’t see how we can enforce that without encrypting the contents and handing out keys to the authenticated (human) readers.

  12. Jeffrey McManus on wrote:

    There’s a big difference between “securing” something and “DRM”. It doesn’t help the discussion when people confuse the two concepts.

    It seems to me that the security options we have today aren’t sufficient, but then I’m no security expert. For Approver, I’d love to have something that wasn’t HTTP auth and that worked broadly across feed readers. I dunno, maybe HTTP auth is sufficient? Hmm.

    Niall, if you see me tonight I’ll tell you the story of what it was like to get eBay to publish their first customer-facing RSS feed back in the day.

  13. Johannes Ernst on wrote:

    Why not use LID or OpenID? E.g. replay attacks are not possible as in case of any username / password.

    There was a long discussion at MashUp Camp on this subject.

  14. kellan on wrote:

    Nik Cubrilovic on September 18, 2006 at 3:30 PM wrote:

    Yikes! Usernames and passwords in URI’s are bad bad bad (storing my passwords in the clear with a third party, no thanks)

    You can argue that its not the most user friendly interface encoding the credentials in the URL, but the actual storing your password in the clear is a limitation of HTTP Auth (Basic, Digest, X-WSSE). So use disposable, auto-generated, unimportant credentials if you’re worried.