My Yahoo! feed API

RSS buttonPowered by My Yahoo!

Yahoo! has developed a backend infrastructure that can be easily deployed across various applications online or on the desktop with full synchronization and feed parsing handled on its servers. Developers could tap into the Yahoo! backend and develop new feed-aware applications quickly and easily on a robust platform already used by millions of users. Yahoo! just needs to publicize the code and make sessions a bit easier but I reverse engineered their code and I’ll give you a primer.

Aggregator developers spend a lot of time dealing with issues such as proper parsing, feed storage, and at later stages providing synchronization between online applications or other desktops for a seamless reading experience across multiple devices. Yahoo! has the infrastructure behind the scenes to power a services-based feed aggregator on any platform based on a (previously) undocumented My Yahoo! API.

You must first submit login credentials to Yahoo! and receive a few cookies. Yahoo! will also generate a web services session ID that will regenerate in a nice XML message when it expires. Requests are handled by a pool of API servers located at api[0-3].my.mud.yahoo.com.

Individual feed data

/rss/Content/V3.0/getFeedData

Yahoo! serves feed data to its applications through a format built on top of RSS 2.0. You can pass the API parameters such as the maximum number of items to retrieve, your desired date format, the level of processed content to return, and the requested ordering of your results. Yahoo! returns a fully processed and cleaned-up feed with extra metadata.

Yahoo! tracks when the feed and each individual post was created, modified and updated. They add information such as whether the feed contains the full content of my post and if I am a podcaster. Rich media such as images or audio content is contained in separate content elements. Yahoo! even creates a special server-side unique identifier for each post, saving aggregators a lot of headaches.

Here is the locally stored Yahoo! output of my blog’s RSS feed compared to my actual feed.

You might want to check out the live feed data for my blog after you have logged-in to Yahoo!.

All an independent developer would have to do is style the returned clean feed data and only deal with one data format plus some custom elements. A lot easier than typical development.

Synchronization

/ymws?m=SetMetaData

Yahoo! synchronizes its list of feeds between applications using SOAP messages. Each client application appears to be assigned an identifier and a version number as well as unique user information tied to a session.

Here is the My Yahoo! synchronization SOAP message with my personal account information removed. Requests are processed using mod_gsoap.

Entire feeds may be marked as read using a resetUnseen element inside a SOAP message. Yahoo! does send some communication back to the server when each post is selected in the Yahoo! Mail view but the individual IDs do not correlate with the internal post ID. Given Yahoo!’s current interfaces synchronizing read status at a per-item level might not make much sense.

Yahoo!’s mail servers can serve as an example endpoint.

Adding and deleting feeds

/rss/Subscriptions/V3.0/addUserSubs

It is possible to add individual subscriptions using an individual API call with a client property, feed URL, and a web service ID.

Here is an example query adding Scripting News to your Yahoo! aggregation space.

/rss/Subscriptions/V3.0/delUserSubs

You can similarly remove the same feed. Here is an example query removing Scripting News from your Yahoo! aggregation space.

Conclusion

I think it is possible to construct a new aggregator interface completely powered by Yahoo! on the backend and extended by individual developers for the best user experience and ease-of-use. Developers could also store data locally in a desktop aggregator for online access or to allow better search over time.

I think Yahoo! should develop a desktop aggregator powered by its APIs. In my limited testing I believe it is possible to build a Yahoo! desktop aggregator for Mac OS X using Cocoa, WebKit, and CoreData. I’ve been meaning to talk to the RSS folks at Yahoo! about desktop client possibilities and there’s nothing like a lengthy blog post detailing my current progress to get the conversation started.

  • Posted
  • Updated at
  • Comments [5]

5 comments

Commentary on "My Yahoo! feed API":

  1. James Holderness on wrote:

    The first thing I noticed when looking at Yahoo’s output of your blog was that it wasn’t even close to being valid RSS. That may be the reason the API wasn’t documented.

  2. Niall Kennedy on wrote:

    James,
    They probably did not namespace the output since they are the only ones consuming the data.

  3. James Holderness on wrote:

    Namespacing is only one of several problems. They converted your title into invalid HTML. They converted your pubDate entries into invalid W3CDTF. It’s not unreasonable to assume that they’ve broken other stuff that just isn’t immediately obvious in your feed.

    Now none of this is really a problem if they are the only ones consuming the data, but I got the impression from your post that you were suggesting other people could make use of it too. I’m just not convinced that another incompatible, unspecified RSS format is such a great idea.

  4. Bob Aman on wrote:

    Herm. FeedTools made a brave attempt at parsing. The only field it seems to get wrong is the description element, probably because of the CDATAs. I’ll probably eventually fix that. It handled the dates and titles without issue.

    Still, I really don’t see the point. The RSS landscape is already messy enough as it is.

  5. MultiZ on wrote:

    Great post. Do you know if Yahoo is accepting partnerships through their API?