Yahoo! has developed a backend infrastructure that can be easily deployed across various applications online or on the desktop with full synchronization and feed parsing handled on its servers. Developers could tap into the Yahoo! backend and develop new feed-aware applications quickly and easily on a robust platform already used by millions of users. Yahoo! just needs to publicize the code and make sessions a bit easier but I reverse engineered their code and I’ll give you a primer.
Aggregator developers spend a lot of time dealing with issues such as proper parsing, feed storage, and at later stages providing synchronization between online applications or other desktops for a seamless reading experience across multiple devices. Yahoo! has the infrastructure behind the scenes to power a services-based feed aggregator on any platform based on a (previously) undocumented My Yahoo! API.
You must first submit login credentials to Yahoo! and receive a few cookies. Yahoo! will also generate a web services session ID that will regenerate in a nice XML message when it expires. Requests are handled by a pool of API servers located at api[0-3].my.mud.yahoo.com.
Individual feed data
Yahoo! serves feed data to its applications through a format built on top of RSS 2.0. You can pass the API parameters such as the maximum number of items to retrieve, your desired date format, the level of processed content to return, and the requested ordering of your results. Yahoo! returns a fully processed and cleaned-up feed with extra metadata.
Yahoo! tracks when the feed and each individual post was created, modified and updated. They add information such as whether the feed contains the full content of my post and if I am a podcaster. Rich media such as images or audio content is contained in separate
content elements. Yahoo! even creates a special server-side unique identifier for each post, saving aggregators a lot of headaches.
Here is the locally stored Yahoo! output of my blog’s RSS feed compared to my actual feed.
You might want to check out the live feed data for my blog after you have logged-in to Yahoo!.
All an independent developer would have to do is style the returned clean feed data and only deal with one data format plus some custom elements. A lot easier than typical development.
Yahoo! synchronizes its list of feeds between applications using SOAP messages. Each client application appears to be assigned an identifier and a version number as well as unique user information tied to a session.
Here is the My Yahoo! synchronization SOAP message with my personal account information removed. Requests are processed using mod_gsoap.
Entire feeds may be marked as read using a
resetUnseen element inside a SOAP message. Yahoo! does send some communication back to the server when each post is selected in the Yahoo! Mail view but the individual IDs do not correlate with the internal post ID. Given Yahoo!’s current interfaces synchronizing read status at a per-item level might not make much sense.
Yahoo!’s mail servers can serve as an example endpoint.
Adding and deleting feeds
It is possible to add individual subscriptions using an individual API call with a client property, feed URL, and a web service ID.
Here is an example query adding Scripting News to your Yahoo! aggregation space.
You can similarly remove the same feed. Here is an example query removing Scripting News from your Yahoo! aggregation space.
I think it is possible to construct a new aggregator interface completely powered by Yahoo! on the backend and extended by individual developers for the best user experience and ease-of-use. Developers could also store data locally in a desktop aggregator for online access or to allow better search over time.
I think Yahoo! should develop a desktop aggregator powered by its APIs. In my limited testing I believe it is possible to build a Yahoo! desktop aggregator for Mac OS X using Cocoa, WebKit, and CoreData. I’ve been meaning to talk to the RSS folks at Yahoo! about desktop client possibilities and there’s nothing like a lengthy blog post detailing my current progress to get the conversation started.