Google Reader API

Google Reader logo

Google Reader is an online feed aggregator with heavy use of JavaScript and pretty quick loading of the latest feed data from around the web. Google’s AJAX front-end styles back-end data published in the Atom syndication format. The data technologies powering Google Reader can easily be used and extended by third-party feed aggregators for use in their own applications. I will walk you through the (previously) undocumented Google Reader API.

Update 10:40 p.m.:Jason Shellen, PM of Google Reader, called me to let me know that Google built its feed API first and the Google Reader application second as a demonstration of what could be done with the underlying data. Jason confirmed my documentation below is very accurate and Google plans to release a feed API “soon” and perhaps within the next month! Google Reader engineer Chris Wetherell has also confirmed the API in the comments below.

A reliable feed parser managed by a third party lowers the barrier to entry of new aggregator developers. Google and its team of engineers and server clusters can handle the hard work of understanding feeds in various formats and states of validation, allowing developers to focus on the interaction experience and other differentiating features. You can also retrieve and synchronize feed subscription lists with an established user base that could be in the millions, providing a better experience for users on multiple devices and platforms. Google Reader’s “lens” provides only one view of the available data.

Google Reader users are assigned a 20-digit user ID used throughout Google’s feed system. No cookies or session IDs are required to access this member-specific data. User-specific data is accessible using the google.com cookie named “SID.”

Feed retrieval

/reader/atom/feed/

Google converts all feed data to Atom regardless of its original publication format. All RSS feed post content appears in the summary element and unlike the My Yahoo! backend I found no additional metadata about the feed containing full posts but Google does publish content data where available.

You may request any feed from the Google Reader system using the following URL structure:

You may specify the total number of feed entries to retrieve using the n parameter. The default number of feed items returned is 20 (n=20).

Google strips off all the data it does not render in Reader. Stripped data includes namespaced data such as Apple iTunes podcast data and Yahoo! Media RSS, additional author data such as e-mail and home URL, and even copyright data.

Subscription list

/reader/atom/user/[user id]/pref/com.google/subscriptions

Google Reader’s feed subscription list contains a user’s current feed subscriptions as well as past deleted subscriptions. Each feed is contained in an entry complete with feed URL, published and updated dates, and user-specific tags, if present. Current subscriptions are categorized as a reading list state. You may request the full list of feeds by setting the complete to true.

Here is a copy of my Google Reader subscription list with my user ID zeroed out. I am not subscribed to my RSS feed (index.xml) and I have added tags to my Atom feed. Each listed feed contains an author element which appears to be empty regardless of declarations within the original feed. Perhaps Google plans to add some feed claiming services, but its own Google blog has no affiliated author information.

Reading list

/reader/atom/user[user id]/state/com.google/reading-list

My favorite feature of the Google Reader backend is direct access to a stream of unread entries across all subscribed feeds. Google will output the latest in a “river of news” style data view.

Here is a sample from my limited subscription set. You may specify the total number of entries you would like Google to return using the n parameter — the default is 20 (n=20).

Read items only

http://www.google.com/reader/atom/user/[user ID]/state/com.google/read

You can retrieve a listing of read items from Google Reader. You might want to analyze the last 100 items a user has read to pull out trends or enable complete search and this function may therefore be useful. You may adjust the number of items retrieved using the n parameter — the default is 20 (n=20).

Reading list by tag

/reader/atom/user/[user id]/label/[tag]

You may also view a list of recently published entries limited to feeds of a certain tag. If you have tagged multiple feeds as “marketing” you might want to request just the latest river of news for those marketing feeds. The returned feed contains both read and unread items. Read items are categorized as read (state/com.google/read) if you would like to hide them from view. The number of returned results may be adjusted using the n parameter.

Starred items only

/reader/atom/user[user id]/state/com.google/starred

Google Reader users can flag an item with a star. These flagged items are exposed as a list of entries with feed URL, tags, and published/updated times included. You may specify the total number of tagged entries to return using the n parameter — the default value is 20 (n=20).

Google treats starred items as a special type of tag and the output therefore matches the tag reading list.

Add or delete subscriptions

/reader/api/0/edit-subscription

You may add any feed to your Google Reader list using the Google Reader API via a HTTP post.

  • /reader/api/0/edit-subscription — base URL
  • ac=["subscribe" or "unsubscribe"] — requested action
  • s=feed%2F[feed URL] — your requested subscription
  • T=[command token] — expiring token issued by Google. Obtain your token at /reader/api/0/token.

Add tags

/reader/api/0/edit-tag

You may also add tags to any feed or individual item via a HTTP post.

  • /reader/api/0/edit-tag — base URL
  • s=feed%2F[feed URL] — the feed URL you would like to tag
  • i=[item id] — the item ID presented in the feed. Optional and used to tag individual items.
  • a=user%2F[user ID]%2Flabel%2F[tag] — requested action. add a tag to the feed, item, or both.
  • a=user%2F[user ID]%2Fstate%2Fcom.google%2Fstarred — flag or star a post.
  • T=[special scramble] — three pieces of information about the user to associate with the new tag. Security unknown and therefore unpublished.

Conclusion

It is possible to build a your own feed reader on top of Google’s data with targeted server calls. You can power an application both online and offline using Google as your backend and focus on building new experiences on top of the data. Advanced functionality is available with a numeric Google ID and some variable tweaks.

Google has built the first application on top of this data API, the Google Reader lens, and judging from their choice of URLs the lens may not be Google’s last application built on this data set. I like the openness of the data calls and think the Google Reader APIs are simple enough to bootstrap a few new applications within Google or created by third-party developers.

Update: The Google Feed API now provides official Google endpoints for most of the data explained in this 2005 post.

46 comments

Commentary on "Google Reader API":

  1. Greg Linden on wrote:

    This is pretty cool, Niall. Nice work tracking all this down.

    But, is it really true that people can build on this?

    This was undocumented by Google and seems to be intended for their own apps. Using this seems risky, I’d think, with high likelihood of getting shut down if you built anything that was available publicly or generated noticeable levels of traffic.

  2. mparaz on wrote:

    My educated guess:

    Google will open up the Google Accounts login infrastructure. Once logged in, you can make these calls legitimately.

    For non-logged in users, there could be an server-side HTTP equivalent of the Maps API (which is for Javascript rendering)

    Nice work, pitting G against Y! in the race for openness!

  3. Niall Kennedy on wrote:

    I think Yahoo! and Google are interested in opening up their feed backend in the same ways they expose maps data and other information. The first step is to expose the data, ask “why not?” and allow other developers to express their interest in using the data.

    I sent a note to Scott Gatz, My Yahoo! lead, last week but I have not heard back from anyone at Yahoo! about my previous post. I also sent off an e-mail to Jason Shellen, Google Reader PM, this evening inviting conversation.

    Google Maps API came after the JavaScript had been reverse-engineered and posted across the Internet with significant interest. I’d like to start a conversation about feed APIs and generate some interest in the data before a formal program is announced.

  4. Chris Wetherell on wrote:

    Hi, Niall. Nice work on the API analysis! I’m certain Jason would be happy to jump into the conversation – I can as well, if that’s all right.

    So, to clarify. To date the API has been undocumented by us to date to help prevent those oh-my-gawd-the-url-changed headaches that would’ve happened if we’d changed something. So developers should note that some of the URLs might still change but that we’re getting close to stability and we’re happy for anyone to build atop the framework who is interested (and doesn’t find managing auth issues daunting) as that’s been our goal and intent with the API for Reader.

    Other neat stuff I didn’t see listed here:

    There’s also a way to exclude tags/labels by appending another query to the URL as xt=[tag]. For example, you could exclude read items from some collections of items by appending: ?xt=user/[000...]/state/com.google/readSome tags include specific domain information because we’ve been planning on an approach that’s flexible for other people’s (read: non-Google) uses.

    And I can confirm that you’re right about the lens – it’s only one of several planned approaches. Based on a series of user studies during early development we’ve been planning from the start a system that can support several distinct UIs for feed reading that would be developed by us – I actually have two other views on my dev build of Reader – but am more excited that others would be able to build clients that work best for them. One of the most exciting ideas for me for any feed reading API is that we (and perhaps other big companies) could possibly work on the more non-obvious and experimental solutions for syndication transformation since third-party developers are going to be excellent at supplying oft-requested and well-understood modes of interaction.

    A short (and incomplete) list of stuff to do includes:
    Passing media enclosure data
    Atom 1.0 compliance (if you notice we’re getting close)

    Some of the other Reader team might post here, too, as everyone’s been directly engaged with the API – it’s been a chief part of the discussion, planning, and design for this li’l project. After we finalize the remaining bits we’ll definitely post to our blog and document the framework.

    – Chris Wetherell
    Google Reader Engineer

  5. ihavenoname on wrote:

    What happens when you retrieve all unread items? Do they all get marked read? If that’s the case then the API is pretty much useless from a synchronization perspective. If I try to sync to my handheld and then only read some of the items (typical usage scenario) and then try to continue reading on the web there will be a problem because items that I have not read in actuality have been marked as read. I don’t see any facility for marking individual items read/unread.

  6. Laurence Gonsalves on wrote:

    Items are not marked as read automatically, so it’s possible to fetch an item without reading it. You can mark them as read using the same facility that’s used for starring. Look for “flag or star a post” above. Just replace “starred” with “read” when passing the “a” parameter.

    By the way, the comment “No cookies or session IDs are required to access this member-specific data” is incorrect.

  7. Niall Kennedy on wrote:

    Thanks Laurence. I rerequested all URLs without an SID cookie and Google returned a status code 403 and “permission denied” in the entity.

    I updated my original text to reflect this information.

  8. jcburns on wrote:

    The value of my SID cookie from Google.com is a string that is 161 characters long and starts with
    DQAAAGgAAABRnt… …so where is this 20 digit value?

  9. Niall Kennedy on wrote:

    jcburns,
    Google assigns each user a 20 digit user ID that would not change over time, and a session ID (SID) you can delete, regenerate, and expire.

  10. Andrei Lopatenko on wrote:

    2 Chris Wetherell

    Hi, Chris
    is there any information which kind of Google Reader API is going to be provided by Google?
    Will you support a streaming model (event-based)?
    I am curious since i like to develop intelligent interfaces and I am RSS-addict. I am quite unhappy about flexibility and search capabilities of many RSS-reader and I want to develop a query interface for Google Reader

  11. Steven on wrote:

    I hope the xt=[tag] functionality makes it into lens soon. The ability to show only things NOT tagged nsfw is a vitally important feature.

  12. Mike on wrote:

    One more comment…. Has anyone figured out how to get a complete list of unread entries for all feeds you are subscribed to?

    This
    /reader/atom/user[user id]/state/com.google/reading-list

    seems to just give me a copy one entry from each feed I am subscribed to. I am looking to get a feed of all entries from all feeds (read or unread).

  13. Sergio Nunes on wrote:

    Hi,

    It seems that URL based authentication works. Try: http://username:password@url
    Replace [username] and [password] with your Google Reader data and [url] with one of the above urls.

    This is great for server side development.

    Regards!

  14. Kannan on wrote:

    I am having some problems with google reader, although i must say that the google reader is extremely fast.

    I want an entire feed to be labelled for example, rugby.com/atom.xml should be labelled as sports and etc. This would mean that I would read my “tech-blogs” labels first and then others.Although it is not very desirable like the above thing, it would be great if we can have rules like gmail to categorize stuff.

    Thanks (And as usual Google Rocks)

  15. mparaz on wrote:

    It looks like even retrieval requires a login now. I was hoping to use this backend for an AJAX login-free aggregator.

  16. mohan on wrote:

    Hi,
    can you please tell about the user id in the URL
    /reader/atom/user/[user id]/pref/com.google/subscriptions

    wher can i get it?

    thanks
    mohan

  17. Nitin Nanivadekar on wrote:

    Thanks a million
    i am building an app that user feed reader as an auxiliary function.
    This fits my bill perfectly.

    -nitin nanivadekar

  18. Darren Kulp on wrote:

    @mohan, jcburns:
    View source on your Google Reader page and search for _USER_ID.

  19. Jamsi on wrote:

    I’m looking at creating a feed reader that works with the Google reader API (mainly so I can download a list of items to read on the train ..) but has anyone made such an application?

    I can’t imagine it to be too hard.

  20. Muzaffer Coruh on wrote:

    Hi,

    The number of returned results may be adjusted using the n parameter.”

    this is really nice but I wonder which parameter is used to call next 20 items…

  21. Mark on wrote:

    This information is great. Thanks so much. I’m having a little trouble understanding how to star or add tags to entries as these links seem to be hidden somewhere deep inside the javascript, css, or otherwise. Can you provide any further information that will help me piece this together?

    I’m working on a pre-reader that will filter duplicate news items from different sources as well as other unwanted content, since this doesn’t seem to be in lens yet. This page has been a great help. Thanks!

  22. Yasha Davidov on wrote:

    Just few things more:
    1. You can access only unread items using ttp://www.google.com/reader/atom/user/[user ID]/state/com.google/unread

    2. You can access all items via http://www.google.com/reader/atom/
    (i suppose you must be loged in)

  23. Yasha Davidov on wrote:

    It seems that com.google/unread always gives you an empty list.
    That was my mistake.

    But you can omit user id and replace it with “-“. If the user is logged in this will work.

  24. Rick on wrote:

    There is a way to request the number of unread feeds?

  25. Venu on wrote:

    For shared items:

    /reader/atom/user/[user id]/state/com.google/broadcast

  26. Edwin Khodabakchian on wrote:

    Hi,
    Does anyone know if there is a way to apply multiple tags to a feed as part of a single HTTP request. I have tried the approach of calling multiple times the request which adds a single tag to a feed and I think that I am running into a concurrency bug because I do not get a consistent behavior from the backend.
    Thanks,
    Edwin

  27. ptman on wrote:

    They still haven’t released the API, have they?

  28. jack on wrote:

    when I try the /subscriptions feed it says “oops! that wasn’t supposed to happen.”
    does anybody have any solution to that ?

  29. ptitcali on wrote:

    Hi

    actually I’ve tried the same, it doesn’t work. I think this is a so-called “oh-my-gawd-the-url-changed” ^^

    by the way, I’ve the same error (oops, that wasn’t supposed to happen) with edit-tag and edit-subscription, even if I put the right parameters.

    Do we have any info about when google is going to officially release the API?

    Thank you Nail for your great work!!

  30. ptitcali on wrote:

    Hi
    @jack I guess they changed the address… I’ve found this
    /reader/api/0/subscription/list
    maybe it is the same!! ^^

    ++

  31. Jack Tanner on wrote:

    Did the API change? The n=[int] parameter doesn’t work for me on /reader/atom/feed/[url] any more.

  32. anjanesh on wrote:

    ptitcali is right.

    /reader/atom/user/-/pref/com.google/subscriptions
    doesnt work any more.

    /reader/api/0/subscription/list
    does.

    How can we find the list of parameters & their functions for the new api ?

  33. HB on wrote:

    does anyone know if we can use Authsub with Google Reader API — as opposed to using the the Auth Cookie which would required getting the user login/password.

  34. Manoj Bist on wrote:

    Hi Chris,
    Any plans to ‘unofficially’ support Google’s authsub for Google Reader any time soon?
    Supporting Authsub will remove huge burden of handling google user’s password for anyone trying to develop applications over google reader.
    MB

  35. Tim on wrote:

    How are you guys authenticating against /reader/api/0/subscription/list?

    I tried some basic http authentication but its not working; it returns a 400 status and this msg:

    Server Error:The server encountered a temporary error and could not complete your request.Please try again in 30 seconds.

    (this method works for getting say a list of google bookmarks or something from search history)

    import httplib2
    login = “timothy.broder@gmail.com”
    password = “*****”
    url = ‘http://www.google.com/reader/api/0/subscription/list’

    h = httplib2.Http()
    h.add_credentials(login, password)

    resp, content = h.request(url, “POST”, body=”nt”, headers={‘content-type':’text/plain’} )

    print content

  36. Joe Lazarus on wrote:

    Niall, do you know if there’s a way to use the Google Reader API to get the RSS contents of a particular item within a feed? I see that you provide an example above of how to view the feed for this blog. I’m wondering if there’s a way to view just this post, for example, without your blog header, sidebar, or all the other posts in your feed.

  37. Ken on wrote:

    This is phenomenal. When will the API be released or stabilized?

  38. Mariano Kamp on wrote:

    Hey Niall,

    great article. Thanks for putting in the time to investigate.

    Do you have any idea how to tell the API what you’ve seen already?
    I am wondering how it can be prevented that all the items are downloaded again, even though they have been seen already.

    Something along the lines of passing in a timestamp?!

  39. rolf on wrote:

    Any way to mark a news as read through the reader api?

  40. jackphelps on wrote:

    Does anyone know how to solve this…?I’m trying to roll my own aggregator by running feeds through the API to normalize the contents. Works great for most feeds. However, some feeds use url parameters to deliver their contents… example: “http://img.perezhilton.com/?feed=rss2″. The API sees this and thinks the feed address is just “http://img.perezhilton.com/”, and that the “?feed=rss2″ is a parameter for IT, rather than the feed’s server. Escaping the ? does not work… anyone have any clues?
    Thanks to Niall for the awesome work.

  41. Elad on wrote:

    Another little something: you can get a feed of any Google Reader user’s publicly shared items without authenticating, like so:
    http://www.google.com/reader/public/atom/user/[userID]/state/com.google/broadcast