February 2007 Archives

  1. Feb26

    Six Apart widget podcast with Byrne Reese

    Last Friday I visited Six Apart's headquarters in San Francisco to talk about widgets with Byrne Reese. Byrne is the former product manager of TypePad, currently a product manager of Movable Type, and a developer of plugins and widgets used in both products. Byrne and I talked about the current state of widgets in Six Apart's four blogging products: TypePad, Movable Type, LiveJournal, and Vox.

    Our 25-minute conversation about Six Apart widgets is available as a 11 MB audio download. I will summarize a few highlights from our conversation below.

    Listen to Niall Kennedy interview Byrne Reese of Six Apart within this page using Flash.

    TypePad

    TypePad logo

    TypePad is a hosted blog service with basic and advanced templates available to authors depending on their paid subscription level. Basic templates take advantage of a drag-and-drop sidebar manager, letting blog authors insert and rearrange widgets from the TypePad widget directory without directly editing the underlying template HTML. TypePad accounts at the Pro level or above can edit their template HTML directly through TypePad advanced templates, placing any widget code anywhere on their pages.

    TypePad authors can browse available widgets in the TypePad widget directory from the TypePad admin interface or select a compatible widget from TypePad partner site Widgetbox. Widget publishers can add widget content directly from their site using the TypePad widget API and a valid partner service key.

    Web feeds and advertising widgets are the most popular widgets on TypePad. Authors like to make money, and they like to integrate content from across multiple sites of interest or other areas of online activity such as their latest photos on Flickr or a music playlist from Last.fm.

    Movable Type

    Movable Type logo

    Byrne developed the Sidebar Manager plugin for Movable Type, integrating the drag-and-drop simplicity of module management into the administrative interface of self-hosted Movable Type software. The Sidebar manager plugin was built into Movable Type's core code starting with version 3.3. Movable Type treats widgets as a special type of template module, grouping appropriate sidebar content for your blog homepage, category listings, or individual entry pages.

    Movable Type plugins allow third-parties to integrate content and functionality directly into the blogging application and output their content directly into each generated HTML page. Movable Type supports static generation of blog files using Perl, or dynamically generated pages using PHP (or both), letting plugin and widget developers utilize available features to create the best possible integration experience.

    Byrne has not seen many developers taking advantage of the plugin + widget management features of Movable Type, but perhaps it's just an education hurdle.

    LiveJournal

    LiveJournal logo

    LiveJournal has a user base that is very security focused, taking a conservative stance on the trade-offs required when including third-party code on their journals. In the past year LiveJournal has opened up a little bit, whitelisting a few widget publishers for inclusion in LiveJournal templates. LiveJournal compiles its user templates, removing any unknown JavaScript entered by an author before the file is saved to disk. A whitelisted widget passes through this filter and is available to LiveJournal's over 12 million authors.

    External widgets can only be added to LiveJournal after the code is reviewed and approved by Six Apart staff. The best way to get whitelisted for inclusion on LiveJournal is to initiate a partner discussion with the Six Apart business development team.

    Vox

    Vox is a closed system and a very controlled environment, and only Six Apart authored widgets are currently available on this new blogging system. Vox may open up in the future to allow third party widget content in its author admin interface.

    Summary

    These are just some of the topics covered in my 25-minute podcast with Byrne Reese of Six Apart. Listen to the full audio to hear our thoughts on popular widgets, widget business models, and what widget functionality we might expect from Six Apart in the future.

    I hope to make this type of developer interview a regular feature, providing direct information about developing widgets on popular widget platforms.

  2. Feb26

    Universality of the web widget

    Netvibes announced a "Universal Widget API" at last week's Future of Web Apps conference in London, promising a write-once run anywhere widget environment using an open-source widget runtime. The new widget system encourages publishers to author widgets using the Netvibes API and extend the reach of their content beyond the Netvibes user base through an adaptable wrapper. In this post I'll walk through some of the differences between widget deployment endpoints from the publisher's point of view, explaining just a few ways a widget must adjust its dialect and structure to adapt and optimize in different widget environments.

    1. Manifests
    2. Inline and collapsed widgets
    3. Storing local variables
    4. Platform look and feel
    5. Requesting remote data
    6. Internationalization and localization
    7. Libraries
    8. Summary

    The manifest

    Each widget platform requires some sort of manifest file, a definition of a widget's contents, file, and author information for interpretation by the widget platform and its widget directory. A manifest file will commonly collect pointers to files within the widget package (CSS, JavaScript, widget preview image), a description of the widget (title, summary, author), and any libraries or non-baseline features you would like to include with each widget render.

    The widget description is easily standardized, collecting an author's name, e-mail, and URL, the widget's homepage on the web, and a description of the widget's function. The widget title is a bit more difficult, as some publishers would like to dynamically update the title to display a total number of unread mail messages, the current weather, or an active search term.

    Inline and collapsed widgets

    Most web widget engines offer an "inline" option, placing your widget code directly into the page instead of inside of the default placement within a sub-page (iframe). Inline widgets can change the page's background color, interact with other widget content on the page, create a dynamic widget title, and more. Permissioning inline access varies by platform but can be a useful option for your widget.

    Collapsed Microsoft gadget weather

    Microsoft gadgets support a collapsed display mode for inline gadgets, displaying extra information about a gadget's content within the gadget title bar. In the example above I've collapsed the Windows Live weather gadget, displaying a weather icon and today's temperature high's and lows instead of a multi-day forecast. A collapsed mode allows more widget content in the same available space.

    Storing local variables

    Apple Dashboard widget weather location

    Your widget may require some configuration to function at its best. You might ask the user to input a ZIP Code, a user ID, or a favorite baseball team to deliver the appropriate content in your widget. You might also allow customization of colors within the widget frame or text, the total number of items to display (3 days of weather, the latest 5 San Francisco Giants news stories, etc.), or whatever else might be applicable to your widget's display and customization.

    Widget variables and configurations are stored on the widget platform's servers, letting users input their preferences once and have the same configuration across multiple computers. The default method of preference input is usually a text input field, but you can provide a drop-down list of possible options, radio buttons, checkboxes, and many other choices.

    Platform look and feel

    Each widget platform typically has its own design, styling a widget window, creating a specially styled button, or generally matching the look and feel of the surrounding environment. Windows Vista Sidebar will appreciate an Aero Glass look-and-feel, Mac users will look for Aqua-styled buttons, Windows Live gadgets might display a FancyButton, etc. Each widget can adopt the default settings of the web browser, explicitly bind with the target platform's UI, or package a custom CSSfile.

    Requesting remote data

    Web widgets typically request data through the hosting platform's proxy, supplying a remote URL to a specialized data type handler and specifying a preferred response such as XML or JSON. The platform caches each request by default for a period of time (usually about 30 minutes) to increase performance and lighten the load of all the Google Personalized Homepage users subscribed to BoingBoing's RSS feed or requesting earthquake updates for northern California.

    Here's an example of how developers can request an feed on three different personalized web homepages:

    Google
    _IG_FetchFeedAsJSON('http://example.com/rss.xml', CallbackFn, 5)
    Netvibes
    Ajax.Request(NV_XML_REQUEST_URL + 'url?' + escape(http://example.com/rss.xml), {method:'get', onSuccess:CallbackFn})
    Windows Live
    Web.Network.createRequest(Web.Network.Type.XML, 'http://example.com/rss.xml', {proxy:'rss', numItems:5}, CallbackFn)

    In the example above I specified a web feed URL, a callback function, the total number of feed items I would like returned, and I would like the data returned to my callback function as a JSON object. Each platform also has request proxies for generic data, XML, JSON request, etc. Your get, post, put, and delete proxied functionalities may vary by platform.

    Internationalization and localization

    Hello world. Hola mundo. G'day mate.

    The newest web products are launching to a global audience, finding popularity beyond English-speaking borders and becoming the hot thing in France, Poland, or Japan. Some widget platforms support internationalization and localization bundles, allowing a widget author to define unique phrases sent to English, French, Polish, or Japanese users. The widget platform interprets the language requested by each user's browser combined with the international domain being accessed and passes this information along to any widget willing to listen. A Google Canada user might view your widget and request the content in French for example. Google Personalized homepage currently supports 15 languages throughout its widget platform.

    Libraries

    Widget platforms can contain a set of basic features available to each widget but expand their capabilities through additional libraries. Google gadgets can request an analytics module to track widget views, add tabs, and more. My Yahoo! widgets might tie into already loaded YUI libraries. Windows Live Gadgets even have a preferred method of string concatenation (StringBuilder).

    A universal widget could load the appropriate library at each destination on demand, or provide a generalized runtime to load similar functionality through its own modules.

    Summary

    The immediate concern with any write-once run anywhere solution is an abstracted non-native feel for each deployment. Like the promises of Java and wxWidgets for desktop applications, cross platform solutions can sometimes feel a bit alien and heavy when compared to a native application. Developers can extend the reach of their applications quickly and easily, trading off a native look-and-feel, performance and memory utilization, and more for a broader list of compatible platforms.

    A universal widget might lose out on advertising opportunities available in each widget platform's gallery. Your widget may not be listed in the Apple Dashboard gallery, the Google Gadgets directory, Windows Live Gadgets Gallery, etc. Users of each platform typically are directed to these directories to find new pieces of content and it's a free form of advertising for your widget and your brand.

    I think universal widgets combined with a good statistics tool will help new widget developers better understand their deployed audience and tweak their offerings over time to deliver the best possible experience for each platform. Widget developers and publishing sites will want to better understand the parent environment of their widget (Google homepage, Live.com, Netvibes, tec.), web browser and version, language, country, and much more. Netvibes will likely see an increase in widgets available for their platform and abstract some of the work needed to stay on top of the changes at each widget platform.

    There are many more differences between platforms than I've covered in this post, but I hope it gives you a good idea of what it means to integrate well across multiple points of deployment.

  3. Feb22

    Yahoo! centralizes its JavaScript network with free hosting

    Yahoo! is opening up the JavaScript powering its websites a bit more tonight, encouraging developers to directly reference libraries on its servers from within their webpages. Yahoo! User Interface Hosting opens up versioned access to the popular YUI Library, creating faster load times for sites across the web using Yahoo's optimized, geo-distributed, and reliable data centers.

    Yahoo! UI hosting sample code

    Many websites utilize common libraries for JavaScript development, creating a drop-down menu, file retrieval, or chart rendering using a library such as Prototype, script.aculo.us, dojo, and many others. If five Ruby on Rails sites utilize the same script.aculo.us library for effects you'll have to download the same file(s) five times from each of the five different domains. Centralized resources such as YUI Hosting create a single download source requiring one file download regardless of the number of sites taking advantage of the YUI library.

    Yahoo! is a global company and spends a lot of money serving up web content as fast as possible in London, San Francisco, or Tokyo. The central YUI files are on that same network, creating a shorter path from a user's browser to required files needed to enhance a website. Pulling files from a separate domain also creates an opportunity for more parallel content downloads, circumventing the two requests per host limit in Firefox and Internet Explorer.

    Yahoo! will be logging each request and its page origination, so if you are worried about privacy and providing pageview numbers to outside sources the hosted version of YUI may not be for you (grab a download, host your own).

    A web widget feature

    Version 2.2 of YUI, released on Tuesday, includes support for a new global variable named YAHOO.env. Web widget developers can reference this variable to determine if YUI is already present on the page for additional functionality or before loading a conflicting library. It's a useful feature for blog sidebars, letting your widget peacefully co-exist with a del.icio.us, Flickr, or MyBlogLog widgets/badges without unnecessarily weighing down the page.

    Summary

    I think the Yahoo! Interface Library will continue to gain traction thanks to its heavy development, extensions, and documentation. It's already being used by large sites such as The Wall Street Journal and SmugMug and across the revamp of the Yahoo! network, which are some key votes of confidence important in new technology adoption.

  4. Feb16

    Google hosting scalability conference June 23

    Google will host a conference this summer focused on building scalable websites. The one-day conference takes place on Saturday, June 23, at the Google offices in Seattle, home to Google Webmaster Tools, Google Talk, and a few other teams.

    The conference organizers are currently accepting applications for 45-minute talks, and hopefully we'll be able to learn a little more about concepts behind MapReduce, Bigtable and some large-scale data management issues from the Google staff.

    Registration is not yet open, and it's unclear whether the conference will be held in the downtown sales office or the engineering office in Kirkland, so don't make travel plans quite yet.

  5. Feb12

    Nokia releases H.264 video podcatcher

    Nokia N95 video playback

    Nokia announced a new mobile feed reader focused on video today at the 3GSM conference in Barcelona. The Nokia Video Center comes pre-installed on new Nokia Series 60 devices such as the newly announced N95 and N93i but is also available as a separate download for compatible devices. You can load videos onto the phone through your home computer or receive updates over-the-air at HSDPA speeds or using open WiFi while you're on the go.

    Nokia Video Center

    The Video Center software supports H.264 videos, meaning videobloggers have even more reasons to create specially formatted and mini-sized (320x240) versions of their show for iPods and Nokia phones. YouTube will soon release its videos in H.264 according to the Nokia announcement. Media RSS is supported (and encouraged) and publishers can send users a special MMS to assist in the subscription process.

    The Nokia N95 comes with 160 MB of internal storage with up to 2 GB available via a microSD memory card. Mobile videos will make most sense for people with a large memory card always in their smartphones since internal storage will fill up fast.

    The Nseries phones are more popular in Europe than the U.S. -- they are quad-band but no carriers here offer them -- but the popularity of the world's most popular cell phone brand might drive some new video subscribers in regions where DMB is already popular and built-in to most new phones.

  6. Feb08

    Netvibes module developer collects web credentials, personal content

    A French security blogger gained access to private user data on personal homepage service Netvibes last weekend, exposing stored usernames and passwords for popular integrated web services as well as user content loaded in the page. The blogger's account has since been deleted from Blog*Spot (currently cached on Yahoo!), but he provided extended details to French blog Le blog de ¥€$ (English translation). Netvibes has since claimed to patch "a security vulnerability in webnotes" exploited by this developer. I alluded to some of these issues with stored user information, phishing, and general brand confusion in a post two months ago about the popularity of available widgets regardless of their makers.

    Netvibes sample modules Gmail eBay webnote

    An external developer created a Netvibes module and submitted it for inclusion in the Netvibes Ecosystem module directory. A Netvibes employee examined and approved the submitted module for inclusion in the directory. The remotely-hosted module was then altered by the developer to retrieve stored preferences from other configured modules and store information from other modules loaded in the page such as the contents of a webnote, the user's latest Gmail messages, upcoming appointments and contacts, etc. The developer stored this data in a remote database and later examined his collected findings.

    Each Netvibes module is rendered inline, meshing the markup generated by the module with the rest of the page's content. A module developer is encouraged to access only their own module's content using a special Netvibes variable, but any developer can request other content on the page through standard JavaScript or the Prototype JavaScript framework.

    A developer can choose to store and retrieve small pieces of data through the Netvibes servers such as a ZIP Code, color preference, or the username and password to remote web services. This personal data is stored in the Netvibes database and authenticated using a token stored in a Netvibes.com cookie.

    This external developer was able to access other rendered content on the page, including content stored in other modules such as a user's latest e-mails or text stored in a webnote. A Netvibes employee stored his login credentials for an internal development wiki inside of a text note on his homepage, and the third-party developer was able to read this information and access data stored inside Netvibes development servers. The developer did not access the Netvibes.com storage methods directly, but was able to gain access through the internal database.

    Other web widget homepages place modules inside of an iframe by default, creating a page within a page with restricted access to other content. It's possible to create inline content on services such as Microsoft's Live.com or Google Personalized Homepage but it raises an extra user warning when granting this higher level of access. A widget directory might also store an approved snapshot of the developer's module code on their own controlled domain for quick, dependable access and reliability. Typically you want to separate the widget homepage and the widget storage into separate domains to restrict access to cookies and other information bound by a domain name.

    Update 2/10: Netvibes will roll out a new widget system over the next month to deal with these types of security issues according to Netvibes lead API developer François Hodierne.

    Tips for installing new widgets and modules

    Gmail widget prompt

    The exposure of this Netvibes user data is a reminder of the tradeoffs between the demand for content and the trustworthiness of the mini-applications we add to our websites and desktops. We might be eager to unblock our PayPal account when we receive a supposed e-mail alert, so eager that you might not even recognize the unfamiliar URL requesting that data with ill will. Widget users (and toolmakers) need to apply similar caution when adding special 200 pixel squares to their homepages and blogs as well, as they are allowing a new publisher to access both the data on the page and the data he or she configures within the module. Web developers should really be using authentication proxies such as Google AuthSub or Yahoo! BBAuth instead of creating their own input boxes for user credentials on those networks.

    Netvibes module install warning

    The best bet for end-users lies in the widget directories for their platform of choice or on the websites of already trusted brand. A Gmail module produced by Netvibes or Google is likely to be more secure than a third party module and provide trusted storage of credentials and secure over-the-wire direct access to your remote data.

  7. Feb07

    Yahoo! Pipes remixes the syndicated web

    Yahoo! released Yahoo! Pipes tonight, a visual editing interface for web feed manipulation and reconstruction. The 5-person Pipes team, part of the Yahoo! TechDev incubation group, spent about 5 months developing the product to help people better remix the syndicated content they find online.

    Yahoo! Pipes lets any Yahoo! registered user enter a set of data inputs and filter their results. You might splice a feed of your latest bookmarks on del.icio.us with the latest posts from your blog and your latest photographs posted to Flickr. You might automatically translate your favorite news sources to your native language, or only receive the 1 out of 20 news stories from your local paper that reference your town or local schools. A traditional web feed lets you select your news from a set menu, while tools like Yahoo! Pipes let you build your own dish with only the ingredients you care about.

    Yahoo Pipes sample edit interface

    The editing interface connects pre-configured modules and their option, creating a new feed accessible as RSS, Atom, or JSON. Anyone can share their modules, or clone the work of others to tinker a few things and enable their own customizations.

    Yahoo! Pipes opens up some interesting possibility for feed aggregators, letting users filter out unwanted content affecting their experience. Pipes opens up a few feeds that were not practical for a human to read in the past, either due to a high volume or possibly a foreign language. My favorite operator is the location extractor which analyzes an item's text attempting to identify addresses, locations, or the URLs of popular mapping services.

    Publisher Concerns

    Yahoo! Pipes has implications for web publishers, changing the reliability of delivered content, the relationship with the end user, and the polling frequency of a mashup that may or may not be actively utilized.

    Yahoo! Pipes makes it easy to remove advertising from feeds or otherwise reformat your content. I already know a few publishers who hold back the publishing the full content of their posts for fear of easy resyndication and brand dilution, and if Pipes becomes popular publishers might hold back a bit further or ban Yahoo! Pipes outright. A Yahoo! Mail user searching for a new feed subscription will likely choose an identical feed labeled "No Ads!!!" associated with their favorite brands.

    One of Yahoo!'s sample pipes, Aggregated News Alerts, uses the Technorati search API and republishes a key issued to an individual user. A site such as Technorati can increase that user's allowed queries per day, but they lose control over the issued unique key and its use.

    The Pipes troubleshooting section lists three ways of blocking the tool from using your feeds: modify your Apache settings to block User-Agent "Yahoo Pipes", add a new element to your feed, or send Yahoo! an e-mail asking them to manually add your URLs to a blocked list and verify your authority to make such a request. The suggested meta element added to your XML creates invalid feed markup and might cause your feed to stop appearing in some strict renderers.

    Summary

    Overall I really like Yahoo! Pipes, it's intuitive interface, and its "View Source" approach to building your own web services. I think a lot of people will build interesting new things using the service, and it ties in nicely with services such as Yahoo! Alerts. It's a pretty solid product from the Advanced Products group, leveraging web feeds as a simple web service.

Niall Kennedy Niall Kennedy is a web technologist in San Francisco, California in the United States. I am very interested in the world of... MORE »

Search this weblog:

Subscribe:

Latest feature: Widget development

  1. Web widget formats
  2. Widget data update formats
  3. Web widget optimization
  4. Scale Google Gadget content

Archives: Popular Categories

  1. Widgets
  2. Programming
  3. Business
  4. Feed Aggregators
  5. Search

Sites: More from Niall