Recently in Business Category

Business news and tips.

  1. Mar15

    Measuring efficiency in the cloud

    Electricity meter

    In the world of cloud computing every action has a cost. Every HTTP request fires off a chain of actions, each uniquely measured on a variety of billable meters. Gone are the days of idle or unused resources on our local servers. Cloud computing charges by the sip (when sips are available) aligning business goals of resource efficiency and its cost. The cloud computing world shares many similarities with the plug-in and go world of electricity, including the need to run green for the sake of resources and cost savings. What can the world of green energy teach us about the future of cloud computing? How can we measure computing resources in the cloud for efficiency, replacement costs, and cost savings? I shared a few ideas on green clouds at last week's Ignite at ETech.

    Marginal resources at a marginal cost

    Cloud computing marginal cost components

    At the most basic level cloud computing is a marginal measure of resource consumption across processor time, memory use, disk use, and bandwidth consumption. Cloud consumption meters are much more precise, measuring every process and its dependent system APIs. Wasteful programs, processes, or external libraries carry a real and measurable cost. Upgrading to the latest version of an application or library, complete with bug fixes and inefficiencies can save real money each month. We might even choose between similar software packages based on their measured efficiency.

    Google App Engine measures each HTTP method, differentiates between database selects and updates, and even measures the system cost of each e-mail message. Below is a highly simplified break-down of App Engine's consumption meters detailing available resources in a day. App Engine users may purchase additional access to most APIs.

    MeterAllowed
    Request Handling
    CPU Time6:30
    HTTP requests1,333,328
    Outgoing bandwidth1 GB
    Incoming bandwidth1 GB
    HTTPS requests1,333,328
    Secure outgoing bandwidth1 GB
    Secure incoming bandwidth1 GB
     
    DB
    Database calls10,368,000
    Database CPU time62:07
    Database size1 GB
    Inserts or Updates12 GB
    Selects116 GB
     
    Memcache
    Calls8,640,000
    add, set10 GB
    fetch50 GB
     
    Email
    Within Google5,000
    Outside Google2,000
    Size of message body61.4 MB
    Number of attachments2,000
    Size of attachments102 MB
     
    External requests
    HTTP requests657,084
    POST, PUT4 GB
    GET4 GB
     
    Dynamic images
    Transformations2,592,000
    Source images1 GB
    Manipulations5 GB
     
    Deployments250

    App Engine is extremely precise in its application meters and I only included a small sampling! A reduction in resource consumption carries real cost savings and opens up additional headroom for other processes within your application.

    Writing efficient code for the cloud

    The specialized cloud stack and its meters exposes code inefficiencies that may have gone undetected in a standard hosting environment. Programmers who learn the inner workings of their virtual machine under an environment of constraints will ultimately write better code in any system. Java developers with experience on mobile have operated under device constraints that serves them well in an environment of assumed excess. A cloud programmer taught to tune his code for the interpreter and its inner workings will similarly benefit inside and outside the cloud.

    Coding inside a constrained environment such as App Engine has changed how I write web applications. I was able to look through Guido van Rossum's code, notice different styles and techniques, and inquire about his coding style. It turns out my code was wasteful in ways I had not considered, yet by observing the virtual machine's architect and tuner I learned how to provide the right processing hints and optimizations to speed up code and reduce its resource consumption. Training engineers for cloud computing isn't a specialized investment competing with the cost of machines; it's a long-term investment in style and proficiency that should pay off in a measurable way under a metered runtime environment.

    Measuring efficiency of packaged applications

    What if installable web applications carried their own cloud efficiency ratings similar to Energy Star ratings placed on home appliances? We will have Energy Star ratings for web servers and datacenters beginning in May to create a direct comparison of energy usage across server vendors. Similar measurements can be applied to installable software packages or library dependencies in the cloud.

    WordPress cloud rating mockup

    The mockup above shows a possible cloud rating for WordPress inside a PHP cloud instance. A potential WordPress customer could compare efficiency ratings and total cost of operation of WordPress's code base over the course of a month or a year. He might then compare WordPress against products of similar functionality but varying operational costs such as Drupal or Serendipity. Measuring inefficiencies could motivate software vendors to reduce waste in their products to speed up execution times and save customers real money. Directly measuring resource consumption in this way motivates change up and down the value chain.

    We are already somewhat familiar with energy ratings in our daily lives. We evaluate a new refrigerator or washing machine based on its initial price as well as its total cost to operate and repair. Applications bundled into a machine instance for easy deploys could be similarly measured in a direct to cloud and managed cloud provisioning structure.

    Rolling back the meter

    Solar panels, wind turbines, Google AdSense

    Google, Microsoft, and Amazon operate cloud computing farms and advertising platforms to pay back the cost of operation. I expect AdSense, AdCenter, and Amazon Associates programs will offer discount premiums for customers generating ad revenue from the same company powering their web presence. Google and Microsoft could also make calls to their own APIs free or cheap to cloud customers since such requests would not need to touch the public Internet.

    Consumption dashboards

    Tendril Vantage customer portal

    Cloud dashboards of the future might provide insights into our software in the same ways smart meters and home monitoring solutions hope to measure our electricity use. Electricity monitoring startups are trying to raise awareness around resource consumption, highlight wasteful outliers, and ultimately affect change. Cloud computing companies such as Google can apply lessons learned from funding smarter power meters to cloud computing dashboards of the future.

    Summary

    In the world of cloud computing every action has a direct and measurable cost. Companies can calculate the savings or a business decision such as increasing cache times from 5 minutes to 10 minutes or new code decisions such as dependent library updates. The meters of the cloud will make us much more aware of our server computing consumption and provide new motivations for change.

    Cloud computing changes the infrastructure we use to power our applications. It also changes how we program by introducing constraints, optimization rewards, and systems designed for parallelization and scale. Some of our fear over change and lock-in is a lack of familiarity in operating at scale across distributed nodes. Programming against cloud computing systems retrains software engineers for a world of symmetric multiprocessing and better prepares us for our a future of multiple computing processes in our racks or in the cloud.

  2. Mar14

    The anatomy of cloud computing

    Cloud computing is changing the way we provision hardware and software for on-demand capacity fulfillment. Lately I have been thinking about the ways on-demand servers, storage, and CDNs are changing the way we develop web applications and make business decisions. Gone are the days of idle CPUs, empty memory, or unused drive space. The cloud charges us for what we use as we use it (assuming capacity is available). In this post I will provide an overview of the cloud hosting landscape with a particular focus on cloud utilization by web companies. I will walk through a managed infrastructure stack and examine a few major business targets.

    1. The hardware
    2. The platforms
    3. The managed cloud stack
      1. High availability
      2. Security
      3. Stable, efficient OS
      4. Programming Language Business Logic
    4. The client layer
      1. Attached storage
      2. Database
      3. Cache
    5. Cloud consumers
      1. Web application developers
      2. Back office tasks
      3. Disaster recovery
    6. Summary

    The hardware

    In 1943 Thomas J. Watson of IBM famously proclaimed "there is a world market for maybe five computers." Today we look back and laugh at such a proclamation but the statement really did hold up for approximately 10 years. Into the 1950s IBM designed computers for a possible market of 20 companies, of which 5 were expected to purchase such a machine. In 1953 IBM was pleasantly surprised to find 18 of 20 companies purchased the IBM 701, provind the business of back office processing and a new division for the tabulating giant.

    Last week Rick Rashid of Microsoft was quoted as saying around 20 percent of the world's servers are sold to a handful of companies: Microsoft, Google, Yahoo!, and Amazon. Three of those four companies are cloud resellers, renting small slices of their compute farms to businesses all over the world. 198 megawatt datacenters may be the new mainframe, with consumption units charged in minutes and bytes much like the time sharing relationships of the 1970s.

    IBM again caught my interest last year with its Kittyhawk project from Jonathan Appavoo, Volkmar Uhlig, and Amos Waterland in New York. IBM is currently researching ways to repurpose the massively parallel Blue Gene supercomputers for the datacenters of the Web. It's possible your future web application will run on a computer originally designed for gene sequencing and nuclear weapons testing.

    Hardware and data operations are again consolidating towards major players. These specialist providers are building at a scale and specialization most web businesses can't match. On-demand infrastructure of the cloud makes it cheaper and more efficient to outsource needed operational function to teams of experts already keeping some of the largest web companies in the world running every day.

    The platforms

    Microsoft and Google are the newest entrants into the cloud computing arena, focusing their efforts their respective programming languages of expertise. Microsoft's Windows Azure services platform will likely be the best platform for C# and ASP.Net development as it is tuned by the creators of .Net, IIS, and SQL Server. Google has similarly applied its expertise in the Python language and distributed web nodes to its Google App Engine product. The App Engine cloud is tuned by top contributors to the Python language including its BDFL, Guido van Rossum. App Engine utilizes custom Google software, Google Front End and Megastore, for web serving and storage. Cloud developers on either platform are using a similar set of hardware and software as the proven web-scale platforms of Live.com and Google. I expect Google App Engine will add support for Java in the near-future, their second major language offering and the most popular language among Google's own services.

    Language specialists are building managed stacks on top of generic cloud platforms such as Amazon Web Services' EC2. Engine Yard sells a custom, managed AMI optimized for the Ruby language and its Rails framework. Rackspace's Mosso subsidiary and others optimize for the latest versions of PHP + MySQL, attracting performance-minded applications in search of a tuned cloud instance. I am not aware of any major language contributors of Ruby or PHP employed at either company but the platforms do attempt to find their own niche among a broad offering of scalable hosting providers.

    Amazon's EC2 is the most well-known cloud computing provider and, as previously mentioned, the baseline service for other companies building value-added solutions. The AMI, a machine image formatted deployment in the Amazon cloud, is the basic building block of EC2 virtualization and the primary interaction point of Amazon's customers. Amazon resells premium operating system and application packages on behalf of companies such as Microsoft, IBM, and Oracle but it's possible such specializations will instead be absorbed by the software publishers themselves as they roll out their own hosted clouds (such as Azure or IBM Blue Cloud).

    The cloud computing software stack is trending towards an integrated, managed experience maintained by some of the top contributors to each programming language and related components. More generic cloud platforms will need to stay up-to-date with managed technologies on their platform and/or establish a strong reseller relationship to more specialized cloud managers.

    The managed cloud stack

    Cloud stack illustration

    Managed cloud providers handle an entire stack of infrastructure needed to deliver web applications at scale. A solid cloud computing environment abstracts the basics of a computing environment away from the implementors and lets them focus on adding value with each new application. Managed cloud hosting providers need to offer the following basic layers to stay relevant in a web developer's world.

    High availability

    Any web application needs to be available to legitimate visitors from all over the world. A true cloud creates spans the entire globe, defeating the speed of light on behalf of its customers with a server point of presence in multiple simultaneous locations. The cloud provider needs to effectively receive and route incoming requests to the appropriate virtualized application instance on behalf of its customers.

    Google and Microsoft replicate each application instance to multiple physical locations. AT&T Synaptic Hosting spans multiple locations for its enterprise customers.

    Security

    Web applications should be protected from intrusion and abuse at the network layer. In a cloud computing world application security is a lot like click fraud in advertising: every bad action carries a marginal cost. Cloud providers need to guard customers against potential external abuse and intrusion.

    Google, Microsoft, and Amazon have their eyes on many incoming requests each day. Google serves App Engine requests off the same hardware handling Google Front End, keeping bad requests away from search, ads, and your apps.

    Stable, efficient OS

    Web applications rely on a stable, efficient operating system to interface with hardware, manage filesystems, and allocate resources. The cloud server operating system is a stripped down version of standard installations without a need for direct hard drive interfaces or other peripherals.

    Amazon EC2 AMI Quick Start

    Amazon EC2 highlights the operating system behind every machine image. Older versions of Fedora and Windows Server are the default "quick start" options available to each new account. Google and Microsoft clouds run on custom operating systems tailored for web use. Windows Azure is a stripped-down version of the latest Windows Server. Google runs a Linux-based OS tuned by its infrastructure team.

    Programming Language Business Logic

    Every managed cloud platform includes a dynamic language virtual machine and an appropriate web services gateway. Language functions too closely associated with the parent operating system and its libraries are stripped away, leaving only a pure operating environment for a machine interpreter. External dependencies such as GNU tools and custom compilers will not function within the cloud language abstraction layer. Cloud services bundle a dynamic language runtime into an easily spawned instance for standard and efficient interpretation across many application instances.

    Google App Engine supports most functions of the Python language with additional support for the Django framework, WebOb, and PyYAML. Developers may replace these built-in libraries with newer or customized versions at an additional performance and usage cost. App Engine passes web requests into the programming language environment through the Web Server Gateway Interface.

    The cloud client layer

    Attached storage

    Cloud applications don't operate in a vacuum. Dynamic applications persist their application state and logic through database and file storage. In the cloud world the database and the file server are cloud services unto themselves, operating in an isolated and specialized layer. This isolation makes the storage layer swappable from the rest of the cloud stack and presents new opportunities for competition.

    Static files fall into two major categories based on their planned consumption. Files under 1 MB in size can be consumed by most clients in a single request, matching the expected simple request/response model of the platform. Files over 1 MB in size need to be broken into more manageable parts, or ranges, for a sequenced download. Static cloud storage can be broken up into differing solutions by file size or file type, providing the best possible solution for the storage and delivery task at hand.

    Google App Engine offers static file storage separate from its dynamic runtime. App Engine supports up to 1,000 files and has a 10 MB HTTP response limit.

    Amazon Web Services offers static file serving through its Simple Storage Service (S3) origin server and CloudFront CDN services. Amazon allows private and public file storage and can even charge individual users of third-party services for their use through DevPay.

    Attached storage is by far the most diverse service offering for companies evaluating a specialized solution. I prefer storage providers with widely supported file management APIs, smart settings for MIME types and caching HTTP headers, and a primary functionality of serving files our to the worldwide web. I expect popular storage providers will bundle more CDN services in the future through an exclusive up-sell partnership. I also expect a new class of storage middleware optimized for minimizing files, cleaning up images, or transcoding video will set up new programmable front-ends backed by popular storage providers.

    Database storage

    Databases are the preferred way of persisting structured data powering web applications. Cloud service providers have tuned and rewritten database functionality for the cloud, opening up new opportunities for scalable data services across multiple dynamic application instances. Cloud databases are distributed, replicated, and largely transactional. Cloud databases can be separated from the rest of the cloud stack through RESTful APIs between different vendors but there is a definite latency advantage to coupling of data and its interpreter.

    Microsoft offers SQL Server as a web service as part of the Azure services stack. Google App Engine offers Megastore, an abstraction layer on top of BigTable, as a service API within an App Engine instance or as a separate remote API. Amazon's SimpleDB brings together EC2 processing with S3 data storage. Greenplum offers PostgreSQL as a stand-alone cloud offering.

    Cloud databases are typically more limited in functionality than their local counterparts. App Engine returns up to 1000 results. SimpleDB times out within 5 seconds. Joining records from two tables in a single query breaks databases optimized for scale. App Engine offers specialized storage and query types such as geographical coordinates.

    The database layer of a cloud instance can be abstracted as a separate best-of-breed layer within a cloud stack but developers are most likely to use the local solution for both its speed and simplicity.

    Cache

    Our web applications receive multiple requests for the exact same resource. We should be able to place a pre-assembled version of our web pages, images, and XHR data into a local memory cache for fast serving on multiple requests. On our own servers we frequently use memcached, Varnish, Squid, etc. The cloud stack should include a storage cache as its first layer of request processing.

    Google App Engine includes a memcache API written by Brad Fitzpatrick, creator of memcached. Windows Azure will supposedly support Velocity caching in the near future.

    Cloud consumers

    Clouds reflected on building

    The target market of a cloud computing platform will affect its stack completeness, feature sets, and future support. Cloud terminology seems to be thrown around as a magical buzzword but there are major usage cases emerging.

    Web application developers

    New web applications start small and may sometimes experience exponential growth on a worldwide basis. Web developers evaluating the cloud stack are likely starting from scratch without the concerns of switching from a legacy system or alternate implementation.

    Cloud computing abstracts tiered architecture, operations planning, and other nuances from companies specializing in bring new ideas to market quickly. Web developers prefer a cloud stack tuned for fast web performance. Geographically distributed dynamic instances are important at least as an upgrade option to protect a new business from a rewrite at varying levels of scale.

    I believe cloud providers offering a complete managed stack will attract web development specialists to their platform. Google App Engine, Mosso, and Windows Azure compete in this space.

    Back office tasks

    Enterprise applications are moving out of the local server closet and into the cloud. Medium- to large-sized companies are replacing in-house maintenance of machines and applications with software and infrastructure as a service. Project management, employee tracking, payroll, and many other common functions have made their way into the software-as-a-service realm. More customized applications will migrate to cloud hosting and take their place alongside the anchor tenants of the groupware and collaboration suites.

    Windows Azure, Salesforce's Force.com, and Google App Engine show strong promise as integrated back office add-ons. Microsoft and Google already have a solid footing in enterprise groupware services through Exchange Online and Google Apps respectively. Force.com can be closely tied to the popular Salesforce CRM application for sales and marketing teams.

    More generic back office functions can operate on any cloud hosting provider with a properly maintained disk image. A new class of hosting provider operates as an abstraction layer between multiple clouds by maintaining the appropriate images and deployment scripts for any given task. Companies such as Aptana, CohesiveFT, RightScale, and many others span multiple cloud hosting providers with a single management interface. Cloud management companies can monitor multiple providers and create spot pricing market for computing resources.

    Back office solutions represent the largest possible growth area for cloud hosting providers. Platforms with strong existing anchor tenants can add on new services combining software-as-a-service and infrastructure-as-a-service. Generic cloud hosting providers will likely be tapped for general tasks directly or though a cloud management layer.

    Microsoft is promoting its cloud hosting solutions through its partner channels. Microsoft partners receive a 12% commission on the first year of revenues and 6% commission on all future revenues. Google offers a 20% discount to Google Apps Authorized Resellers over the life of the account.

    Excess capacity

    Hosting solutions need to scale up to meet peak demand. Peak demand could occur for an hour each day, one day a year (Black Monday in the retail sector), or one month out of twelve (college basketball playoffs). Cloud computing lets businesses pay only for what they use when they use it. Servers are not sitting around in your datacenter depreciating in value and consuming resources while you wait for peak load to occur.

    Excess capacity needs may be predictable and cyclical, allowing a business to integrate cloud computing into their computing workflow with ease. Generic cloud computing platforms offer the best migration costs as businesses clone their own local machine images for execution in a cloud computing environment.

    Disaster recovery

    Business operations need to stay online when catastrophe strikes. An earthquake in California, a hurricane in Florida or Texas, or a power outage anywhere in the world could knock your business offline instantly. A hot backup in the cloud spins up when your primary site is down. An on-demand backup facility is a lot cheaper than physical investments as companies invest in contingency planning.

    Amazon Web Services recently introduced reserved machine instances for companies who must be absolutely sure they will be able to operate in an environment of strained cloud capacity. Reserved instances receive priority allocation of cloud resources in exchange for an upgrade fee and lower monthly usage charges. Reserved instances are the VIP treatments of the cloud hosting world.

    Demand response programs are common in utility sectors such as electricity. Businesses can opt to be the last ones kicked off the grid in a low-capacity environment in exchange for higher consumption costs.

    Summary

    Cloud computing is picking up steam and there are a few early winners. The most promising solutions from large vendors are still in a technology preview stage but should be open for general use by the end of the year. Startups developing new applications should pick the best solutions provider based on the strength of their stack offering and usage pricing. Some cloud layers can easily be abstracted to best-of-breed solutions.

    I hope you enjoyed this summary of the world of cloud hosting! There is a lot going on and this post just scratches the surface of how our computing world is changing.

  3. Jan27

    How will Twitter make money?

    Twitter bird on a nest of cash

    Micro-blogging service Twitter will celebrate its third birthday in March and may have a revenue model to support the company over the long-term. Last month Twitter CEO Evan Williams told Kevin Maney of Porfolio.com the company will kick off new revenue streams by March 2009 to avoid raising another round of venture capital funding. Twitter's deeply engaged community would love to see a sustainable business develop around the site, its services, and the community. In this post I will take a deeper look at Twitter and its revenue potential as publicly hinted by its founders.

    1. What is Twitter?
    2. SMS revenue
    3. Brand monitoring
    4. Summary

    What is Twitter?

    Blog platform

    Twitter is a hosted blogging platform that limits blog posts to 140 characters or less. Short Twitter messages were designed as an archived status message communicated to friends via instant message, text message, or even radio bursts. Twitter's 140-character message fits within a SMS message's 140 octets for UTF-8, a design contraint that has spurred new content creation with little effort.

    Feed reader

    Twitter builds feed reading directly into the blogging service. Members subscribe to other Twitter accounts ("follow" in the Twitter vernacular) to receive updates in a centralized timeline. Twitter members create new posts referencing other members and their posts or comment privately via direct message, creating a publishing system that feeds off each uniquely assembled list of content.

    Twitter also operates a near real-time search engine against its public content. The search engine receives direct updates from the Twitter blogging service over a streaming API interface nicknamed "the firehose." Twitter currently allows trusted external partners to consume this full streaming update of its data. Other consuming clients are limited to a public timeline snapshot of 20 Twitter updates every 60 seconds.

    Twitter Services Inc.

    Technically there are two Twitters on record with the state of California: Twitter Inc. and Twitter Services Inc. The difference may be a remnant from Twitter's beginning in incubator Obvious Corp. or it could be a sign of the company's plans to operate its services business separate from the blogging platform.

    SMS revenue

    Nokia N79 SMS message Twitter revenue share

    Twitter currently pays vanity short code fees and SMS fees for mobile-terminated and mobile-originated messages in the United States and Canada. Twitter accepts mobile-originated messages to a long number in England. An intermediary such as Sybase 365 sits in between Twitter servers and the cellular networks of AT&T, T-Mobile, Sprint, Verizon, Bell Canada, Rogers, and more. A subset of Twitter users take advantage of SMS update notifications but this feature is currently a drain on Twitter cash reserves.

    SMS fees typically reverse at volume, with carriers paying application providers for SMS revenue generation. A carrier such as AT&T might charge 5 cents per SMS and pass a fraction of a cent per message to large applications such as Yahoo! or Google. Twitter could be approaching a SMS volume that provides bargaining power with carriers for mobile-terminated and mobile-originated message revenue domestically and mobile-originated message revenue internationally. New hire Kevin Thau, Twitter's Director of Mobile Business, will likely be creating closer partnerships with mobile carriers to lower Twitter's SMS spend and possibly extract revenue from this feature.

    Brand monitoring

    Twitter has publicly mentioned future revenue extraction opportunities from businesses on Twitter. Dell directly measures revenue generated by its multiple Twitter channels. Comcast offers "digital care" for its customers through a special Twitter account. Whole Foods discusses groceries with over 22,000 subscribers. Twitter has yet to extract value from the thousands of brand connections it currently enables. Brand monitoring will likely be the first corporate product from Twitter.

    Twitter dashboard mockup

    Brands monitor conversations on Twitter through specially-crafted search queries. Jet Blue might track all conversations around its brand, for example, likely leading to customers or potential customers experiencing issues in need of real-time solutions or clarity. Twitter does not currently provide much information about the people behind such brand mentions directly in search results, leading to a extra required steps for brand managers. What might brands want in real-time brand management dashboard for Twitter?

    • Better context around the person behind each brand mention. Who are they, where do they live, and how many people subscribe to their updates on Twitter?
    • Easy integration with existing CRM and customer support systems. Companies should be able to track existing customers on Twitter by importing a list of e-mail addresses.
    • Brokered communications channel (direct messages) to Twitter members who are not already subscribed to the brand's Twitter account. Similar to LinkedIn's paid inMail feature.
    • Account analytics. Twitter does not currently share the number of times a particular message or profile was viewed, even in channels it controls such as web or widget views.
    • Sponsored account suggestions. Twitter suggests other members you might want to follow. These suggestions appear, or have appeared at some point in the past, on the Twitter homepage, public timeline, and individually-tailored suggestions. Twitter has even experimented with small textual call-outs linking to another account in every member's sidebar. These sponsored listings are currently untapped, under-marketed, or both.

    Potential competition

    The limited data sharing of full Twitter updates limits the open competition to data services developed by Twitter in-house. Nielsen BuzzMetrics, Biz360, TNS Cymfony, and many others currently tap into broad social media streams for brand insights. Twitter could compete with these dashboards in a more real-time environment, resell their solution to larger monitoring firms, or both.

    Summary

    Twitter plans to roll out revenue-generating services to serve corporate customers within the next two months. The company is also rumored to be raising additional capital that could extend its runway before Twitter needs real revenue under its wings. I believe SMS revenue-sharing and brand monitoring will provide Twitter revenue in 2009 with additional paid features made available to enthusiast users later this year. Twitter's best path to realized revenue may be an acquisition but large companies always like a bird with wings.

  4. Nov06

    Syndication and Widgets Primer

    The publishing world is continuously evolving, creating new opportunities for plugged-in companies to reach new audiences like never before. Today's publishers need to think beyond the fixed location of their website and fully integrate with the large hubs of user activity on the desktop, mobile phone, social networks, blogs, and web pages at large. Syndication and widgets power new opportunities to carry content beyond the walls of a single site and into some of the largest brands in the world yet some publishers still haven't gotten the message. I recorded a 1-hour video presentation earlier this week to better explain the syndication and widget landscape to web publishers. This summary document helped shape the Widget Summit program and new publisher opportunities.

    Widgets and Syndication presentation capture

    A widget in its simplest sense breaks apart a website into its essential components, broadcasts those components to anyone who will listen, and reassembles the content on a remote system while tapped into local resources. It's a bit like writing your website's front end in a remote location powered by local assembly methods, cached resources, and rich interactions. I compare syndication and widgets to television broadcasting and international shipping: we build our products to exact specifications to take advantage of new audiences and standardized transports.

    Reach larger audiences

    Widgets let publishers take their content to the audience instead of waiting for the audience to come to them. Throughout my presentation I used Twitter as an example. Twitter attracted approximately 2.3 million unique visitors in the U.S. in September according to Nielsen Online. That's a pretty strong audience but it's tiny compared to the repeat daily activities of Facebook, MySpace, Google, or everyone reaching Windows Vista. We package up and redistribute our content to reach these larger audiences spending time away from our site instead of waiting for a new visit into a fixed domain.

    In the presentation I broke syndication strategy into two major components: syndicate data using Atom and piece your data back together on major platforms of interest using widgets. I dive into a few example of major feed reading and widget platforms, and even spent some time on advanced topics such as contextual awareness.

    I am still experimenting with creating presentations for online video distribution but I hope you enjoy these multimedia conversations.

  5. Sep16

    Inside the iPhone App Store acceptance process

    Apple's iPhone OS App Store is a little over two months old and already the focus of both hype and fear among members of the press. KPCB has already invested more than $30 million through its iPhone-specific fund. Established companies are writing iPhone applications for the first time. A few applications have been banned, as expected with most platforms. Apple's relative secrecy regarding the iPhone platform and distribution policies have caused market uncertainties in need of some further clarity. In this post I will examine the iPhone OS 2.0 platform and the iPhone App Store from the point of view of Apple and other hosted storefront providers.

    As I write this post there are over 3400 applications available from the iTunes App Store. 90% of those apps are available for both iPhone and iPod touch. 76% of App Store listings require payment ($1 or more). Developers may distribute an iPhone OS 2.0 application directly to handsets using their own infrastructure or distribute through Apple's App Store built-in to iTunes and iPhone OS.

    1. Inside the App Store
    2. Podcaster rejection
      1. Submitting to App Store
    3. Ad-Hoc Distribution
    4. Summary

    Inside the App Store

    Apple's iPhone App Store connects developers large and small to millions of iPhone OS devices around the world. Apple handles payment processing, international business licenses, distribution, marketing, and delivery of applications to iPhone and iPod Touch devices over WiFi, cellular data, and tethered experiences. App developers may list items for sale in Apple's App Store in exchange for a consignment fee of 30%.

    Carrying third-party content inside your store and on your devices does carry some risk. Developers pay to access the iPhone developer program, sign their applications with unique keys, and assume some of the support burden for their applications. There are a few obvious reasons why a platform such as iPhone might choose not to carry an application in its storefront:

    • Chargebacks. Buyers frequently return your product for reasons including buyer's remorse or just receiving a different product than they expected. The "I Am Rich" $1000 iPhone app carries a heavy chargeback risk.
    • Insufficient differentiation. App authors should be able to submit an application to App Store and expect there won't be a knock-off product sold directly alongside. Open-source applications can swap out an application title and submit the app as their own without adding new functionality.
    • Misleading marketing, including trademarks. Don't misrepresent yourself or your product or cause obvious confusion.
    • Horrible customer experience. Apple will recommend interface designers who can assist you with visual aspects of your application. Long load times or heavy resource utilization might will make both you and the platform look bad.

    Podcaster rejection

    The Podcaster application was initially rejected for redistribution through Apple's iPhone App. I downloaded a copy of the app, version 1.0.9b, directly from the developer, installed it on my iPhone, and reviewed the app from the point of view of the Apple based on existing publications and guidelines.

    Podcaster navigation screen

    Navigating the tab bar to content sections such as newest podcasts or featured podcasts brings up a splash advising 3-5 minute wait times before content is returned. Red flag. My previous search input does not disappear when I change navigation modes.

    Podcaster for iPhone add podcast confirmation

    The developer used a single button action sheet with a vibration action to confirm each new podcast download. Apple adds a badge to the Downloads tab for a similar action in iTunes. The user has to dismiss a confirmation sheet for every new addition.

    Podcaster 1.0.9 compared to iPod podcasts 2.1

    Pictured above is a view of the same podcast in Podcaster and iPod. Duplicate functionality when tethered but Podcaster does provide over-the-air updates direct from the handset. Podcaster has some obvious errors with text display in the regular table view but the display does work. Displaying the sound disclosure indicator on every row is a bit overkill: Apple only includes this disclosure indicator when the audio file is currently playing. Small fit and polish issue.

    Submitting to App Store

    Podcaster submitted the first release candidate of their iPhone application to iTunes Connect, App Store's web-based management tool, on August 14. Application release notes are available through the developer's Twitter feed. The version submitted to Apple in mid-August had problems downloading and playing many podcasts including FeedBurner's redirect URLs or large downloads (likely a Range issue). On September 12 Podcaster heard back from Apple regarding their App Store submission. The $4.95 application was not accepted for distribution through App because it it duplicates the functionality of the Podcast section of iTunes.

    The developers resubmitted Podcaster to iTunes Connect on September 13 with a new description before launching a media campaign and ad-hoc distribution.

    Ad-Hoc Distribution

    iPhone application distribution

    Apple designed Ad-Hoc distribution for direct distribution of iPhone OS applications. Each application build is limited to 100 provisioned handsets. Ad-Hoc distribution is a primary distribution mechanism for beta testers, classrooms, workshops, or corporate environments that do not desire worldwide distribution through App Store.

    As of yesterday afternoon Podcaster had provisioned 1130 devices for distribution across 12 different copies of the application hosted on Google Code. Each new uploaded build included up to new 100 authorized devices after the publisher received payment via PayPal. It's stretching the Ad-Hoc distribution model a bit but the application may have collected approximately $11,000 over the weekend through suggested donations of $10 per handset. At the time of writing Apple has not pulled the application or developer certificates from their central certificate authority.

    Summary

    It's not always clear what Apple is up to but there are legitimate reasons to not carry an item for sale in your store. iPhone OS will continue to be a popular development platform even if Apple is overwhelmed with interest and developer support. The best developer information comes from within the confines of ADC, WWDC, and iPhone developer programs, and other privileged access forums restricted by Apple and its partners.

  6. Aug21

    Intel and Yahoo! announce Widget Channel for HDTV

    Flickr on Widget Channel

    The Internet is coming to your TV, reclaiming your split attention span from the other gadgets around the house. Intel announced its latest effort to power your living room yesterday with new media processors, reference designs, and software stacks that may eventually find their way into the cable boxes, Blu-ray players, and home media centers of 2010. Intel partnered with Yahoo! to deliver Internet-connected widgets, advertising, and content to potential partners with a software stack branded The Widget Channel. Yahoo! spent about two years customizing Yahoo! Widget Engine for high-definition televisions and hardware-accelerated graphics displays. Yahoo! will pitch its widget engine for televisions, display advertising integration, and customizable widget gallery to cable operators, television manufacturers, and other major consumer electronics companies as Yahoo! seeks a prominent role in what it calls the "Cinematic Internet."

    I visited the Intel Developer Forum in San Francisco yesterday for a first-hand look at the new prototype widgets platform. I was lucky enough to bump into Eric Kim, Intel's SVP and General Manager of the Digital Home group, and recorded a 8-minute overview of Widget Channel. I've embedded the walk-through below. High-resolution snapshots of individual Widget Channel widgets are available on my Flickr account.

    Intel Widget Channel demonstration
    1. Yahoo! Widgets on TV
    2. From reference design to reality
    3. Picking apart the pieces
    4. Summary

    Yahoo! Widgets on TV

    Widget developers may be already familiar with the Yahoo! Widget Engine, also known as Konfabulator. This desktop engine started out on the Mac, ported to Windows, and now runs inside the Linux-based Intel TV platform. The Widget Engine team is a part of Yahoo!'s Connected Life division which also includes Yahoo! Mobile -- powered by Blueprint widgets -- and DVR acquisition Meedio. Yahoo! has tied its data APIs to TiVo and Windows Media Center in the past, accessible as a full-screen application after a deep dive through the device's navigation options. The Widget Channel and its alpha-blended snippet dock complementing the main viewing experience of your TV is a radical departure from past Yahoo! partnerships in the space, an obvious result of designing an experience from the ground-up instead of bolting onto other vendor's solutions.

    Widget developers can build new widgets for Widget Channel using most of the same resource bundles and runtimes used on the desktop Konfabulator engine. XML manifest files define widget metadata, preferences, and screen UI. JavaScript powers on-screen interactions and dynamic data. Yahoo! Widget Engine includes a WebKit run-time, which will hopefully be ported by Yahoo! and Intel to support hardware accelerated CSS and other nice features on the new software stack.

    Widgets written for the new Yahoo! Widget Engine for TV must conform to four major UI modes: snippet content on the bottom dock, sidebar content, full-screen display, and background processing. Docked snippets are more than just minimized widgets: viewers can cycle through multiple snippets inside a single widget such as weather in various cities or a sports scores. Most sidebar displays mocked-up by Yahoo! used an accordion design pattern to collapse multiple content sections inside a small space. Full-screen experiences match the full-screen designs of the Web. The demo widget for Flickr uses their newly redesigned Flash slideshow display for a familiar look and feel from desktop to living room.

    I couldn't get a solid answer from Intel regarding how tightly Widget Channel was tied to Yahoo!'s Widget Engine. Intel wants to sell its new consumer electronics system-on-a-chip, the media processor CE 3100, far and wide with or without the Yahoo! engine. I expect the underlying platform contains a native widget layer and programming environment with tighter integration but more programming complexity than Yahoo!'s engine in much the same way NVIDIA Preface bolsters its platform offering with a Windows Sideshow gadgets run-time.

    From reference design to reality

    Gigabyte MD300 DVP rear

    Yesterday's announcement from Intel and Yahoo! is merely a reference design showing off what both companies hope is the future of Internet-enabled consumer electronics. The Yahoo! Widget Engine for TV still needs a lot of work and there are currently no shipping products implementing the hardware and software stack demonstrated yesterday. Cable companies, television manufacturers, and other consumer electronics companies will evaluate the stack over the next year for possible inclusion in products shipping next decade.

    Intel is trying to displace consumer electronics chipsets already in production from IBM and NVIDIA. Sony, Toshiba, and IBM worked together to create the Cell multiprocessor already powering the PlayStation 3 and built-in to the next generation of televisions. NVIDIA chipsets are inside cable boxes from Scientific Atlanta and others. The upcoming Java-based tru2way cable software platform is already under active development by consumer electronics companies and software vendors. Widget Channel could operate as an additional layer on top of tru2way, as mentioned in Comcast's press release yesterday. The word "Yahoo" does not appear in the Comcast press release and Comcast has only announced their intent to evaluate the new reference design against their own Java-based offerings in 2009.

    Picking apart the pieces

    Intel Widget Channel stack

    Widget Channel is a Linux-based operating system with platform software and middleware provided by channel partners. The Widget Engine is one of the available software options on the device. Carrier-specific back-end services including reporting, storage, security, and developer certificate verification reside within the carrier network with possible add-ons such as display advertising powered by Yahoo! or others. Each Widget Channel implementation can choose its own Widget Gallery service and white-listed widgets and possibly receive extra content from a compatible widget gallery offering served by Yahoo!.

    Yahoo! has a good opportunity to serve display advertisements, sell premium widget placements and certifications, and promote its own content within each Widget Channel deployment. It may be possible for Yahoo! and Google to program their own compatible widget and advertising layers on top of the base Intel platform to replace or compete with Yahoo!. Major features such as contextual widgets and advertising layers have yet to be developed for the platform, leaving new opportunities for other Internet companies to step in with their own swappable components inside the software stack.

    Summary

    Yahoo! Widget Gallery home screen

    The Intel Widget Channel provides a peek inside the connected future of our living rooms. Consumer electronics companies and large carriers from the cable and satellite industries want to participate in the premium content offerings available through Internet-connected electronics and new software stacks from chip vendors could help bootstrap new services. Intel wants to sell more chips, Yahoo! wants to serve more ads, and cable companies want to boost subscription revenues without major investments in new infrastructure.

    Intel admits interactive services on the television has been a popular goal over the last 10 years but without measured success. [T]he rate of adoption has so far been disappointing, studies show that consumers remain receptive to the concept. New levels of broadband penetration combined with high-definition viewing could change consumer adoption but we are still a few years out from real adoptable implementations. We'll have to wait and see what hardware and services are announced in 2009 before spending too much development time against a reference design.

  7. Jan17

    Upgrade your Google Analytics tracker

    Google Analytics logo

    Google released a new version of its Google Analytics tracking code in December after a two-month limited beta. The new Google Analytics tracker is a complete rewrite of JavaScript inherited from the Urchin acquisition in 2005 and the first time the two products have been officially decoupled. The existing version of Google Analytics tracker, urchin.js, has been deprecated but should continue to function until the end of 2008. Google will only roll out new features on the new ga.js tracker. If you currently track website statistics using Google Analytics you should upgrade your templates to take advantage of the new libraries.

    What changed?

    The new Google Analytics tracker supports proper JavaScript namespacing and more intuitive configuration methods (e.g. _setDomainName instead of _udn). My tests show about a 100 ms faster execution even with a 24% increase (1514 bytes) in file size (ga.js is also minified).

    The new tracking code makes advanced features a lot more accessible. You can now track a page on multiple Google Analytics accounts, which should help user generated content sites integrate their author's Google Analytics IDs alongside the company's own tracking account. The new event tracker lets you group a set of on-page related actions such as clicking a drop-down menu or typing a search query (very useful for widgets). Ecommerce tracking is now a lot more readable. You can read about all the tracker changes in the Google Analytics migration guide PDF.

    Implementation

    Switching your site tracker is pretty simple. Trackers are now created as objects and configured before the page is tracked.

    <script type="text/javascript" src="http://www.google-analytics.com/ga.js"></script>
<script type="text/javascript">
var pageTracker=_gat._getTracker('UA-XXXXXX-X');
pageTracker._initData();
pageTracker._trackPageview();
</script>

    That's it. You are now running the new Google Analytics tracker. You'll need to swap in your Analytics account and profile IDs, which should be pretty easy to spot in your existing code.

    Summary

    Google Analytics tracking code is completely rewritten for faster on-page behavior that plays well with others. The old tracker will be deprecated within a year, and new features are only available to users running the new code. Existing Google Analytics users should swap out their tracking code to take full advantage of this free stats tool.

  8. Jan09

    FeedDemon and NetNewsWire are now free

    NewsGator is giving away desktop feed readers FeedDemon, NetNewsWire, and NewsGator Inbox. The company hopes to regain any loss of revenue from its desktop business with new enterprise sales leads and better attention metadata. The company announced the change in pricing in a press release today and a blog post by founder Greg Reinacker.

    NewsGator's desktop feed readers previously cost about $30 each and faced some commoditization through feed reading software bundled with modern operating systems, office suites, or competitive open-source solutions. Windows client FeedDemon needs to compete with feed reading capabilities built-in to Windows Vista and Internet Explorer 7 or open-source clients such as RSS Bandit. Apple client NetNewsWire competes with Mail.app in Leopard and open-source freeware such as Vienna. NewsGator Inbox competes directly with Outlook 2007. Online competitors such as Google Reader are starting to deliver desktop-like speeds in an always up-to-date, always available model.

    NewsGator differentiates its desktop client offerings from the competition through the NewsGator Online hub. Each client filters its requests for feed data through the centralized online service and synchronizes each user's list of subscriptions, read/unread items, shared snippets, and more. NewsGator plans to use the extended user base available via its free clients to fine-tune relevancy and other metrics available through uniquely identifiable attention data.

    [B]y using your data, in combination with aggregate data from other users, we can deliver a better experience for everyone. And that’s a good thing - both for us and for you.

    Each desktop application can also sync with a local activity hub NewsGator is selling within enterprises. They hope free tools will infiltrate corporate America to generate new sales leads and internal advocates for bigger licensing fees.

    Summary

    NewsGator's move to free is an interesting risk for a changing business. Competitors such as Attensa do not have a similar strength in the desktop client space, and NewsGator will continue to worry about Microsoft shipping an update to SharePoint that could shake up their enterprise market. In the mean time thousands of consumers will be able to download quality software for free, and the small desktop clients can continue developing cool new features funded by enterprise usage.

    Update: Nick Bradbury, creator of FeedDemon, shares his thoughts on the freebies on his blog.

  9. Dec01

    Facebook cleanses Pages of supposed fakesters

    Facebook Pages iconFacebook is proactively deleting Pages and other content from its site in an attempt to limit fake listings created by unauthorized entities. The new enforcement procedures started on Thursday night with many Facebook users receiving notifications of their deleted pages and an apparent violation of the Facebook Pages Terms of Service. The main issue seems to be individuals on Facebook proving they have the authority to create a Page for their company, band, or product. Facebook is now requesting a yet unspecified amount of documentation from each user before they create a Facebook Page to avoid future deletions.

    Yesterday morning many brands sat down in front of their computers and learned Facebook had deleted their pages from the social network. Facebook called into question the identity of athletes, startup company founders, web hosting companies, video game development houses, and television networks maintaining pages on Facebook to connect with their fans. Electronic Arts was no longer engaging fans of its video games on Facebook, Fox television shows disappeared. Athletes such as David Beckham were no longer kicking it with their fans. The new proactive fake Pages sweep by Facebook has isolated brands and celebrities currently evaluating the Facebook social network and its benefits. This early negative experience will likely harm Facebook's attempts to reach out to brands for advertising and promotional opportunities on Facebook.

    We need a document showing that you (or your company, which we will need proof of affiliation with, as well) have the rights to represent these companies and individuals. A document on the company letterhead would be a good start. You can email all documentation to advertise@facebook.com, and we will then be able to assist you.

    Facebook is asking each person creating a new Facebook Page to first contact Facebook's advertising department via e-mail with documentation stating you have the right to create the page. If your request is approved Facebook staff will "make a note on your account so that [the page] won't be removed" according to an e-mail I received from Facebook's sales staff. I have submitted requests for Facebook to reinstate pages for two products I own, Startup Search and Widget Summit. I have also submitted a copyright counter notification just in case someone filed a copyright infringement notice against one of my own sites and logos although I received no notification of copyright violation from Facebook for any pages, only statements questioning my authority to create such content.

    What is a Facebook Page?

    Facebook Page Blockbuster

    Facebook Pages were introduced on November 6 as part of the new Facebook Ads suite of products. Facebook profiles correspond to an individual person while Facebook Pages are a specialized type of Facebook profile created for a local business, brand, product, non-profit organization, celebrity figure, or other entity. Facebook Pages are administrated by one or more Facebook members and allow anyone to become a "fan" of a product, company, or service.

    Brands such as Blockbuster might create a page on Facebook to connect fans of its video rental service and reach new customers. Blockbuster can distribute its custom Facebook application through this channel and help these new fans share their video rental history throughout the social graph. Blockbuster can also accelerate its fan growth through targeted advertisements on Facebook.

    A band or celebrity might create a Facebook Page to let its marketing staff promote and measure their online identity. David Beckham could have an individual profile on Facebook, but the friends requests would become overwhelming and lose all personal meaning. Instead Beckham can create a Facebook Page with multiple administrators and managers experimenting with how to best connect with fans online.

    The Fakester Problem

    Social networks have always been plagued by fake accounts, popularized under the "Fakester" term during the Friendster era. Anyone can create spammy social network profiles for a popular Christmas toy, pharmaceutical drug, or an actress in the news. These pages take advantage of the celebrity or brand power of another entity for personal gain and may confuse online visitors or hurt an online reputation.

    A trademark and service mark is one way we protect online brands and establish authorized uses. Companies often take control of domain names in the hands of domain squatters through legal enforcement afforded trademark owners.

    Some social networks choose to play a continual game of Whac-A-Mole with fake profile creators, trimming the unwanted parts of its user base to keep a clean community of real people conducting meaningful interactions. Most social networks choose not to review new profiles and instead wait for a reported violation from a brand or copyright owner through established channels such as the Digital Millennium Copyright Act and its safe harbor provisions. Google and Second Life regularly deal with reported fakesters, trademark infringements, and copyright violations and have established well-documented processes to deal with each request.

    Permission-based inclusion

    Facebook Pages terms of service state "Facebook does not review Facebook Pages to determine if they were created by an appropriate party" yet recent actions and statements by the company indicate a Page is considered fake and subject to removal by Facebook unless written documentation is filed with the company to assure your authorization to create such content on behalf of all involved entities. There are a few big problems to this approach that will likely cause companies to walk away instead of submit papers for each employee and service provider.

    Local business authorization is expensive or impossible

    Facebook would like to create local listings Pages covering bars, restaurants, bookstores, and other places of business in your local town. The owner of my favorite cafe uses a Gmail address and my uncle runs his auto repair shop from Comcast services. They both have filed appropriate false name forms (doing business as) with their local county governments but the hassle of locating or reissuing these documents for a social network such as Facebook is a prohibitive barrier to entry. VeriSign will walk into your startup's office, verify your existence, and issue an extended validation web browser certification for $1500 if you have the money and the patience to endure that level of verification.

    Big brand, many managers

    Companies have teams of individuals working on a product or service both inside and outside the company. A smaller website such as RockYou might have a marketing consultant, public relations firm, and a product team interested in engaging a larger audience on Facebook or other networks. A video game such as Rock Band is developed by Harmonix, published by MTV, distributed by Electronic Arts, marketed to the press by a public relations firm, and marketed to an online audience by a specialized social marketing firm.

    The number of people involved in a particular brand, product, or service makes verifying each business and its employees a prohibitive cost of engagement. Like the $1500 browser certification, companies will only go through the hassle if they have a significant opportunity for large returns on their investments. Facebook and other social networks are still in the experimental stage at many large brands.

    The techie friend

    Less technically skilled businesses and brands rely on the help of a techie friend for many tasks in the techie world. My local cafe might not know how to log-in to a website, claim their business, and input their hours of operation. Traditionally the techie friend role for local listings was the phone company calling every business for yellow page listings and perhaps a few ads. Earlier this year Google started paying $10 for each local business listing created by its users going door to door snapping photographs and collecting local business data.

    It is difficult to verify the techie friend since they are helping out someone who was not Internet or social network savvy enough to fill out the appropriate online forms or mailers. I had not heard of any restaurant or cafe listings removed from a local database until Facebook's recent cleaning spree.

    Protecting your company and brand

    There are a few things your company and staff can do to decrease the chance Facebook might remove your employees, products, or advertisements from its site.

    1. Create a Facebook Network for your company and its employees.

      Facebook Networks associate Facebook members with their current or past places of employment. Company membership will be validated by e-mail address, allowing anyone with a "yourcompany.com" e-mail address to join your corporate Facebook network.

      Web hosting company Joyent recently had their Facebook Page deleted even though the company had an established business partnership with Facebook. Joyent's CTO and VP of Marketing were Page administrators yet there was no Joyent company network and therefore no established link to help their case. The executives had no official Facebook validation and verifications on their account and were listed as regular members of the San Francisco network.

    2. Submit a documentation request to Facebook's Advertising department, asserting your ability to create a Facebook page for your brand. Name each of your employees or outside partners you would like to grant explicit permission to administer or contribute to your Facebook Pages or content.

      Facebook requires documentation from "all companies involved" showing each Page administrator has been authorized to represent the product. The exact documentation and assertion required from Facebook is still unclear, but a pro-active approach should help protect your Pages from deletion.

      Facebook also offers people-friendly URLs for some verified pages. Your page could have a URL of facebook.com/blockbuster instead of Facebook page 5973937214 after verification.

    3. Create a Facebook ad.

      Paying Facebook a few dollars might help ensure the long-term existence of your Page. Fraudsters might be less likely to spend money promoting a false page or entering identifying information such as a credit card in Facebook's system. An advertisement history associated with your page may be a positive signal indicating your engaged interest in the success of your Page.

      Facebook advertisements have a minimum cost of a penny per click and a daily ad budget of $5. If you would like to associate an ad with your Page for little to no money just target an obscure group for a 1-day ad campaign, such as Texas vegetarians who prefer Kobe beef, to help ensure your fees never approach even $5.

    Summary

    Facebook is a closed network and the company reserves all right to determine when, where, or how you or your brand might exist on its site. It's a shaky ground to enter but the promise of millions of eagerly waiting customers act as siren calls to brands entering the wild west world of social networking and user-generated media. Large brands such as Coca-Cola are pulling back from their planned involvement and instead opting for a "wait and see" attitude, and other brands may follow their lead as Facebook works out the kinks of each new system.

  10. May30

    Startup Search: tracking the web startup ecosystem

    Tonight I am unveiling a new site tracking the startup ecosystem. It's a directory and analytics tool I've personally wanted for a long time, and I know others will enjoy. Introducing Startup Search.

    Startup Search screenshot

    Startup Search tracks Web startups, their products, key employees, investment firms, and investment partners. Startup Search also tracks the success of each product since it was first introduced to the world, using publicly available metrics pulled into a single page. It's a research tool, a discovery engine, and a fact-filled directory of our little Web startup world. I'll walk through a few features.

    Directory data

    Startup Search is a data-driven website tracking facts and figures about the entire web industry. You might be familiar with a particular web product, but who is the parent company? Where do the founders live and work every day? Have they taken funding, and if so, from what firms or investors?

    In today's web directory data about startups and their employees was locked behind a paywall. A service such as VentureOne might call your company on behalf of a paying venture capitalist and ask questions about the company to help build a profile. Startups would never see this data, only provide information to someone they may never meet on behalf of a venture capitalist they may never meet. I want to change the flow of information, placing more power in the hands of anyone who would like to blog about, take a job with, or invest in some of the companies Startup Search covers.

    Startup Search also covers some data you might never find inside of an existing directory. Who are the 12 angel investors in Dogster who collectively contributed $1 million? Who is the team behind Blinksale? Who is Felicis Ventures?

    There may be some entirely new data areas covered as the directory expands. Good feedback will reveal just how much has changed.

    Statistics

    Each product is tied to a set of statistics I call buzz and traffic. Buzz measures the level of conversation around a product such as links from the web and from blogs, or mentions of the product's name in blog posts or web searches. Traffic accumulates daily Alexa data together with monthly data from Quantcast and Compete into one single page.

    I enjoyed deconstructing the Alexa data and putting it all back together again. All data is licensed using the appropriate APIs for each provider. I hope to increase my data coverage and site features over the next few days at Google Developer Day and Where Camp.

    More to come

    I wanted to release the product and iterate out in the open based on the tons of feedback available. I plan to introduce more features for startups who would like to claim their profile on the site. I'd also like to introduce more tools to help people track the business aspect of their startup such as tracking new statistics activity or perhaps researching interesting partners.

    There is a lot more to be constructed from all the underlying data available at Startup Search. I could sort every possible venture capital investor based on their political affiliations and donations, or compute a possible burn rate based on a company's business headquarters.

    Summary

    I hope you enjoy Startup Search and the tools contained within. The companies and features available now are just a beginning, with many iterations and expansions to come.

    Startup Search is powered by Python, Django, and YUI. Bryan Veloso of Revyver designed the site using his new love for grid design. The site is currently sponsored by venture capital firm True Ventures for the month of June, and I hope to continue supporting some site costs through run-of-site sponsorships for the near future. There are many eggs and APIs coming together behind-the-scenes, and I'll likely discover a few more likely sources of inspiration over the next few days.

    Enjoy. startupsearch.org

Niall Kennedy Niall Kennedy is a web technologist in San Francisco, California in the United States. I am very interested in the world of... MORE »

Search this weblog:

Subscribe:

Recently Popular

  1. Rewriting Twitter for web best practices
  2. The story behind Google Chrome
  3. Google App Engine for developers
  4. Sniff browser history for improved user experience
  5. Data Portability, Authentication, and Authorization

Archives: Popular Categories

  1. Widgets
  2. Programming
  3. Business
  4. Feed Aggregators
  5. Search

Sites: More from Niall