On Monday Google launched Google App Engine, a hosted dynamic runtime environment for Python web applications inside Google's geo-distributed architecture. Google App Engine is the latest in a series of Google-hosted application environments and the first publicly-available dynamic runtime and storage environment based on large-scale propriety computing systems.
Google App Engine lets any Python developer execute CGI-driven Web applications, store its results, and serve static content from a fault-tolerant geo-distributed computing grid built exclusively for modern Web applications. I met with the App Engine's team leads on Monday morning for an in-depth overview of the product, its features, and its limitations. Google has been working on the Google App Engine since at least March 2006 and has only just begun revealing some of its features. In this post I will summarize Google App Engine from a developer's point of view, outline its major features, and examine pitfalls for developers and startups interested in deploying web applications on Google's servers.
What is Google App Engine?
Google App Engine is a proprietary virtualized computing suite covering the major common components of a modern web application: dynamic runtime, persistent storage, static file serving, user management, external web requests, e-mail communication, service monitoring, and log analysis. The Google App Engine product offers a single hosted production web server stack hosted on Google's custom-designed computers and datacenters distributed around the world.
Google App Engine is a managed hosting environment with a tightly managed stack running in a machine-independent environment. It simplifies the deployment and management of your web application software stack while constraining you to a specific stack. When I start a new web development project today I have to first setup a tiered system to effectively handle site growth:
- Purchase dedicated servers or virtualized slices. Estimate necessary CPU, memory, disk space, etc. at each tier.
- Configure a web server for dynamic content. Install Python and its eggs, Apache HTTPd and extra modules such as modwsgi. Configure and tweak each. Open appropriate ports. Listen.
- Setup a MySQL database server and choose the appropriate storage engine. Configure MySQL, add users, add permissions. Tweak and optimize.
- Add an in-memory caching layer for frequently accessed dynamic content.
- Monitor your uptime and resource utilization with Ganglia and/or other tools on each machine.
- Serve static files such as JavaScript, CSS, and images from a specialized serving environment such as Amazon's Simple Storage Service.
- Turn your static server into an origin server for a CDN with points of presence close to your website's users.
- Connect each piece of the stack, keep its software updated to avoid security vulnerabilities, and hopefully respond to all website requests in less than a second.
- Dedicate work hours and expertise to all the above. Hire outside assistance if needed.
- Don't go broke trying.
Your tiers will expand as your new web application gains popularity. Your single-server tiers become load-balanced services, message bus broadcasts and listeners, and distributed cache arrays at scale. You'll probably spend time rearchitecting your application at each stage of growth to incorporate for these new resource demands if you can afford the time, expertise, and effort.
Google App Engine is a new and interesting solution for Python developers interested in adding features, not servers. Google spends hundreds of millions of dollars developing its custom infrastructure with 12-volt power supplies tapped into a hydro-electric dam next door and fat fiber pipes owned by local governments carrying requests and responses to their proper home. Google's physical infrastructure is vast array of highly optimized web machines, and we'll now be able to see how such infrastructure performs across more generic applications on App Engine.
Freemium hosting model
Google App Engine is a "freemium" business model offering basic features for free with paid upsells available for application developers exceeding approximately 5 million pageviews a month. This resource quota approximately matches the Google Analytics 5 million pageview limit. Google Analytics customers may currently exceed this limit if they maintain an active AdWords account with a daily advertising budget of $1 or more. The Google App Engine team plans to introduce pricing and service level agreements for additional resources, priced in a pay-as-you-go marginal resource structure, once the product leaves its limited 10,000-person preview period later this year.
| Quota Type | Limit / day |
|---|
| HTTP requests | 650,000 |
| Bandwidth In | 9.77 GB |
| Bandwidth Out | 9.77 GB |
| CPU megacycles | 200 million |
| E-mails | 2,000 |
| Datastore calls | 2.5 million |
| External URL requests | 160,000 |
Google publishes these quotas and provides administrative monitoring tools. The quotas are just a guideline as Google may cut off access to your application if you receive a traffic spike of an unspecified duration. The Google App Engine quota page specifies:
If your application sustains very heavy traffic for too long, it is possible to see quota denials even though your 24-hour limit has not yet been reached.
Google App Engine already failed the Techcrunch effect and appears the platform is currently unable to handle referral traffic loads from a popular blog or news site typically associated with a product launch. The traffic spike cutoffs make me think twice about hosting anything of value on App Engine.
The team
The Google team behind App Engine has a long history in developer services. Team members include some of the top Python experts in the world, financial transaction specialists, and developer tool builders.
- Python creator Guido van Rossum wrote the App Engine SDK and ported the Python runtime and Django framework for the new environment. Google App Engine is Guido's first full-time project at Google after his Noogler project Mondrian.
- Technical lead Kevin Gibbs previously worked on the the SashXB Linux development toolset and multiple RPC projects at IBM before he created Google Suggest in 2004.
- Developer Ryan Barrett wrote the BigTable datastore implementation and related APIs. Previously Ryan was tech lead on Moneta, Google's transaction processing platform and customer data store.
- Product lead Paul McDonald has worked on Google Checkout, AdWords, and a Web-based IDE named Mashup Editor (all strong candidates for App Engine inclusion).
- Product manager Peter Koomen has previously authored papers on natural language search and semantic analysis.
The list above is just a sampling of the full-team behind App Engine.
Feature limitations
Google App Engine is not without its faults. Applications cannot currently expand beyond the quota's ceiling. It's still unclear how an application will dynamically scale on App Engine once it leaves the farm leagues, and at what cost.
A few major issues include:
- Static files are limited to 1 MB. App Engine does not support partial content requests (Accept-Ranges).
- Cron jobs and other long-life processes are not permitted.
- Applications are not uniquely identifiable by IP address, leading to a lack of identification for external communications. Applications may suffer from bad neighbor penalties from API providers upset at another app on the service.
- No SSL support. No IP address complicates signing, but port 443 is open for requests. You can rely on Google services (and branding) for trusted login and possibly future payments.
- No image processing. Python Imaging Library relies on C, and is therefore not a possible App Engine module.
- Google user accounts. Site visitors are very aware of your choice in web hosts each time they attempt to logon to your application. I feel like this flow makes your application seem less professional, but may be a reasonable trade-off. Google will store your user data and potentially mine its data for better ad targeting.
Summary
Overall I am quite impressed with Google App Engine and its potential to remove operations management and systems administration from my task list. I am not confident in Google App Engine as a hosting solution for any real business while the host is in preview stage but those concerns may be alleviated once the product is ready for real customers and real service-level agreements.
Python developers have just been granted a few superpowers for future projects. As an existing Python and Django developer I know how difficult it can be to find a managed hosting provider with modern Python support. Many hosts are years behind, running Python 2.3. I am excited App Engine already features the programming tools I use every day, with a few modifications for their proprietary systems. App Engine should introduce more developers to Python and the Django framework and hopefully cause other web hosts to provide better Python support as well.
