Social computing has changed the way we interact with the Web. Our information consumption and production benefits from the participation of the crowd in its various forms, creating niche audiences and new types of curators independent of space and time. We're connected to local experts on hiking, cooking, parenting, programming, and much more. Yet social media extends beyond the realm of content creators, bolstered by the comments, ratings, rankings, sharing, and reading masses that help us find the content we seek.
Forrester Research released a report last week, Social Technographics, detailing levels of social media participation among 10,000 adults and youth. Their sample panel provides new insights and statistics into how users are currently engaging in social media activities, and the motivations which might drive such participation.
Last Friday I sat down with Forrester Research analyst Charlene Li to discuss her report's findings and its implication for business on the Web. You can read more about the topics of our social media trends and engagement discussion on my podcast site, and view select results from the research report. Our podcast on social media trends is 20 minutes in length, a 9 MB download.
Tip: The full Social Technographics research report costs $279 and is available as a downloadable PDF. The accompanying free PowerPoint slide deck contains key statistics and other summary data you might find useful while keeping your wallet in your pocket.
Rich Internet applications are stepping out of the web browser and onto the desktop, helped along by a new set of toolkits. Web developers are able to code against desktop resources using familiar languages and toolkits such as JavaScript, Ruby on Rails, or HTTP interactions. Offline access for web applications is about much more than planes, trains, and automobiles -- it can accelerate performance and integrate with established desktop interactions as well.
Offline web application capabilities are about more than a missing Internet connection. Application data is stored on a local hard drive instead of a far away datacenter, boosting your load times. Web applications become searchable components of the local operating system, displayed inside a Windows Vista Search result or Mac OS X Spotlight. Your application data might become fully integrated with desktop calendar, address book, or web feed platforms, exposed to any requesting application including mobile phone synchronization or personal backups.
Summary
The offline web application space is a hot topic of discussion which may or may not apply to your product. Is offline access a graceful enhancement on top of your existing application? Are customers clamoring for it? Will you take your application offline using Adobe Apollo, Firefox 3, Joyent Slingshot, XULRunner, or Zimbra Offline? Those are just a few of the toolkits we know about this month, yet more are coming.
It's time to demystify. I hope you enjoy my podcast with Brad Neuberg, one of the experts in the space of offline access for web applications, as a quick way to get your head around some of these larger issues in the future of web application development.
Last Friday I visited Six Apart's headquarters in San Francisco to talk about widgets with Byrne Reese. Byrne is the former product manager of TypePad, currently a product manager of Movable Type, and a developer of plugins and widgets used in both products. Byrne and I talked about the current state of widgets in Six Apart's four blogging products: TypePad, Movable Type, LiveJournal, and Vox.
Our 25-minute conversation about Six Apart widgets is available as a 11 MB audio download. I will summarize a few highlights from our conversation below.
TypePad
TypePad is a hosted blog service with basic and advanced templates available to authors depending on their paid subscription level. Basic templates take advantage of a drag-and-drop sidebar manager, letting blog authors insert and rearrange widgets from the TypePad widget directory without directly editing the underlying template HTML. TypePad accounts at the Pro level or above can edit their template HTML directly through TypePad advanced templates, placing any widget code anywhere on their pages.
TypePad authors can browse available widgets in the TypePad widget directory from the TypePad admin interface or select a compatible widget from TypePad partner site Widgetbox. Widget publishers can add widget content directly from their site using the TypePad widget API and a valid partner service key.
Web feeds and advertising widgets are the most popular widgets on TypePad. Authors like to make money, and they like to integrate content from across multiple sites of interest or other areas of online activity such as their latest photos on Flickr or a music playlist from Last.fm.
Movable Type
Byrne developed the Sidebar Manager plugin for Movable Type, integrating the drag-and-drop simplicity of module management into the administrative interface of self-hosted Movable Type software. The Sidebar manager plugin was built into Movable Type's core code starting with version 3.3. Movable Type treats widgets as a special type of template module, grouping appropriate sidebar content for your blog homepage, category listings, or individual entry pages.
Movable Type plugins allow third-parties to integrate content and functionality directly into the blogging application and output their content directly into each generated HTML page. Movable Type supports static generation of blog files using Perl, or dynamically generated pages using PHP (or both), letting plugin and widget developers utilize available features to create the best possible integration experience.
Byrne has not seen many developers taking advantage of the plugin + widget management features of Movable Type, but perhaps it's just an education hurdle.
LiveJournal
LiveJournal has a user base that is very security focused, taking a conservative stance on the trade-offs required when including third-party code on their journals. In the past year LiveJournal has opened up a little bit, whitelisting a few widget publishers for inclusion in LiveJournal templates. LiveJournal compiles its user templates, removing any unknown JavaScript entered by an author before the file is saved to disk. A whitelisted widget passes through this filter and is available to LiveJournal's over 12 million authors.
External widgets can only be added to LiveJournal after the code is reviewed and approved by Six Apart staff. The best way to get whitelisted for inclusion on LiveJournal is to initiate a partner discussion with the Six Apart business development team.
Vox
Vox is a closed system and a very controlled environment, and only Six Apart authored widgets are currently available on this new blogging system. Vox may open up in the future to allow third party widget content in its author admin interface.
Summary
These are just some of the topics covered in my 25-minute podcast with Byrne Reese of Six Apart. Listen to the full audio to hear our thoughts on popular widgets, widget business models, and what widget functionality we might expect from Six Apart in the future.
I hope to make this type of developer interview a regular feature, providing direct information about developing widgets on popular widget platforms.
Rasmus Lerdorf led OSCON attendees through a series of optimizations for modern web applications using PHP at O'Reilly's Open Source conference today. Most programmers use default installations and configurations for their web applications and never really dig deep within their stack or their own code to optimize page load and latency. The full slides from Rasmus's talk are available online and I recorded audio of the entire session from the front row.
Rich web applications make better use of browser user interfaces through the use of less visible round-trips to the server (through AJAX or other methods). These new interaction models place increased importance on your server's ability to respond quickly and frequently to requests, allowing your web app to feel like a desktop app. Rasmus walked through each of the essential components of a rich web app, optimizing each step along the way with the goal of supporting 500,000 users and their 1700 requests/second as seamlessly as possible with less hardware.
Once your web application in complete Rasmus recommends using http_load to evaluate your application performance. How many requests per second can you handle? What is the latency of these requests? Your choices will reflect your app's ability to scale.
Once you've grabbed some statistics it's time to dig a bit deeper and understand which processes are responsible for the slowdown. Valgrind is an emulation tool allowing you to step through your Apache server, Zend components, and PHP wrappers to determine the pain-points in your application. You may notice unexpected processes firing off such as SSL for local database connections that are unnecessarily weighing down your app. KCachegrind is useful for visualizing these processes.
Prepared statements can be a bit slow in PHP and skip the query cache. Rasmus recommends using PDO::ATTR_EMULATE_PREPARES in PHP 5 for more optimal data access.
APC is an alternative cache allowing for better query execution. Note: include_once and require_once don't play nice with opcode caches right now and changing to require. This behavior should be fixed in future versions of APC. APC has a no-stat mode and if you give it absolute paths you can skip the stat() call.You can also store PHP variables in shared memory such as your config.ini file.
PHP 5 adds better XML based on libxml2. SimpleXML is a good tool to load a full XML file and map elements. PHP 5 contains a new SOAP extension written in C and will have an Axis-based version available in a few months.
I may up a full transcript of the talk, but until then you can walk through the slides from Rasmus's talk.
Widgets are taking over the web, small pieces at a time. Big web destinations are opening their templates to custom configurations by users and pre-configurations with special partners. Mix and match your favorite content from around the web on your personal start page from Microsoft, Google, or Netvibes. Share a few live and always updating bits of information in your blog sidebar using widgets on WordPress or TypePad.
Small(er) businesses can leverage the huge distributions of users across most of the top web properties. Over half of the top Internet companies for home users currently open up their pages for easy setup of your content.
Each of the companies highlighted in yellow has an potential audience waiting for content and information from your webapp. What fresh content and information would you like to present to your users and their readers anywhere at any time?
Widgets are simply composed of HTML with optional CSS for styling and JavaScript for rich interaction. A few sites require an XML to tie all your files together as well.
Widgets can include the latest inbound links to your blog as tracked by Technorati, your current availability on an IM network, or the top stories on Digg. Some widgets can be extended to your desktop using software from Microsoft, Google, Yahoo!, Apple, and others.
New ecosystems are springing up around widget-enabled sites. Companies such YouTube and Slide receive the majority of their traffic from embedded content on MySpace. Google has created an IDE for their gadgets. Startup company PostApp is creating an entire business around widget syndication. It's only just the beginning of widget services across the web.
Om Malik and I discuss in more detail the current state of the widget web in this week's PodSession named Widgetization of the Web. The podcast is 23 minutes in length, a 10 MB download.
Seth doesn't think the big portals have added a new feature that users have been clammoring for in years. Chinese company Tencent has captured about two-thirds of the Chinese market with its QQ client. Estonian company Skype has enjoyed tremendous success with its P2P and encrypted conversation technologies. Meanwhile large companies have relied on acquisitions to stay fresh, such as Yahoo! integrating Dialpad and Microsoft adding FolderShare to their latest releases.
Meebo has noticed a sharp decline in instant message traffic during World Cup games involving England or Brazil. The world pauses to watch the game before returning to their computers for instant analysis.
This week's PodSession, Instant Messaging. The podcast is 24 minutes in length, a 11 MB download.
Last night Om and I sat down with Matt Mullenweg, lead developer of open-source blogging software WordPress and a recent founder of Automattic to record our latest PodSession. Automattic is a software services company centered around the WordPress blogging platform.
We chatted about how to successfully scale a new web application. WordPress.com currently hosts about 200,000 blogs with mirrored hosting in San Diego and Dallas. Matt and I agreed that it's best not to over-optimize at the beginning but instead sit back and watch the actual usage of your web application to fine-tune. Check out Cal Henderson's new book, Building Scalable Web Sites for over 300 pages to help keep your site online.
Om tried to shake things up with a Ruby on Rails vs. PHP showdown, but again it didn't work. David Heinemeier Hansson and I talked about the same issue in our 37signals podcast last December. Pick a programming language you and your engineers are comfortable with, or you can easily pick up based on existing skills. Using existing programming libraries in a given language may sway your decision.
We talked about spam in the form of fake blogs and/or comment spam. The Akismet plugin has stopped over 40 million spam messages from blog owners and is also being used to identify the creation of spam blog accounts on WordPress.com. With the recent integration of blog search results to mainstream media sites such as Time.com and the Associated Press blog spam now has a new level of visibility and motive for attack. Hopefully startup companies are keeping on top of the problem and related items for "Bush" won't be overwhelmed with camgirls.
Microsoft, Nintendo, and Sony are showing off their latest gaming consoles this week at E3 in Los Angeles. Xbox 360, Wii, and PlayStation 3 each feature an ethernet jack and online gaming experiences built-in. These new devices are increasing demand for HDTV, broadband, and networking gear in the home. Game developers are able to push out new content after the game has shipped, or introduce smaller, more niche games through a console maker's online marketplace. The connected console is changing the way games are made and causing consumers to rethink their living rooms.
Om and I cover these topics and more in this week's PodSession, Video games gunning for bandwidth. The podcast is 21 minutes in length, a 10 MB download
I slipped in some lyrics from rapper Big Pun towards the beginning if you want a good laugh.
I few weeks ago Tom Raftery interviewed me over Skype as part of his weekly PodLeaders podcast. We chatted about Technorati, Macs, Microsoft, podcasting, and many more topics I've been meaning to blog about.
Tom is a fellow Irishman and some of my answers are aimed at a European audience. I've briefly answered the questions from the podcast below. You can listen to the entire interview for my full responses to questions from Tom and his listeners.
I recorded the interview from my home at 8 a.m. after a brief cup of tea, so hopefully it all makes sense.
Can you tell us first off about your background? - 0:36
I've been in vertical search for about 7 years and blogging even longer. I've worked in shopping search, institutional investment search, small business search, and most recently in blog search.
I've never been too impressed with Froogle, and neither has the market. Froogle is about #7 or #8 in the space and PriceGrabber and NexTag are somewhere in the top 5. Kelkoo still dominates in Europe and might be the most familiar shopping comparison site for Tom's listeners.
Om and I would often chat about telcom and the changing Web. We both like being exposed to new ideas, and our conversations were so interesting I wanted to share them with the world. I interviewed Om a few times about VoIP, broadband, and some other topics he likes to cover on his blog, and eventually we decided to create a brand out of it and podcast weekly. He used to call it "lazy blogging" but I think he's come around to podcasting as a unique medium.
It is interesting how you don't always see eye to eye - 2:58
That's what makes it good! We report, you decide...
You recently left Technorati and joined the guys in Microsoft. Does that feel a bit like going over to the Dark side? - 3:43
I think we view companies with large market share as "the dark side" due to their ability to move a market and throw their weight around a bit in not so great ways. Microsoft did some pretty stupid and anticompetitive things I don't agree with in the late 90s such as threatening businesses such as OEMs not marching to the Microsoft desired beat.
The Windows Live initiative is a chance to rethink how average Internet users interact with online data and that's an exciting opportunity to me.
When are you starting with Microsoft? - 5:42
I started on Monday, April 24, a few days after my interview with Tom.
Will it mean you having to move from San Francisco to Seattle? - 5:57
No, I'll continue to live in San Francisco. I like being at the center of activity in the tech world here in the Bay Area and I'm introduced to new ideas all the time. I'm working in Microsoft's Silicon Valley offices right now but hope to have an office in San Francisco in the near future.
How does Live.com differentiate itself from Google's start page and MyYahoo? - 7:00
Live.com contains gadgets similar to what you might find on the Google personalized homepage for weather, stock quotes, RSS feeds, or whatever you want. Microsoft gadgets can run not only on your browser home page, but also inside the Windows Vista sidebar. In the future you could run the same gadgets inside a toolbar or on your laptop's external screen.
You are a Mac user. How will Windows Live work with a Mac, I mean OneCare won't work with a Mac, will it? - 10:31
Windows Live is a broad brand name but the websites and services should work with Macs and Firefox/Safari. OneCare, Messenger, and toolbar
Is there a business model around Windows Live that you are aware of? - 12:08
Advertising. Microsoft calls their new advertising platform AdCenter and it will deliver targeted ads across all the new websites. An online version of OneCare might have safety-related ads such as Volvo on the same page.
There are rumours emerging about Windows Live offering truckloads of disk space. Do you know anything about that? - 13:47
Yeah, I think it will happen. I blogged about some of the possibilities, and "truckloads" is all relative. We'll keep wanting more and more space as more media becomes available as digital downloads or ripping technology improves.
What will you be bringing to Microsoft and what would you like to change in their attitude towards RSS/Atom - 16:23
I think about feed syndication technologies all the time. I geek out about the future but don't forget about many of the potential users out there who would love the benefits of technologies we enjoy if only we could make the technologies easy enough to use. Microsoft has a few hundred million online users and I think they will enjoy reading feeds as part of their daily lives.
What do you think of RSS/Atom/ and other "standards?" Do you think there should be one standard to unite them all? - 18:14
Ah, politics! From a feed reader perspective we're going to have to parse and make sense of popular formats of publication. If China decides to invent their own XML syndication format tomorrow because existing methods were not invented in China we'd still support reading the data.
I don't think there will be "one format to unite them all" anytime soon. I think Atom, as an IETF standard, will be the preferred standard of governments swapping data such as Europe's interoperable delivery programs. RSS 2.0 has been in the wild for about three years now and has a large and proven deployment.
How will that affect Microsoft's extension of RSS? - 21:09
Microsoft introduced some XML namespaces regarding ordering lists (simple list extensions) and synchronizing data between devices (simple sharing extensions). Both proposals have Creative Commons licenses and, based the conversations I've had with Ray Ozzie, are intended as first drafts and idea-starters.
If you had some advice for some aspiring bloggers who wanted to improve the technical sophistication of their blog, what would it be? - 21:43
I get technical sometimes, but I also enjoy explaining complex things to new audiences. If you're going to be technical I suggest picking a specific topic area and diving deep into it over and over again. Use that topic as an example of technology applications, helping people understand its uses.
As I understand it, you are going in to do some major work on RSS. If that's true then is this for IE7? Or is this part of a longer term strategy regarding RSS inside MSFT apps? - 23:30
The Windows RSS platform is a part of Internet Explorer 7 for Windows XP SP 2 and above, including Windows Vista. I'm working on an online platform connecting online and desktop services with common feed syndication tools and technologies. I live online, and IE7 is on the desktop.
Did he have an inkling of an offer from Microsoft before he quit Technorati? If not, I find it interesting that he had enough confidence in his own "brand," as represented by his blog, to feel he could make that leap into the unknown. - 24:39
The position at Microsoft did not exist when I announced I was leaving Technorati. I left Technorati with a few opportunities in motion, including my own startup. Leaving Technorati was a bit of a leap into the unknown, but I knew the job market was strong enough and was not worried too much.
Bif wants to know how you think the U.S. will do in the World Cup? - 31:30
It's going to be difficult to keep morale high playing Czech Republic (ranked #2) and Italy (ranked #14) in your first two games. There will be so many Czech and Italian fans in the stands it will feel like a home game for both squads. The U.S. has a strong squad with some good depth, and we have a lot more players in Europe with good weekly competition than we had during the last World Cup. If Ireland almost beat Brazil last year, anything is possible.
How do you keep up with information management? What tools do you use? - 33:51
I try out new feed readers all the time but I mostly use NewsFire and NetNewsWire on my Mac and FeedDemon on my PC.
Will you have to give up the Mac working for MS? Or will you use Bootcamp? - 35:40
I use Windows at work to take advantage of Outlook, Exchange, and intranet sites that work best in Internet Explorer. I still use my MacBook Pro all the time. I bought my first PowerBook so I could have a different work and home computing experience. Coming home to a Mac doesn't make me feel like I am doing more work. It's fun.
Want to found a startup! First you should go to school and learn a few things. Last Saturday's Startup School at Stanford brought together about 600 people from across the country and in some cases the world to learn what it takes to take a business from zero to profit without going broke. Om and I both attended and talked about the event and the current environment for new startups in this week's PodSession.
The event was organized by Y Combinator, a venture firm focused on seed funding for geeky projects. Students in the Y Combinator program can try startup life for three months with some seed funding and see if their ideas and execution are good enough for the next round of funding and clients. I interpret the venture firm's goals as "drop out of college and flip your company to Yahoo!" based on partner Paul Graham's past essays.
The day featured half-hour speeches from successful founders, industry observations from Om Malik and Tim O'Reilly, and advertisements from lawyers, venture capitalists. Google had a 30-minute recruiting pitch complete with multiple slides on their benefits package and some press release highlights.
Paul Graham's lessons for startups speech is available on his site. I learned more about Flickr's early marketing from Caterina, including fanatical involvement in message boards, making introductions within the community based on profile data, and rewarding people for their viral marketing efforts. Joshua Schachter of del.icio.us talked about building projects in your spare time and waiting for things to take off before leaving your well-paying job. Om talked about companies and products that caused "behavior change" such as the iPod or online e-mail and change the way we live our lives.
This week's PodSession, Startup School, is 24 minutes in length, a 11 MB download.
