I LOVE my Canon 5D MK II. Check out this video I filmed with it last week at the LeWeb Conference. Make sure you watch it in HD (might buffer if you don’t have a good connection, it’s best if you open the video on its own URL and make sure the HD button is clicked).

This was filmed using nothing but the Canon with 17-40mm F4.0 lens. Total cost? About $3,800. No extra gadgets other than I use a $150 Bogen/Manfrotto monopod to steady the camera.

YouTube’s video quality is spectacular.

How big a deal is this? Well, Mike Arrington and I interviewed Chad Hurley, cofounder of YouTube, and at the end of the interview we talk about the impact that HD will have (you can see Mike Arrington’s Techcrunch post and video here). Here’s that interview (here’s the URL for that video, watch it on its own page for best quality):

The thing is, for “pro” quality now you only need a $4,000 system but you can get pretty damn good quality with a Kodak Zi8 or a FlipCam. The Kodak works great at 720p quality and OK at 1080p. But those cameras are only $200. The fact that you can do decent HD for $200 and distribute those videos for free is a revolution.

Welcome to the age of HD video. Where’s yours?

One problem with this HD age is you can really see when I haven’t done something technically well. For instance the Chad Hurley video is slightly not focused properly (the 5D isn’t easy to focus in video mode). But the fact that I can do this at all is incredible. Thanks, too, to LeWeb for putting in high speed lines so I could upload these two videos from the conference floor.

When Rob La Gesse, Graham Weston, and Lew Moorman hired me at Rackspace (the world’s largest web hosting company) they said they expected me to challenge them to do new things. I haven’t done enough of that, but last week sitting in the audience listening to Twitter’s API Chief Ryan Sarver got me excited and that’s what this post is about (I haven’t talked with anyone inside Rackspace about this yet, so this is NOT something Rackspace is officially doing — it’s just something I would love to see them do and, because Twitter is a public service I figured it’d be interesting to have a conversation about this idea out in public rather than try to keep it behind closed doors).

What did he do? He announced that Twitter was going to make the real time firehose available to everyone (you can watch video of his speech here). That was the biggest news at LeWeb, even though most people didn’t understand the impact.

What is the firehose feed? It’s what businesses like Seesmic, Tweetdeck, Peoplebrowsr, and FriendFeed get from Twitter: it is the full stream of Tweets (more than 10gigs a day). The problem is that to deal with it you need to be expert at dealing with large, changing databases, and if your business gets popular you must be able to build scalable databases that can serve out millions of queries per day. That’s beyond most developer’s skill sets, but that shows there’s opportunity for new kinds of businesses to build on top of Twitter. Which is why I wrote this note.

I’ve been watching a new kind of business evolve thanks to Twitter’s real time stream. One example is StockTwits. A few months back I interviewed Howard Lindzon, founder. The problem is that he had to hire developers, setup servers, and all that. It cost him a lot more to start that business than $100 and took a lot longer than 15 minutes.

Let’s say you have a brain fart in the middle of the night like I did last night. I’d love to start a micro business that tracks discussion of Half Moon Bay Restaurants. I’d love to watch for Tweets that are actually happening inside and around our few restaurants. But that kind of business isn’t that profitable, so I can’t afford to hire a developer and raise investments to make it happen.

But what if I could go to Rackspace Cloud, click “start a Twitter business” and start filling out a wizard. Something like:

Track Tweets from these locations: [ ] < -Import a list of restaurant locations, or hand enter them.
Track Tweets with these keywords: [ ] <-Import a list of restaurant names, like Pasta Moon, plus a few other keywords, like "restaurant" or "eating."
Join these Tweets with APIs from these services [ ] <-Here's how the business will make money, we'll get people to reserve tables with OpenTable, which will get us a kickback.
Track sentiment using these phrases: [ ] <- Things like "service sucks" or "great meal" or "having fun" or other phrases you come up with.
Build page using this design [ ] <- Give me some choices of how the page will look, and give me some options for changing colors, background image, header images, etc.
Display on our directory [ ] <- Of course we'll have a directory, wouldn't we? Something like an app store where you can add other people's Twitter apps to your page, with revenue sharing, of course!

I'm sure there's lots of other details such a business will need to enter too, but this gives you a taste of what a new Twitter business could do and how easy it could be to build such a business.

Now what would Rackspace need to implement to make 15-minute $100 microbusinesses happen?

1. A clonable database of the Twitter firehose feed. After all, if you have a brainstorm about a new business you want to see results immediately upon starting up your business and you'd want to iterate it and see what the effects of changing various things are. In the example above, for instance, maybe if you are too strict with the geolocations you won't get any results, so you'll have to be looser about what your page will accept. You'll need to see the results on a few months' worth of data and you'll need to see your changes in real time. If you have to wait a few days to see results you'll lose interest and won't finish building a micro business like this.

2. Needs to be "cloud computing" simple. If you go to the Rackspace Cloud you can start up a new server in minutes just by entering in your credit card data. It's simple and anyone can do it. Same needs to be here.

3. Needs to have an upgrade path. If 100,000 of these simple businesses get created a few will get popular, like StockTwits has. That's when you will need to hire real developers to add some other features and build out iPhone apps and all the rest of the things a modern business needs now.

4. Needs to have Rackspace's famous service. Run into trouble, or need some help building something out, even at 2 a.m. on a Saturday? Gotta have the same business service Rackspace is famous for.

What kinds of things will Rackspace need from Twitter to make this happen (if it happens at all)?

1. A decent service agreement that will let Rackspace host Twitter's firehose real time stream and clone it many, many times. Also, needs to have a way to keep a database of several months' worth of Tweets (I'd prefer to have a database of all Tweets, but I understand that Twitter doesn't want to enable shadow Twitters or real-time search engines).

2. Access to all the metadata around Tweets. The real value in Twitter is no longer just the Tweets themselves, but new businesses will need access to what lists people are on, how many times the tweet was retweeted, geolocation data, and more.

3. If Twitter comes out with a "supertweet" (a new display surface/metadata payload that gets included with each Tweet) then we'll need access to that and we'll need a clean revenue-sharing agreement for any ads or affiliate networks placed on this supertweet.

4. Need an easy to understand, and affordable, licensing agreement for the firehose feed.

Anyway, what do you think of this idea? Would you use it if it were built? What other kinds of things would you like to do with Twitter's real-time firehose feed?