Mike Arrington notes that Google is walling off its news garden and keeping other services from spidering it.

Mike is right to point this out as hypocrisy. Google is making money off of other people’s work and wants to have some exclusivity.

Imagine if they did this with Blogger. To tell you the truth I’m very shocked Google hasn’t behaved like this earlier.

Here’s why walled gardens are important to companies (and why we hate them).

Let’s say we were going to develop a competitor to Google News or TechMeme.

Now, I’ve been reading quite a few feeds. Probably about 1/4 as many as the Google News algorithm, but enough to understand how to build a competitor.

There are a few stages to figuring out what is important news.

1) Parse the various parts of a post and put it into buckets. For instance, look at this item about GigaOm getting a new CEO. There’s the headline and the text. Then let’s look at the component parts. There’s who wrote it “CEO Smack”. There’s the subject matter “GigaOm” and “new COO.” Then there’s the body where you probably can find a variety of other important terms. “San Francisco” “Blog” “Om Malik” “Growth” “Paul Walborsky” “sales, operations, conferences” “Hercules Technology Growth Capital” and a link to TechCrunch.
2) Look at the inbound links. In this case there are none, but this is a ranking mechanism. More links means more important news.
3) Look at the comments. Are there any? How fast were they received? How many?
4) Use a human news judgment on the source of the news (is CEO Smack more or less important than, say, News.com, or TechCrunch?)
5) Look at how many times this story has been written about on blogs elsewhere in the past few hours.
6) Look for “news” verbs like “just released” or “new” or “beta” or “exclusive.”
7) Look for “news nouns” like “Microsoft” “Google” “Apple” “Facebook.” (Or whatever company you wanted to track).

Are there other things to study? Not many.

So, how do you get a better display of news than anyone else?

1) Get readers to vote. AKA Digg.
2) Get readers to add comments. AKA new Google stuff.
3) Track readers’ clicking behavior. If you know everyone is clicking on Paris Hilton stories instead of some new software for Facebook, wouldn’t that be valuable to you?
4) Get readers to send in their own news.

Are there many other ways to get a better display?

Now, how do you keep a better display? Easy. Wall it off. Keep your competitors from using the stuff that makes your display special. That way they’ll have to figure out a way to get it on their own rather than just spidering your display results and using that as a bootstrap to build upon.

Personally, the more I look at it, the more I understand what Google’s doing.

How about you?

The reason we hate them? Exactly because of that reason. We can’t bootstrap off of them and build something better.

Oh, and we don’t like it that companies are making profits off of our work. It +is+ our work that is building TechMeme and Google News, isn’t it? Yes. So why not share the profits back with the people who are helping make the system special?

Evil or not? That is the question.

Dare Obasanjo says all he sees on his Facebook news feed is my stuff.

I’m sorry.

I get even with Dare, though. I added his blog post to my news feed so the rest of my 4,600 friends will see Dare’s post. 🙂

UPDATE: I’m sitting next to Ryan Stewart (Adobe evangelist) at the Ignite Seattle event. He just signed into Facebook and only two of my items are on his Facebook news feed. So, maybe Dare needs better friends. Heheh.

UPDATE2: I changed many of my Facebook settings to only show most of my items to my friends. That should make it less “noisy” for other people.

[podtech content=http://media1.podtech.net/media/2007/08/PID_012136/Podtech_TomRolander.flv&postURL=http://www.podtech.net/home/3801/the-rest-of-the-story-how-bill-gates-beat-gary-kildall-in-os-war-part-1 &totalTime=3575000&breadcrumb=44609486ca1a4f08927df317b2a5c221]

You’ve heard the stories about how Microsoft and Bill Gates got the operating system business with IBM and how Gary Kildall and Digital Research lost the deal.

But I’ve always wondered about why Gary was out flying that day.

So when I got a chance to sit down with Gary Kildall’s best friend and FLYING PARTNER that day I jumped at the chance. That’s Tom Rolander who held a key role inside Digital Research (the folks who made CPM which, back before the IBM PC, was one of the most popular personal computer OS of the day — my dad had a CPM card for our Apple IIs so we could run software designed for it).

This is still the biggest business story in the tech industry. It is one that business school students will study for a long time.

It’s a story of arrogance. Legal misjudgments. Misjudging the players. And an abiding deep friendship that comes through.

If there’s a piece of video that will probably outlast me this is it.

Actually there’s four pieces. The first hour you meet Tom and hear the story of when IBM came to visit. That’s the interview that was put up today.

The second piece takes us to a restaurant where Tom tells lots of fun early industry stories.

The third piece takes us on a tour of Pacific Grove which is where Digital Research was located. We take you to the house where IBM visited Digital Research.

The fourth piece is where Tom introduces you to his new company, Crossloop, which is developing software to enable you to help others with their computer problems.

One interesting thing you’ll learn?

Bill Gates and Microsoft didn’t want the operating system business and sent IBM down to Digital Research.

Oh, and thank you to Tom and Mrinal Desai of Crossloop. He wrote me a few weeks ago on Facebook and said that his new boss was the one flying with Gary Kildall that fateful day.

Have you ever blown a multi-hundred-billion-dollar business deal? Me neither.

But now you can say you’ve met one of the guys who can say that.

Another thing you’ll learn? Why we all owe a debt of gratitude to Gary Kildall for the modern operating system architecture.

There’s a lot more, but it’s better just to watch the videos. Hope you find this as interesting as I do.

Oh, and if someone can post these to Gary Kildall’s wikipedia page, I’d be most grateful.

The beginning of the video brings introductions — I started filming the minute we got out of my car (you meet Buzz Bruggeman, CEO of ActiveWords, and Patrick Scoble, my son, and Mrinal). I think this is interesting stuff so we don’t edit it. The meat of the story starts up at about 16 minutes into the video, but I think you’ll find the rest of the conversation interesting. It’s one of the most interesting conversations I’ve ever been a part of.