I think we need a new blog search engine and a new blog community service. The ones we have now just really suck. Sorry to Google BlogSearch. Sorry to Technorati. Sorry even to my favorite TechMeme, TailRank, or to Digg. No, Slashdot didn’t do a good job either.
Here’s your assignment: find me the best blogs from Demo Fall 2006.
Within 10 minutes.
I just spent an hour at it and it’s amazingly difficult to pick out who has the best blogs on a single topic.
Let’s start on blog search.
Search Google’s Blog Search service (did you even know they had a blog search service?) for the words “Demo 2006.” Now, you get a nice list of all sorts of blogs. But, is there anything on any of these posts that tells you one post is better than another? No. You just get a freaking long list and there’s even some noise in this result set. See “Military Recruiting Station Demo.”
Which brings me to a way Google could improve. Why doesn’t it ask me “are you looking for recruiting Demos or are you looking for DemoFall 2006?” It could easily constrain the search down a bit. Search on “DemoFall 2006” with quotes, for instance, and you get a much cleaner resultset.
But, what I really want is a human touch. I want a wiki! Is there one? If there were we could all link to stories about Demo and we could add editorial commentary onto those links.
Lacking a wiki, though, we need a hierarchy.
So, let’s head over to Technorati. I like the UI on Technorati a lot more than Google Blog Search. It’s amazing that Google hasn’t invested any time on that yet. It’s why I still go back to Technorati.
Oh, and I just tried its “authority” ranking. Here, compare. “DemoFall 2006” with a lot of authority. That brings back 61 results. Change the drop down setting to “Any Authority” and you get 411 results.
The problem is, authority is based on global links, not links about a specific post from DemoFall. So, it’s really not as useful as it might seem. In practice, though, it works better than you might expect since people who have built a good reputation and earned a lot of links over time do tend to have the better blog posts.
But, now, let’s change the rules. Let’s find the best products, as chosen from the bloggers? Finding that is nearly impossible. You have to read every blog.
I’d like to be able to read all the blog posts about the new Zing, for instance, and have a UI that would let me read “positive reviews” and “negative reviews.”
None of the news sources or blog search engines give you anything like that.
But, how do you find the best posts from an event? How do you figure out what was important that happened there? What would you like to know about what happened at Demo yesterday and today? Using today’s tools just totally leaves me unsatisfied.
Scobleizer 
Robert, Did you try Ask.com (Don’t they own Bloglines?)
I don’t know if it’s any better, but they sure are quick to pick up new content, so sorting by most recent gets good results, including this post currently at number 1 most recent.
http://www.ask.com/blogsearch?q=%E2%80%9CDemo+2006.%E2%80%9D&t=a&s=f&bql=&qsrc=2103&rpp=10
LikeLike
Robert, Did you try Ask.com (Don’t they own Bloglines?)
I don’t know if it’s any better, but they sure are quick to pick up new content, so sorting by most recent gets good results, including this post currently at number 1 most recent.
http://www.ask.com/blogsearch?q=%E2%80%9CDemo+2006.%E2%80%9D&t=a&s=f&bql=&qsrc=2103&rpp=10
LikeLike
I hadn’t seen Ask for a long time. Damn, it rocks! Thanks for that.
LikeLike
One of the challenges is that what you find to be the best may not be what someone else believes are the best. For example, with DEMO in particular, my blog (Pardon the Disruption) may be a good source if you are curious about web apps, but if you’re into hardware or security mine probably sucks because I don’t understand those areas as well and can’t explain them the same way I can with web apps.
LikeLike
I hadn’t seen Ask for a long time. Damn, it rocks! Thanks for that.
LikeLike
One of the challenges is that what you find to be the best may not be what someone else believes are the best. For example, with DEMO in particular, my blog (Pardon the Disruption) may be a good source if you are curious about web apps, but if you’re into hardware or security mine probably sucks because I don’t understand those areas as well and can’t explain them the same way I can with web apps.
LikeLike
Try blogdigger.com
blogdigger
LikeLike
Try blogdigger.com
blogdigger
LikeLike
Okay. I just tried blogdigger.com. At the top of the results list are huge google ads. They take up 1/3 to 1/2 of the visible results window before they show you any of the real results. That just lost my interest in the site.
LikeLike
Okay. I just tried blogdigger.com. At the top of the results list are huge google ads. They take up 1/3 to 1/2 of the visible results window before they show you any of the real results. That just lost my interest in the site.
LikeLike
Carson: thanks for the plug!
Jason H: I apologize that the ads were distracting; we try to set them so that they are distinguishable from the main content but also there for our users. It’s tricky, because sometimes they do provide useful additional information, but sometimes do not, and we have to try to find a balance. I’d like to follow up with you to get your thoughts on how we could do it better.
Scoble: Regarding postive/negative comments about things, try Opinmind, they’ve got a cool sentiment search engine that is powered by blogs.
Regarding the other issues that you bring up, these are all important; the problem is that each one of these issues is essentially a seperate problem to be solved. Finding the best products is different than finding all the posts that mention specific keywords, both of which are different detecting sentiment. I think some of them are best handled manually, for example, the best posts from DEMO would probably be best compiled by someone from the conference who knows in advance who is there, has collected a list of their RSS feeds, and reads them in an aggregator and re-blogs or links to the best ones (or the wiki scenario you suggest).
Regarding the mixture of authority on specific subjects and cleaning noise out of the result set, we’re working on a solution to this that I think will help solve the problem. I’d be happy to give you more details, if you’re interested.
LikeLike
Carson: thanks for the plug!
Jason H: I apologize that the ads were distracting; we try to set them so that they are distinguishable from the main content but also there for our users. It’s tricky, because sometimes they do provide useful additional information, but sometimes do not, and we have to try to find a balance. I’d like to follow up with you to get your thoughts on how we could do it better.
Scoble: Regarding postive/negative comments about things, try Opinmind, they’ve got a cool sentiment search engine that is powered by blogs.
Regarding the other issues that you bring up, these are all important; the problem is that each one of these issues is essentially a seperate problem to be solved. Finding the best products is different than finding all the posts that mention specific keywords, both of which are different detecting sentiment. I think some of them are best handled manually, for example, the best posts from DEMO would probably be best compiled by someone from the conference who knows in advance who is there, has collected a list of their RSS feeds, and reads them in an aggregator and re-blogs or links to the best ones (or the wiki scenario you suggest).
Regarding the mixture of authority on specific subjects and cleaning noise out of the result set, we’re working on a solution to this that I think will help solve the problem. I’d be happy to give you more details, if you’re interested.
LikeLike
for best results I tried feedster, just used demofall 2006, w/o quotes, check it out, they all seem fresh; http://www.feedster.com/search/demofall%202006
LikeLike
for best results I tried feedster, just used demofall 2006, w/o quotes, check it out, they all seem fresh; http://www.feedster.com/search/demofall%202006
LikeLike
Robert, you just described a good use case for microformats (near term) and the Semantic Web (longer term). Explicit data like event descriptions and positive/negative reviews (of events, blog posts and products) can go in hEvent and hReview microformat HTML. If the material was aggregated not just as “a bunch of content with a little bit of metadata” but a “a bunch of interlinked data” it could subsequently queried (GRDDL, RDF & SPARQL are in the frame for this).
The Wiki idea makes sense for a one-off event, but given that your main requirements could be fulfilled from the (decentralized) blogosphere, it should really be redundant.
Not sure why you suggest a hierarchy – information about the event and the products isn’t hierarchical (nor is the Web).
LikeLike
Robert, you just described a good use case for microformats (near term) and the Semantic Web (longer term). Explicit data like event descriptions and positive/negative reviews (of events, blog posts and products) can go in hEvent and hReview microformat HTML. If the material was aggregated not just as “a bunch of content with a little bit of metadata” but a “a bunch of interlinked data” it could subsequently queried (GRDDL, RDF & SPARQL are in the frame for this).
The Wiki idea makes sense for a one-off event, but given that your main requirements could be fulfilled from the (decentralized) blogosphere, it should really be redundant.
Not sure why you suggest a hierarchy – information about the event and the products isn’t hierarchical (nor is the Web).
LikeLike
Greg: I appreaciate the need to differentiate the google ads from the content. However, the format of the Ads is distracting and kills too much real estate before thew user gets to content they are really seeking.
I would recommend using a vertically smaller and wider Ad at the top of the search results screen. For example, a Leaderboard (728×90) or a Banner (468×60) Ad would probably work perfectly and not be too distracting from the search result content.
You could even change the color of the Ad background if you really wanted to make it standout, but personally I would have the Ad backgroupnd match the site background.
Feel free to email me at [systemsengineer {at} yahoo {dot} com] regarding the Ads or any other site feedback for blogdigger, if you would like. I would be happy to give any feedback you would like from me.
LikeLike
Greg: I appreaciate the need to differentiate the google ads from the content. However, the format of the Ads is distracting and kills too much real estate before thew user gets to content they are really seeking.
I would recommend using a vertically smaller and wider Ad at the top of the search results screen. For example, a Leaderboard (728×90) or a Banner (468×60) Ad would probably work perfectly and not be too distracting from the search result content.
You could even change the color of the Ad background if you really wanted to make it standout, but personally I would have the Ad backgroupnd match the site background.
Feel free to email me at [systemsengineer {at} yahoo {dot} com] regarding the Ads or any other site feedback for blogdigger, if you would like. I would be happy to give any feedback you would like from me.
LikeLike
I think that part of the problem is that “demo” is a common word.
This was the reason why I hated the scripting language “expect”. Very common word, very hard to direct your search towards it.
LikeLike
I think that part of the problem is that “demo” is a common word.
This was the reason why I hated the scripting language “expect”. Very common word, very hard to direct your search towards it.
LikeLike
just go to Cnet blogs or ZDnet blogs and do a (internal) site search
LikeLike
just go to Cnet blogs or ZDnet blogs and do a (internal) site search
LikeLike
#12 has a point. Often it’s easier just to find a few quality bloggers and listen to them. Sure you’ll miss some nuggets but how much of your life do you want to spend looking for them, until better tools are developed?
LikeLike
#12 has a point. Often it’s easier just to find a few quality bloggers and listen to them. Sure you’ll miss some nuggets but how much of your life do you want to spend looking for them, until better tools are developed?
LikeLike
Yes, there are some improvements to be made for searching the blogosphere. I usually use a combo of Technorati, Google blog search, Ask Bloglines, Blogpulse, and Icerocket. As with normal searches on the internet, I’d often need to use several search engines, I don’t expect that to be too different in the blogosphere.
But, what I do wish for is a blog search engine that will also crawl the comments of blog posts — I want to search blog comments — I’m discovering some very valuable content in the comment section, but get the impression that blog search engines don’t crawl that.
LikeLike
Yes, there are some improvements to be made for searching the blogosphere. I usually use a combo of Technorati, Google blog search, Ask Bloglines, Blogpulse, and Icerocket. As with normal searches on the internet, I’d often need to use several search engines, I don’t expect that to be too different in the blogosphere.
But, what I do wish for is a blog search engine that will also crawl the comments of blog posts — I want to search blog comments — I’m discovering some very valuable content in the comment section, but get the impression that blog search engines don’t crawl that.
LikeLike