Why does searching for Twitter lists suck so much?

Uncategorized 5 Minutes

Let’s say you want to find a Twitter list on a topic like, say, “tech.” Where would you go?

For me there are two sites that cover Twitter lists:

1. Listorious.
2. Tlists.

Now Listorious’ home page uses tags to build a directory. But let’s ignore the directory for a while. I’m digging into search because I’ve seen a couple of different apps coming later this summer that are using search to bring new lists and the entrepreneurs were complaining about the quality of the search results.

Even doing something simple, like searches for “technology” or “tech” or “startups” shows you that the search results suck. Let’s dig in and explore some ways to improve the searches, shall we?

First, what would be a GREAT list? My biases (I’ve looked at 10s of thousands of lists in the past few months):

1. A list that’s complete. Basically, a list with 500 sources on it beats one with 10 by a country mile, especially if all those Twitter accounts are actually on topic.
2. A list that’s popular. Whenever I’ve found a list that’s popular it almost always is popular for a reason. So, a list with 6,000 followers is generally better than one that only has 100 followers. That’s not always true, Mashable’s lists, for instance, suck, but they are very popular, but generally that’s true and even in Mashable’s case there’s a case for why they are so popular and better than “joe smith’s tech lists”.
3. A list that’s created by a credible and authoritative person or brand. So, one created by the New York Times is better than one that’s created by someone you never heard of.
4. A list that’s curated every day and kept clean. Sometimes Twitter accounts turn out to not cover too much the topic being discussed. It’s pretty easy for a human to look at a list and figure out if it’s clean or not, but it’s hard for machines to do that. Generally this is why popular lists are better than non-popular ones. Humans don’t follow lists full of spammers.

There are other, harder ways, to figure out whether a list is good. If I could code, I’d be looking at how many RT’s or favorites a list’s members gets every day and helping new lists get to the top that way (I wish I had that data, because I’d create new lists that are made up of only the most retweeted people.

But, generally speaking, those four things are great ways to tell quality of lists.

So, now, why does the search features on Listorious and Tlists suck so bad?

Let’s go to Listorious and search for “tech.” I’m using “tech” because it’s something I know a lot about and I know what the best lists are.

First five results:

1. TechNews from Tech Introvert. My score? Low quality. Why? From a guy without high authority and credibility scores in industry. Only has 45 sources on it. Only has 3 followers. Why the hell is it #1?
2. GirlsintechLA. Even worse than #1. Only 20 members on it, is hardly authoritative, nor complete, and only has 4 followers.
3. QR-Code by Nonprofi Tech. Getting even worse.
4. Tech News Brands from me. OK, this one is good, although, to tell you the truth, this is NOT even my best list for someone searching for tech. My “most-influential-in-tech” list has more followers and is more credible.
5. tech by Peter Urbanski. Now at least this one is interesting, is from someone who has been in the tech scene a long time, has 453 members, but only 11 followers.

Now, these belong in the result set SOMEWHERE but NOT in the top five. There are far better lists out there. What’s weird is Listorious’ engine mostly NAILS the results if you click on their tag interface (with one exception, because the Mashable team list is so popular it is #1, which shows the downside of using popularity as the only score — that list isn’t even Mashable’s best list). Compare that for “tech” and you’ll see that the lists you’ll find are FAR more authoritative and credible. Why is this? Why does the search interface suck so bad? Keep in mind, tags are great for areas like tech where there’s lots of results, but what if you are searching for something that hasn’t had a lot of tagging behavior, like plumbing? Also, keep in mind that tags can be gamed too. So, they are less resistant to spammers and commercial interests or people with large teams of people who can tag a list with tons of tags.

Let’s go over to TLists and see if they do any better.

One thing you’ll notice first is that Tlists ONLY bets on search. No tagging here.

Let’s search on “Tech” and see what the first results are.

One thing immediately is that Tlists’ system is actually trying to both understand and display what people on each lists are tweeting about. It also shows how many Tweets per day are coming from that list, how many members each list has, and how many followers each list has. That’s a MUCH better display right away. But the relevancy of the lists still sucks, especially when you compare it to Listorious’ tag results. Let’s dig in:

1. Tech & Science. This is a super list, which is a great idea. They bring in the most listed Tweeters on 450 lists. This list isn’t very popular, only has 79 followers, and isn’t made by someone very credible on its face, but when you look into it this list was algorithmically produced and is very good. Much better than Listorious’ search efforts. Someone finding this list would be well served.
2. Tech Journalists by Huffington Post. Very credible source. Not very popular, though, only has 46 followers. Not very complete. 26 members. For comparison, my Tech News People list has 499 members and 3,143 followers. On every measure my list is better than the Huffington Post one. But why is Huffington Post’s result here? Are they paying for a high result? We don’t know.
3. Tech Geeks Directory by Susan Elaine Cooper. OK, this is where the list goes south. Not from someone authoritative. Not very popular (only 35 followers). Not very complete (45 members). Why is this list even NEAR the top of search results? Can anyone tell me?
4. twist-callers by JT Keller. This list doesn’t deserve to be on one about tech. 42 members. 19 followers.
5. Technology News by The Job Guy. Another crappy list. 30 members. 6 followers.

Why does the search engine show so many crappy results? Why can’t anyone build a search engine for Twitter lists that works well? It doesn’t look like they are doing ANY relevancy ranking and, even, makes the engine smell like it’s getting paid off to put certain results at the top. Not good. What do you think? Have you tried any searches? Do your results match mine? (I’ve done a variety of searches on both engines and they all suck like the result for “tech” which SHOULD bring back some awesome lists, like Listorious’ tag results do).

Published by Robert Scoble

I help entrepreneurs build their technology business' story, help with getting ready for investors, with other launch plans, and many other strategic things that can help your new startup. Call to talk: +1-425-205-1921 (text first).

Published

26 thoughts on “Why does searching for Twitter lists suck so much?”

  2. Tunkrank is a pretty clever way of ranking the authority of a twitter user (http://tunkrank.com/), but so far as I know nobody has applied it to lists. A simple (and probably not-quite-right) approach might be if someone ranked every list by the sum of the TunkRank of each of the sources/contributors to the list…..

    Like

  3. That probably would be a better way than what Tlists and Listorious are doing, which seems to be no relevancy/authority/credibility/completeness/popularity rankings at all.

    Like

  4. I didn't agree with your criteria for quality lists, but I do wish they were easier to not only find, but search within. Manually pulling in tweets from the api and regex'ing them for matching patterns skipping white spaces took a little effort to play nice on both server and client side.

    Like

    1. We at inagist.com have tried to work on one objective of making lists around a topic, to find the most relevant and talked about tweets on a topic. We have tried to do a hybrid of curating a list and then running a trend analysis on the lists to identify the tweets with most conversations. We have done it for about 20 different topics in our channels. We saw a pretty good result set of trending conversations real time on these topics. You can take a look at worldbiz.inagist.com for business related information as a sample. We are still refining this to eliminate irrelevant content.

      Like

  7. We've focused our search energies on helping you find people/accounts rather than lists. For your example our search results for “tech” accounts (which is our default view) is pretty strong http://listorious.com/search/people?q=tech&sear… — and congrats on being on the first page of results by the way. From the people search, we link to the tag pages (e.g. http://listorious.com/tags/technology) which as you point out have strong list results.The list search page you linked to is a a secondary search mode that we haven't gotten around to updating yet. We'll work it into our main search or revise it shortly.

    Like

  8. True, as Mashable's results show (their “team” list is popular, but not very good). But we have to start SOMEWHERE and judging by popularity brings a lot better quality list today than not judging anything at all.

    Like

  9. I'm honored to be on the first page of results, but, seriously, that result set is hardly the best Twitterers in tech, myself included. But that's a different search problem anyway. Oh, and why do you say “people” when there are so many brands there? I wish you could separate brands into a separate search result page.

    Like

  13. I'd love to separate brands and people! We just say people for simplicity because “Account Search” sounds strange. Even Twitter's top bar says “Find People”.Might be a cool feature for Twitter to allow people to declare their account as person, brand, media, government, etc. Or if Twitter has no plans to do it, maybe we will.

    Like

  14. They're given room for such things in the metablock (although they're tweet based) Maybe that's their upgrade path, community adopted norms.

    Like

  16. Nice article, although I am not sure I agree with more is better idea. Yeah 10 sources seems a little small, however list with large numbers are not always better especially the more specific the category is. Many times I find that list with a lot of sources tend to wander away from the category they are listing.

    Like

  17. In answer to Mark's question – Yes search engines do index lists – just tried Googling one of mine that has no competition and the results filled the page. Lists ought to be a good way to manage streams from those you follow but unfortunately I've reached the limit – 20 just isn't enough. What I have found is that as soon as a list is created followers arrive for the subject in question, which in the main appear to be marketing something so are they just looking to get added to as many lists as possible, rather than following for the content?However, don't rely on any of my lists to find what your looking for – yet!What I would really like is the ability to list my tweets and RTs according to the content, which varies widely. Obviously that could be done with different profiles, although that is against the rules. The ideal would be that individual tweets could be created under specific sub-profiles according to content and others could choose to follow either just one or as many of the sub-profiles as they wish, which they could also then add to lists as they deem appropriate. This would then make gregory's job of creating relevant lists of lists that much easier and indeed it would improve the chances of the content in those lists actually being relevant a much greater percentage of the time.

    Like

  18. Twitter lists are still ridiculously underrated. A fantastic curating system that I hope more people continue to jump on. I don't know about y'all but it certainly allows me to de-clutter.

    Like

  19. That is both wrong and true. Figuring out when it's wrong and true is why you need many different variables to study. But, GENERALLY more sources is better than fewer in my experience.

    Like

  20. I'm not really sure what you mean, Robert. I usually agree with you on everything. But I just checked Tlists.com com for some pretty specific interests of mine: neuroscience, boston dining, clinical trials, and autism and I got back perfect matches. What queries did you use?

    Like

  23. Hi Scoble, thoughtful piece, as usual.You may take a look at Listiki (http://listiki.com), an Openfund startup publicly launched a few days ago.It's not focused solely on Twitter, but list creation in general and we believe that it delivers what it takes for a list to stand out: collaboration (essentially a wiki) and a different paradigm for editing (plain vanilla drag and drop).I'd love to hear your thoughts and comments on Listiki, thanx a lot.

    Like

  25. This doesn't strike me as a viable business idea – providing human-curated Twitter lists for people like @scobleizer who expect to use the service for free.

    Like

Comments are closed.