Paul Montgomery’s post “Clone the Memeorandum API” is sure to get onto Memeorandum in a few hours. The thing is he thinks the algorithm that Gabe Rivera developed for Memeorandum is fairly simplistic.
Oh, Paul, you sound a bit like Tim Bray tonight!
If only developing quality software was so simplistic. You might not know Gabe, but I do. He’s a former compiler writer. Used to work for Intel. He’s one smart cookie. His algorithms might be simple to explain, but they aren’t so simple to implement. For instance, do you know how he filters out noise? What happens if I link to a political blog? Why doesn’t that show up on Memeorandum? Ever consider that one? I’ve talked with Gabe about it. The algorithm he wrote takes care of it.
Doing Memeorandum took him more than a year. Are you a former compiler writer? Expect it to take longer then and not work as well.
I want to see a “personal Memeorandum” too, though, just to be clear. But thinking that it’s a simple problem just denigrates Gabe who took more than a year off of work — without pay — to implement Memeorandum.
The other thing you need to know is that opening API’s increases a systems’ POTENTIAL insecurity quite a bit. So doing so must be done with great thought. Lots of threat modeling. Lots of testing. Not easy stuff. I bet that turning on an API will require Gabe to do even more work than he’s already done.
Scobleizer 
Well we have to ackowledge Gabe’s fine work, thanks Paul for speaking out on my behalf of questioning the need for this API, I agree Robert that it is hard as freaking hell to do what he did. Even more so that he isn’t getting paid by it.
But you have to think of it like this, he is working by himself, why isn’t the GYM finding him and picking him up so he can work in a team, with equally talented people.
Hundred of man hours seems like nothing when you have a team of 10 people.
Gabe is definitly a special guy, and what he did to the blogsphere should be ackowledged and implemented into search technologies.
LikeLike
Well we have to ackowledge Gabe’s fine work, thanks Paul for speaking out on my behalf of questioning the need for this API, I agree Robert that it is hard as freaking hell to do what he did. Even more so that he isn’t getting paid by it.
But you have to think of it like this, he is working by himself, why isn’t the GYM finding him and picking him up so he can work in a team, with equally talented people.
Hundred of man hours seems like nothing when you have a team of 10 people.
Gabe is definitly a special guy, and what he did to the blogsphere should be ackowledged and implemented into search technologies.
LikeLike
First you attack Tim Bray and now this?
What’s with all the vitriol?
LikeLike
First you attack Tim Bray and now this?
What’s with all the vitriol?
LikeLike
Memeorandum works better than anything I’ve seen yet for very quickly tracking the tech news buzz that’s out there. As it breaks it rolls into Memeorandum. Although I know that it is at it’s core an algorithm. I’ve got to believe that on some level at some point human editing comes into play — perhaps not, but it’s damn impressive if that’s the case.
Either way I’ve yet to see anything that tracks what I’m interested in in one place as efficiently. If someone thinks they can copy it and build something just as good I’d be happy to try it out — but Memeorandum does have a huge advantage being first to market with this.
LikeLike
Memeorandum works better than anything I’ve seen yet for very quickly tracking the tech news buzz that’s out there. As it breaks it rolls into Memeorandum. Although I know that it is at it’s core an algorithm. I’ve got to believe that on some level at some point human editing comes into play — perhaps not, but it’s damn impressive if that’s the case.
Either way I’ve yet to see anything that tracks what I’m interested in in one place as efficiently. If someone thinks they can copy it and build something just as good I’d be happy to try it out — but Memeorandum does have a huge advantage being first to market with this.
LikeLike
Isn’t that always the way though with the great performmers, atheletes, etc? Through hard work, practice, brilliance, and thought they make it look simple and easy.
Ever watch a great basketball player like Larry Bird or Michael Jordan? They make it look effortless as they set up their team mates for the easy basket. A really great software designer makes software that has an intuitive interface that takes care of all of the mundane tasks that would otherwise need to be done by the programmer implementing the API.
Gabe is one of those great ones and has done an excellent job. Thanks Gabe and Robert for recognizing his work.
LikeLike
Isn’t that always the way though with the great performmers, atheletes, etc? Through hard work, practice, brilliance, and thought they make it look simple and easy.
Ever watch a great basketball player like Larry Bird or Michael Jordan? They make it look effortless as they set up their team mates for the easy basket. A really great software designer makes software that has an intuitive interface that takes care of all of the mundane tasks that would otherwise need to be done by the programmer implementing the API.
Gabe is one of those great ones and has done an excellent job. Thanks Gabe and Robert for recognizing his work.
LikeLike
I like this topic,I like this topic,I like this topic,I like this topic,really!!!!!!!!!!
LikeLike
I like this topic,I like this topic,I like this topic,I like this topic,really!!!!!!!!!!
LikeLike
here is better than http://www.blogger.com
LikeLike
here is better than http://www.blogger.com
LikeLike
asdfasfsafsad dosdf .com sadt
LikeLike
asdfasfsafsad dosdf .com sadt
LikeLike
Contrary to what Paul, and the other people who took “business math” think, an algorithm doesn’t have to implement complex, “high-level” mathematics to be complicated. Most genetic algorithms don’t use much math at all. Maybe multiplication or division to weight the probability of some mutations. It’s only when you try to cram a complex, non-linear problem, like PageRank, into a linear solution that you end up with lots and lots of square roots, summations, and many members of the greek alphabet.
The biggest question to be answered is “What would a Memeorandum API do?” What would it look like? What would the inputs and outputs be? And, most importantly, what would you DO with a Memeorandum API? Just create your own personal Memeorandum? Is that all?
LikeLike
Contrary to what Paul, and the other people who took “business math” think, an algorithm doesn’t have to implement complex, “high-level” mathematics to be complicated. Most genetic algorithms don’t use much math at all. Maybe multiplication or division to weight the probability of some mutations. It’s only when you try to cram a complex, non-linear problem, like PageRank, into a linear solution that you end up with lots and lots of square roots, summations, and many members of the greek alphabet.
The biggest question to be answered is “What would a Memeorandum API do?” What would it look like? What would the inputs and outputs be? And, most importantly, what would you DO with a Memeorandum API? Just create your own personal Memeorandum? Is that all?
LikeLike
The other thing you need to know is that opening API’s increases a systems’ POTENTIAL insecurity quite a bit. So doing so must be done with great thought.
Security by obscurity. How apt from a Microsoft employee.
LikeLike
The other thing you need to know is that opening API’s increases a systems’ POTENTIAL insecurity quite a bit. So doing so must be done with great thought.
Security by obscurity. How apt from a Microsoft employee.
LikeLike
algorithm, natural language :
for each crappy subscription I subscribe to
if (text contains “politics”) no follow
else (text contains “Scoble Ass Birtney Spears Google”) add one link to every ten posts
end for
LikeLike
algorithm, natural language :
for each crappy subscription I subscribe to
if (text contains “politics”) no follow
else (text contains “Scoble Ass Birtney Spears Google”) add one link to every ten posts
end for
LikeLike
Robert,
I’ve got no doubt at all that Gabe is a smart guy. However, there are other smart people in the world. One man year of work – even from a great developer – does not consititute a significant barrier to entry. In other words, if there is an appetite to replicate what Memeorandum does from a technical point of view, it would be easy to do.
As an aside, it’s interesting that you say Gabe is a former compiler writer. In my experience, compiler writers often find it difficult to deal with problems involving natural language (and other types of complex, ambiguous, error-rich, and generally “messy” data). I’m not sure why this is. It may be something to do with the fact that they’re used to wanting to formulate problems in terms of grammars that aren’t a good fit with natural language.
Having said all that, what I would say to developers (Open Source or otherwise) is that they should focus on innovating, not on copying other people’s ideas: that’s a lame thing to do…
LikeLike
Robert,
I’ve got no doubt at all that Gabe is a smart guy. However, there are other smart people in the world. One man year of work – even from a great developer – does not consititute a significant barrier to entry. In other words, if there is an appetite to replicate what Memeorandum does from a technical point of view, it would be easy to do.
As an aside, it’s interesting that you say Gabe is a former compiler writer. In my experience, compiler writers often find it difficult to deal with problems involving natural language (and other types of complex, ambiguous, error-rich, and generally “messy” data). I’m not sure why this is. It may be something to do with the fact that they’re used to wanting to formulate problems in terms of grammars that aren’t a good fit with natural language.
Having said all that, what I would say to developers (Open Source or otherwise) is that they should focus on innovating, not on copying other people’s ideas: that’s a lame thing to do…
LikeLike
Ok, so how do you square…open/clone the Goggle API with close/isolate the Memeorandum API? And “Security thru Obscurity” is so mid 1980s, geesh, even MFST has updated it to “Security via ‘Proper’ Reporting”.
I don’t see what all the emotional hot-flashing is about Memeorandum, just take a smug-egohead narrow A-list grouping of people/bloggers, play random weighting games, output random bits. If anything it only serves as a glaring example of the incestuous-linking nature of blogs, and it bothers me to no end that the original sourcing usually gets lost in a thicket of blog yabberings. Link link link, and then this other person link links to this and then another someone link link links to that; it becomes a giant game of ‘telephone’, where the meaning and context gets wholly lost. And tracking back the original sourcing thru the dense web forest is more effort than worth most of the time. Memeorandum is placing you in the middle of the maze, saying XYZ bloggers opinions are heavier than the originator of said material. I can’t stand Memeorandum, for real news searching, it’s totally flawed.
And then see, it will create MORE incestuous-linking, as bloggers will go, ‘Oh so that’s the hot topic’, and then link/trackback and comment further creating yet another vicious full-circle.
It’s politics page is a nice but grand failed effort, as heck, even the horrid Google News trumps Memeorandum totally. And it be but a pale comparison to say, BBC News, Roll Call or the Drudge Report. Humans will always trump algorithms.
Your feed blog rocked…problem is it takes time and effort, but like anything of sustaining value, eventually becomes professional, or it dies off.
LikeLike
Ok, so how do you square…open/clone the Goggle API with close/isolate the Memeorandum API? And “Security thru Obscurity” is so mid 1980s, geesh, even MFST has updated it to “Security via ‘Proper’ Reporting”.
I don’t see what all the emotional hot-flashing is about Memeorandum, just take a smug-egohead narrow A-list grouping of people/bloggers, play random weighting games, output random bits. If anything it only serves as a glaring example of the incestuous-linking nature of blogs, and it bothers me to no end that the original sourcing usually gets lost in a thicket of blog yabberings. Link link link, and then this other person link links to this and then another someone link link links to that; it becomes a giant game of ‘telephone’, where the meaning and context gets wholly lost. And tracking back the original sourcing thru the dense web forest is more effort than worth most of the time. Memeorandum is placing you in the middle of the maze, saying XYZ bloggers opinions are heavier than the originator of said material. I can’t stand Memeorandum, for real news searching, it’s totally flawed.
And then see, it will create MORE incestuous-linking, as bloggers will go, ‘Oh so that’s the hot topic’, and then link/trackback and comment further creating yet another vicious full-circle.
It’s politics page is a nice but grand failed effort, as heck, even the horrid Google News trumps Memeorandum totally. And it be but a pale comparison to say, BBC News, Roll Call or the Drudge Report. Humans will always trump algorithms.
Your feed blog rocked…problem is it takes time and effort, but like anything of sustaining value, eventually becomes professional, or it dies off.
LikeLike
Hello Robert – Your comments areas is a scary, dark and angry place – it’s like you carry all these angry lost souls on to each new post. How on earth do you put up with it each and every day? Kind of like a lame fightclub but with RSS and CAPS.
Anyway I digress, as a lot of people read your words could you please reword/re-explain/refactor the ‘Opening up an API increases a systems POTENTIAL insecurity’ bit please? That’s, well, wrong and I think most of the security world now agrees it’s pretty wrong too.
Thanks,
David
LikeLike
Hello Robert – Your comments areas is a scary, dark and angry place – it’s like you carry all these angry lost souls on to each new post. How on earth do you put up with it each and every day? Kind of like a lame fightclub but with RSS and CAPS.
Anyway I digress, as a lot of people read your words could you please reword/re-explain/refactor the ‘Opening up an API increases a systems POTENTIAL insecurity’ bit please? That’s, well, wrong and I think most of the security world now agrees it’s pretty wrong too.
Thanks,
David
LikeLike
David: good Merlot! 🙂
Um, what part of the security world says that’s wrong? Anytime you open a public API you increase the surface area where a hacker could potentially get in and attack you.
Doing that must be done very carefully and with great forthought. All of our teams here now have to do threat modeling, for instance. And every team has to go through security reviews for public APIs.
LikeLike
David: good Merlot! 🙂
Um, what part of the security world says that’s wrong? Anytime you open a public API you increase the surface area where a hacker could potentially get in and attack you.
Doing that must be done very carefully and with great forthought. All of our teams here now have to do threat modeling, for instance. And every team has to go through security reviews for public APIs.
LikeLike
Robert: Cheers, if you’re having one!
Well that’s fair enough from an exposure/surface area point of view, I just took your comment to mean (perhaps mistakenly) that the actual *algorithm* may be more secure if never revealed, when it fact it is often the case that opening up an algorithm to more eyes is considered a ‘good’ thing to do.
Thanks,
David
LikeLike
Robert: Cheers, if you’re having one!
Well that’s fair enough from an exposure/surface area point of view, I just took your comment to mean (perhaps mistakenly) that the actual *algorithm* may be more secure if never revealed, when it fact it is often the case that opening up an algorithm to more eyes is considered a ‘good’ thing to do.
Thanks,
David
LikeLike
Short term opening an API makes it more insecure.
What Microsoft still hasn’t figured out (and that Open Source has) is that opening an API in the long run FORCES the API to become more secure
LikeLike
Short term opening an API makes it more insecure.
What Microsoft still hasn’t figured out (and that Open Source has) is that opening an API in the long run FORCES the API to become more secure
LikeLike
If my post came across as in any way disrespectful of Gabe, I am sorry. That was not my intention, I have the utmost respect for him.
I do not “think” the Memeorandum algorithm is “fairly simplistic”. I don’t know how complex is it, and I stated that clearly. That’s why I asked the question. To find out. I know Gabe watches my blog because he pops up in the comments a lot, so I was hoping to get some sort of answer to the question. Understandably he dodged the question, as is only prudent for him to protect his intellectual property.
As to your questions… “For instance, do you know how he filters out noise?” – he has a carefully maintained whitelist, that’s pretty simple. “What happens if I link to a political blog? Why doesn’t that show up on Memeorandum?” – he has separate lists for www. and tech., that’s also pretty straightforward. I don’t know that that is the way he does it, but that seems to me to be the most efficient method. Tinfinger will be trying that out to see if it works, in any case.
Some questions still remain: if the Memeorandum algorithm is so complex, can a “personalised” version of it be possible? Is it possible to expose the algorithm and allow users to fiddle with the operands and put weightings on the individual variables? Or will the only workable solution be a bunch of abstracted sliders, as in Yahoo Mindset or MSN Result Ranking?
I’m not a compiler writer, I never worked for Intel. I am, to use a popular word these days, an “amateur” when it comes to programming. However, I’m a quick learner, and I’m collecting data to pass on to my business partner Tai (a real programmer). I expected Gabe not to give up all his secrets, but I was hoping other people would pick up the meme and speculate openly, as you have Robert (thanks mate!). I’m learning. That was the purpose of my post.
LikeLike
If my post came across as in any way disrespectful of Gabe, I am sorry. That was not my intention, I have the utmost respect for him.
I do not “think” the Memeorandum algorithm is “fairly simplistic”. I don’t know how complex is it, and I stated that clearly. That’s why I asked the question. To find out. I know Gabe watches my blog because he pops up in the comments a lot, so I was hoping to get some sort of answer to the question. Understandably he dodged the question, as is only prudent for him to protect his intellectual property.
As to your questions… “For instance, do you know how he filters out noise?” – he has a carefully maintained whitelist, that’s pretty simple. “What happens if I link to a political blog? Why doesn’t that show up on Memeorandum?” – he has separate lists for www. and tech., that’s also pretty straightforward. I don’t know that that is the way he does it, but that seems to me to be the most efficient method. Tinfinger will be trying that out to see if it works, in any case.
Some questions still remain: if the Memeorandum algorithm is so complex, can a “personalised” version of it be possible? Is it possible to expose the algorithm and allow users to fiddle with the operands and put weightings on the individual variables? Or will the only workable solution be a bunch of abstracted sliders, as in Yahoo Mindset or MSN Result Ranking?
I’m not a compiler writer, I never worked for Intel. I am, to use a popular word these days, an “amateur” when it comes to programming. However, I’m a quick learner, and I’m collecting data to pass on to my business partner Tai (a real programmer). I expected Gabe not to give up all his secrets, but I was hoping other people would pick up the meme and speculate openly, as you have Robert (thanks mate!). I’m learning. That was the purpose of my post.
LikeLike
Paul,
This is the first time I have seen or heard of you.
My initial impressions are that if you are such an “amateur” and you are “learning”, maybe you should communicate with more humility and approach others who are infact not amateur with a little more respect then perhaps you may learn a thing or 2 (and I’m not just talking about technology)
Scoble, sorry for the personal comments on your blog but it is how I feel.
B4L
LikeLike
Paul,
This is the first time I have seen or heard of you.
My initial impressions are that if you are such an “amateur” and you are “learning”, maybe you should communicate with more humility and approach others who are infact not amateur with a little more respect then perhaps you may learn a thing or 2 (and I’m not just talking about technology)
Scoble, sorry for the personal comments on your blog but it is how I feel.
B4L
LikeLike
Bruno, I don’t think I’m showing anyone any disrespect. I am an amateur in comparison to such esteemed company, yes, but this isn’t the priesthood we’re talking about – programmers do not deserve (and most would not want) to be treated with undue deference. I don’t know about you, but I learn far more from robust conversation than by plaintively asking for divine providence. I would never attack people, but I make no apologies in attacking problems.
BTW I don’t know you either, Bruno, and you didn’t link to your blog.
LikeLike
Bruno, I don’t think I’m showing anyone any disrespect. I am an amateur in comparison to such esteemed company, yes, but this isn’t the priesthood we’re talking about – programmers do not deserve (and most would not want) to be treated with undue deference. I don’t know about you, but I learn far more from robust conversation than by plaintively asking for divine providence. I would never attack people, but I make no apologies in attacking problems.
BTW I don’t know you either, Bruno, and you didn’t link to your blog.
LikeLike
With such an eloquent grasp of the English language, one would think you could contribute more than the inane babble you’ve been espousing of late.
All I’m merely suggesting is that your manner strikes me counter productive to “attacking problems” and engaging in real dialogue – rather, you speak with a tone of pompous arrogance which does not really offer value to the community nor insight dialogue.
In any case, this is not the attack people channel, so I won’t ramble on about my dislike of your approach or lack of contribution to the blog or software community.
For the record:
1) I am a “real” programmer.
2) I don’t have a blog.
If for some reason you wish to pursue this conversation further, my email address is bruno4life@gmail.com lets not consume scobles blog with this kind of thing.
B4L
LikeLike
With such an eloquent grasp of the English language, one would think you could contribute more than the inane babble you’ve been espousing of late.
All I’m merely suggesting is that your manner strikes me counter productive to “attacking problems” and engaging in real dialogue – rather, you speak with a tone of pompous arrogance which does not really offer value to the community nor insight dialogue.
In any case, this is not the attack people channel, so I won’t ramble on about my dislike of your approach or lack of contribution to the blog or software community.
For the record:
1) I am a “real” programmer.
2) I don’t have a blog.
If for some reason you wish to pursue this conversation further, my email address is bruno4life@gmail.com lets not consume scobles blog with this kind of thing.
B4L
LikeLike