The comprehensive guide to upgrading – or replacing – Twitter
Twitter is a rather new communications service, wildly popular in the technology blogging and podcasting communities. There are close to a million registered accounts or users, but I’d guess the active users number in the low-mid five figures. Even at that low usage, Twitter is on overload, plagued with outages and data loss.
Scaling Twitter is a huge challenge. Doing so will involve changing just about every aspect of what Twitter is. A number of commentators have suggested lesser fixes, but none that I’ve seen is apt to work. (Generally, they forget that UI options will need to change as usage grows.) However, I think I’ve come up with an approach that would indeed work, for:
- Arbitrarily high levels of public Twitter use.
- Twitter integration with other communication tools such as instant messaging or IRC-style chat.
- Enterprise or integrated personal/enterprise use of Twitter.
The sections below cover:
- Future metadata needed by Twitter “tweets” (i.e., posts)
- Filtering enhancements Twitter will need as usage scales (and could greatly use already today)
- Present and future Twitter use cases
- Twitter CEP and database architecture (almost everybody else I see writing about Twitter gets this wrong)
- Enterprise Twitter
- Twitter’s competitive vulnerabilities
| Categories: Microblogging, Social software and online media, Twitter | 15 Comments |
A game theorist’s view of Microsoft/Yahoo
Edit: Microsoft/Yahoo could easily end up being an all-cash deal.
Larry Dignan encourages a game theoretic view of the Microsoft/Yahoo merger, following Trip Chowdhry. I actually have a Ph.D. in game theory, so I’ll bite. 🙂
In most negotiation games — including pretty much all in which money can change hands — there’s one outcome that makes the most sense for all concerned. They should agree to that outcome, and haggle about nothing except price.* In this case, the best outcome for Microsoft and Yahoo is a quick Microsoft takeover of Yahoo. That’s what I thought all along, due to a whole lot of Microsoft/Yahoo synergies. Michael Arrington reports, in confirmation, that there are no viable alternative bidders.
*A fancy way of saying that is “The feasible set has a continuous and effectively one-dimensional Pareto frontier.”
In such cases, the haggling over price depends a lot on each side’s “threat point” — i.e., their fallback position, and the (un)desirability of that fallback position for each side. Yahoo’s fallback position is probably one or more aggressive deals with other major internet players. Merely outsourcing its search business to Google would be stupid. Selling the search business to Google could fetch a wonderful price, because Google would be even more entrenched — but for exactly that reason, it would surely fail to pass antitrust muster. That’s why the Amazon idea that’s been floated is so crucial; a Yahoo/Amazon merger would actually be synergistic in its own way, and hence could command a price at least somewhat competitive with Microsoft’s offer.
As for Microsoft — despite successes in individual Internet areas, it has consistently failed to build a coherent Internet business. Yahoo has its own issues, obviously, but on the whole it’s maintained pretty decent Internet status even as its technological efforts have been consistently disappointing. If Microsoft doesn’t buy Yahoo, it probably needs to buy somebody else with a consistent record of Internet leadership, such as Amazon. That would also involve paying a large premium. And here’s a twist: If Amazon for any reason wants to sell to fellow Washingtonian Microsoft at a big premium, it’s best move may be to sabotage the Microsoft/Yahoo deal somehow.
One final note: If Yahoo outsources its search business to Google, the possibility of a Microsoft deal is gone forever. Microsoft can not be assured of winning a waiting game, the way Oracle outlasted Peoplesoft.
Bottom line: The Microsoft/Yahoo deal should and probably will happen, and Yahoo should and probably will be able to squeeze Microsoft for more money than has first been offered.
Please sign up for our feed!
Technorati Tags: Trip Chowdhry, game theory
| Categories: Microsoft, Yahoo | 2 Comments |
Coveo highlights
I talked yesterday with enterprise search vendor Coveo. Here are some highlights.
- Coveo spun out of Copernic a few years ago. The only relationship between the companies now is that Coveo licenses Copernic’s desktop search product.
- Coveo has 60 employees.
- Coveo has 5-600 customers, including lots of big-name companies.
- Coveo’s pitch boils down to “inexpensive, easy to install, and no-apologies functionality.” Actually, Coveo also claims superior relevance and performance, but I’m not going to comment much on those until I have a chance for a more technical discussion.
- Example of ease of set-up: Coveo says Factiva downloaded the product on a Monday, called up and bought it on Thursday, and deployed it in production that Friday. This may be a growing industry trend. Attivio also features a “download first, talk to us second” distribution model. So do vendors of other kinds of “platform” software such as database management systems, application servers, or complex event/stream processing.
- Average selling price: $50K. Everything is included for one price unless it requires bundled third-party software (as is the case for audio, video, and OCR search).
- Coveo claims 90% head-to-head win rates vs. Google OneBox and Microsoft Sharepoint search. Generally, customers have other search products too (I guess that’s obvious, since Coveo has only been around 2-3 years). Sometimes they even have all-you-can-eat licenses to competitive products, but buy from Coveo anyway. Rule of thumb: Nobody’s head-to-head win rate is truly as high as they like to think, but companies that think their rate is 90% generally are doing quite well.
- Coveo cites a strong demand for text search of relational databases. Based on specific examples cited, this seems to mean text fields such as call center notes.
- Coveo offers audio/video search. Really, it’s just an audio search technology; what’s being searched on in videos is the audio part. And the audio search boils down to a speech-to-text transcription, with a search of the resulting text. Coveo’s key claim is that the error-laden text you get from speech-to-text conversion is sufficient for useful searching. Specifically, you do best searching for unusual words, such as proper names. In the case of telephone calls, which are low quality – perhaps 32 kb/sec – Coveo says there’s only 10-20% accuracy in word transcription. However, Coveo also says that the words that do come through are exactly the unusual ones most usefully searched on.
- Coveo also says that its speech-to-text lexicon is initially strengthened by text crawls. In general, while I didn’t ask, I would guess that the easy-installation story involves a fair amount of automated lexicon enhancement.
| Categories: Audio and video search, Coveo, Enterprise search, Search engines | 5 Comments |
Microsoft, Yahoo, and innovation
Bill Burnham argues that a Microsoft/Yahoo merger would drive down M&A prices. Marc Andreesen disagrees. His argument is essentially twofold:
- Microsoft and Yahoo were never more than a small part of the exit opportunity anyway.
- A merged Microsoft/Yahoo will be so slow-moving it will create more opportunities for competition than it destroys.
Andreesen certainly knows about slow-moving behemoths making wasted acquisitions; Netscape was acquired by two companies (AOL and Sun) that both dribbled away the parts they respectively acquired.* However, I think he and a lot of other observers are missing something this time — the Microsoft/Yahoo synergies are too large to ignore.
*The legalities of the merger were a lot more complicated than that, but in essence AOL got the “internet” piece of Netscape and Sun got the enterprise side.
Given the opportunity, here are some reasons I think integration would go a lot better than most people think: Read more
| Categories: Enterprise search, Microsoft, Search engines, Social software and online media, Yahoo | 1 Comment |
Sturgeon’s Law, and the future technology of social technology
Social technology has been hugely important to me since 1991. I met Linda on a Prodigy bulletin board. Blogging is crucial to my business. Mailing lists have led Linda and me to two vacations, most of our computer gaming, multiple TV shows (especially Buffy/Angel), and a whole lot of books. I find LinkedIn useful at times, and for the past few weeks I’ve been Twittering up a storm. My love life, work, and entertainment all are rooted in technology that gets people communicating with each other.
I’m not just saying that for street cred. My experiences also illustrate two important points – people use many different kinds of social technology, and social technology is very important to them. When you feel or hear negatives about MySpace, Facebook, LinkedIn, Twitter, blog reading or whatever – those are indictments of particular services or technologies, not of online social networking in general. Read more
| Categories: Blogosphere, Microblogging, Social software and online media, Twitter | 4 Comments |
19 Microsoft/Yahoo synergies that could revolutionize the Internet
Many – perhaps most — commentators on Microsoft’s bid for Yahoo are thoroughly missing the point. The most interesting part of Microsoft’s bid for Yahoo isn’t the horse-race retrospective “How did they screw up so much as to need each other?” It’s not the incipient bidding war for Yahoo. And it’s certainly not the antitrust implications.
The Microsoft/Yahoo combination could revolutionize the Internet. I’m serious. The opportunities for huge synergies might just be enough to blast the merged companies out of their current uncreative, Innovator’s Dilemma funks. Search is open for radical transformation in user interface, universal search relevancy, Web/enterprise integration, and just about everything to do with advertising and monetization. Email stands to be utterly reinvented. Portals and business intelligence have only scratched the surface of their potential. And social networking is of course in its infancy.
Here’s an overview of where some synergies and opportunities for a combined Microsoft/Yahoo lie. Read more
| Categories: Enterprise search, Google, Microsoft, Search engines, Social software and online media, Spam and antispam, Website filtering, Yahoo | 15 Comments |
Implications of Microsoft’s bid for Yahoo
As I write this, Microsoft has just announced an offer to acquire Yahoo. Early responses from the likes of Danny Sullivan, Henry Blodget, the Download Squad, TechCrunch, Raven SEO, Mashable, and others seem to boil down to:
- Wow.
- Both sides needed it.
- Yahoo wasn’t going anywhere fast on its own.
- Microsoft wasn’t going anywhere fast in search on its own.
- This may be enough critical mass to matter.
- Conference call at 8:30 am
I’ll try to be a bit more analytical than that, but this is still going to be quick. Assuming the deal goes through:
- Microsoft will recombine both parts of the old FAST/alltheweb.com Therefore, Microsoft will be able to use the same technology for web and enterprise search, to the extent that such commonality makes sense.
- I’d expect Microsoft to try to differentiate its technology via faceted/structured search. That’s a FAST strength.
- The old FAST search-as-BI dream might become pretty appealing to Microsoft/Yahoo.
- In a non-search point, Microsoft is strong in games and Yahoo is strong in fantasy sports. Look for some synergies.
- There sure would be a whole lot of non-Windows technology inside Microsoft. 🙂
Basically, Microsoft is a company that’s a lot more sophisticated in its thinking about user interfaces and experiences than Yahoo is. That’s where the really interesting competitive innovation would be most likely to occur.
| Categories: Enterprise search, FAST, Microsoft, Search engines, Structured search, Yahoo | 6 Comments |
The biggest text analytics company you probably never heard of
I caught up with Expert System S.p.A. last week. They turn out to be doing $10 million in text technology annual revenue. That alone is surprising (sadly), but what’s really remarkable is that they did it almost entirely in the Italian market. As you might guess, that figure includes a little bit of everything, from search engines to Italian language filters for Microsoft Office to text mining. But only $3 ½ million of Expert System’s revenue is from the government (and I think that includes civilian agencies), and under 30% is professional services, so on the whole it seems like a pretty real accomplishment. Oh yes – Expert Systems says it’s entirely self-funded.
As of last year, Expert System also has English-language products, and a couple of minor OEM sales in the US (for mobile search and semantic web applications). German- and Arabic-language products are in beta test. The company says that its market focus going forward is national security – surely the reason for the Arabic – and competitive intelligence. It envisions selling through partners such as system integrators, although I think that makes more sense for the government market than it does vis-a-vis civilian companies. In February the company is introducing a market intelligence product focused on sentiment analysis.
Expert System is a bit of a throwback, in that it talks lovingly of the semantic network that informs its products. Read more
| Categories: Application areas, Competitive intelligence, Enterprise search, Expert System S.p.A., Ontologies, Search engines, Text mining | Leave a Comment |
Forrester says 2008 is the year of enterprise social software
We all know how “The Year of X” kinds of predictions go. Still, when I read that Forrester Research says enterprises are ready to seriously adopt wikis and message forums, it made sense to me. Email threads — via Notes/Exchange or otherwise — aren’t doing the job any more. It’s time to go straight to communally-created web pages.
Personally, I think it’s also time to further replace email disasters, by having broadcasts over something like an enterprise version of Twitter. Clearly, enterprise Twitter would have to have a lot more tagging, group filtering, and automated censorship — ::sigh:: — than current public Twitter. But that all fits very well into the CEP-based architecture (or some near equivalent) that I believe to be the future of Twitter anyway. So would a complete integration between enterprise Twitter and point-to-point enterprise instant messaging.
| Categories: Social software and online media, Twitter | Leave a Comment |
Google is putting more emphasis on phrases
I don’t know how pronounced this trend is, but Google web search seems to be putting more emphasis on phrases than it used to.
For starters, Google doesn’t always ignore stopwords. The Fly and Fly produce different search results. Beyond that, “or” is sometimes assumed to be a word you’re searching on, not an operator — for an example, try live free or die and see the line of text that comes back under the search box. (I’m not sure whether this ever works for “and” as well — even Sanford and Son returns the usual harangue that “the AND operator is unnecessary”.) This is all a pretty clear indicator that Google is looking at phrases. Bill Slawski’s patent-analysis-heavy SEO blog has a lot more to say on that subject, specifically on an indexing scheme that addresses the problems that indexing stopwords in might otherwise cause.
Also, there’s a direct series of patents on “Phrase-Based Indexing.”
Finally, although I don’t recall a link, there seems to be a belief that:
- Google is using or moving to Latent Semantic Indexing (LSI)
- Word-based LSI is patented by somebody else.
| Categories: Google, Search engines | 3 Comments |