Update: Autonomy/Verity merger
I had a couple of very interesting calls with Autonomy last week. One message I got was that they do not want to be pigeonholed in search, which they think on the whole is a primitive way of dealing with “unstructured information.” Nonetheless, my first post based on those calls will indeed focus on text indexing and search. You see, I wrote quite skeptically about the Autonomy/Verity merger when it was announced, and I’d like to amend that with an updated opinion. Autonomy’s claims can be summarized in part by the following:
1. Long before acquiring Verity, Autonomy supposedly offered full plug-compatibility with Verity’s text indexing product. (At varous times the Verity product was called “Topic” and “K2”.) This claim is almost credible. After all, there was no “secret sauce” in getting the result set; it was all tokenization and boolean operations and proximity and so on. The reason I’m a little skeptical about the full claim is that the same can’t be said for ranking the results. There, each vendor indeed had/has some secrets which aren’t so easy to reverse-engineer.
2. At the time of the acquisition and for a while before, Autonomy’s underlying engine (IDOL) was simply better than Verity’s. At least, that’s the way the kernels stacked up. When I was closest to Verity, which was for a couple of years including 1997, the kernel group consisted of about 6 guys, led by Phil Nelson. All except one left, and the remaining one wasn’t Phil. Verity never recovered. I bet Dennis McEvoy would have fixed things given enough time, but if memory serves he was there less than a year. (In general, Verity seemed to suffer from brutal turnover.)
The most specific and obvious point is that IDOL is multithreaded, while K2 Classic is singlethreaded. Beyond that, IDOL and K2 have similar architectures (inverted index on words, etc.), although IDOL overall tracks more info (all the Bayesian stuff, and maybe it also was more advanced in recognizing parts of documents, headers, etc. — this is another point where I’d be curious to get clarification.)
3. IDOL is now the upgrade path for K2 users. Well, in light of the above — duh. Of course it is. And even if plug-compatibility wasn’t perfect when the companies were independent, it surely was easy to achieve once Autonomy could learn the ranking functions used in Verity’s product.
Comments
Leave a Reply