Enterprise Search: Not Exactly Crazy but Close
April 13, 2020
I think I started writing the first of three editions of the Enterprise Search Report in 2003. I had been through the Great Search Procurement competition for the US government’s search system. The original name for the service was FirstGov.gov (the idea was that the service was the “first” place to look for public facing government information. The second name was USA.gov, and it was different from FirstGov because the search results were pulled from an ad supported Web index.
The highlight of the competition was Google’s losing the contract to Fast Search & Transfer. (Note: The first index exposed to the public was the work of Inktomi, a company mostly lost in the purple mists of Yahoo and time.) Google was miffed because Fast Search & Transfer had teamed with AT&T and replied to the SOW with some of the old fervor that characterized the company before Judge Green changed the game. I recall one sticking point: Truncation. In fact, one of the Google founders argued with me about truncation at a search conference. I pointed out that Google had to do truncation whether the founders wanted to or not. My hunch is that you don’t know much about truncation and what it contributes. I won’t get into the weeds, but the function is important. Think stemming, inflections, etc.
I examined more than 60 “enterprise” search systems, including the chemical structure search systems, the not-so-useful search tools in engineering design systems like AutoCAD, and a number of search systems now long forgotten like Delphis and Entopia, among others.
I have also written “The New Landscape of Search” published by Pandia and “Successful Enterprise Search Management” with Martin White, who is still chugging along with his brand of search expertise. Of course, I follow search and retrieval even though I have narrowed my focus to what I call intelware and policeware. These are next-generation systems which address the numerous short coming of the oversold, over-hyped, and misunderstood software allowing a commercial enterprise to locate specific items of interest from their hotchpotch of content.
In this blog, Beyond Search/DarkCyber I write about some enterprise search systems. In general, I remain very critical of the technologies and the mostly unfounded assertions about what a search-and-retrieval system can deliver to an organization.
With this background, I reacted to “Enterprise Search Software Comparison” with sadness. I was not annoyed by the tone or desire to compare some solutions to enterprise content finding. My response was based on my realization about how far behind understanding of enterprise search’s upsides and downsides, the gap between next-generation information retrieval systems and the “brand” names, and the somewhat shallow understanding of the challenges enterprise search poses for licensees, vendors, and users.
The write up “compares” these systems as listed in the order each is discussed in the source article cited above:
- IBM Watson Discovery
- Salesforce Einstein Search
- Microsoft Search
- Google Cloud Search
- Amazon Kendra
- Lucidworks
- AlphaSense.
Each of these system merits a couple of paragraphs. For comparison, the discussion of systems in the Enterprise Search report typically required 15 or more pages. In CyberOSINT, I needed four pages for each system described. I had to cut the detail to meet the page limit for the book. A paragraph may be perfect for the thumb typing crowd, but detail does matter. The reason is that a misstep in selecting enterprise software can cost time and money and jobs. The people usually fired are those serving on the enterprise search system procurement team. Why? CFOs get very angry when triage to make a system work costs more than the original budget for the system. Users get angry when the system is slow (try 120 seconds to find a document in a content management system and then learn the document has not been indexed), stakeholders (the investment in search cannot be recovered without tricks, often illegal), and similar serious issues.
Let’s look at each of these systems described in the write up. I am going to move forward in alphabetical order. The listing in the source implies best to worst, and I want to avoid that. Also, at the end of this post, I will identify a few other systems which anyone seeking an enterprise search system may want to learn about. I post free profiles at www.xenky.com/vendor-profiles. The newer profiles cost money, and you can contact me at benkent2020 at yahoo dot com. No, I won’t give you a free copy. The free stuff is on my Xenky.com Web site.
AlphaSense. This is a venture backed company focused on making search the sharp end of a business intelligence initiative. The company is influenced by Eric Schmidt, the controversial Xoogler. The firm has raised about $100 million. The idea is to process disparate information and allow users to identify gems of information. AlphaSense competes with next-generation information services like DataWalk, Voyager Labs, and dozens of other forward looking firms. Will AlphaSense handle video, audio, time series data, and information stored on a remote workers’ laptop? Yeah. To sum up: Not an enterprise search solution; it is a variant of intelware. That’s no problem. AlphaSense is a me too of a different category of software.
Amazon Kendra. Amazon has a number of search solutions. This is Lucene. Yes, Lucene can deliver enterprise search; however, the system requires a commitment. Amazon’s approach is to put enterprise search into AWS. There’s nothing quite like the security of AWS in the hands of individuals who have not been “trained” in the ways of Amazon and Lucene.
Google Cloud Search. This is the spirit of the ill fated Google Search Appliance. The problems of GSA are ameliorated by putting content into the Google Cloud. What’s Google’s principal business? Yep, advertising. Those Googlers are trustworthy: Infidelity among senior managers raises this question, “Can we trust you to keep your body parts out of our private data?” You have to answer that question for yourself. (Sorry. Can’t say. Legal eagles monitor me still.)
IBM Watson Discovery. Okay, this is Lucene, home brew, and acquired technology like Vivisimo. Does it work? Why not ask Watson. IBM does have robust next-generation search, but that technology like IBM CyberTap is not available to the author of the article or to most commercial organizations. So IBM has training wheels search which requires oodles of IBM billable hours. Plus the company has next-generation information access. Which is it? Why not ask Watson? (If you used ITRC in the 1980s, you experienced my contribution to Big Blue. Plus I took money. None of that J5 stuff either.)
Salesforce Einstein Search. If a company puts its sales letter and contacts into this system, one can find the prospect and the email a salesperson sent that individual. Why do company’s want Salesforce search? When a salesperson quits, the company wants to make sure it has the leads, the sales story, etc. There are alternatives to Salesforce’s search system. Why? Maybe there are sufficient numbers of Salesforce customers who want to control what’s indexed and what employees can see? Just a thought.
Microsoft Search. I would like to write about Microsoft Search. (Yep, did a small thing for this outfit.) I would like to identify the acquisitions Microsoft completed to “improve” search. I would like to point out that Microsoft is changing Windows 10 search again. But that’s the story. One flavor of Microsoft Search is Fast Search & Transfer. It is so wonderful that a competitive solution is available from outfits like Surfray, EPI Server, and even Coveo (yep, the customer support and kitchen sink vendor). Why? Microsoft Search is very similar to the Google search: Young people fooling around in order to justify their salaries and sense of self worth. The result? I particularly like the racist chat bot and the fact that Microsoft bought Fast Search & Transfer as the criminal case for financial fraud was winding through Norway’s court system. Yep, criminal behavior. Why? Check out my previous write ups about Fast Search & Transfer.
Lucidworks. Okay, I did some small work for this outfit when it was called Lucid Imagination. Then the revolving door started to spin. The Lucene/Solr system collected many, many millions and started its journey to … wait for it… digital commerce and just about anything that could be slapped on open source software. Can one “do” enterprise search with Solr? Sure. Just make sure you have money and time. Lucidworks’ future is not exactly one that will thrill its funding sources. But there is hope for an acquisition or maybe an IPO. Is Lucidworks a way to get “faceted search” like Endeca offered in 1998? Sure, but why not license Endeca from Oracle? Endeca has some issues, of course, but I wanted to put a time mark in this essay so the “age” of Lucidworks’ newest ideas are anchored with a me-too peg.
What vendors are not mentioned who can implement enterprise search?
I will highlight three briefly, just to make clear the distortion of the enterprise market that this article presents to a thumb typing millennial procurement professional:
- Exalead spawned a number of interesting content companies. One of them is Algolia. It works and has some Exalead DNA.
- SearchIT is an outfit in Europe. It delivers what I consider a basic enterprise search system.
- Maxxcat produces a search appliance which is arguably a bit more modern than the Thunderstone appliance.
- Elastic Elasticsearch. This is the better Compass. How many outfits use Elasticsearch? Lots. There’s a free version and for-fee help when fans of Shay Bannon get stuck. Check out this how to.
There are others, of course, but my point is that mixing apples and oranges gives one a peculiar view of what is in the enterprise search orchard. It is better to categorize, compare and contrast systems that perform “enterprise search” functions. What are these? It took me 400 pages to explain what users expect, what systems can deliver, and the cost/engineering assumptions required to deliver a solution that is actually useful.
Search is hard. The next-generation systems point the way forward. Enterprise search has, in my opinion, not advanced very far beyond the original Smart system or IBM STAIRS III.
PS. Notice I did not use the jargon natural language processing, semantics, text analytics, and similar hoo haa. Why? Search has a different meaning for each worker in quite distinct business units. Do you expect a chemical engineer looking for Hexamethylene triperoxide diamine to use a word or a chemical structure? What about a marketing person seeking a video of a sales VP’s presentation at a client meeting yesterday? What about that intern’s Instagram post of a not-yet-released product prototype? What about the information on that sales VP’s laptop as he returns to his home office after a news story appeared about his or her talk? What about those human resource personnel data files? What about the eDiscovery material occupying the company’s legal team? What about the tweet a contractor sent to a big client about the cost of a fix to a factory robot that trashed a day’s production? What about the emails between an executive and a sex worker related to heroin? (A real need at a certain vendor of enterprise search!) Yeah! Enterprise search.
Stephen E Arnold, April 13, 2014
Ancient Search Recipes: Bread Pork Chops
December 16, 2019
I noted a report in the Times of Israel titled “Cache of Crypto-Jewish Recipes Dating to Inquisition Found in Miami Kitchen.” One of the recipes explained how to make a pork chop from bread and milk. (Dairy? Guess so.) Here’s what you and I can whip up using this ancient recipe:
The cookbook contains information which the author “didn’t think to question the idiosyncratic customs her mother and grandmothers practiced in the kitchen.”
By coincidence, my news alert spit out this article in the same list: “The Growth of Cognitive Search in the Enterprise, and Why It Matters.”
Magic. Bread pork chops created from zeros and ones.
Search matters. Cognitive search matters more. Who buys? The enterprise.
The write up recycles the equivalent of the break pork chop formula. Mix jargon, sprinkle with the notion of federated data, and bake until the checks clear the bank.
The article is fascinating, and it overlooks a few milestones in the history of enterprise search. What for example? Glad you asked:
- Forrester, the Wave folks, has created a report for its paying customers which reveal that search is now cognitive, able to tap dark data, and ready for prime time. Again! The Wave returns.
- Big companies are into search, including Microsoft with its Fast inspired solution and Amazon Kendra with an open source how de doo to Elastic and LucidWorks. Some use old spices; others, open source flavoring with proprietary special seasonings.
- Outfits which have been around for more than a decade like Coveo are now smarter than ever in their decade long effort to pay off their patient investors
- Autonomy gets a nod despite the interesting trial underway in the UK.
The point is that enterprise search is going to be in the news whether anyone wants to revisit hyperbole which makes the chatter around artificial intelligence and quantum computing seem rational and credible.
Here’s a quick refresher about why untapped data in an organization is likely to remained untapped or at the very least not tapped by vendors of smart key word search systems:
First, data are in silos for a reason. No enterprise search system with which I am familiar can navigate the permissions and access controls required to put siloed data in one index. There’s a chance that the Amazon blockchain permissions system can deliver this, but for now, the patents are explanations and federated enterprise search is a sales pitch.
Amazon Enterprise Search: Kendra
December 6, 2019
Years ago I worked on a small project for a company connected to the film industry. At one of those Hollywood “lunches”, a person pointed across the restaurant and said, “That’s Kendra.” I had zero idea who or what a Kendra was. It turned out that “Kendra” was famous, a star. She was a Playboy bunny! She looked like most of the other female appearing types in the room.
Amazon’s Kendra is not a Playboy bunny. Kendra is Amazon’s new online enterprise search service. It looks pretty much like all the other online enterprise search services in the room.
There’s a difference. This enterprise search service is mounted on the Amazon platform, and it has open source goodness, some proprietary fashion flair, and hooks into numerous good looking advanced AWS services.
Amazon says in “Amazon Kendra”:
Amazon Kendra is a highly accurate and easy to use enterprise search service that’s powered by machine learning. Kendra delivers powerful natural language search capabilities to your websites and applications so your end users can more easily find the information they need within the vast amount of content spread across your company.
I am not sure what “accurate” means, but it sure differentiates the service from the odd ball results some enterprise search solutions deliver. The “easy” part is also relative and subjective. Which of AWS’s more than 170 functions and services does Kendra get along with? Too soon to tell.
Some observations:
- The enterprise search vendors who have convinced venture capitalists to invest hundreds of millions in enterprise search and retrieval may be curious about Amazon’s sudden aggressiveness
- The enterprise search companies themselves now have to decide: Put services on AWS or go elsewhere despite the costs and resources required
- The AWS customers may want to kick the tires of the AWS service and postpone a procurement of a venture funded old school search engine. (Talking about NLP and machine learning is one thing. Delivering productized services from an AWS dashboard is another.)
Net net: For organizations struggling to federate and provide blockchain centric information access management, Amazon’s Kendra might look quite fetching.
Stephen E Arnold, December 6, 2019
ElasticSearch: A Push for Log File Revenue?
November 13, 2019
Elasticsearch had a moment in the sun. For several years it was the go to way to deploy a low cost, reasonably usable search and retrieval system. Then came the venture funding to a clump of outfits talking about search in terms of jargon that would have made a 1950s’ Madison Avenue executive reach for the bottle in his desk drawer.
To get a sense of the pressure exerted on ElasticSearch, navigate to “How to Get Started with Kibana.” Why battle with the tens of millions in fresh cash stuffing the pockets of Algolia, Coveo, and LucidWorks, among other 21st century enterprise search vendors with a penchant for buzzwords?
Do the pivot and keep one’s hand on the Elasticsearch throttle.
The write up explains:
Kibana is a powerful tool for visualizing data in Elasticsearch.
The article provides a sunny overview of how
you can explore practically any type of data, from text documents to machine logs, application metrics, ecommerce traffic, sensor telemetry, or your company’s business KPIs.
A KPI is, if you did not know, is a key performance indicator. What’s performance? What’s key? Heck, what’s an indicator? There you go. Modern methods for Kibana.
Net net: Elasticsearch is making it clear that it too is moving beyond search.
Search, however, has been moving beyond itself for decades. But my tagline for anything to do with the interesting and slightly sordid past of enterprise search is, “Who cares?”
Elasticsearch stakeholders.
Stephen E Arnold, November 13, 2019
Coveo: A 15 Year old $1 Billion Start Up Unicorn in Canada!
November 6, 2019
I read “Coveo Raises US$172M at $1B+ Valuation for AI-Based Enterprise Search and Personalization.” The write up states:
Search and personalization services continue to be a major area of investment among enterprises, both to make their products and services more discoverable (and used) by customers, and to help their own workers get their jobs done, with the market estimated to be worth some $100 billion annually. Today, one of the big startups building services in this area raised a large round of growth funding to continue tapping that opportunity.
Like Elastic, Algolia, and LucidWorks, Coveo is going to have to generate sufficient revenues to pay back its investors. Perhaps the early supporters have cashed out, but the new money is betting on the future.
Coveo was founded in Quebec City more than a decade ago. The desktop search company Copernic spun off Coveo in 2004. The original president was Laurent Simoneau. Mr. Tetu is an investor with great confidence in enterprise software, and he has become the “founder”, according to the write up. In April 2018, Coveo obtain about $100 million from Evergreen Coast Capital.
DarkCyber recalls that Coveo has moved from Microsoft-centric search to search as a service to customer experience and now personalization.
In 2005, I wrote this about the upsides of the Coveo approach in the Enterprise Search Report I compiled for an outfit lost to memory:
Coveo is a reasonably-priced, stable product. Any organization with Microsoft search will improve access to information with a system like Coveo’s. Microsoft SharePoint customers will want to do head-to-head comparisons with other “solutions” to Microsoft’s native search solution. Coveo has a number of features that make it a worth contender. Other benefits of the Coveo approach include:
- Web-based administration tool allows straightforward configuration and monitoring of the system.
- Automatic indexing of new and updated documents in near real-time.
- Includes linguistic and statistical technologies that can identify the key concepts and the key sentences of indexed documents. Provides automated document summaries for faster reading and filtering.
- Groups information sources into collections for field-specific searches.
- The product is attractively priced.
- Tightly integrated with other Microsoft products and Windows-based security regimes.
- Customer base has grown comparatively quite rapidly and customers tend to speak well of the product.
I noted these considerations:
The software is Windows-centric – both in terms of its own software as well as document security settings it tracks – which may be an issue with certain types of organizations. You will have to assign permissions to index to allow the ASP.NET worker process user to access the index. The task is simplified, but it can be overlooked. Administrative controls are presented without calling attention to actions that require particular attention. Coveo is still however able to search content on any operating system, application, or server. Other drawbacks of the Coveo search system include:
- There is limited software development support to allow customization or extensions of the core technology to other applications, although the company is expanding the product’s reach through Dot Net-based APIs.
- When the system is installed and its defaults accepted, the “Everyone” group is enabled. Administrators will want to customize this setting. A wizard would be a useful option for organizations new to enterprise search.
- No native taxonomy support, except through partner Entrieva.
- Achieving scalability beyond hundreds of millions of documents requires appropriate resources.
My final take on the company was:
Coveo Enterprise Search meets many distinct needs of the small and medium-sized business that has standardized on the Microsoft platform, while still providing a few critical advanced search capabilities. Perhaps more importantly, CES minimizes search training, system maintenance, and other cost “magnets” that typically accompany an enterprise search deployment.
Like a handful of other products in this report, you can test Coveo out first, via a free download of a document-limited version.
The challenge for Copernic is to make enough sales and to generate robust sustainable income. This is the uphill run that Algolia, Elastic, LucidWorks, and probably a number of other enterprise search vendors face. Perhaps an outfit like Xerox will buy up, which would be one way to get the investors their money?
DarkCyber wishes Coveo the best. But a start up unicorn? No, that is not exactly correct for a 15 year old outfit. This push to make the investors smile is not for the faint hearted or those who have a solid grasp of the formidable enterprise search options available today. Plus there are outfits like Diffeo and other next generation information access systems available for free (Eleasticsearch) or bundled with other sophisticated information management tools (Amazon, search, managed blockchain, workflows, and a clever approach to vendor lock in.)
One tip: Don’t visit Quebec City in February during a snow storm.
Stephen E Arnold, November 6, 2019
Stephen E Arnold, November 5, 2019
Enterprise Search: Two Interesting Assertions
September 23, 2019
Judging from the uptick in the estimates of the size of the enterprise search market, the “find documents” purveyors are generating public relations outputs.
Two items DarkCyber found interesting:
LucidWorks occupies a position of distinction. The company is the “sole visionary” in the enterprise search universe. According to “one of the world’s leading independent technology research and advisory firms,” LucidWorks — now check this wording — “LucidWorks believes its strong product offering and clear understanding of the needs of enterprise buyers in the Insight Engine market will enable the company to serve a wide variety of industries well into the future.” For more of this “alone at the top” razzmatazz, navigate to this Yahoo news item.
The second item is a bit more modest. Economic Times reported this fascinating factoid:
iQuanti is rapidly building its customer success and solutions teams in the region to activate its growth plans. Meanwhile, globally, iQuanti’s patented enterprise search channel platform ALPS was named the second-best search software in the U.S by the Drum Search Awards in 2018. ALPS uses proprietary data science and machine learning models to build predictive enterprise-level roadmaps that deliver strong ROI. iQuanti has also featured in the 2019 Inc. list of fastest-growing private companies in the US, for a record fifth time.
DarkCyber assumes that the top search engine is LucidWorks.
Now you know or maybe not.
Stephen E Arnold, September 23, 2019
Can a Well Worn Compass Help Enterprise Search Thrive?
September 4, 2019
In the early 1990s, Scotland Yard (which never existed although there is a New Scotland Yard) wanted a way to make sense of the data available to investigators in the law enforcement sector.
A start up in Cambridge, England, landed a contract. To cut a multi year story short, i2 Ltd. created Analyst’s Notebook. The product is now more than a quarter century old, and the Analyst’s Notebook is owned by IBM. In the span of five or six years, specialist vendors reacted to the Analyst’s Notebook functionalities. Even though the early versions were clunky, the software performed some functions that may be familiar to anyone who has tried to locate, analyze, and make sense of data within an organization. I am using “organization” in a broad sense, not just UK law enforcement, regulatory enforcement, and intelligence entities.
What were some of the key functions of Analyst’s Notebook, a product which most people in the search game know little about? Let me highly a handful, and then flash forward to what enterprise search vendors are trying to pull off in an environment which is very different from what the i2 experts tackled 25 years ago. Hint: Focus was the key to Analyst’s Notebook’s success and to the me-too products which are widely available to LE and intel professionals. Enterprise search lacks this singular advantage, and, as a result, is likely to flounder as it has for decades.
The Analyst’s Notebook delivered:
- Machine assistance to investigators implemented in software which generally followed established UK police procedures. Forget the AI stuff. The investigator or a team of investigators focused on a case provided most of the brain power.
- Software which could identify entities. An entity is a person, place, thing, phone number, credit card, event, or similar indexable item.
- Once identified, the software — influenced by the Cambridge curriculum in physics — could display a relationship “map” or what today looks like a social graph.
- Visual cues allowed investigators to see that a person who received lots of phone calls from another person were connected. To make the relationship explicit, a heavy dark line connected the two phone callers.
- Ability to print out on a big sheet of paper these relationship maps and other items of interest either identified by an investigator or an item surfaced using maths which could identify entities within a cluster or an anomaly and its date and time.
Over the years, other functions were added. Today’s version offers a range of advanced functions that make it easy to share data, collaborate, acquire and add to the investigative teams’ content store (on premises, hybrid, or in the cloud), automate some functions using IBM technology (no, I won’t use the Watson word), and workflow. Imagery is supported. Drill down makes it easy to see “where the data came from.” An auditor can retrace an investigator’s action in order to verify a process. If you want more about i2, just run a Bing, Google, or Yandex query.
Why am I writing about decades old software?
The reason is that is read an item from my files as my team was updating my comments about Amazon’s policeware for the October TechnoSecurity & Digital Forensics Conference. The item I viewed is titled “Thomson Reuters Partners with Squirro to Combine Artificial Intelligence Technology and Data to Unlock Customer Intelligence.” I had written about Squirro in “Will Cognitive Search (Whatever That Is) Change Because of Squirro?”
I took a look at the current Squirro Web site and learned that the company is the leader in “context intelligence.” That seemed similar to what i2 delivered in the 1990s version of Analyst’s Notebook. The software was designed to fit the context of a specific country’s principal police investigators. No marketing functions, no legal information, no engineering product data — just case related information like telephone records, credit card receipts, officer reports, arrest data, etc.
Squirro, founded in 2012 or 2013 (there are conflicting dates online) states that the software delivers
a personalized, real-time contextual stream from the sea of information directly to your workplace. It’s based on Squirro’s digital fingerprint technology connecting personal interests and workflows while learning and refining as user interactions increase.
I also noted this statement:
Squirro combines all the different tools you need to work with unstructured data and enables you to curate a self-learning 360° context radar natural to use in any enterprise system. ‘So What?’ Achieving this reduces searching time by 90%, significantly cutting costs and allows for better, more effective decision-making. The highly skilled Swiss team of search experts has been working together for over 10 years to create a precise context intelligence solution. Squirro: Your Data in Context.
Well, 2013 to the present is six years, seven if I accept the 2012 date.
The company states that it offers “A.I.-driven actionable Insights,” adding:
Squirro is a leading AI-platform – a self-learning system keeping you in the know and recommending what’s next.
I’m okay with marketing lingo. But to my way of thinking, Squirro is edging toward the i2 Analyst’s Notebook type of functionality. The difference is that Squirro wants to serve the enterprise. Yep, enterprise search with wrappers for smart software, reports, etc.
I don’t want to make a big deal of this similarity, but there is one important point to keep in mind. Delivering an enterprise solution to a commercial outfit means that different sectors of the business will have different needs. The different needs manifest themselves in workflows and data particular to their roles in the organization. Furthermore, most commercial employees are not trained like police and intelligence operatives; that is, employees looking for information have diverse backgrounds and different educational experiences. For better or worse, law enforcement intelligence professionals go to some type of training. In the US, the job is handled by numerous entities, but a touchstone is FLETC. Each country has its equivalent. Therefore, there is a shared base of information, a shared context if you will.
Modern companies are a bit like snowflakes. There’s a difference, however, the snowflakes may no longer work together in person. In fact, interactions are intermediated in numerous ways. This is not a negative, but it is somewhat different from how a team of investigators worked on a case in London in the 1990s.
What is the “search” inside the Squirro information retrieval system? The answer is open source search. The features are implemented via software add ons, wrappers, and micro services plus other 2019 methods.
This is neither good nor bad. Using open source reduces some costs. On the other hand, the resulting system will have a number of moving parts. As complexity grows with new features, some unexpected events will occur. These have to be chased down and fixed.
New features and functions can be snapped in. The trajectory of this modern approach is to create a system which offers many marketing hooks and opportunities to make a sale to an organization looking for a solution to the ever present “information problem.”
My hypothesis is that i2 Analyst’s Notebook succeeded an information access, analysis, and reporting system because it focused on solving a rather specific use case. A modern system such as a search and retrieval solution that tries to solve multiple problems is likely to hit a wall.
The digital wall is the same one that pushed Fast Search & Transfer and many other enterprise search systems to the sidelines or the scrap heap.
Net net: Focus, not jargon, may be valuable, not just for Squirro, but for other enterprise search vendors trying to attain sustainable revenues and a way to keep their sources of funding, their customers, their employees, and their stakeholders happy.
Stephen E Arnold, September 4, 2019
The New Lingo of Enterprise Search
August 28, 2019
Enterprise search is back. My Google Alert has been delivering market research reports which tell me that finding information is huge. Plus, there have been some announcements about funding which have surprised me. Examples include:
- Capacity raised $13.2 million. Source: DarkCyber
- LucidWorks snagged an additional $100 million. Source: Globe News Wire
- Squirro pulled in additional funds, but the timing of the Salesforce investment and additional funding of this Zurich based company remains a bit of a mystery. Source: Venture Lab
These are just three examples plucked from my box of note cards about search vendors.
What’s interesting is the lingo, the jargon, and the argot these outfits are using. Frankly the plumbing is usually open source, a fact which the companies bury beneath the blizzard of buzzwords.
Here are some examples:
AI powered
actionable insights
artificial intelligence
cloud
cognitive
connect the dots
data mining
fusion
information mining
machine learning
natural language
pattern detection
platform
self learning
transform
The problem with the vendors collecting investment funds are easy to identify:
- The content processed is text. The unstructured information in videos, podcasts, messaging apps like WhatsApp, images like chemical structures and engineering drawings, etc. are not included.
- Indexing content residing on cloud platforms may work today, but as market dynamics shift, access to that content my be blocked or prohibited by regulations in certain countries
- Federation, on-the-fly so that real time information is available remains a challenge which typically requires script fiddling or new content filters
- Configuration of “smart” systems is not significantly different from the complex, time consuming, and expensive procedures which added friction to some Autonomy, Convera, Fast Search & Transfer, and similar systems’ deployment
- Maintenance is an issue, micro services work well in a low latency environment. Under loads, the magic of sub three second response can disappear
- Search remains an idiosyncratic solution. Many departments require specific features. As a result, enterprise search — regardless of the wrappers around open source information retrieval systems — is a series of customizations.
To sum up, enterprise search has failed to deliver for more than 50 years. Despite the optimism that investors have for “finding the next Google”, enterprise search vendors will find themselves hitting a revenue ceiling just as Autonomy, Fast Search, and similar firms did.
The fix was acquisitions and allegations of financial fancy dancing. If we assume that investors still dream of a 10x or higher return, is it possible that LucidWorks can generate sufficient revenue to pull off an IPO or a sale like Exalead, Vivisimo, and other search vendors were able to complete before the hammer fell?
This is an important question because new enterprise search vendors are popping up like mushrooms. The incumbents like Attivio, Coveo, Mindbreeze, and Sinequa are also trying to smash a ball over the fence.
Net net: Enterprise search appears to be putting on the worn slippers last used by the founders of Fast Search & Transfer. Maybe Microsoft will buy another enterprise search vendor? The problem is that enterprise search is easy to make visible with marketing LED lights. Delivering sustainable revenues is a far greater challenge when Amazon is a competitor and a platform enabler.
What happens when Amazon competes more aggressively, raises its prices, or bundles text search into another of its services?
Answer: Nothing particularly beneficial for the investors in new and improved enterprise search solutions based on Lucene/Solr and dusted with disco glitter.
Stephen E Arnold, August 28, 2019
Enterprise Search: AI and a Low Spend
August 26, 2019
DarkCyber Read “Capacity Raises $13.2 Million to Index Emails, Files, and More with AI.” The company was founded in 2017. We noted this passage:
Capacity (formerly Jane.ai), [is] a startup developing a platform that indexes data from apps, teams, and more and enables users to search through the corpus using natural language.
Plus, the system learns and improves over time.
The company’s funding to deliver AI, multi-source enterprise search is “over $21 million.”
One of the founders is CEO David Karandish, formerly the CEO of Answers.com. He is quoted as saying:
[Capacity] is an intuitive, intelligent AI-powered Teammate who gives employees instant access to the information they need to do their jobs well.
The indexing system can process content from such systems as:
- ADP human resource information
- Box
- NetSuite
- Google Gmail
- Microsoft Exchange
- Microsoft OneDrive
- Sage human resource information
- Salesforce
- ServiceNow
- Zendesk
The system includes “a chatbot with natural language processing capabilities that integrates with popular messaging apps such as Slack and Skype.”
We noted this statement:
Capacity can deliver company-wide announcements, like daily news and event notifications, and onboard new hires by providing access to forms that need to be completed. For customers with websites that have FAQ sections, it can be made public-facing to help cut down on customer service requests.
If Capacity can deliver, outfits like LucidWorks will have some explaining to do to its investors.
Stephen E Arnold, August 26, 2019
Enterprise Search and Grease Management
June 7, 2019
I see some crazy stuff. Every once in a while, a really crazy item crosses my desk. The example I wish to highlight today is called “Enterprise Search Software Market to depict huge growth, Key Methodologies, Top Players: SharePoint, IBM, Lucidworks, Microsoft FAST, Oracle, Amazon CloudSearch, Apache Lucene, Attivio.” My hunch is that rolling in Amazon and Microsoft cloud revenues will make almost any market look like Popeye the Sailor Man. The reality is that enterprise search came and went in a blaze of litigation and embarrassment. Some of the exhaust seems to be emanating from the Hewlett Packard litigation related to the former medical device maker’s acquisition of an enterprise search vendor.
Enterprise search has overpromised and under delivered for about 50 years. Elsewhere I have recounted the adventures of services which most people don’t recall or simply knew nothing about. Remember InQuire, the service with forward truncation? A more recent fumble is the disappearance of those cheerful yellow Google Search Appliances, its staff, and the marketing collateral promising an end to the misery of traditional enterprise search solutions.
The buzz has not died down at at Reports Monitor. You can read their remarkable news release at this link. Forget the incredible hyperbole of “huge growth.” Hello, Reports Monitor, one can download a perfectly good enterprise search system from open source repositories. There are low cost systems available from outfits like Funnelback. You can get a next generation system from vendors of intelware. Don’t recognize the term? Don’t worry. These vendors don’t know what enterprise search means. And there are some companies which this report does not list as players. Want these names? Sorry, that’s information for which I charge a fee. Believe me. Reports Monitor and perhaps you, gentle reader, don’t know about these companies either.
What causes me to write about a report which is a bit on the wild side? How about this passage:
Key Insights:
- Complete in-depth analysis of the Grease Management in Commercial Kitchens
- Important changes in market dynamics.
- Segmentation analysis of the market.
- Emerging segments and regional markets.
- Historical, on-going, and projected market analysis based on volume and esteem.
- Assessment of niche industry players.
- Market share analysis.
- Key strategies of major players.
Yep, grease management. Now we’re getting to the heart of slippery data and even more slippery reports about enterprise search. The report provides region-wise data. Great stuff.
News flash: Enterprise search left the dock and took on water. Some outfits torpedoed their investors, customers, and partners. Others have tried to become business intelligence, analytics, even customer service support systems. Did not work too well.
Why?
Enterprise search is not a general purpose application. Significant work is necessary to make it possible for employees to find information in what are silos or in oddball lingo. Furthermore important people like lawyers, product researchers, and big wheels like to keep their information secret. An enterprise search system has failure baked in unless it is tailored to a quite specific problem. But at that point why not buy an eDiscovery system, a lab notebook system, or a niche solution for the eager beavers in marketing?
Maybe I am too harsh on the grease management angle. That may be closer to the truth than Reports Monitor realizes.
Stephen E Arnold, June 7, 2019
-
- Subscribe to Beyond Search
Feature archive
News archive
-
