Remote Work and Enterprise Search: Implement Now!
April 28, 2020
The US and other countries has been shut down for more than a month. Companies of all sizes are struggling for revenue. The shift to WFH (work from home) is not exactly going on as smoothly as paint at a pre lockdown Peugeot plant.
The enterprise search idea articulated by a person once affiliated with IBM Watson is a stunner. You can get the full scoop in the online publication RTInsights. (No, this RT is not part of the Russian propaganda system.)
“Making Remote Work More Effective with Enterprise Search” argues that the WFH crowd can be productivity pythons. Forget the kids, the loneliness, the hassles with shopping, the security goblins, and the fear of losing one’s job. Put them out of your mind, WFH’ers. You can be a productivity python.
Sort of.
First, your employer — assuming you have one — must have an enterprise search in place. Failing that, your employer must spend money to license a suitable service. Hey, why not Sinequa, the French system which also does Big Data, analytics, and phenomenal marketing.
Now there are a couple of very minor issues to address; for example:
- Conducting a content inventory, determining what information can be accessed by an authorized WFH’er.
- The security and access controls must be defined, put in place, tested, and deployed.
- The indexing cycles must be determined because WFH’er presumably put in their 12 hour days across time zones, from a variety of computing devices, and in chunks. (Someone has to remove the Amazon packages from the door step before a bad actor removes the inviting parcels with a smile logo.)
- A workflow for getting employee generated content into the system and then getting the “real time indexing” which vendors stress their system performs to index in a reliable manner.
- Assisting employees who use the WFH system and cannot find the document a colleague said was in the system on the Zoom call that ended five minutes ago. The basic questions are, “Where is the document? When will it be available? Who’s in charge of this clown car?
Second, candidate information must be located, vetted, and converted to a format that the enterprise search system can process. Videos, audio files, images, and proprietary file formats may be a bit of a challenge in terms of time and resources.
Third, the system must be made to work. No, I mean it, deliver results employees or authorized users need. How many enterprise search systems deliver on this final point?
The write up explains:
Almost all knowledge-intensive organizations have a digital workplace that includes enterprise search, which connects employees to the content they need to complete a given task. Companies typically either deploy a rudimentary open-source kit that relies on search queries using keywords or a larger ecosystem like Microsoft, Google, or IBM, which tend to exclude content and data stored outside of the ecosystem.
What?
Oh, here’s the point:
Now is the time for organizations to think about the way employees access content platforms and how that is impacting employee productivity, knowledge sharing, and competitive advantage.
Based on the research Martin White and I did for oru book Successful Enterprise Search Management, the time required to deploy an enterprise search system was measured in months, often years. Tossing in the WFH requirement is going to add more time and cost to those sensitive to data access, indexing cycles, optimization, and other easy-to-ignore factors.
The benefits of providing enterprise search for WFH’ers remind me of the IBM Watson promises about smart software: Failure and massive costs, a loss of stakeholder value, and the distinction to be removed from Houston’s cancer hub.
To sum up, Sinequa’s sales pitch wrinkled the DarkCyber forehead a tiny bit:
- Glittering generalities about off site access to certain content is not something one just “thinks about.” Real management effort is required to avoid loss of trade secrets, sensitive information, and data which may be subject to government restrictions.
- The data supporting the assumption “better, faster, cheaper search yields more productivity (whatever that is). There is zero evidence that WFH’ers will be more or less productive if enterprise search is available. Right now, finding information is more like a Zoom call, not a session online hunting through results lists and waiting for results lists to appear.
- Phishing and other exploits. Security is not automatic. Security takes work. Oracle tried to sell its search system with Oracle security. No one in my experience was prepared to go through the hoops necessary to implement secure search. The result silos. What’s the cost for the WFH cohort? Probably more than some organizations are able to pay. (The May 12, DarkCyber video news program profiles a free-for-now open source solution to certain types of exploits. That’s a solution for those with handy infosec skills.)
Most applications used by WFH’ers include some type of search function. When information is not available, send an email or, better yet, hop on a Zoom call. And don’t forget Google, the millennials’ Swiss Army knife for information, or some social media scanning.
Enterprise search has not created productivity pythons in the more than 50 years information retrieval systems have been available.
Net net: Using Covid, WFH, and rusted buzzwords like enterprise search may not move the revenue meter. Invoking the tired, cheers-for-hire outfits like Gartner and IDC won’t do the job either. New types of information access systems are available. For examples, check out CyberOSINT: Next Generation Information Access. Even millennials will find some of these newer systems a refreshing findability option. As for enterprise search, its day in the sun faded with vendors’ inability to deliver results for licensees. Don’t believe me? Just ask former customers of Delphis, Entopia, Fast Search & Transfer, and the other precursors of today’s laborers in the search-and-retrieval Incan potato terraces.
Stephen E Arnold, April 28, 2020
Austrian Enterprise Search Gets Upgrade
April 23, 2020
When it comes to search innovation, people think about Silicon Valley or Japan. Austria, however, has its own high tech search solution from Iphos IT Solutions. Open PR shares news about Iphos IT Solutions’ newest search endeavor in the article, “SearchIT 2020-Even More Features & Functions For The Enterprise Search Solution searchIT.”
SearchIT 2020’s upgrade comes at an important time, because more employees are working from home during the COVID-19 pandemic. They need quick access to their company’s network from self isolation. The enhanced features and new function for SearchIT 2020 will make telecommunication seamless and guarantee employees are as productive, if not more, during quarantine.
Based on specially designed AI, SearchIT 2020 offers:
“SearchIT indexes and processes internal company data from a wide variety of sources, such as databases, file servers, mail servers, or various cloud storage sources. Finding data across all these sources is made easy with the full-text search function. The high degree of automation as well as many additional features not only provide users with an efficient and secure search tool, but also with the possibility of easy-to-handle document management, knowledge management, or the creation of comprehensive data archives.”
Since searchIT 2020 is built on AI, it does more than basic enterprise search applications. It allows users to build automatic data archives, starting data with third parties, and media monitoring. New features that are included in the upgrade include integrations of new search sources, dynamic report creation, meta and special searches with extended query syntax, preview function, explorer view, and automated language recognition.
Iphos IT Solutions provided solutions for large corporations to small and medium sized businesses. The pricing varies for all projects, but it is made to be affordable for any business while providing large scale services.
Whitney Grace, April 23, 2020
Enterprise Search: Not Exactly Crazy but Close
April 13, 2020
I think I started writing the first of three editions of the Enterprise Search Report in 2003. I had been through the Great Search Procurement competition for the US government’s search system. The original name for the service was FirstGov.gov (the idea was that the service was the “first” place to look for public facing government information. The second name was USA.gov, and it was different from FirstGov because the search results were pulled from an ad supported Web index.
The highlight of the competition was Google’s losing the contract to Fast Search & Transfer. (Note: The first index exposed to the public was the work of Inktomi, a company mostly lost in the purple mists of Yahoo and time.) Google was miffed because Fast Search & Transfer had teamed with AT&T and replied to the SOW with some of the old fervor that characterized the company before Judge Green changed the game. I recall one sticking point: Truncation. In fact, one of the Google founders argued with me about truncation at a search conference. I pointed out that Google had to do truncation whether the founders wanted to or not. My hunch is that you don’t know much about truncation and what it contributes. I won’t get into the weeds, but the function is important. Think stemming, inflections, etc.
I examined more than 60 “enterprise” search systems, including the chemical structure search systems, the not-so-useful search tools in engineering design systems like AutoCAD, and a number of search systems now long forgotten like Delphis and Entopia, among others.
I have also written “The New Landscape of Search” published by Pandia and “Successful Enterprise Search Management” with Martin White, who is still chugging along with his brand of search expertise. Of course, I follow search and retrieval even though I have narrowed my focus to what I call intelware and policeware. These are next-generation systems which address the numerous short coming of the oversold, over-hyped, and misunderstood software allowing a commercial enterprise to locate specific items of interest from their hotchpotch of content.
In this blog, Beyond Search/DarkCyber I write about some enterprise search systems. In general, I remain very critical of the technologies and the mostly unfounded assertions about what a search-and-retrieval system can deliver to an organization.
With this background, I reacted to “Enterprise Search Software Comparison” with sadness. I was not annoyed by the tone or desire to compare some solutions to enterprise content finding. My response was based on my realization about how far behind understanding of enterprise search’s upsides and downsides, the gap between next-generation information retrieval systems and the “brand” names, and the somewhat shallow understanding of the challenges enterprise search poses for licensees, vendors, and users.
The write up “compares” these systems as listed in the order each is discussed in the source article cited above:
- IBM Watson Discovery
- Salesforce Einstein Search
- Microsoft Search
- Google Cloud Search
- Amazon Kendra
- Lucidworks
- AlphaSense.
Each of these system merits a couple of paragraphs. For comparison, the discussion of systems in the Enterprise Search report typically required 15 or more pages. In CyberOSINT, I needed four pages for each system described. I had to cut the detail to meet the page limit for the book. A paragraph may be perfect for the thumb typing crowd, but detail does matter. The reason is that a misstep in selecting enterprise software can cost time and money and jobs. The people usually fired are those serving on the enterprise search system procurement team. Why? CFOs get very angry when triage to make a system work costs more than the original budget for the system. Users get angry when the system is slow (try 120 seconds to find a document in a content management system and then learn the document has not been indexed), stakeholders (the investment in search cannot be recovered without tricks, often illegal), and similar serious issues.
Let’s look at each of these systems described in the write up. I am going to move forward in alphabetical order. The listing in the source implies best to worst, and I want to avoid that. Also, at the end of this post, I will identify a few other systems which anyone seeking an enterprise search system may want to learn about. I post free profiles at www.xenky.com/vendor-profiles. The newer profiles cost money, and you can contact me at benkent2020 at yahoo dot com. No, I won’t give you a free copy. The free stuff is on my Xenky.com Web site.
AlphaSense. This is a venture backed company focused on making search the sharp end of a business intelligence initiative. The company is influenced by Eric Schmidt, the controversial Xoogler. The firm has raised about $100 million. The idea is to process disparate information and allow users to identify gems of information. AlphaSense competes with next-generation information services like DataWalk, Voyager Labs, and dozens of other forward looking firms. Will AlphaSense handle video, audio, time series data, and information stored on a remote workers’ laptop? Yeah. To sum up: Not an enterprise search solution; it is a variant of intelware. That’s no problem. AlphaSense is a me too of a different category of software.
Amazon Kendra. Amazon has a number of search solutions. This is Lucene. Yes, Lucene can deliver enterprise search; however, the system requires a commitment. Amazon’s approach is to put enterprise search into AWS. There’s nothing quite like the security of AWS in the hands of individuals who have not been “trained” in the ways of Amazon and Lucene.
Google Cloud Search. This is the spirit of the ill fated Google Search Appliance. The problems of GSA are ameliorated by putting content into the Google Cloud. What’s Google’s principal business? Yep, advertising. Those Googlers are trustworthy: Infidelity among senior managers raises this question, “Can we trust you to keep your body parts out of our private data?” You have to answer that question for yourself. (Sorry. Can’t say. Legal eagles monitor me still.)
IBM Watson Discovery. Okay, this is Lucene, home brew, and acquired technology like Vivisimo. Does it work? Why not ask Watson. IBM does have robust next-generation search, but that technology like IBM CyberTap is not available to the author of the article or to most commercial organizations. So IBM has training wheels search which requires oodles of IBM billable hours. Plus the company has next-generation information access. Which is it? Why not ask Watson? (If you used ITRC in the 1980s, you experienced my contribution to Big Blue. Plus I took money. None of that J5 stuff either.)
Salesforce Einstein Search. If a company puts its sales letter and contacts into this system, one can find the prospect and the email a salesperson sent that individual. Why do company’s want Salesforce search? When a salesperson quits, the company wants to make sure it has the leads, the sales story, etc. There are alternatives to Salesforce’s search system. Why? Maybe there are sufficient numbers of Salesforce customers who want to control what’s indexed and what employees can see? Just a thought.
Microsoft Search. I would like to write about Microsoft Search. (Yep, did a small thing for this outfit.) I would like to identify the acquisitions Microsoft completed to “improve” search. I would like to point out that Microsoft is changing Windows 10 search again. But that’s the story. One flavor of Microsoft Search is Fast Search & Transfer. It is so wonderful that a competitive solution is available from outfits like Surfray, EPI Server, and even Coveo (yep, the customer support and kitchen sink vendor). Why? Microsoft Search is very similar to the Google search: Young people fooling around in order to justify their salaries and sense of self worth. The result? I particularly like the racist chat bot and the fact that Microsoft bought Fast Search & Transfer as the criminal case for financial fraud was winding through Norway’s court system. Yep, criminal behavior. Why? Check out my previous write ups about Fast Search & Transfer.
Lucidworks. Okay, I did some small work for this outfit when it was called Lucid Imagination. Then the revolving door started to spin. The Lucene/Solr system collected many, many millions and started its journey to … wait for it… digital commerce and just about anything that could be slapped on open source software. Can one “do” enterprise search with Solr? Sure. Just make sure you have money and time. Lucidworks’ future is not exactly one that will thrill its funding sources. But there is hope for an acquisition or maybe an IPO. Is Lucidworks a way to get “faceted search” like Endeca offered in 1998? Sure, but why not license Endeca from Oracle? Endeca has some issues, of course, but I wanted to put a time mark in this essay so the “age” of Lucidworks’ newest ideas are anchored with a me-too peg.
What vendors are not mentioned who can implement enterprise search?
I will highlight three briefly, just to make clear the distortion of the enterprise market that this article presents to a thumb typing millennial procurement professional:
- Exalead spawned a number of interesting content companies. One of them is Algolia. It works and has some Exalead DNA.
- SearchIT is an outfit in Europe. It delivers what I consider a basic enterprise search system.
- Maxxcat produces a search appliance which is arguably a bit more modern than the Thunderstone appliance.
- Elastic Elasticsearch. This is the better Compass. How many outfits use Elasticsearch? Lots. There’s a free version and for-fee help when fans of Shay Bannon get stuck. Check out this how to.
There are others, of course, but my point is that mixing apples and oranges gives one a peculiar view of what is in the enterprise search orchard. It is better to categorize, compare and contrast systems that perform “enterprise search” functions. What are these? It took me 400 pages to explain what users expect, what systems can deliver, and the cost/engineering assumptions required to deliver a solution that is actually useful.
Search is hard. The next-generation systems point the way forward. Enterprise search has, in my opinion, not advanced very far beyond the original Smart system or IBM STAIRS III.
PS. Notice I did not use the jargon natural language processing, semantics, text analytics, and similar hoo haa. Why? Search has a different meaning for each worker in quite distinct business units. Do you expect a chemical engineer looking for Hexamethylene triperoxide diamine to use a word or a chemical structure? What about a marketing person seeking a video of a sales VP’s presentation at a client meeting yesterday? What about that intern’s Instagram post of a not-yet-released product prototype? What about the information on that sales VP’s laptop as he returns to his home office after a news story appeared about his or her talk? What about those human resource personnel data files? What about the eDiscovery material occupying the company’s legal team? What about the tweet a contractor sent to a big client about the cost of a fix to a factory robot that trashed a day’s production? What about the emails between an executive and a sex worker related to heroin? (A real need at a certain vendor of enterprise search!) Yeah! Enterprise search.
Stephen E Arnold, April 13, 2014
Ancient Search Recipes: Bread Pork Chops
December 16, 2019
I noted a report in the Times of Israel titled “Cache of Crypto-Jewish Recipes Dating to Inquisition Found in Miami Kitchen.” One of the recipes explained how to make a pork chop from bread and milk. (Dairy? Guess so.) Here’s what you and I can whip up using this ancient recipe:
The cookbook contains information which the author “didn’t think to question the idiosyncratic customs her mother and grandmothers practiced in the kitchen.”
By coincidence, my news alert spit out this article in the same list: “The Growth of Cognitive Search in the Enterprise, and Why It Matters.”
Magic. Bread pork chops created from zeros and ones.
Search matters. Cognitive search matters more. Who buys? The enterprise.
The write up recycles the equivalent of the break pork chop formula. Mix jargon, sprinkle with the notion of federated data, and bake until the checks clear the bank.
The article is fascinating, and it overlooks a few milestones in the history of enterprise search. What for example? Glad you asked:
- Forrester, the Wave folks, has created a report for its paying customers which reveal that search is now cognitive, able to tap dark data, and ready for prime time. Again! The Wave returns.
- Big companies are into search, including Microsoft with its Fast inspired solution and Amazon Kendra with an open source how de doo to Elastic and LucidWorks. Some use old spices; others, open source flavoring with proprietary special seasonings.
- Outfits which have been around for more than a decade like Coveo are now smarter than ever in their decade long effort to pay off their patient investors
- Autonomy gets a nod despite the interesting trial underway in the UK.
The point is that enterprise search is going to be in the news whether anyone wants to revisit hyperbole which makes the chatter around artificial intelligence and quantum computing seem rational and credible.
Here’s a quick refresher about why untapped data in an organization is likely to remained untapped or at the very least not tapped by vendors of smart key word search systems:
First, data are in silos for a reason. No enterprise search system with which I am familiar can navigate the permissions and access controls required to put siloed data in one index. There’s a chance that the Amazon blockchain permissions system can deliver this, but for now, the patents are explanations and federated enterprise search is a sales pitch.
Amazon Enterprise Search: Kendra
December 6, 2019
Years ago I worked on a small project for a company connected to the film industry. At one of those Hollywood “lunches”, a person pointed across the restaurant and said, “That’s Kendra.” I had zero idea who or what a Kendra was. It turned out that “Kendra” was famous, a star. She was a Playboy bunny! She looked like most of the other female appearing types in the room.
Amazon’s Kendra is not a Playboy bunny. Kendra is Amazon’s new online enterprise search service. It looks pretty much like all the other online enterprise search services in the room.
There’s a difference. This enterprise search service is mounted on the Amazon platform, and it has open source goodness, some proprietary fashion flair, and hooks into numerous good looking advanced AWS services.
Amazon says in “Amazon Kendra”:
Amazon Kendra is a highly accurate and easy to use enterprise search service that’s powered by machine learning. Kendra delivers powerful natural language search capabilities to your websites and applications so your end users can more easily find the information they need within the vast amount of content spread across your company.
I am not sure what “accurate” means, but it sure differentiates the service from the odd ball results some enterprise search solutions deliver. The “easy” part is also relative and subjective. Which of AWS’s more than 170 functions and services does Kendra get along with? Too soon to tell.
Some observations:
- The enterprise search vendors who have convinced venture capitalists to invest hundreds of millions in enterprise search and retrieval may be curious about Amazon’s sudden aggressiveness
- The enterprise search companies themselves now have to decide: Put services on AWS or go elsewhere despite the costs and resources required
- The AWS customers may want to kick the tires of the AWS service and postpone a procurement of a venture funded old school search engine. (Talking about NLP and machine learning is one thing. Delivering productized services from an AWS dashboard is another.)
Net net: For organizations struggling to federate and provide blockchain centric information access management, Amazon’s Kendra might look quite fetching.
Stephen E Arnold, December 6, 2019
ElasticSearch: A Push for Log File Revenue?
November 13, 2019
Elasticsearch had a moment in the sun. For several years it was the go to way to deploy a low cost, reasonably usable search and retrieval system. Then came the venture funding to a clump of outfits talking about search in terms of jargon that would have made a 1950s’ Madison Avenue executive reach for the bottle in his desk drawer.
To get a sense of the pressure exerted on ElasticSearch, navigate to “How to Get Started with Kibana.” Why battle with the tens of millions in fresh cash stuffing the pockets of Algolia, Coveo, and LucidWorks, among other 21st century enterprise search vendors with a penchant for buzzwords?
Do the pivot and keep one’s hand on the Elasticsearch throttle.
The write up explains:
Kibana is a powerful tool for visualizing data in Elasticsearch.
The article provides a sunny overview of how
you can explore practically any type of data, from text documents to machine logs, application metrics, ecommerce traffic, sensor telemetry, or your company’s business KPIs.
A KPI is, if you did not know, is a key performance indicator. What’s performance? What’s key? Heck, what’s an indicator? There you go. Modern methods for Kibana.
Net net: Elasticsearch is making it clear that it too is moving beyond search.
Search, however, has been moving beyond itself for decades. But my tagline for anything to do with the interesting and slightly sordid past of enterprise search is, “Who cares?”
Elasticsearch stakeholders.
Stephen E Arnold, November 13, 2019
Coveo: A 15 Year old $1 Billion Start Up Unicorn in Canada!
November 6, 2019
I read “Coveo Raises US$172M at $1B+ Valuation for AI-Based Enterprise Search and Personalization.” The write up states:
Search and personalization services continue to be a major area of investment among enterprises, both to make their products and services more discoverable (and used) by customers, and to help their own workers get their jobs done, with the market estimated to be worth some $100 billion annually. Today, one of the big startups building services in this area raised a large round of growth funding to continue tapping that opportunity.
Like Elastic, Algolia, and LucidWorks, Coveo is going to have to generate sufficient revenues to pay back its investors. Perhaps the early supporters have cashed out, but the new money is betting on the future.
Coveo was founded in Quebec City more than a decade ago. The desktop search company Copernic spun off Coveo in 2004. The original president was Laurent Simoneau. Mr. Tetu is an investor with great confidence in enterprise software, and he has become the “founder”, according to the write up. In April 2018, Coveo obtain about $100 million from Evergreen Coast Capital.
DarkCyber recalls that Coveo has moved from Microsoft-centric search to search as a service to customer experience and now personalization.
In 2005, I wrote this about the upsides of the Coveo approach in the Enterprise Search Report I compiled for an outfit lost to memory:
Coveo is a reasonably-priced, stable product. Any organization with Microsoft search will improve access to information with a system like Coveo’s. Microsoft SharePoint customers will want to do head-to-head comparisons with other “solutions” to Microsoft’s native search solution. Coveo has a number of features that make it a worth contender. Other benefits of the Coveo approach include:
- Web-based administration tool allows straightforward configuration and monitoring of the system.
- Automatic indexing of new and updated documents in near real-time.
- Includes linguistic and statistical technologies that can identify the key concepts and the key sentences of indexed documents. Provides automated document summaries for faster reading and filtering.
- Groups information sources into collections for field-specific searches.
- The product is attractively priced.
- Tightly integrated with other Microsoft products and Windows-based security regimes.
- Customer base has grown comparatively quite rapidly and customers tend to speak well of the product.
I noted these considerations:
The software is Windows-centric – both in terms of its own software as well as document security settings it tracks – which may be an issue with certain types of organizations. You will have to assign permissions to index to allow the ASP.NET worker process user to access the index. The task is simplified, but it can be overlooked. Administrative controls are presented without calling attention to actions that require particular attention. Coveo is still however able to search content on any operating system, application, or server. Other drawbacks of the Coveo search system include:
- There is limited software development support to allow customization or extensions of the core technology to other applications, although the company is expanding the product’s reach through Dot Net-based APIs.
- When the system is installed and its defaults accepted, the “Everyone” group is enabled. Administrators will want to customize this setting. A wizard would be a useful option for organizations new to enterprise search.
- No native taxonomy support, except through partner Entrieva.
- Achieving scalability beyond hundreds of millions of documents requires appropriate resources.
My final take on the company was:
Coveo Enterprise Search meets many distinct needs of the small and medium-sized business that has standardized on the Microsoft platform, while still providing a few critical advanced search capabilities. Perhaps more importantly, CES minimizes search training, system maintenance, and other cost “magnets” that typically accompany an enterprise search deployment.
Like a handful of other products in this report, you can test Coveo out first, via a free download of a document-limited version.
The challenge for Copernic is to make enough sales and to generate robust sustainable income. This is the uphill run that Algolia, Elastic, LucidWorks, and probably a number of other enterprise search vendors face. Perhaps an outfit like Xerox will buy up, which would be one way to get the investors their money?
DarkCyber wishes Coveo the best. But a start up unicorn? No, that is not exactly correct for a 15 year old outfit. This push to make the investors smile is not for the faint hearted or those who have a solid grasp of the formidable enterprise search options available today. Plus there are outfits like Diffeo and other next generation information access systems available for free (Eleasticsearch) or bundled with other sophisticated information management tools (Amazon, search, managed blockchain, workflows, and a clever approach to vendor lock in.)
One tip: Don’t visit Quebec City in February during a snow storm.
Stephen E Arnold, November 6, 2019
Stephen E Arnold, November 5, 2019
Enterprise Search: Two Interesting Assertions
September 23, 2019
Judging from the uptick in the estimates of the size of the enterprise search market, the “find documents” purveyors are generating public relations outputs.
Two items DarkCyber found interesting:
LucidWorks occupies a position of distinction. The company is the “sole visionary” in the enterprise search universe. According to “one of the world’s leading independent technology research and advisory firms,” LucidWorks — now check this wording — “LucidWorks believes its strong product offering and clear understanding of the needs of enterprise buyers in the Insight Engine market will enable the company to serve a wide variety of industries well into the future.” For more of this “alone at the top” razzmatazz, navigate to this Yahoo news item.
The second item is a bit more modest. Economic Times reported this fascinating factoid:
iQuanti is rapidly building its customer success and solutions teams in the region to activate its growth plans. Meanwhile, globally, iQuanti’s patented enterprise search channel platform ALPS was named the second-best search software in the U.S by the Drum Search Awards in 2018. ALPS uses proprietary data science and machine learning models to build predictive enterprise-level roadmaps that deliver strong ROI. iQuanti has also featured in the 2019 Inc. list of fastest-growing private companies in the US, for a record fifth time.
DarkCyber assumes that the top search engine is LucidWorks.
Now you know or maybe not.
Stephen E Arnold, September 23, 2019
Can a Well Worn Compass Help Enterprise Search Thrive?
September 4, 2019
In the early 1990s, Scotland Yard (which never existed although there is a New Scotland Yard) wanted a way to make sense of the data available to investigators in the law enforcement sector.
A start up in Cambridge, England, landed a contract. To cut a multi year story short, i2 Ltd. created Analyst’s Notebook. The product is now more than a quarter century old, and the Analyst’s Notebook is owned by IBM. In the span of five or six years, specialist vendors reacted to the Analyst’s Notebook functionalities. Even though the early versions were clunky, the software performed some functions that may be familiar to anyone who has tried to locate, analyze, and make sense of data within an organization. I am using “organization” in a broad sense, not just UK law enforcement, regulatory enforcement, and intelligence entities.
What were some of the key functions of Analyst’s Notebook, a product which most people in the search game know little about? Let me highly a handful, and then flash forward to what enterprise search vendors are trying to pull off in an environment which is very different from what the i2 experts tackled 25 years ago. Hint: Focus was the key to Analyst’s Notebook’s success and to the me-too products which are widely available to LE and intel professionals. Enterprise search lacks this singular advantage, and, as a result, is likely to flounder as it has for decades.
The Analyst’s Notebook delivered:
- Machine assistance to investigators implemented in software which generally followed established UK police procedures. Forget the AI stuff. The investigator or a team of investigators focused on a case provided most of the brain power.
- Software which could identify entities. An entity is a person, place, thing, phone number, credit card, event, or similar indexable item.
- Once identified, the software — influenced by the Cambridge curriculum in physics — could display a relationship “map” or what today looks like a social graph.
- Visual cues allowed investigators to see that a person who received lots of phone calls from another person were connected. To make the relationship explicit, a heavy dark line connected the two phone callers.
- Ability to print out on a big sheet of paper these relationship maps and other items of interest either identified by an investigator or an item surfaced using maths which could identify entities within a cluster or an anomaly and its date and time.
Over the years, other functions were added. Today’s version offers a range of advanced functions that make it easy to share data, collaborate, acquire and add to the investigative teams’ content store (on premises, hybrid, or in the cloud), automate some functions using IBM technology (no, I won’t use the Watson word), and workflow. Imagery is supported. Drill down makes it easy to see “where the data came from.” An auditor can retrace an investigator’s action in order to verify a process. If you want more about i2, just run a Bing, Google, or Yandex query.
Why am I writing about decades old software?
The reason is that is read an item from my files as my team was updating my comments about Amazon’s policeware for the October TechnoSecurity & Digital Forensics Conference. The item I viewed is titled “Thomson Reuters Partners with Squirro to Combine Artificial Intelligence Technology and Data to Unlock Customer Intelligence.” I had written about Squirro in “Will Cognitive Search (Whatever That Is) Change Because of Squirro?”
I took a look at the current Squirro Web site and learned that the company is the leader in “context intelligence.” That seemed similar to what i2 delivered in the 1990s version of Analyst’s Notebook. The software was designed to fit the context of a specific country’s principal police investigators. No marketing functions, no legal information, no engineering product data — just case related information like telephone records, credit card receipts, officer reports, arrest data, etc.
Squirro, founded in 2012 or 2013 (there are conflicting dates online) states that the software delivers
a personalized, real-time contextual stream from the sea of information directly to your workplace. It’s based on Squirro’s digital fingerprint technology connecting personal interests and workflows while learning and refining as user interactions increase.
I also noted this statement:
Squirro combines all the different tools you need to work with unstructured data and enables you to curate a self-learning 360° context radar natural to use in any enterprise system. ‘So What?’ Achieving this reduces searching time by 90%, significantly cutting costs and allows for better, more effective decision-making. The highly skilled Swiss team of search experts has been working together for over 10 years to create a precise context intelligence solution. Squirro: Your Data in Context.
Well, 2013 to the present is six years, seven if I accept the 2012 date.
The company states that it offers “A.I.-driven actionable Insights,” adding:
Squirro is a leading AI-platform – a self-learning system keeping you in the know and recommending what’s next.
I’m okay with marketing lingo. But to my way of thinking, Squirro is edging toward the i2 Analyst’s Notebook type of functionality. The difference is that Squirro wants to serve the enterprise. Yep, enterprise search with wrappers for smart software, reports, etc.
I don’t want to make a big deal of this similarity, but there is one important point to keep in mind. Delivering an enterprise solution to a commercial outfit means that different sectors of the business will have different needs. The different needs manifest themselves in workflows and data particular to their roles in the organization. Furthermore, most commercial employees are not trained like police and intelligence operatives; that is, employees looking for information have diverse backgrounds and different educational experiences. For better or worse, law enforcement intelligence professionals go to some type of training. In the US, the job is handled by numerous entities, but a touchstone is FLETC. Each country has its equivalent. Therefore, there is a shared base of information, a shared context if you will.
Modern companies are a bit like snowflakes. There’s a difference, however, the snowflakes may no longer work together in person. In fact, interactions are intermediated in numerous ways. This is not a negative, but it is somewhat different from how a team of investigators worked on a case in London in the 1990s.
What is the “search” inside the Squirro information retrieval system? The answer is open source search. The features are implemented via software add ons, wrappers, and micro services plus other 2019 methods.
This is neither good nor bad. Using open source reduces some costs. On the other hand, the resulting system will have a number of moving parts. As complexity grows with new features, some unexpected events will occur. These have to be chased down and fixed.
New features and functions can be snapped in. The trajectory of this modern approach is to create a system which offers many marketing hooks and opportunities to make a sale to an organization looking for a solution to the ever present “information problem.”
My hypothesis is that i2 Analyst’s Notebook succeeded an information access, analysis, and reporting system because it focused on solving a rather specific use case. A modern system such as a search and retrieval solution that tries to solve multiple problems is likely to hit a wall.
The digital wall is the same one that pushed Fast Search & Transfer and many other enterprise search systems to the sidelines or the scrap heap.
Net net: Focus, not jargon, may be valuable, not just for Squirro, but for other enterprise search vendors trying to attain sustainable revenues and a way to keep their sources of funding, their customers, their employees, and their stakeholders happy.
Stephen E Arnold, September 4, 2019
The New Lingo of Enterprise Search
August 28, 2019
Enterprise search is back. My Google Alert has been delivering market research reports which tell me that finding information is huge. Plus, there have been some announcements about funding which have surprised me. Examples include:
- Capacity raised $13.2 million. Source: DarkCyber
- LucidWorks snagged an additional $100 million. Source: Globe News Wire
- Squirro pulled in additional funds, but the timing of the Salesforce investment and additional funding of this Zurich based company remains a bit of a mystery. Source: Venture Lab
These are just three examples plucked from my box of note cards about search vendors.
What’s interesting is the lingo, the jargon, and the argot these outfits are using. Frankly the plumbing is usually open source, a fact which the companies bury beneath the blizzard of buzzwords.
Here are some examples:
AI powered
actionable insights
artificial intelligence
cloud
cognitive
connect the dots
data mining
fusion
information mining
machine learning
natural language
pattern detection
platform
self learning
transform
The problem with the vendors collecting investment funds are easy to identify:
- The content processed is text. The unstructured information in videos, podcasts, messaging apps like WhatsApp, images like chemical structures and engineering drawings, etc. are not included.
- Indexing content residing on cloud platforms may work today, but as market dynamics shift, access to that content my be blocked or prohibited by regulations in certain countries
- Federation, on-the-fly so that real time information is available remains a challenge which typically requires script fiddling or new content filters
- Configuration of “smart” systems is not significantly different from the complex, time consuming, and expensive procedures which added friction to some Autonomy, Convera, Fast Search & Transfer, and similar systems’ deployment
- Maintenance is an issue, micro services work well in a low latency environment. Under loads, the magic of sub three second response can disappear
- Search remains an idiosyncratic solution. Many departments require specific features. As a result, enterprise search — regardless of the wrappers around open source information retrieval systems — is a series of customizations.
To sum up, enterprise search has failed to deliver for more than 50 years. Despite the optimism that investors have for “finding the next Google”, enterprise search vendors will find themselves hitting a revenue ceiling just as Autonomy, Fast Search, and similar firms did.
The fix was acquisitions and allegations of financial fancy dancing. If we assume that investors still dream of a 10x or higher return, is it possible that LucidWorks can generate sufficient revenue to pull off an IPO or a sale like Exalead, Vivisimo, and other search vendors were able to complete before the hammer fell?
This is an important question because new enterprise search vendors are popping up like mushrooms. The incumbents like Attivio, Coveo, Mindbreeze, and Sinequa are also trying to smash a ball over the fence.
Net net: Enterprise search appears to be putting on the worn slippers last used by the founders of Fast Search & Transfer. Maybe Microsoft will buy another enterprise search vendor? The problem is that enterprise search is easy to make visible with marketing LED lights. Delivering sustainable revenues is a far greater challenge when Amazon is a competitor and a platform enabler.
What happens when Amazon competes more aggressively, raises its prices, or bundles text search into another of its services?
Answer: Nothing particularly beneficial for the investors in new and improved enterprise search solutions based on Lucene/Solr and dusted with disco glitter.
Stephen E Arnold, August 28, 2019
-
- Subscribe to Beyond Search
Feature archive
News archive
-
