Elastic: The Add Value to Open Source Outfit Bounces Along
November 25, 2020
Elastic Adds New Features to Enterprise Search, Observability, and Security Solutions
Search and data-management firm Elastic has some new features to crow about. BusinessWire posts “Elastic Announces Innovations Across its Solutions to Optimize Search and Enhance Performance and Monitoring Capabilities.” One new tool is Kibana Lens, a visual data analysis tool with a drag-and-drop interface described as intuitive. There is also a beta launch of the searchable snapshots, an efficient way to manage data storage tiers with searchable snapshots. The press release tells us:
“New expanded Elastic Observability features, including user experience monitoring and synthetics, give developers new tools to test, measure, and optimize end-user website experiences. The launch of a new dedicated User Experience app in Kibana provides Elastic customers with an enhanced view and understanding of how end users experience their websites. In addition, Elastic customers can use the new user experience monitoring feature to review Core Web Vitals, helping website developers interpret digital experience signals. Elastic users can also leverage a dev preview release of synthetic monitoring in Elastic Uptime to simulate complex user flows, measure performance, and optimize new interaction paths without impact to a website’s end users. The combination of these two new observability features gives Elastic customers a deeper view of their customers’ digital experience before and after a site update is deployed.”
See the write-up for its list of specific updates and features to Elastic’s Enterprise Search, Observability, Security, Stack, and Cloud products. Built around open source software, the company prides itself on its user-friendly products that have been adopted by major organizations around the world, from Cisco to Verizon. Elastic began as Elasticsearch Inc. in 2012, simplified its name in 2015, and went public in 2018. The company is based in Mountain View, California, and maintains offices around the world.
Cynthia Murrell, November 25, 2020
Enterprise Search: Still Crazy after All These Years
November 20, 2020
This is not old wine in new bottles. This is wine in those weird clay jars with the nifty moniker “amphora” filled with Oak Leaf Vineyards Sauvignon Blanc White Wine. Cough, cough.
CMS Wire gets it correct when it declares, “Scanning and Selecting Enterprise Search Results: Not as Easy as it Looks.” The article doesn’t even approach the formation of a query—finding the right wording then tweaking filters and facets to produce a manageable list. Here we are only looking at the next step. Though the task seems simple on its surface—scan a list of results and select the most relevant ones—writer Martin White explains why it is not so straightforward.
First is scanning results. Users’ perceptual speed differs, so for some folks (like those who are dyslexic, for example) the process can be so tedious as to make searching pointless. White tells us that inconvenient fact is often overlooked in the discussion of search functionality. Also under-considered is the issue of snippet length. A bit of research has been performed, but it involved web pages, which are themselves more easily scanned and assessed than content found in enterprise databases. Those documents are often several hundred pages long, so ranking algorithms often have trouble picking out a helpful snippet. Some platforms serve up a text sequence that contains the query term, others create computer-generated summaries of documents, and others reproduce the first few lines of each document. Each of these approaches is imperfect. Still others produce a thumbnail of a whole page that contains the search term, and that probably helps many users. However, there are accessibility problems with that method.
White concludes:
“We know from recent research that people may make different decisions from the information they perceive initially as relevant based on their expertise. Equally, most search metrics are based around the notional relevance of the results being presented in response to a query. If the true value of relevance cannot be well judged from the snippet, that calls any metrics associated with query performance (especially precision) into question.
“There are no easy solutions to the issues raised in this column. In the quest for achieving an acceptable user experience the points to consider are:
*Are the techniques used by the search application to create snippets appropriate to the types of content being searched?
*Can the format of snippets be customized by the user?
*How easy is it to scan and assess results from a federated search?
“In the final analysis, it doesn’t matter how sophisticated the search technology is (in terms of semantic analysis, etc.). What matters is if the user can make an informed judgment of which piece of content in the results serves their information requirement, reinforces their trust in the application and maintains the highest possible level of overall search satisfaction.”
Sigh. It seems the more developers work on enterprise search, the more complicated it is to effectively operate. The field has been at it for 50 years, and is still trying to deliver something useful. Still crazy after all these years too.
PS. Our esteemed check writer (Stephen E Arnold) wrote a book about enterprise search with the author of the source document. No wonder this essay seemed weirdly familiar. I had to proofread what turned out to be prose that made the Oak Leaf stuff welcome at the end of an editing day. Cough, cough, eeep.
Cynthia Murrell, November 20, 2020
Survey Says Data Governance Is Important. But What Is Data Governance?
November 20, 2020
Here’s what the Google says governance means: The action or manner of governing. Okay, but what exactly is governing. Google says: Having authority to conduct the policy, actions, and affairs of a state, organization, or people.
Okay, now let’s add the magic word “data,” which is a plural, not a single thing. (That’s what datum means, right?)
Google says: Facts and statistics collected together for reference or analysis.
Let’s put the information together, shall we?
An organization uses authority to conduct policy, actions and affairs to deal with facts and statistics for reference or analysis.
Why care? The answer is found in “Businesses Positive about Data Governance but Still Struggle with Privacy Concerns.”
Okay, now we have linked dealing with information and privacy. This is getting interesting or is it? I go with the “not interesting,” but let’s plod forward in the write up.
A vendor of search and retrieval software sponsored a research project conducted by Standard & Poor 451 Research. Note: That report is titled “Pathfinder Report Market Intelligence: Information Driven Compliance and Insight. Two Sides of the Same Coin.” I am not sure about the “coin” metaphor, compliance, insight, and pathfinding. But no one ever accused me of understanding mid-tier consulting firms, sponsored research, and 18 year old vendors of proprietary search and retrieval software.
The 451 outfit tapped its pool of “survey responders” and discovered:
72 percent of enterprises believe data governance is an enabler of business value rather than a cost center.
Okay, that’s a lot of enterprises, assuming the sample was statistically valid, the questions not shaped, and the data analysis of the survey responses was performed on the up and up. But sponsored research is different from the often wonky academic research churned out by professors and work-from-home students. That’s better, right?
I learned:
- One in four organizations have more than 50 distinct data silos
- 37 per cent of respondents say having relevant information automatically displayed, when the team needs it, would benefit them the most in the pursuit of automation.
- Budget, privacy issues, and expertise are barriers.
How does one deal with data silos, which I assume is “governance”? How does one deal with security? Privacy? How does an enterprise search company cope with the assorted sixes and sevens of data in an organization; for example, tweets, encrypted messages, images, geospatial data, videos, and information which must be kept isolated from the grubby “let’s federate information” crowd? (Why must some data be isolated? Find an attorney. Ask her what happens if information in a legal matter is out of her span of control.)
What’s the net net of the mid-tier consulting outfit’s report? Here it is:
Success requires alignment of business objectives by looking for common-denominator requirements across business units.
Let me be clear: Enterprise search is not the solution to problems with an “authority to conduct policy, actions and affairs to deal with facts and statistics for reference or analysis.”
Enterprise search is information retrieval, data governance no matter how much a marketer wishes it were. Enterprise search vendors have been struggling for relevance because Lucene/Solr are good enough and users want information to address right now business issues. Library style lists of stuff to read or look up may not ring the chimes of a thumb typing user.
Want the full report? Go here. Please, keep marketing and governance separate. Statistics 101 offered some useful guidelines. Some, however, did not pay attention. You will have to register. Marketing is still marketing.
Stephen E Arnold, November 20, 2020
Voyager Search Tapped for USDA Search and Discovery Project
November 4, 2020
Low-profile enterprise search company Voyager Search just made an important deal with a high-profile government agency. AIThority announces, “New Light Technologies and Voyager Search Team Win New Contracts with the U.S. Department of Agriculture to Implement Data Search and Discovery Solutions.” Voyager’s partner in the project, New Light Technologies (NLT), is a consulting firm working in the areas of cloud tech, cybersecurity, software development, data analytics, geospatial tech, and scientific R&D. The write-up reports:
“Access to accurate information is crucial to the department’s mission to support sustainable agriculture production and protection of natural resources. Both NLT and Voyager Search bring many years of experience developing award-winning federal data integration and dissemination platforms and will build federated data search solutions to index and link disparate cloud-based and on-prem data sources, including large repositories of imagery and geospatial data files that are used for a variety of analytical reporting and data dissemination systems, such as the Global Agricultural Information Network, Global Agricultural & Disaster Assessment System, Crop Explorer, and the Geospatial Data Gateway. Leveraging NLT and Voyager Search’s Professional Services Department and Vose technology which provides robust spatial search capabilities, the team’s solution will enable users to search for data, content, and documents by who, what, when, and where. Together, the team is providing the technology and services to advance a modern data architecture for the department that will support improved information flow, security, and analysis as well as power the Artificial Intelligence (AI) and Machine Learning (ML) of the future.”
“Voyager” is a popular name for a business, so do not confuse Voyager Search with other enterprises like digital innovation firm Voyager, manufacturer Voyager Industries, or even the Voyager Company that pioneered DC-ROM production back in the day. Vose is the name of Voyager Search’s platform that will be used for the USDA project, but the company also offers Server, essentially Vose for larger implementations, and ODN (Open Data Network), a searchable global-content catalog. Both products build on Vose’s “smart spatial search” technology. Based in Redlands, California, Voyager Search was founded in 2008.
Cynthia Murrell, November 4, 2020
Persistent: Enterprise Search and Cloud Expertise
October 22, 2020
I checked my enterprise search files. Sure enough, Persistent Systems is in the enterprise search game. You can get a sense of the firm’s consulting orientation if you download and study “An Essential Primer on Enterprise Search Evaluation.” Yep, evaluation. Most organizations have employees who need to locate information: Text, videos, PowerPoints on clever sales professionals’ work laptops, documents generated by the less-than-forthcoming legal department, and information about recreational softball in the Era of the Rona. We noted that the company acquired Capiot. This is a company which provides integration services. To sum up, “enterprise search” appears to be a consulting services operation at Persistent. With a workable search solution available as open source, renting people who can allegedly make search perform magic tricks seems logical. But what about rich media, tweets, silos of data, and uncooperative sales professionals who tweak slide decks moments before making a pitch to up the chances for a sale? Let’s not dig too deeply into the contents of the “Essential Primer,” shall we? Enterprise search appears to be a synonym for “consulting.”
Stephen E Arnold, October 22, 2020
Another Crazy Enterprise Search Report
October 18, 2020
“Enterprise Search Market Investment Analysis | Dassault Systemes, Oracle, HP Autonomy, Expert System Inc.” may be a knock out report, but its presentation of the company’s nuanced understanding is like hitting an egg with a feather. The effort appears to be there, but the result is an intact egg.
You can learn about this omelet of a report at this link. The publisher is PRnewsleader, which seems to be one of the off brand SEO centric content outputters.
The first thing I noticed about this report was the list of vendors in the document; to wit:
Coveo Corp.
Dassault Systèmes
Esker Software
Expert System
HP Autonomy
IBM Corp.
Lucidworks
Marklogic
Microsoft
Oracle
Perceptive Software
Polyspot and Sinequa
SAP
What jumped out at me was the inclusion of Polyspot and Sinequa. Polyspot was acquired years ago by an outfit called oppScience. The company offers Bee4Sense and list information retrieval as a solution. As far as I know, oppScience is a company based in Paris, not on a street once known for fish sales. Sinequa is a separate company. True, it once positioned itself as an enterprise search developer. That core capability has been wrapped in buzzwordery; for example, “insight platform.” Therefore, listing two companies incorrectly as one illustrates a minor slip up.
I also noticed the inclusion of Esker Software. This company is a process automation outfit, and it says that it has an artificial intelligence capability. (Doesn’t every company today?) Esker is into the cloud, and its search technology is a bullet point, not the white paper/journal article/rah rah approach used by Lucidworks.
And what about Elasticsearch? What about Algolia (former Dassault Exalead DNA I heard)? What about Voyager Search? What about Maxxcat? And there are other vendors.
What’s amusing is that the authors of this report are able to set forth:
forecasts for Enterprise Search investments till 2029.
Okay, that’s almost a decade in the Era of the Rona. I am not sure what’s going on tomorrow. Predicting search in 2029 is Snow Crash territory. But I am confident the authors of this report are intrepid researchers who just happened to overlook the Polyspot Sinequa mistake. What else has been overlooked?
Stephen E Arnold, October 18, 2020
Hulbee Is In the Enterprise Search Derby
June 18, 2020
Enterprise search should be an easy out-of-the-box, deployable solution, but more often it is a confusing mess. Companies like Hulbee Enterprise Search develop search programs that delete the guesswork and immediately function:
“Hulbee Enterprise Search not only provides a simple search software, but also consolidates our experience and knowledge, which has been accumulated for over 17 years and combines intelligent search, format diversity, different corporate infrastructures, security, etc. in areas such as document management.
Our goal is to create a timely software technology for you that meets all security requirements. We would be very pleased if you test our software. Request a Proof of Concept.
Our software complements existing software products from other manufacturers such as SharePoint, Exchange, DMS etc. through the innovation of the search. It is thus not a competition, but an addition to and completion of the optimal search in the company.”
The purpose of enterprise search is to quickly locate information, so it can be employed by a business. Information includes structured and unstructured data, so enterprise search needs to be robust and smart enough to filter relevant results. Search must also be compliant with security measures, especially as more businesses host their data on clouds.
Enterprise search solutions like Hulbee must be flexible enough to adjust to changing security measures, but also continue to offer the same and better features for search.
Customization is key to being a contender in the marker for enterprise search.
Whitney Grace, June 18, 2020
AWS Kendra: A Somewhat Elastic Approach to Enterprise Search
May 12, 2020
Elastic, Shay Banon’s Version 2 of Compass, has a hurdle to jump over. Elasticsearch has been a success. The Lucene-centric “system” which some call ELK has become a go-to solution for many developers. Like Lucidworks (It does?) and many other “enterprise search and more” vendors, Elasticsearch delivers information retrieval without the handcuffs of options like good old STAIRS III or Autonomy’s neuro-linguistic black box.
Amazon took notice and has effectively rolled out its own version of enterprise search based on … wait for it … the open source version of Elastic’s Elasticsearch. The service has been around since Amazon hired some of the Lucidworks (It does?) engineers more than five years ago after frustration with the revolving doors at that firm became too much even by Silicon Valley standards. Talk about tension. Yebo!
Amazon has reinvented Elasticsearch. The same process the Bezos bulldozer has used for other open source software has been in process for more than 60 months. Like the system’s Playboy bunny namesake, Kendra has a few beauty lines in her AWS exterior.
A tweak here (access to Amazon’s smart software) and a tweak there (Amazon AWS pricing methods), and the “new” product is ready for prime time, ready for a beauty contest against other contestants in the most beautiful IR system in the digital world.
“Amazon Launches Cognitive Search Service Kendra in General Availability” reports:
Once configured through the AWS Console, Kendra leverages connectors to unify and index previously disparate sources of information (from file systems, websites, SharePoint, OneDrive, Salesforce, ServiceNow, Amazon Simple Storage Service, relational databases, and elsewhere).
Does this sound like federated search or the Palantir Gotham approach to content?
Well, yes.
The reason is that most enterprise search vendors like Coveo, Attivio, X1, IBM Omnifind (also built on Lucene), and dozens of other systems make the same claims.
The reality is that these systems do not have the bits and pieces available within a giant cloud platform with quite a few graduates of an Amazon AWS training program ready to plug in the AWS solution. For example, if a government agency wants the search in Palantir, no problem. Palantir deploys on AWS. But if that government agency wants to use Amazon’s policeware services and include search, there’s Kendra.
You can get a free copy of the DarkCyber Amazon policeware report’s executive summary by requesting the document at this link.
What does Amazon bring to the enterprise search party?
The company has more than 200 services, features, component, and modules on the shelf. Because enterprise search is not a “one size fits all”, the basic utility function has to fit into specific enterprise roles. For most enterprise search vendors, this need for user function customization is a deal breaker. Legal doesn’t want the same search that those clear minded home economics grads require in the marketing department. Microsoft SharePoint offers its version of “enterprise search” but paints over the cost of the Microsoft Certified professionals who have to make the search system work Fast. (Yep, that’s sort of an inside search joke.)
Amazon AWS provides the engine and the Fancy Dan components can be plugged in using the methods taught in the AWS “learn how to have a job for real” at a company your mom uses to shop during the pandemic. Amazon and Microsoft are on a collision course for the enterprise, and the Kendra thing is an important component.
The official roll out is capturing headlines, but the inclusion of Lucene-based search invites several observations:
- Despite AWS’ pricing, an Amazon enterprise search system allows the modern information technology professional to get a good enough service with arguably fewer headaches than other options except maybe the SearchBlox solution
- Enterprise search becomes what it has been for most organizations: A utility. Basic information retrieval is now an AWS component and that component can be enhanced with SageMaker, analytics, and other AWS services.
- Amazon wins even if Kendra does not win the hearts and minds of IBM Omnifind, Inbenta, and Algolia users. Why? Most of the cloud based enterprise search vendors support the AWS platform. What are the choices? The wonky HP cloud? The “maybe we will kill it” Google Cloud? Azure, from the outfit that cannot update Windows 10 without killing user computers who activate game mode? Plus, dumping Kendra for another TV star inspired search system is easy. Chances are that, like Palantir, AWS hosts and supports that competitive system too.
Net net: The fight with Microsoft is escalating. The Bezos bulldozer will run over open source outfits and probably some AWS customers. But Kendra’s turning her gaze on the bountiful revenues of Microsoft in the enterprise. Will Amazon buy a vendor of Word, PowerPoint, and Excel clones?
Exciting times, maybe not just because of enterprise search? Why did those defectors from Lucidworks (It does?) embrace Lucene and not SOLR? Maybe they did that too?
Stephen E Arnold, May 12, 2020
The Crazy Search Market Report: May 4, 2020 Edition
May 4, 2020
DarkCyber is pondering a new feature called “Crazy Search Report.” The “search” refers to enterprise search. The “crazy” refers to the assertions and marketing hoo-hah in news releases about the steady stream of in depth analysis of the market for enterprise search.
Yeah, I know that sounds crazy. Well, that’s why the crazy search report will be useful. We will identify the producer of the report and include some content from the news releases issued to cheerlead for these five figure “believe it or not” compilations.
Let’s look at the first report in the series.
This is called “Global Coronavirus Impact and Implications on Enterprise Search Market Research Report 2019 Analysis and Forecast To 2029.” The publicity was generated by something called factmr.com on May 1, 2020.
Here’s the hook paragraph, that’s the one that will make you buy the report from Research Moz. Is this outfit in the same league as Bain, Booz, Allen, and McKinsey? You decide:
Companies in the Enterprise Search market are vying suggestive steps to tackle the challenges resulting from the COVID-19 (Coronavirus) pandemic. Exhaustive research about COVID-19 is providing present-day techniques and alternative methods to mitigate the impact on Coronavirus on the revenue of the Enterprise Search market.
The news release also talks about the enterprise search landscape. I remember writing a book, published by Pandia Press, called “The Landscape of Enterprise Search.” Happy coincidence maybe?
Are these some factoids to make you want to buy this report? Sure, for example:
…The report ponders over the various factors that are likely to impact the overall dynamics of the Enterprise Search market over the forecast period (20XX-20XX) including the current trends, business expansion opportunities and restraining factors amongst others. As per the market report suggested by ResearchMoz.us, the global Enterprise Search market is expected to register a CAGR growth of ~XX% during the forecast period and attain a value of ~US$XX by the end of 20XX.
Okay, the numbers are left out. You have to pay before you get the alleged factoids. That definitely makes me lust after a copy not.
What’s interesting is the list of companies allegedly profiled and X-rayed in the document. Note: I alphabetized the company names, but the Moz outfit does not bother with this convention. My comments are in parentheses.
Attivio Inc (business intelligence maybe?)
Concept Searching Limited (Microsoft add in from far, far way from the US)
Coveo Corp (customer support and assorted buzzwords)
Dassault Systemes (product engineering search)
Expert System Inc (semantic utility, focus on mobile)
Google (not in the search business)
Hyland (ISYS Search which dates from the 1980s)
IBM Corp (Lucene plus Vivisimo plus home grown code and Watson. I can’t forget Watson.)
Lucid Work (Typo. The company’s name is Lucidworks.)
Micro Focus (Autonomy)
Oracle (Secure Enterprise Search, RightNow, Endeca, and others)
Marklogic Inc (XML database company with proprietary extensions)
Microsoft (Fast Search & Transfer plus assorted acquisitions and home grown tomfoolery)
SAP AG (Who knew?)
X1 Technologies (desktop search and eDiscovery originally from Idealab year ago)
How much is the report? The news release does not say, but we held our breath and clicked the Research Moz link and learned that the document costs $3,900.
That’s it. Crazy stuff for a crazy market sector.
Stephen E Arnold, May 4, 2020
Enterprise Search Craziness: Destiny Adjacency
May 1, 2020
The enterprise search vendors are not to blame. The finger of ineptitude writes boldly:
Enterprise Search Market each qualitative and quantitative records analysis to provide an overview of the destiny adjacency around Enterprise Search Market for the forecast duration, 2020-2025.
You can read the original at this link. Enterprise search has tried a number of snappy phrases to make a utility the potent heart of a 21st century enterprise; for example:
- Semantic meaning
- Natural language processing
- Artificial intelligence
- Precision, recall, and relevance. Yikes, delete those loser words.
DarkCyber believes that “destiny adjacency” is the all-time leader in the meaningless baloney fest that is pulled into the orbit of enterprise search.
Yep, “destiny adjacency”. Maybe a T shirt? A tattoo?
Stephen E Arnold, May 1, 2020
-
- Subscribe to Beyond Search
Feature archive
News archive
-
