Attivio: Dines on Data Dexterity
March 7, 2016
Attivio was founded by some former Fast Search & Transfer executives. Attivio also had a brush with a board member who found himself in a sticky wicket. Quite a pedigree.
I read “Enterprise Search Takes Its Place at the Big Data Table.” The write up is built upon an interview with the chief executive officer of Attivio. Nice looking fellow who had a degree in music and marketing and an MBA from Wharton, the institution which helped educate Donald Trump.
What caught my attention were these points in the write up. My observations are in italics:
- Enterprise search has been around for two decades. [Nah, enterprise search is closing in on 50 years of fun and delight.]
- Enterprise search “finds unstructured content housed in file shares like SharePoint and other content management systems, in email archives, and in the content repositories of applications like customer relationship management. [Yep, and that is part of the problem with enterprise search. The bulk of the systems I have examined do not handle video, audio, binaries, and odd ball file types like those in ANB format very well or not at all. Plus users expect comprehensive results updated in near real time presented in a form which allows instant use.]
- Enterprise search does analytics and accelerated data discovery. [Yep, if the customer licenses a system like BAE NetReveal, the Palantir platform, or another industrial-strength fusion vendor.]
What I found interesting was the phrase “reducing the time to insight.” There is a suggestion from Attivio and from other vendors that their systems process digital content in a super fast mode.
In our testing, we have found that throughput for new content can require considerable investment in engineering and processing capability. Furthermore, dealing with flows from intercepts or other high volume content sources, most enterprise search systems cannot handle:
- Processing large flows of content in a matter of minutes. Hours or days is a more suitable time unit
- Updating the index or indexes
- Integrating real time data into search results, reports, and visualizations in a dynamic manner.
That’s why outfits who are emulating Palantir-style information access use open source search and then invest hundreds of millions in specialized engineering, interfaces, and fusion technologies.
Enterprise search vendors chasing Palantir-type systems are delivering what marketers find quite easy to describe. Here’s an example:
Not only that, but many enterprises can only “see” 10 percent of their data. The other ninety percent remains hidden—dark data. Data is often locked in silos, and it’s just too time-consuming to get it out. And making connections across structured, semi-structured, and unstructured information to serve to a BI tool is a completely manual, slow process – although highly valuable for developing strategic insights. Organizations that can cross this chasm will be poised to transform productivity, mitigate risks, and seize market opportunities.
The only hitch in the git along is that systems which handle “dark data” are available now. There are outfits able to handle “dark” data today. True, these are not based on enterprise search concepts because the core of a utility function is not a solid foundation for next generation information access. There are platforms which deliver actionable outputs. Even more interesting is that the US government is funding research to develop next generation systems designed to leap frog Palantir, i2, DCGS-A, and many other solutions.
Why?
Marketing is one thing. Delivering a system which works reliably, exhibits consistency, and integrates with work flows is a work in progress.
The notion that a Fast-type system can deliver what a Palantir-type system does is something I believe is wordsmithing. Watson does wordsmithing; others deliver next generation information access. Has Attivio hit a home run with its new positioning? Is the Attivio solution a starter for the Hickory Crawdads? My hunch the folks investing $70 million in Attivio want to start for the Boston Red Sox this year. Play ball.
Stephen E Arnold, March 7, 2016
Venture Backed Search Vendors Face Exciting 2016
February 12, 2016
I read “The State of Venture Capital.” I thought, “Oh, ho, here comes the tightening of the thumbscrews. The idea is simple. Insert fingers and turn the crank. My hunch is that the device will focus the attention of person whose fingers are in the business end of the gizmo.
In the write up, I learned that in the next two years, folks should expect:
- Increased loss ratios
- Most flat rounds
- More down rounds
- More structured rounds
- Relatively harder to raise capital
- VCs marking-to-market showing some movements south
I like the reference to the movement south.
How does this relate to the search and content processing outfits which have sucked in tens of millions in venture funding? Three items for which I will be watching:
- More market repositioning. Think predictive analytics, data lakes, cloud solutions, and artificial intelligence. Talk is cheap. If talk generates a license deal, that’s the upside.
- Downsizing. I know that growth is all the rage, but I think that some vendors will have no choice except cutting back on expenses. Full time hires become contract workers. Trade show participation becomes a webinar which is archived and the promoted as a resource.
- Dance card shuffling. In an effort to generate leads and from the leads some real license deals, companies will join up. Others will departner and find another entity with which to dance.
Which search vendors will survive? The last big shake out winnowed the likes of Convera, Delphes, Entopia, and Siderean. The acquisition boomlet moved Autonomy, Exalead, ISYS Search, and Vivisimo into the safe havens of larger organization. Who will buy today’s market leaders? Other vendors will have no choice but go quiet. The last time I checked Dieselpoint it was still in business. Sophia Search? Intrafind? X1?
Which company is the next Autonomy? Elastic, Recommind, IBM Watson?
My view is that 2016 will be exciting for some folks.
Stephen E Arnold, February 12, 2016
Google Search Appliance: Like Glass It Broke
February 8, 2016
I read “So Long Google Search Appliance.” Farewell, happy yellow and blue boxes. So long integrators who have been supporting these wildebeests for a decade. Au revoir easy-as-pie search.
According to the write up:
The tech giant told its reseller and consulting partners the news via email on Thursday, noting that they can continue to sell one-year license renewals for existing hardware customers through 2017, but that they will be unable to sell new hardware. Renewals will end in 2018.
I recall writing about the Google Search Appliance when I was reporting about enterprise search for specialist publications. I was the first or one of the first to run down the pricing for the wonky boxes. I pointed out that a redundant multi million document system would ring the Google cash register in the high six figures with seven figures not out of sight. I thought I mentioned that the number of engineeers supporting the GSA had dwindled to a couple of folks. I thought I pointed out that the assumption a Web search system would work like a champ on corporate content was a wild and crazy notion.l
Like so many others who assumed enterprise search was not a tough problem, the Alphabet Google thing has bailed. Google essentially failed to revolutionize enterprise search. Cheaper and more usable appliances are available, including products from Maxxcat and Thunderstone. There are reasonable cloud solutions. And there is a cornucopia of outfits offering repackaged open source systems. Heck, if one pokes around long enough, a bold enterprise can license a system from companies with proprietary information access systems; 3RDi, Fabasoft, Lexmark, etc.
What will organizations do without the Google Search Appliance? Yard sale, Goodwill?
Stephen E Arnold, February 8, 2016
3RDi for Enterprise Search
February 5, 2016
Health and medical search need an upgrade? T/DG 3RDi might be just what the doctor ordered. You search blues will disappear when you have natural language processing, semantic search, search relevancy, search analytics, research tools, and data integration. Very comprehensive it seems.
T/DG offers 3RDi. Now try to search for these entities. To locate the services firm offering the 3RDi system, one has to figure out how to make Bing, Google, and Yandex point to the correct entities.
Naming products and companies is tricky. Let me save you the hassle of wading through false drops.
- T/DG means “The Digital Group,” an outfit founded in 1999 and operating from New Jersey.
- 3RDi means “relevant, deep insights.” (I don’t know what the 3 means.)
The search system appears to be a “platform” based on open source technology. Here’s a block diagram of 3RDi:
Source: The Digital Group, 2015
The company’s most recent push is health care. The search system performs the type of functions which I associate with a system like the ones Autonomy and Fast Search & Transfer described in the late 1990s. There is also a hefty dose of “platformitis.” The idea is that a licensee can use the system to meet the needs of users. The support for controlled vocabularies is helpful in domain specific deployments, but these have to be maintained, which can be a financial and resource burden for some licensees.
3RDi embraces the semantic marketing jargon enthusiastically; for example, this diagram shows how “knowledge” and “semantics” make the “experience” work for licensees:
Source: The Digital Group, 2015
Users of the system do not have to deal with results lists. The system presents information in a visual manner; for example:
Source: The Digital Group, 2015
In short, 3RDi appears to deliver the type of utility I associate with systems from outfits like BAE Systems and Palantir.
If your organization wants an open source system with the bells and whistles found in seven figure platforms, you may want to explore 3RDi.
The urls you need are:
I assume that the company will make the “3” clearer going forward. There is a live demo available. You will need to register. The system balks at non commercial domains like my Yahoo account.
The recent marketing push given 3RDi signals that the enterprise search sector is alive and well. As the company says, “Start experiencing.” I wonder what the “3” means.
Stephen E Arnold, February 5, 2016
Proprietary Enterprise Search: False Hopes and Brutal Costs
December 21, 2015
At lunch the other day, the goslings and I engaged in what I thought was a routine discussion: The sad state of the enterprise search market.
I pointed out that the “Enterprise Search Daily” set up by Edwin Stauthamer was almost exclusively a compilation of Big Data articles. Enterprise search, although the title of the daily, was not the focal point of the content.
Enterprise search is a cost black hole. R&D, support, customization, and bug fixes gorge on money and engineers. Instead of adding value to an enterprise system, search becomes the reason the CFO has a migraine and why sales professionals struggle to close deals.
I said, “Enterprise search has disappeared.”
One of the goslings asked, “What’s happened to the proprietary search systems acquired by some big companies?”
We were off an running.
The goslings mentioned that Dassault Systèmes bought Exalead and the brand has disappeared from the US market. IBM bought Vivisimo, and the purchase was explained as a Big Data buy, but the company and its technology have disappeared into the Great Blue Hole, which is today’s IBM. Hummingbird bought Fulcrum, and then OpenText bought Hummingbird. Open Text owns Information Dimension’s BASIS, BRS Search, and its own home brew search system. Oracle snapped up Endeca, InQuira, and RightNow in a barrage of search binge shopping. Lexmark—formerly a unit of Big Blue—bought ISYS Search Software and Brainware. Then there was the famous purchase of Fast Search & Transfer by Microsoft and the subsequent police investigation and the charges filed against a former executive for fancy dancing with the revenue numbers. And who can forget the $11 billion purchase of Autonomy by IBM. There have been other deals, and the goslings enjoyed commenting on this.
I called a halt to the lunch time stand up comedy routine. The executives of these companies were trying to do what they thought was best for their [a] financial future and [b] for their stakeholders. Some of these stakeholders had suffered through revenue droughts and were looking for a way out of the sea of red ink enterprise search vendors generate with aplomb.
The point I raised was, “Does the purchase of a proprietary enterprise search system?” make a substantive contribution to the financial health of the purchasing company.
Canada Based Coveo Predicts the Future of Enterprise Search
December 17, 2015
Coveo, an enterprise search vendor once closely focused on Microsoft SharePoint, offered some predictions about 2016 and search.
What does the Québec City based search vendor presage? The answer appears in “Coveo Releases 2016 Enterprise Search Predictions.” Here you go:
- Machine learning. IBM will definitely support this prediction or their public relations advisors will.
- Cybersecurity. Yep. This is a hot area. Some of the companies pushing new boundaries in search are outfits like Diffeo, Tiberian, and Recorded Future (backed by the Alphabet Google outfit). Key word search does not sleep in the same king sized bed as these outfits in my opinion.
- Real time engagement and learning for digital transformation. I thought of Yahoo. The company has several thousand people working on search. How is that working out for the Xoogler running Yahoo?
- Proactive recommendations. Okay, this is the selective dissemination or SDI thing. SDIs are useful but these cannot be annoying. Does anyone remember Desktop Data? I do.
- Search makes systems more intelligent. Well, maybe. search is a utility. The more successful of the intelligence automation outfits uses subcomponents of search; for example, entity extraction, metadata generated on the fly, and relationship maps. These are, in my opinion, more important than old school search.
The write up includes a remarkable quote. I have placed this gem in my folder for future reference. Here is the statement I highlighted in bold red marker ink:
2016 will be a pivotal year for enterprise search. Organizations now recognize the strategic value of using intelligent search-based applications to drive more customer and employee engagement.
Based on the information available to me as we complete The Dark Web Dilemma, 2016 is going to be a bit different from 2015 when venture money flowed like water to outfits in the content processing game. My thought about the future is that companies which have ingested oodles of cash have to generate revenue growth, demonstrate a path to profitability, or face a Convera, Delphes, Entopia, or Siderean Software type future.
What’s that type of future?
A sell out or a shut down. My hunch is that the stakeholders in the Coveo play are looking for a buyer just like Lexmark.
Stephen E Arnold, December 17, 2015
Big Data and Enterprise Search: The Caution Lights Are Flashing
December 15, 2015
I read “How You Should Explain Big Data to Your CEO.” The write up included a link which triggered thoughts of how enterprise search dug itself a hole and climbed. Unable to extricate itself from a problem enterprise search vendors created, the entire sector has been marginalized. In some circles, enterprise search is essentially a joke. “Did you hear about the three enterprise search vendors who walked into a bar?” The bartender says, “What is this? Some kind of joke?”
The link pointed me to a Slideshare (owned by the email and job hunting champion LinkedIn). That presentation, “5 Signs Your Big Data Project is Doomed to Fail,” could have been borrowed from one of my talks about enterprise search in 2001. It was not, but the basic message was identical: Big Data has created a situation in which there are some challenges here and now.
The presentation was prepared by Qubole (maybe pronounced cue ball?). Qubole is a click to query outfit. This means that reports from Big Data are easy to generate.
Here are the problems Big Data faces:
- Failed implementations. Qubole asserts that 87 percent of the Big Data implementations are flops
- 73 percent of executive describe the Big Data project as flop
- 45 percent of Big Data projects are completed
These data are similar to the results of “satisfaction” with enterprise search solutions.
Why? Qubole asserts:
- Inaccurate project scope
- Inadequate management support
- No business case
- Lack of talent (in search the talent may be present but overestimates its ability to deal with enterprise search technologies and processes)
- “Challenging tools.” I think this means that in the Big Data world there are lots of complexities.
What can one charged with either search or Big Data tasks do with this information?
My view is, “Ignore it.”
The “can do” spirit carries professionals forward. Hiring a consultant provides some job protection but does little to reverse the failure and disappointment rate.
My view is that the willingness of executives to grab at a magic solution presented by a showman marketer overrides failure date. The arrogance of those involved create a “that won’t happen to us” belief.
Who is to blame? The company for believing in baloney? The marketer for painting a picture and showing a Hollywood style demo? The developers who created the Big Data solution, knowing that chunks were not complete or just did not work before the ship date? The in house engineers who lacked self knowledge to understand their own limitations?
Everyone is in the hole with the enterprise software vendors. The hole is deep. Magic solutions are difficult to pin down. The future of Big Data is likely to parallel to some degree the dismal track record of enterprise search. Fascinating. I can hear the mid tier consultants and the handful of remaining enterprise search vendors asserting that Qubole’s points are not applicable to their specific situation.
Yep, and I believe in the tooth fairy and Santa.
Stephen E Arnold, December 15, 2015
Paper.Li Enterprise Search Daily Not Updating
December 11, 2015
Update December 12, 2015–The Enterprise Search Paper.li is updating again. The content is primarily about Big Data.
The Enterprise Search Daily, which shifted from enterprise search to Big Data, is not updating. Is it a Paper.li problem or a lack of suitable information about search?
The archives are available from this link. If the page resolves, click on the Archives and you can view a Search Daily by clicking on a calendar. My hunch is that the flow of information about enterprise search slowed in 2015. Marketers have embraced Big Data even if their systems do not really handle lots of information without developing indigestion. Earlier this year, I heard that a real publisher was going to introduce a newsletter about enterprise search. I am not sure if anyone will notice.
If the Enterprise Search Daily turns up in my Overflight system again, I will note the date. For now, fans of enterprise search will have to entertain themselves by running queries on the topics related to search. Big Data is not a useful search phrase. Everybody does Big Data according to the marketers and the venture firms funding search and retrieval. What about the conference organizers who run for fee shindigs to discuss enterprise search? Yikes. I am delighted to be beyond search.
Stephen E Arnold, December 11, 2015
Enterprise Software Survey: ZDNet Content Marketing
December 8, 2015
“Research: Enterprise Software Rising with 69 Percent Usage” caught my attention. I like numbers which suggest two thirds adoption of software which seems ubiquitous to me. So the write looks like news. It sounds like news. Sort of news I think.
The write up is a marketing pitch for a new study by ZDNet and its for fee research arm Tech Pro Research. “Enterprise Software: Advantages, Opportunities, Challenges” studies enterprise software. I am not exactly sure what “enterprise” means in the ZDNet world. My hunch is that the term embraces commercial enterprises which buy products from the ZDNet advertisers. I could be wrong, of course, because enterprise may mean, the local dry launderette entrepreneur.
The article discloses some interesting factoids, which I assume ZDNet and a host of enterprise software sales people believe are solid gold. None of the gold foam stuff.
I noted these points:
Enterprise software has a definition I found surprising. I learned:
Enterprise software encompasses a variety of functions, including asset management, business intelligence, CRM utilities, data processing, databases, financial applications, identity management, retail software, process management and resource planning. It can run on either individual machines or on centralized servers, whether in-house, located in the cloud or a hybrid combination.
No, enterprise search, no analytics, no Big Data, and no enterprise cyber security. Yikes. The mid tier consultants are probably perspiring in their home offices due to these omissions.
I found this factoid interesting: nine percent of the sample do not currently use an enterprise software solution but are considering one in the next year and 22 percent do not use enterprise software and are not planning on changing their stripes next year. That works out to 31 percent of the sample not doing the enterprise software thing even though, I assume, the companies in the sample were enterprises. I said, “Huh. Imagine that.”
Even more puzzling was the list of enterprise software deployed in the last 12 months by the sample.
- Human resources
- Storage
- Databases, note the plural
- Big Data
- Mobile
Now the paragraph I quote about the functions of enterprise software did not include Big Data. What’s the scoop, ZDNet. Is Big Data an enterprise application or not? My view is that Big Data is for the folks who understand analytic-type stuff. Most enterprises have precious few of these types of people. Another fascinating point.
The preferred vendors identified in the write up came at me from left field. I admit I am not in the know like the real journalists at ZDNet. Here’s that listing:
- Adobe Systems
- Dropbox
- LinkedIn (my goodness)
- Microsoft
But the un-preferred vendors is more intriguing. This group includes:
- CA Technologies
- Oracle
- Red Hat
- SAP
I assume that each of these companies will be really thrilled to meet with the ZDNet ad sales professionals.
If you want more, you will be able to explore the opportunities by diving into the for fee version of the study.
Stephen E Arnold, December 7, 2015
Has Enterprise Search Drowned in a Data Lake?
December 6, 2015
I had a phone conversation with a person who unluckily resides in a suburb of New York City. Apparently the long commute allows my contact to think big thoughts and formulate even bigger questions. He asked me, “What’s going to happen to enterprise search?”
I thought this was a C minus questions, but New Yorkers only formulate A plus questions. I changed the subject to the new Chick Fil-A on Sixth. After the call, I jotted down some thoughts about enterprise search.
Here for your contemplation are five of my three comments which consumed three legal pad sheets. I also write small.
Enterprise Search Is Week Old Celery
In the late 1990s when the Verity hype machine was rolling and the Alphabet boys were formulating big thoughts about search, enterprise search was the hot ticket. For some techno cravers, enterprise search was the Alpha and Omega. If information is digital, finding an item of information was the thrill ride ending in a fluffy pile of money. A few folks made some money, but the majority of the outfits jumping into search either sold out or ended up turning off the lights. Today, enterprise search is a utility and the best approach is to use an open source solution. There are some proprietary systems out there, but the appeal of open source is tough to resist. Remember. Search is a utility, not a game changer for many organizations. Good enough tramples over precision, recall, and relevance.
New Buzzwords and the Same Old Tune
Hot companies today do not pound their electric guitar with the chords in findability. Take an outfit like Palantir. It is a search and information access outfit, but the company avoids the spotlight, positions its technology packages as super stealthy magic insight machines. Palantir likes analytics, visualizations, and similar next generation streamlined tangerine colored outputs. Many of the companies profiled in my monograph Cyberosint are, at their core, search systems. But “search” is tucked into a corner, and the amplified functions like fancy math, real time processing, and smart software dominate. From my point of view, these systems are search repackaged and enhanced for today’s procurement professionals. That’s okay. But search is still search no matter what the “visionaries” suggest. Many systems are enterprise search wrapped in new sheet music. The notes are the same.
Big Data
I find the Big Data theme interesting. The idea of processing petabytes of data in a blink is future forward. The problem is that the way statistical procedures operate is to sidestep analyzing every single item. I can examine a grocery list of 10 items, but I struggle when presented with a real time updating of that list with trillions of data points a second. The reality of Big Data is that it has been around. A monk faced with copying two books in a couple of days has an intractable Big Data problem. The love of Hadoop and its extended family of data management tools does not bring the black sheep of the information family into the party room. Big Data requires pesky folks who have degrees in statistics or who have spent their youth absorbed in Mathematica, MatLab, SPSS, or SAS. Bummer. Enterprise search systems can choke on modest data. Big Data kills some systems dead like a wasp sprayed with Raid.
Real Time
For a client in the UK, I had to dig into the notion of real time. Guess what the goslings found. There was not one type of real time information system. I believe there were seven distinct types of real time information. Each type has separate cost and complexity challenges. The most expensive systems were the ones charged with processing financial transactions in milliseconds. Real time for a Web site might mean anything from every 10 second or every week or so. Real time is tough because no matter what technologies are used to speed up computer activities, the dark shadow of latency falls. When data arrive which are malformed, the real time system returns incomplete outputs. Yikes. Incomplete? Yep, missing info. Real time is easy to say, but tough to deliver at a price an average Fortune 1000 company can afford or one of the essential US or UK government agencies can afford. Speed means lots of money. Enterprise search systems usually struggle with the real time thing.
Automatic, Smart Indexing, Outputs, Whatever
I know the artificial intelligence, cognitive approach to information is a mini megatrend. Unfortunately when folks look closely at these systems, there remains a need for slug like humans to maintain dictionaries, inspect outputs and tune algorithms, and “add value” when a pesky third party cooks up a new term, phrase, or code. Talk about smart software does not implement useful smart software. The idea is as appealing today as it was when Fulcrum in Ottawa pitched its indexing approach or when iPhrase talked about its smart system. I am okay with talk as long as the speakers acknowledge perpetual and include in the budget the humans who have to keep these motion Rube Goldberg confections on point. Humans are not very good indexers. Automated indexing systems are not very good indexers. The idea is, of course, that good enough is good enough. Sorry. Work remains for the programmers. The marketers just talk about the magic of smart systems. Licensees expect the systems to work, which is an annoying characteristic of some licensees and users.
Wrap Up
Poor enterprise search. Relegated to utility status. Wrapped up in marketing salami. Celebrated by marketers who want to binge watch Parks and Recreation.
Enterprise search. You are still around, just demoted. The future? Good enough. Invest in hyper marketing and seek markets which do not have a firm grasp of search and retrieval. Soldier on. There are many streaming videos to watch if you hit the right combination on the digital slot machine.
Stephen E Arnold, December 6. 2015
-
- Subscribe to Beyond Search
Feature archive
News archive
-
