LucidWorks Growth Focuses on the User Experience and Search Features
September 27, 2012
We recently commented on the Forbes’ article titled, “LucidWorks: Bringing Search to Big Data” and the rising usage of Lucene/Solr technology across dominating companies such as Netflix, AT&T, and Twitter. The same article also brings to light the fact that the world of data is shifting from mainly numbers to one that is essentially text-based, thanks to outlets such as social networking.
This is where LucidWorks enters. The company is aware that the Web search box is the key to helping any and all users find the information they are looking for, even if they are inept at programming language. This transition will be bringing Big Data to a larger audience with easy-to-use search features. LucidWorks knows search technology is where Big Data needs to focus and the company plans on becoming the leader for this enterprise. Strides in this direction were made in May with the launch of a big data beta project, LucidWorks Big Data, which certifies and integrates Apache open source components to develop and manage big data applications.
We learn about the company’s plan and focus on user-experience in the article:
“‘Users are missing from the big data conversation,’ argues [Grant Ingersoll, Chief Scientist for LucidWorks.] Paying attention to what users are doing helps improve the real-time, ad-hoc access to the data by improving relevance and search results. The analysis of users’ interaction with the system could also provide, as an interesting by-product, new insights about the business. In other words, what your employees do with your data may tell you a whole lot about how your business is functioning and even where it’s heading.’”
Created in 2008 as Lucid Imagination, the team is adept at managing and developing in the ever-shifting enterprise search marketplace. Changing the company’s business model to become innovators in the open source technology realm and now tackling the global emphasis on Big Data, the developers are aware of what needs to be done to fully contribute and make an impact in the expanding market. With a commitment to innovation and user-experience, we agree with Forbes about the future of LucidWorks:
“LucidWorks may represent a new wave of change, using search—the first ‘killer app’ of the Web—to unlock the value of enterprises’ much expanded big data stores and overflowing organizational memories.”
We are anxious to watch the company continue to grow and become leaders in search, content processing, and Big Data analytics.
Andrea Hayden, September 27, 2012
Sponsored by ArnoldIT.com, developer of Augmentext.
What Is the Most Deployed Search System in the World?
September 26, 2012
I just had a brief chat with LucidWorks. In that call, I learned about a surprising fact presented in the Forbes’ article “LucidWorks: Bringing Search to Big Data.” Here’s the point I noted:
Lucene/Solr is the most deployed search technology in the world, used by companies such as Netflix, AT&T, Sears, Ford, and Verizon. According to Ingersoll, Twitter search is powered by Lucene, handling more than a billion queries a day, with close to four hundred million tweets indexed and available within 50 milliseconds of being posted (see here for a 2010 post about Lucene by the Twitter engineering team).
There may be many search options. Some are free and spin outs of university or personal research projects (Elasticsearch, SearchBox). Others are well backed start ups (Palantir, Centrifuge Systems). Some are hybrids (Basho and Datastax).
If Forbes is correct, there is one vendor poised to disrupt search, analytics, and content processing—LucidWorks. More about this Forbes article in a day or two.
Don C Anderson, September 26, 2012
Sponsored by Augmentext
Deconstructing HP Autonomy and Its Eight Answers
September 26, 2012
All Things Digital ran a story called “Eight Questions for Hewlett Packard Software Head George Kadifa.” Let me nudge aside any thoughts that the interview and the questions were presented as public relations and marketing. I want to view the comments or “answers” as accurate. Once I have highlighted the points which caught my attention, I want to offer some side observations from my goose pond in rural Kentucky.
First, there were two passages which addressed the $12 billion Autonomy purchase.
The first was information about a recent planning meeting. The Autonomy staff were on deck and ready for duty. The key statement for me was this one:
Basically when you look at Autonomy, the core unit is the IDOL Engine, which is the unique capability of meaning-based computing. We’re going to double down on that. In our labs in Cambridge, England, we have 40 or 50 mathematicians writing algorithms. And we’re going to build a team here in the U.S. to productize it and create a platform around it because it has that potential. Frankly, the way Autonomy was managed previously, they put a lot more emphasis into enabling applications, which was fine, but our belief is that there’s a broad agenda, which is creating a platform around meaning-based computing. So we will maintain those apps, but at the same time we’ll open up the capabilities to a broader set of players outside HP.
Makes sense. Pay $12 billion for IDOL. Leverage it.
The second was semi-business school thinking about how to grow Autonomy’s business. Here’s the passage I noted:
In Europe, they tend to make things complex in order to create more value. For example, they saw the IDOL engine as too complex to just give it to people. Instead they thought they should acquire vendors and then create value by enabling applications. Here we take something that’s complex and we ask how we might simplify it in order to give it more scale for a bigger market. So some of that difference was cultural, and some of it was that I think they fell in love with these acquisitions. … We think Autonomy’s technology has broader implications.
I urge you to read the full “eight questions” and the answers. Now my observations:
- Productizing IDOL or any search engine can be difficult. When I use the word “difficult,” I mean time consuming, expensive, and timetable free. Buying a search engine and sticking it in a product or service looks easy. It is not. In fact, IBM has elected to use open source search to provide the basics. Now IBM is working hard to make money from its value add system, the game show winner Watson. There may be a product in “there”, but it is often to find a way to make money. HP has to pay back the $12 billion it spent and then grow the Autonomy business which was within shouting distance of $1 billion.
- The notion that Europeans see the world differently from HP is interesting. I am not sure how European Autonomy was. My view is that Autonomy’s senior management acquired companies and did upselling. As a result, only Autonomy broke through the glass ceilings behind which Endeca, Exalead, ISYS, and Fast Search & Transfer were trapped. Before applying business school logic to Autonomy, perhaps one should look at how other acquired search vendors have paid off. The list is, based on my research, a short one indeed. Microsoft, for example, has made Fast Search a component of SharePoint. With Fast Search nearing or at its end of life, Microsoft faces more search challenges, not fewer. HP may find itself with facing more challenges than it expects.
- The notion of “broader applications” is a popular one. Dassault Systèmes, acquired Exalead, which is arguably better and more recent technology than IDOL. But Dassault’s senior managers continue to look for ways to convert a more modest expenditure for Exalead into a river of revenue. Dassault has a global approach and many excellent managers. Even for such an exceptional firm, search is not what it seemed to be; that is, a broad application which slots into to many customer needs. Reality, based on my research for The New Landscape of Search, is different from the business school map.
HP is making an trip which other companies have taken before. My view is that HP will have to find answers the these questions, which were not part of the interview cited above:
First, how will HP pay off the purchase price, grow Autonomy’s revenue, and generate enough money to have an impact on HP’s net profit? My work has pointed out that cost control is the major problem search vendors face. It takes money to explain a system no matter how productized it becomes. It takes money to support that technology. It takes money to enhance that system. It takes money to hire people who can do the work. In short, search becomes a bright blip on most CFOs’ radar screens. HP may be different, but I am not sure that the cost issue will remain off the radar for very long.
Second, IDOL is a complex collection of software components. The core is Bayesian, but much of the ancillary IDOL are the add ons, enhancements, and features which have been created and applied to base system over the last two decades. Yep, two decades. In search, most of the systems which have figured in big deals in the last two years date from the mid to late 1990s. The more modern systems are not search at all. These new systems leap frog key word search and push into high value opportunities. HP may be forced to buy one of more of these next generation systems just to stay in the “beyond search” game.
Third, HP is a large company and it faces considerable competition in software. What makes HP interesting is that it has not been able to make its services business offset the decline in personal computers and ink. HP now wants to prove that it can make services work, but as the Inquirer pointed out in mid August 2012:
HP’s write-down of EDS might have resulted in just a paper loss – the firm didn’t actually lose $9bn in cash – but it provides an insight into how a decade of mismanagement has left HP in a bad situation. The fact is that HP cannot lay the blame on diminishing PC sales because its enterprise business, printing and services divisions all reported losses, too. For HP to write down the purchase of EDS, a company it paid $13.9bn for just four years ago, strongly suggests that those who were at the helm of HP in the run-up to that acquisition simply had no clue as to how much EDS was really worth and how to incorporate the company into HP. The value of any company can go down over time – just look at AOL, Microsoft or Yahoo – but for an established business such as EDS to be overvalued by almost $10bn just four years after being acquired is nothing short of gross incompetence by HP in both the purchase and the subsequent handling of the firm once it became a part of HP.
I don’t fully agree with the Inquirer’s viewpoint. But one fact remains: HP must demonstrate that it can manage a complex business based on IDOL, a technology which is not a spring chicken. The man who did manage Autonomy to almost $1 billion in sales is not longer with HP. In the history of enterprise search and content processing, Mike Lynch was unique. Perhaps the loss of that talent will continue to impact HP’s plans for a different approach to the market for Autonomy’s technology?
Life extension treatments are available, but these often do not work as expected and can be expensive. Most fail in the end.
Stephen E Arnold, September 25, 2012
Sponsored by Augmentext
Moving to SharePoint 2013: Planning Necessary
September 25, 2012
Many organizations will want to take advantage of the new features, services, and functions of SharePoint 2013 and SharePoint Search 2013. “Planning the Infrastructure Required for the new App Model in SharePoint 2013” makes clear that a shift to Microsoft’s 2013 enterprise systems requires planning and preparation.
Because the name of the solution is the same, many SharePoint administrators may feel that SharePoint 2013 is a routine upgrade. The article points out: “SharePoint 2013 brings with it a brand new application model, which we euphemistically refer to as the ‘app model’ or ‘cloud app model’. “
The scope of the planning required, according to Steve Peschka, includes, the development model, the security model, and the infrastructure. The article jumps from broad themes into quite specific information about modifications to url formation. For an administrator with this specific concern, the information is useful. The recommendation focuses on creating additional Web applications.
Comperio, one of the world’s leading specialists in search and content processing, approaches SharePoint planning by considering the context of the client’s needs, the existing SharePoint implementation, and the requirements the client has which can benefit from the 2013 solution. Comperio’s search engineers can handle the technical details of an implementation, but these are integrated into the roll out of a SharePoint system which considers budget, timetable, and existing resources.
According to Jørn Ellefsen , CEO and founder of Comperio:
Search matters. Our approach is to gather information, analyze the data, and develop a migration plan which focuses on meeting client requirements. Our engineers specialize in the search element of SharePoint for SharePoint migrations. However, our capabilities embrace the preparation and post-migration work that are important to SharePoint licensees.
Comperio’s approach to SharePoint Search reduces the time and cost of a shift from an existing SharePoint installation to the latest version of SharePoint. For more information about Comperio Search’s SharePoint 2013 planning and implementation services, visit www.comperiosearch.com.
Stephen E Arnold, September 25, 2012
Sponsored by Augmentext
Salesforce Incorporates Coveo Enterprise Search
September 22, 2012
ITWorldCanada announces, “Coveo Brings Enterprise Search to Salesforce.com.” The Canadian company will contribute its indexing engine and business intelligence tools to the Salesforce.com cloud. Coveo for Salesforce, which can pull together, index, and analyze unstructured data from multiple sources, will be fully integrated into the popular online customer relationship management (CRM) platform.
The write up tells us:
“Louis Tetu, CEO of Coveo, said the product is the first tool of its kind that is integrated directly into Salesforce. ‘We are enabling an entirely new paradigm to federate information on demand,’ he said. ‘And that paradigm means that we don’t have to move data, we’re just pointing…secure indexes to that information.’
“Users of the technology that need information delivered in real-time, such as customer-facing companies, will be able to get it rapidly — within 100 milliseconds — he added. This will help solve the common problem of consumers dealing with contact centres that cannot pull up their information in a reasonable period of time.”
Yes, that is a real plus. Tetu went on to emphasize that this is no small development– his company has conquered the considerable challenges of operating securely in the cloud. He mentions they also make a special effort to ensure new users can dive in as easily as possible.
Coveo was founded in 2005 by some members of the team which developed Copernic Desktop Search. Coveo takes pride in solutions that are agile and easy to use yet scalable, fast, and efficient.
Cynthia Murrell, September 22, 2012
Sponsored by ArnoldIT.com, developer of Augmentext
Learning about SharePoint Search 2013
September 18, 2012
“Search Is Everywhere! What You Need to Know about Search in SharePoint 2013 Preview” provides a handy summary of new features and major changes in Microsoft enterprise search.
The summary begins with a look at the tweaked document library. The author points out, “Search is everywhere in SharePoint now so it is important to understand how it has changed.”
One important assertion the article makes is, “Search was essentially rewritten from the ground up… The concept of FAST Search for SharePoint servers is gone. The components from FS4SP have made their way directly into SharePoint and we don’t need separate dedicated servers for it (necessarily).
Corey Roth adds:
Aside from cool stuff like drag and drop, document libraries directly leverage the search engine to allow users to filter documents easily. Gone are the days of relying on CAML for simple document library searches. You’ll find out why later in this post on why you can rely on search for those uses as well.
The other topics covered in the summary include the components of the search system; specifically, crawl component, content processing component, index component, analytics component, query processing component, and the search administration component.
The summary provides additional detail about crawling. The summary points out:
Whereas SharePoint 2010, only had protocols for Local Search and OpenSearch 1.1, SharePoint 2013 Preview (as well as SharePoint Online Preview), support for Remote SharePoint servers and Exchange has been added. Since SharePoint and People Search results are served by the same search index now, you can choose which type of results you want here too.
This summary includes information about querying, the user interface, and the API, and some of the new administrative controls. Illustrated with screenshots, we think the article is a useful reference.
For more detail, you will want to take a look at “What’s New within SharePoint 2013 Search.” So far, SharePoint Talk has issue three useful write ups which include screenshots and links to supplementary documents.
The first part is “What’s New within SharePoint 2013 Search? Part 1. This is an overview. The second part has the same name but focuses more on some of the configurable elements; for example, managed properties.
The new features of SharePoint Search 2012 deliver significant benefits to those seeking information. However, implementing and customizing specific search features will require close attention by programmer who are deeply knowledgeable about Microsoft’s technologies. The “cool stuff” is indeed very exciting. Comperio can provide the engineering support to make these functions deliver to the SharePoint’s licensees’ specific requirements.
Comperio’s search engineers continue to work closely with clients, Microsoft partners, and Microsoft itself. As a result, we have compiled deep experience with the most recent version of SharePoint Search. We can implement the specific features of SharePoint Search 2013 that a client requires. More importantly, we can extend the system to deliver particular functions such as seamless interaction with third party enterprise applications. For more information about Comperio, visit the firm’s Web site at www.comperiosearch.com.
Stephen E Arnold, September 18, 2012
Sponsored by Augmentext
Tips Are Not Enough for a Killer SharePoint Search System
September 11, 2012
We read “5 Tips for Turning a SharePoint 2010 Search Center into a Find Center.” The points are useful and include such suggestions as appointing a search administrator, have a SharePoint Search plan, and monitor the search system.
We found this passage interesting:
The default Search Center above might be enough for some sites (hey, the minimalist approach works for Google), but with some work, you can turn this into a Search Center that is useful enough for users to set it as their home page. You can get a lot of good ideas from the white paper How Microsoft IT Deployed FAST Search Server 2010 for SharePoint, especially the Enterprise search center article. This screenshot from the white paper provides an example of what you can do with your Search Center. FAST Search Server will give you a richer experience on the search results page, but the items you see on this Search Center are achievable with SharePoint 2010—or even SharePoint 2007. To create a Search Center that is the go-to place for your users, you can add helpful information such as links to event calendars, corporate news, campus maps, benefits, expense reporting, and research portals.
In our experience, SharePoint Search can deliver high-value services to users throughout an organization. SharePoint, particularly with its distributed and cloud capabilities, can now provide exceptional information access across a wide range of on premises and remote worker use cases.
However, only a handful of consulting services firm have the technical expertise and hands-on experience necessary to deploy a SharePoint solution in a matter of days. Search Technologies has implemented hundreds of SharePoint Search solutions, and the firm’s technical staff knows how to move through a project from its inception to its customization and optimization in an efficient manner.
If you want to move SharePoint to the next level, consider Search Technologies.
Iain Fletcher, September 11, 2012
Sponsored by Augmentext
xx
Information about Search for SharePoint 2013
September 6, 2012
We have been tracking useful sources of information about search for SharePoint 2013. You will find the three minute video prepared by Search Technologies an excellent place to begin. Search Technologies points out that the Fast search technology, acquired by Microsoft in 2008, and it is at the heart of SharePoint 2013. Technologies and ideas from Bing and elsewhere have been added to the mix to provide a comprehensive set of enterprise search capabilities, with plenty of room for customization. Search for SharePoint 2013 includes a rules-based query parsing framework. Search Technologies indicates that pricing has not yet been formally announced, but it is generally assumed that this search functionality will be a standard part of SharePoint 2013. See http://www.searchtechnologies.com/sharepoint-15-search-overview.html.
Microsoft has done a great job of providing information about SharePoint 2013 search. I wanted to make sure you knew that a series of articles is pulling together much of the Microsoft information and adding some insights that could be difficult to locate.
We can point out another useful source of information in this Microsoft document.
The author is Nicki Borell. The first three parts of his coverage of SharePoint 2013 discuss:
- Office 365
- What Happens with Fast in SharePoint 2013
- Search Dictionaries, Query Builder, Query Client Type.
Two more articles will appear in the near future, and these will cover administrative changes and user interface modifications.
Our engineers at Search Technologies track SharePoint 2013 on an hourly basis. We found that the discussion of dictionaries, query builder, and query client type were useful for two reasons:
- The articles include screenshots which make it easy to get oriented in a graphical or PowerShell environment
- There is sufficient descriptive narrative to make clear the specific feature; however, for those working with certain large SharePoint environments, additional explanation might prove useful to some system administrators.
Search Technologies has the deep experience required to handle basic and advanced SharePoint configuration, customization, and integration for any size SharePoint deployment. For more information, visit http://www.searchtechnologies.com/.
Iain Fletcher, September 6, 2012
Sponsored by Augmentext
Kona Data Search Bets on Salesforce as Salesforce Swims Against the Current
August 27, 2012
One-size-fits-all search has become a tough sale. Presumably clever enterprise search vendors have embraced wordsmithing in order to boost sales. Examples range from converting a deduplicating technology into big data, shifting an entity extraction program to business analytics, and presenting XML as a “new” content manipulation tool which slices, dices, and chops with ease.
I learned via a random LinkedIn message that a copy called Kona Data Search. I pinged the company, was promised information, and even followed up (a rare action for the addled goose). After some dithering, I checked out the company’s Web site (which created some choking and stumbling for my so-so Chrome browser) and learned:
Kona Search [for Salesforce] is a relevancy-based text search application for Salesforce, with a Search Results page and a pop-up sidebar, or “Search Gadget,” for persistent display. Relevancy is a way of sorting search results based on how well they match the terms in a user’s query. You may be familiar with relevancy from public web search applications like Google search. KonaSearch applies the same principals to Salesforce objects. Also like the web search applications, KonaSearch highlights the words that match in the results so you can see why an object was included. The current release can search text, dates, and numbers for the main Sales Cloud objects. Immediately following this release will be more Salesforce products, Chatter®, and Microsoft Outlook.
Searching for information on Salesforce is okay. There are problems when one has quite a few employees using Salesforce and a super user needs to pinpoint a specific email or contact interaction chain for something like eDiscovery or checking up on a sales professional who has just resigned.
Kona asserts that it delivers such functionality as:
- Auto suggestion
- Field specific search
- Date and number search
- Entity extraction
- Facets
- Nested Boolean
- Phrase detection
- Spelling correction
- Stemming
- Synonym expansion
- Term biasing (weighting)
- Wildcards
The service costs about $240 per user per year.
In short, Kona includes the basics of what might be called traditional enterprise search. Google’s original search appliance intentionally trimmed such functions from its user interface. The assumption was that enterprise users don’t know how to formulate complex queries and are more interested in slamming in a word or two and getting relevant results. We know that neither traditional enterprise search nor the Google approach has hit home runs over the last few years.
What makes Kona interesting is that it is approaching the market with what appears to be an initial focus on Salesforce. (Versions of Kona for other systems is promised, however.) Now Salesforce is an interesting company, but I heard a rumor that Google considered purchased Salesforce seven years ago. But Google passed. Now Salesforce has to fight the likes of Oracle and smaller companies’ iPad apps to stay in the game. Salesforce has the same cost control problem that is gobbling Amazon’s margins.
According to “Salesforce Losses Swell, Despite Rise in Sales,” high flying Salesforce may have sucked some errant geese into its jet engines. The Register said:
Software-as-a-Service pin-up Salesforce.com reported growing losses despite increased sales. The hosted CRM provider reported a loss of $9.82m on a 34 per cent increase in net sales to $73.6m for the three-month period to 31 July.
Then added, “The company’s costs are increasing as it adds more staff to sell to the enterprise, against rivals such as Oracle and SAP.”
Here in Harrod’s Creek, the river dogs say, “Rising water lifts them boats.” What happens when the water level falls? Will Kona be able to float the Salesforce boat or will Salesforce get stuck in the mud and drag down Kona? We are monitoring the revenue flow gauge.
Stephen E Arnold, August 27, 2012
Sponsored by Augmentext
Enhanced Document Filters from dtSearch
August 21, 2012
We learn that dtSearch is beta testing its product line’s version 7.70 from Technology Magazine in “Beta Enhances dtSearch Document Filters to Display Highlighted.” The press release tells us:
“dtSearch Corp., a leading supplier of enterprise and developer text retrieval along with document filters, announces beta testing of Version 7.70 of the dtSearch product line. The new version adds multiple improvements to the document filters spanning the dtSearch product line. For customers in need of data parsing, conversion and extraction only, the dtSearch Engine APIs (native 64-bit/32-bit, Win/Linux C++, Java and .NET through 4.x) also make the document filters available for separate OEM licensing.”
Besides the dtSearch Engine, available for Windows and .NET or for Linux, the new release also applies to dtSearch Web with Spider, dtSearch Network with Spider, dtSearch Publish, and dtSearch Desktop with Spider.
Users of the new version will find that it supports a wide array of data types, and that image support has been added to Word, PowerPoint, Excel, Access, RTF, and email files. Enhancements have also been made to the multi-level nested configurations, including a new “object extraction” API. The write up also emphasizes the following features: built-in spider functionality; a terabyte indexer; assorted search options; and international language support. See the press release for more details.
Incorporated in Virginia in 1991, dtSearch began its text retrieval R&D back in 1988. Business and government organizations in over 70 countries rely on its wide product line to manage a myriad of data related tasks.
Cynthia Murrell, August 21, 2012
Sponsored by ArnoldIT.com, developer of Augmentext
-
- Subscribe to Beyond Search
Feature archive
News archive
-
