Palantir Advises More Abstraction for Less Frustration
June 10, 2014
At this year’s Gigaom Structure Data conference, Palantir’s Ari Gesher offered an apt parallel for the data field’s current growing pains: using computers before the dawn of operating systems. Gigaom summarizes his explanation in, “Palantir: Big Data Needs to Get Even More Abstract(ions).” Writer Tom Krazit tells us:
“Gesher took attendees on a bit of a computer history lesson, recalling how computers once required their users to manually reconfigure the machine each time they wanted to run a new program. This took a fair amount of time and effort: ‘if you wanted to use a computer to solve a problem, most of the effort went into organizing the pieces of hardware instead of doing what you wanted to do.’
“Operating systems brought abstraction, or a way to separate the busy work from the higher-level duties assigned to the computer. This is the foundation of modern computing, but it’s not widely used in the practice of data science.
“In other words, the current state of data science is like ‘yak shaving,’ a techie meme for a situation in which a bunch of tedious tasks that appear pointless actually solve a greater problem. ‘We need operating system abstractions for data problems,’ Gesher said.”
An operating system for data analysis? That’s one way to look at it, I suppose. The article invites us to click through to a video of the session, but as of this writing it is not functioning. Perhaps they will heed the request of one commenter and fix it soon.
Based in Palo Alto, California, Palantir focuses on improving the methods their customers use to analyze data. The company was founded in 2004 by some folks from PayPal and from Stanford University. The write-up makes a point of noting that Palantir is “notoriously secretive” and that part(s) of the U.S. government can be found among its clients. I’m not exactly sure, though, how that ties into Gesher’s observations. Does Krazit suspect it is the federal government calling for better organization and a simplified user experience? Now, that would be interesting.
Cynthia Murrell, June 10, 2014
Sponsored by ArnoldIT.com, developer of Augmentext
Elasticsearch: Bulldozing Content Processing
June 7, 2014
When I left the intelligence conference in Prague, there were a number of companies in my graphic about open source search. When I got off the airplane, I edited my slide. Looks to me as if Elasticsearch has just bulldozed the search and content sector, commercialized open source group. I would not want to be the CEO of LucidWorks, Ikanow, or any other open sourcey search and content processing company this weekend.
I read “Elasticsearch Scores $70 Million to Help Sites Crunch Tons of Data Fast.” Forget the fact that Elasticsearch is built on Lucene and some home grown code. Ignore the grammar in “data fast.” Skip over the sports analogy “scores.” Dismiss the somewhat narrow definition of what Elasticsearch ELK can really deliver.
What’s important is the $70 million committed to Elasticsearch. Added to the $30 or $40 million the outfit had obtained before, we are looking at a $100 million bet on an open source search based business. Compare this to the trifling $40 million the proprietary vendor Coveo had gathered or the $30 million put on LucidWorks to get into the derby.
I have been pointing out that Elasticsearch has demonstrated that it had several advantages over its open source competitors; namely, developers, developers, and developers.
Now I want to point out that it has another angle of attack: money, money, and money.
With the silliness of the search and content processing vendors’ marketing over the last two years, I think we have the emergence of a centralizing company.
No, it’s not HP’s new cloudy Autonomy. No, it’s not the wonky Watson game and recipe code from IBM. No, it’s not the Google Search Appliance, although I do love the little yellow boxes.
I will be telling those who attend my lectures to go with Elasticsearch. That’s where the developers and the money are.
Stephen E Arnold, June 7, 2014
Software AG Happy About JackBe
May 30, 2014
Business Wire via Sys Con has some great news: “Software AG’s Acquisition Of JackBe Recognized As Strategic M&A Deal Of The Year.” Software AG is a big data, integration, and business process technologies firm driven to help companies achieve their desired outcomes. With the acquisition of real time visual analytics and intelligence software provider JackBe will be the foundation for Software AG’s new Intelligent Business Operations Platform. The acquisition even garnered attention from the Association for Corporate Growth and was recognized as the Strategic M&A deal of the year in the $100 million category.
JackBe will allow Software AG to offers its clients a broader range of enterprise functions in real-time, especially in areas related to the Internet of Things and customer experience management.
“The real-time analysis and visualization of massive amounts of data is increasingly becoming the basis for fast and intelligent business decisions. With the capabilities of JackBe integrated in its Intelligent Business Operations platform, Software AG has been able to provide customers with a comprehensive 360-degree view of operational processes by combining live, historical and transactional data with machine-to-machine communications.”
Purchasing JackBe was one of the largest big data deals in 2013 and it also proves that technology used by the US government can be turned into a viable commercial industry.
Software AG definitely has big plans for 2014. Will they continue to make headlines this year?
Whitney Grace, May 30, 2014
Sponsored by ArnoldIT.com, developer of Augmentext
This is Microsoft Embracing Predictive Analysis
May 19, 2014
Now here’s a valuable use of predictive analytics. Digital Trends reports, “Microsoft to Use Bing Search Data to Predict Outcomes of Reality Shows.” Microsoft announced the initiative in this Bing blog post. It is good to see such an influential company investing its resources in issues that affect the quality of life for all humanity. Writer Konrad Krawczyk tells us:
“Beginning today [April 21], Bing will attempt to forecast the results of shows like ‘The Voice,’ ‘American Idol’ and ‘Dancing With The Stars,’ by scanning search data, along with ‘social input’ from Facebook and Twitter. For instance, if you head over to Bing right now and search ‘American Idol predictions’ like we did, the top of the page will feature a set forecasts for five singers. We’ll refrain from adding in any potential Bing-generated spoilers here, but you’re free to check out what the search engine thinks for yourself.
“‘In broad strokes, we define popularity as the frequency and sentiment of searches combined with social signals and keywords. Placing these signals into our model, we can predict the outcome of an event with high confidence,’ the Bing Predictions Team says in its blog post.
“Microsoft also says that Bing’s predictions incorporate numerous emotionally-driven factors into how it generates predictions, allegedly accounting for biases like favoritism, regardless of how a person’s favorite singer/contestant performs from one week to the next.”
While this example does sum up the gist of predictive analysis, we can think of several areas to which the technology could be better applied. To be fair, the Bing Predictions Team says reality TV is not the pinnacle of its prediction projects. Will the next initiative be aimed at similarly vacuous forecasts?
Cynthia Murrell, May 19, 2014
Sponsored by ArnoldIT.com, developer of Augmentext
SAS Text Miner Gets An Upgrade
May 5, 2014
SAS is a well-recognized player in IT game as a purveyor of data, security, and analytics software. In modern terms they are a big player in big data and in order to beef up their offerings we caught word that SAS had updated its Text Miner. SAS Text Miner is advertised as a way for users to not only harness information in legacy data, but also in Web sites, databases, and other text sources. The process can be used to discover new ideas and improve decision-making.
SAS Text Miner a variety of benefits that make it different from the standard open source download. Not only do users receive the license and tech support, but Text Miner offers the ability to process and analyze knowledge in minutes, an interactive user interface, and predictive and data mining modeling techniques. The GUI is what will draw in developers:
“Interactive GUIs make it easy to identify relevance, modify algorithms, document assignments and group materials into meaningful aggregations. So you can guide machine-learning results with human insights. Extend text mining efforts beyond basic start-and-stop lists using custom entities and term trend discovery to refine automatically generated rules.”
Being able to modify proprietary software is a deal breaker these days. With multiple options for text mining software, being able to make it unique is what will sell it.
Whitney Grace, May 05, 2014
Sponsored by ArnoldIT.com, developer of Augmentext
Big Data: Can the Latest Trend Deliver?
April 25, 2014
If you track Big Data, you will want to read “Why Big Data Is Stillborn (for Now).” The write up hits the highlights of the flickering hyperbole machine that sells fancy math to the government and organizations desperate for a Silver Bullet.
The article asserts:
Most “big data” has to be moved in physical containers. Most data centers do not have excess capacity to handle petabyte level simultaneous search and pattern discovery.
Believe in real time and high speed access? Consider this statement:
Bandwidth, throughput, and how “real time” is defined all come down to the weak link in the chain and we have many ***very weak*** links across the chain and especially in Washington, D.C. The bottom line is always “who benefits?” The FCC decision to destroy net neutrality is in error. The citizen, not the corporation, is “root” in a Smart Nation.
If you wonder why your Big Data investments have yet to deliver a golden goose pumping out 24 caret eggs everyday, check out this write up. Worth reading.
Stephen E Arnold, April 25, 2014
Small Analytics Firms Reaping the Benefit of Investment Cycle
April 23, 2014
Small time analytics isn’t really as startup-y as people may think anymore. These companies are in high demand and are pulling in some serious cash. We discovered just how much and how serious from a recent Cambridge Science Park article, “Cambridge Text Analytics Linguamatics Hits $10m in Sales.”
According to the story:
Linguamatics’ sales showed strong growth and exceeded ten million dollars in 2013, it was announced today – outperforming the company’s targeted growth and expected sales figures. The increased sales came from a boost in new customers and increased software licenses to existing customers in the pharmaceutical and healthcare sectors. This included 130 per cent growth in healthcare sales plus increased sales in professional services.
This earning potential has clearly grabbed the attention of investors. This, is feeding a cycle of growth, which is why the Linguamaticses of the world can rake in impressive numbers. Just the other day, for example, Tech Circle reported on a microscopic Mumbai big data company that landed $3m in investments. They say it takes money to make money and right now, the world of big data analytics has that cycle down pat. It won’t last forever, but it’s fun to watch as it does.
Patrick Roland, April 23, 2014
Sponsored by ArnoldIT.com, developer of Augmentext
Small Analytics Firms Reaping the Benefit of Investment Cycle
April 22, 2014
Small time analytics isn’t really as startup-y as people may think anymore. These companies are in high demand and are pulling in some serious cash. We discovered just how much and how serious from a recent Cambridge Science Park article, “Cambridge Text Analytics Linguamatics Hits $10m in Sales.”
According to the story:
Linguamatics’ sales showed strong growth and exceeded ten million dollars in 2013, it was announced today – outperforming the company’s targeted growth and expected sales figures. The increased sales came from a boost in new customers and increased software licenses to existing customers in the pharmaceutical and healthcare sectors. This included 130 per cent growth in healthcare sales plus increased sales in professional services.
This earning potential has clearly grabbed the attention of investors. This, is feeding a cycle of growth, which is why the Linguamaticses of the world can rake in impressive numbers. Just the other day, for example, Tech Circle reported on a microscopic Mumbai big data company that landed $3m in investments. They say it takes money to make money and right now, the world of big data analytics has that cycle down pat. It won’t last forever, but it’s fun to watch as it does.
Patrick Roland, April 22, 2014
Sponsored by ArnoldIT.com, developer of Augmentext
Imagine the Internet without Search Engines
April 17, 2014
Centrifuge Systems proposes an interesting idea in “Big Data Discovery Without Link Analysis Is Like The Web Without Google.” Centrifuge Systems asks readers of the short article to imagine using the Internet without a search engine. How would we locate information? It would be similar to the librarian’s favorite description of the Internet all the contents of a library spilled on the floor. The article continues to explain that big data without link analysis works the same as the Internet without a search engine.
What is link analysis?
“You can view link analysis as a data discovery technique that reveals the structure and content of information by representing it as a set of interconnected objects. When combined with a visual representation, an investigator can quickly gain an understanding of the strength of relationships and the frequency of contacts and immediately discover new associations. Link analysis offers an intuitive alternative to the traditional relational database formats and BI tools without deep technical expertise.”
It is a convincing analogy. To increase a potential client’s interest, Centrifuge Systems offers a Data Discovery Challenge, where the client is given a free solution. In other terms, it’s a free estimate for services. Big data is full of analytics, but has anyone other than Centrifuge Systems offer rich link analysis?
Whitney Grace, April 17, 2014
Sponsored by ArnoldIT.com, developer of Augmentext
Google Glass and Predictive Analytics
April 13, 2014
I read “I Was Assaulted for Wearing Google Glass.” This is a sad commentary on our times. Will I be assaulted in Harrod’s Creek for driving my Kia Soul and wearing a T shirt that says “ Seavey’s Dog Kennel”? After I read the item, I wondered, “Why can’t Google’s predictive analytics be used to display to a Glass wearer that the assault risk level?” A color coded scheme could be used based on previous Glass users’ encounters, GPS data, and other inputs available to the Google / Recorded Future systems.
I noted this passage:
Why were people laughing at my misfortune or implying I somehow deserved it?
Beats me.
Stephen E Arnold, April 13, 2014
-
- Subscribe to Beyond Search
Feature archive
News archive
-
