Edging Closer to a Predictive Analytics Appliance
October 4, 2015
We live in a hybrid world. Some see virtualization as the future. Other want hardware with smart software able to live in the cloud or down the hall in an on premises facility. The smart software has to figure out what the information flowing to the system and stored within the system means. Humans used to do this work, but volume and other constraints force a rethink. If the information is “FalconStor Adds Cumulus Logic Predictive Analytics to FreeStor” is accurate, a predictive analytics appliance may be at hand. If so, this is an important step. I don’t like the lack of “e”, however.
Stephen E Arnold, October 4, 2015
Google Flu Trends: Smart Software in Action
October 2, 2015
i read “What We Can Learn from the Epic Failure of Google Flu Trends.” I like history. In grade school in the 1950s, there was not much talk about predicting the flu.
Flash forward to 2008. According to the write up:
In 2008, researchers from Google explored this [prediction based on users’ queries] potential, claiming that they could “nowcast” the flu based on people’s searches. The essential idea, published in a paper in Nature, was that when people are sick with the flu, many search for flu-related information on Google, providing almost instant signals of overall flu prevalence.
Then what? Failure. The write up reminded me:
GFT failed—and failed spectacularly—missing at the peak of the 2013 flu season by 140 percent. When Google quietly euthanized the program, called Google Flu Trends (GFT), it turned the poster child of big data into the poster child of the foibles of big data.
The point is that Big Data are going to be darned useful. I agree. For now I will temper my enthusiasm for Google’s beating death and IBM Watson curing cancer. I want to be conservative and get a flu shot.
Stephen E Arnold, October 2, 2015
Massive Analytic: The First Precognitive Analytics Platform
October 1, 2015
Let me reflect a moment. IBM is doing cognitive computing. I am assuming that the on going PR and marketing activities are accurate representation of money making technologies.
Massive goes IBM one better. The Massive Analytic outfit claims on its Web site that it delivers “effortless data driven decisions.” The product or service is Oscar AP, which allows you to “analyze all your data with artificial precognition.”
Interesting. About five weeks ago, I read “SAP, Oracle and HP Don’t Get Big Data, Claims Massive Analytic Chairman.” In the article, I learned:
Large IT vendors such as SAP, Oracle and HP don’t understand how to properly help their customers to make the most of big data, being more concerned about locking them into their ecosystems than providing them with true analytical insight. That’s according to George Frangou, executive chairman and founder of “precognitive data analytics” platform Oscar AP, which Frangou described as “an AI that allows people to foresee the future and the outcome of their decisions” which “makes Minority Report real”.
That reminded me of Recorded Future, an outfit partially funded by the Alphabet Google thing and In-Q-Tel, the US government intelligence community’s investment fund. Recorded Future rolled out in 2008 after a year or so of gestation. Massive Analytic took its first breath in 2010. I assume the wiggle room created by the term “precognitive” allows Massive Analytic to claim the adjective “first.”
The write up about Massive Analytic contained a statement which I found interesting. I circled this in red, gentle reader:
according to Frangou, larger competitors such as SAP, Oracle and HP “don’t get it” when it comes to making the most of big data and analytics. “They don’t get it because the driver for them is to sell kit. … You’re into millions of dollars before you start,” he said, attacking the aggressive sales tactics of the big vendors, which he said are designed solely to sell the product and not to provide support. “And by the way, the actual algorithms don’t scale either, so you’re into lots of people and manual intervention,” he added. Because of this, Frangou said Massive Analytic is “quite unashamedly following a displacement strategy to displace the incumbents because they’re not getting it”. He added that SAP HANA, Oracle Exalytics and HP Haven are essentially the same product because they’re built on the same base code.
It is true that most analytics vendors recycle what the engineers and mathematicians with MBAs learned in their university courses. I am not sure about the “algorithms don’t scale.” There are issues with algorithms, but as the work by SRCH2 shows, there is a great deal of innovation opportunity in optimization.
But the point which I find slightly jarring is the reference to SAP, Oracle, and HP “built on the same base code.”
Well, maybe. SAP uses home brew code (anyone remember TREX), acquired stuff from Business Objects (Inxight), and open source snippets. Oracle uses the wild and crazy home brew code, acquired code from “analytics” outfits like Endeca, and confections from some of the Oracle partner ecosystem. HP—an example for MBA cases studies for the next couple of decades—uses home brew, acquired technology from outfits like Autonomy, and probably scripts written by the Board of Directors and Meg Whitman in their spare time.
What the three companies share is, therefore:
- Code written by employees and contractors
- Code from open source and licensed libraries
- Code from companies acquired in moments of great wisdom.
The wrappers each of these companies exposes to its customers and partners make it easy to use the popular programming conventions, recycle structured query language, and exploit reasonably stable Web conventions.
I would suggest that once one looks under the hood of one of these companies’ projects, there will be a world of differences. There is a simple reason or two.
First, some familiar bits and lots of unfamiliar or downright extraterrestrial methods translate to job security and on going consulting work. Who wants to lose a night Oracle DBA job? Not anyone I know.
Second, enterprise software is about customization. I know the yap about enterprise apps, but these apps are little more than customized scripts to allow a hapless marketer with a degree in home economics to pull down a standard report.
I will leave it to you to unravel the mysteries of precognitive analytics and the assertion that HP, Oracle, and SAP are peas in a pod.
Stephen E Arnold, October 1, 2015
Reddit’s Extended Family
October 1, 2015
I have a problem. I have a Reddit addiction. My addiction is so bad that I once meant to spend five minutes on the news site, when I ended up spending five hours. To control my compulsions, I only allow myself to read the first hundred posts and if I have finished my work, the first two hundred. I am currently in the process to kick the Reddit habit, so I will be a more productive person. But then I came across this article on Chi-Nese: “20 Great Reddit Alternatives You Should Know.”
Just as I thought I did not have enough Web sites on my RSS feed, now I have these lovely alternatives. Here is the scoop:
“Reddit is the most popular social bookmarking site celebrating 10-year anniversary of existence nowadays. Reddit has accumulated over 16 billion up-votes, over 1 billion comments and over 190 million posts, which are – compared to other Reddit alternatives – enormous numbers. Despite the fact that Reddit is a website with a massive number of users and posts, below is a list of international Reddit alternatives that have great potential, and are definitely worth a try!”
Most of these Reddit alternatives are in a foreign language (not English), but some of ones to make the list are Hubski, PushedUp, Qetzl, Voat.co, and 3tags.
I am surprised that Fark did not make the list. Fark is the “original” Reddit, but it focuses on aggregating outlandish news content. There goes my productivity!
Whitney Grace, October 1, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Data Science: Converging to Confusion
September 30, 2015
I read “Two Great Visualizations about Data Science.” There is not too much reading involved. The article provides images of two graphics. The more interesting is “another nice picture about the history of big data and data science.”
Note that in the 2010 column, the separate lines of “technology” have converged into what looks to me like a fur ball.
The diagram captures several important ideas.
First, note that Bayes and Bayesian methods have some continuity. Other numerical approaches are important, but that Bayes has created the equivalent of Gorilla Glue.
Second, progress, particularly after 1990, seems to point to visualization. This is, for me, similar to judges awarding a cake with nice looking icing a blue ribbon without tasting the baker’s confection. Appearances are more important than substance.
Third, the end point of the diagram is a circular image which looks like a 1950s atomic diagram from the old Atomic Energy Forum. I think the image looks like a darned confusing diagram.
I think data science and Big Data are more confusing than they were in 2010. The eccentric orbits are becoming more distorted.
Stephen E Arnold, September 30, 2015
The Many Applications of Predictive Analytics
September 29, 2015
The article on Computer World titled Technology that Predicts Your Next Security Fail confers the current explosion in predictive analytics, the application of past occurrences to predict future occurrences. The article cites the example of the Kentucky Department of Revenue (DOR), which used predictive analytics to catch fraud. By providing SAS with six years of data the DOR received a batch of new insights into fraud indicators such as similar filings from the same IP address. The article imparts words of wisdom from SANS Institute instructor Phil Hagen,
“Even the most sophisticated predictive analytics software requires human talent, though. For instance, once the Kentucky DOR tools (either the existing checklist or the SAS tool) suspect fraud, the tax return is forwarded to a human examiner for review. “Predictive analytics is only as good as the forethought you put into it and the questions you ask of it,” Hagen warns…. Also It’s imperative that data scientists, not security teams, drive the predictive analytics project.”
In addition to helping the IRS avoid major fails like the 2013 fraudulent refunds totaling $5.8 billion, predictive analytics has other applications. Perhaps most interesting is its use protecting human assets in regions where kidnappings are common by detecting unrest and alerting organizations to lock up their doors. But it is hard to see limitations for technology that so accurately reads the future.
Chelsea Kerwin, September 29, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Business Intelligence: A Magical Insight Machine?
September 28, 2015
I found “Thinking Outside the Big Data Black Box: Why BI Isn’t a Magical Insight Machine” interesting. The main point of the write up is that vendors well analytics platforms. The licensees learn that set up, tuning, expertise are required to make these often expensive systems deliver useful outputs.
The write up states:
Big data, or indeed any data, may indeed hold huge value, but it’s often looked at in the wrong way. When we are looking at data – collected from different sources, to address different motivations, with an ever-changing context – we can’t fast track every correlation into an actionable insight. We have to understand where the data comes from, the factors limiting its reliability, its consistency when applied across different sub-groups, and where biases may be lurking. We need to carefully interrogate any correlation, before we can understand whether it represents a truth in the real world.
Bummer. Smart software, flashy Powerpoints, and examples of Hollywood style graphics make data work fun, interesting, game like, right?
Not without effort.
The write up points out:
A straightforward analysis of historical data will spot factors that consistently cause cost overruns. But more sophisticated techniques and a bit of intuition can go much further – for example you may find short planning time is not generally correlated with cost overruns, but it is more strongly correlated with overruns in projects over a certain size. Most importantly, you need to understand why these relationships exist. If one factor consistently reduces costs, can you be confident it will continue to do so in a new market where conditions are different? If you don’t understand your data you can’t make such predictions.
After reading the article, I was shocked. I thought that today’s nifty systems eliminated the requirement to understand data, understand the mathematical option, and provide ready to use outputs.
Disappointed. I thought the quip about business intelligence as an oxymoron was a cheap shot.
Stephen E Arnold, September 28, 2015
Accidental and On-Purpose Insider Threats in Federal Agencies Still Raging
September 28, 2015
The article on Eweek titled Insider Threats a Major Security Issue for Federal Agencies looks at the recent results of a MeriTalk survey investigating federal response to insider threats through interviewing federal IT managers. The results are shocking, with almost 30% of agencies acknowledging data lost to an insider threat in the last year and half of respondents claiming that unauthorized personnel commonly fail to observe protocols. Even worse, most agencies have no tracking in place to recognize what a staffer may have seen or shared, making them virtually incapable of following up on risky behavior in their employees. The article says,
“The most startling finding from the survey is the fact that 45 percent of agencies say they’ve been a target of an attack – malicious or unintentional – yet 50 percent still say employees do not follow all the protocols in place,” Steve O’Keeffe, founder of MeriTalk…”There is also a lack of agreement on the best solution. Frequent, hands-on employee training is the key to preventing these incidents, as well as accountability. However, we are all human and people make mistakes.”
O’Keefe recommends the immediate and comprehensive adoption of better encryption and two-factor authentication to address the issue. But perhaps equally important is continuously updated training, and ongoing training, to avoid the common accidental insider threats.
Chelsea Kerwin, September 28, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
The Dream, the Fantasy: Government More Like the Alphabet Google Thing
September 27, 2015
I read in my dead tree edition of the New York Times “”Making Government Work More Like Google.” A version of this story is / was online at “A Better Government, One Tweak at a Time.” Gentle reader, you may have to pay to access this chunk of “real” journalistic content.
The premise of the write up is that Google knows how to do stuff. The lead example is the Google.com splash page: Clean, simple, ad free, etc.
The article snags the notion that I think emerged from the Ivory Tower at the University of Chicago. The idea manifested money in the form of the best selling “Nudge: Improving Decisions about Health, Wealth, and Happiness.”
Make little changes, and big things can happen. The notion is, according to my ageing memory, part of the Butterfly Effect. Yes, your favorite mathematician Edward Lorenz.
The core idea is that government needs to do more tweaking and nudging. Big things will happen. Maybe not what the whiz kids expect, but something great will definitely, eventually happen, maybe.
Here’s the passage which warranted a big circle:
the big idea is simpler: It’s not about knowing how to do better, it’s about testing what works. Experiment relentlessly, keep what works, and discard what doesn’t. Following this recipe may yield a government that’s just like Google: clear, user-friendly and unflinchingly effective.
A few observations:
- Alphabet Google is another minor tweak at what used to be Google. Tweak? Yep, no big deal. Focus the legal matters at the new unit and move forward with Loon balloons and solving death. Working well I assume.
- The tweaks at Google have not resulted in shifting the firm’s revenues from almost complete dependence on online advertising. Sure, the business model is still pumping out cash, but after 15 years of tweaks, maybe there could be a couple of other revenue streams?
- Lots of tweaks in the social media space at the Alphabet Google thing have done little to make Google a challenger to Facebook. Facebook is doing its same old, same old. But if the tweak thing worked, wouldn’t Orkut have blasted Facebook out of the water as it improved with each nudge of Google’s social media offerings?
Stepping back, the idea that the US government, or any government for that matter, can continuously “improve” like Alphabet Google is interesting.
I assume that the momentum of centuries old bureaucracies will respond to a nudge. The idea is similar to that of pushing an asteroid heading toward earth so the ice ball does not obliterate a chunk of real estate. Great idea.
I am confident that a government working like Google can make this work. Project Runway comes to the procurement procedures of the US, French, or Brazilian government.
Nudge away. A-B test a lot. Rinse. Repeat. Tweak. Etc.
Stephen E Arnold, September 27, 2015
Harsh Criticism of Yahoo
September 24, 2015
Kill dear old Yahoo? IBTimes reports on some harsh words from an ivory-tower type in, “NYU Professor: Yahoo Ought to Be ‘Euthanised’ and Marissa Mayer’s Pregnancy Saved her Job.” It seems marketing professor Scott Galloway recently criticized the company, and its famous CEO, in a televised Bloomberg interview. In his opinion, any website with Yahoo’s traffic should be rolling in dough, and the company’s struggles are the result of mismanagement. As for his claim that the “most overpaid CEO in history” only retains her position due to her pregnancy? Reporter Mary-Ann Russon writes:
“Galloway says that Yahoo would not be willing to face the public backlash that would come from firing a woman in such a position of power who has just announced she is pregnant.
“This is not a stretch since there are still far fewer women in leadership positions than men – as of March 2015, only 24 of the CEOs in Fortune 500 companies are women – and the issue with how companies perceive family planning remains a sore point for many career-minded women (Read: Gamechangers: Why multimillionaire ‘mom’ Marissa Mayer is damned if she does and damned if she doesn’t).
“However, Galloway also pointed the finger of blame for Yahoo’s woes at its board, which he said has been a ‘lesson in poor corporate governance,’ since there have been five CEOs in the last seven years.”
Though Yahoo was a great success around the turn of the millennium, it has fallen behind as users migrate their internet usage to mobile devices (with that format’s smaller, cheaper ads). Though many still use its free apps, nowadays most of Yahoo’s revenue comes from its Alibaba investment.
So what does Galloway recommend? “It should be sold to Microsoft,” he declared. “We should put a bullet in this story called ‘Yahoo’.” Ouch. Can Yahoo reverse their fortunes, or is it too late for the veteran Internet company?
Cynthia Murrell, September 24, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
-
- Subscribe to Beyond Search
Feature archive
News archive
-
