Artificial Intelligence is no Threat to Analytics Jobs
August 21, 2013
As the fields of artificial intelligence and big data analytics continue to grow greater every year, the idea that their paths will converge is not totally farfetched. While this has some worried, we are not among them after reading a recent Extreme Tech article, “Artificial Intelligence Has the Verbal Skills of a Four-Year-Old, Still No Common Sense.”
According to the story:
An artificial intelligence like >this one might have access to a lot of data, but it can’t draw on it to make rational judgements by leveraging implicit facts — things that we all know, but are so obvious we wouldn’t even consider them relevant information. ConceptNet might know that water freezes at 32 degrees, but it doesn’t know how to get from that concept to the idea that ice is cold. This is basically common sense — humans (even children) have it and computers don’t.
This should come as a comfort for anyone working in the analytics field. There is still a strong need for human intellect to interpret all the raw data floating around out there. As this article recently pointed out, right now is actually the best time to start networking in the analytics field. We totally agree.
Patrick Roland, August 21, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Infographics Take on New Analytic Importance
August 21, 2013
We are increasingly living in a big data and analytic society. But when discussing all this information, it’s hard to put a visual with it. Humans are, after all, very sight-oriented. However, that problem is quickly looking like a thing of the past after discovering a recent Make Use Of article, “Create Your Own Infographic about Your Facebook, Twitter and Youtube Use.”
According to the story:
What About Me is a free to use web service that lets you easily analyze how you have been using your Facebook, Twitter, and YouTube accounts. You start by granting the site access to your accounts. Your usage is analyzed and the infographic is generated while you play with some distractingly interactive circles that are displayed.
The infographic that is finally generated shows your interests in terms of percentages, how you react with friends, plus a number of other interesting things about your social networking usage.
This really is the next logical step in infographics. We’ve been lured, as Wired astutely pointed out, by infographics as “link bait” for a long time. It’s time we turned that gaze inward to see what our social habits say about us. This will take off, we predict.
Patrick Roland, August 21, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Next Generation Content Processing: Tail Fins and Big Data
August 19, 2013
Note: I wrote this for Homeland Security Today. It will appear when the site works out its production problems. As background, check out “The Defense Department Thinks Troves of Personal Data Pose a National Security Threat.” If the Big Data systems worked as marketers said, the next generation systems would these success stories provide ample evidence of the value of these Big Data systems?]
Next-generation content processing seems, like wine, to improve with age. Over the last four years, smart software has been enhanced by design. What is your impression of the eye-popping interfaces from high-profile vendors like Algilex, Cybertap, Digital Reasoning, IBM i2, Palantir, Recorded Future, and similar firms? ((A useful list is available from Carahsoft at http://goo.gl/v853TK.)
For me, I am reminded of the design trends for tail fins and chrome for US automobiles in the 1950s and 1960s. Technology advances in these two decades moved forward, but soaring fins and chrome bright work advanced more quickly. The basics of the automobile remained unchanged. Even today’s most advanced models perform the same functions as the Kings of Chrome of an earlier era. Eye candy has been enhanced with creature comforts. But the basics of today’s automobile would be recognized and easily used by a driver from Chubby Checker’s era. The refrain “Let’s twist again like we did last summer” applies to most of the advanced software used by law enforcement and the intelligence community.
[Image file: tailfin.png]
The tailfin of a 1959 Cadillac. Although bold, the tailfins of the 1959 Plymouth Fury and the limited production Superbird and Dodge Daytona dwarfed GM’s excesses. Source: https://en.wikipedia.org/wiki/File:Cadillac1001.jpg
Try this simple test. Here are screenshots from five next-generation content processing systems. Can you match the graphics with the vendor?
Here are the companies whose visual outputs appear below. Easy enough, just like one of those primary school exercises, simply match the interface with the company
The vendors represented are:
A Digital Reasoning (founded in 2000 funded in part by SilverLake. The company positions itself as providing automated understanding as did Autonomy, founded in 1996)
B IBM i2 (industry leader since the mid 1990s)
C Palantir (founded a decade ago with $300 million in funding by Founders fund, Glynn Capital Management, and others)
D Quid (a start up funded in part by Atomico, SV Angel, and others)
E Recorded Future (funded in part by In-Q-Tel and Google, founded by the developer of Spotfire)
Another Information Priority: Legacy Systems
August 16, 2013
The hoohah about cloud computing, Big Data, and other “innovations” continues. Who needs Oracle when one has Hadoop? Why license SPSS or some other Fancy Dan analytics system when there are open choice analytics systems a mouse click away? Search? Lots of open source choices.
Image from http://sageamericanhistory.net/gildedage/topics/gildedage3.html
We have entered the Gilded Age of information and data analysis. Do I have that right?
The marketers and young MBAs chasing venture funding instead of building revenue shout, “Yes, break out the top hats and cigars. We are riding a hockey stick type curve.”
Well, sort of. I read “Business Intelligence, Tackling Legacy Systems Top Priorities for CIOs.” Behind the consultant speak and fluff, there lurk two main points:
- Professionals in the US government and I presume elsewhere are struggling to make sense of “legacy” data; that is, information stuffed in file cabinets or sitting in an antiquated system down the hall
- The problems information technology managers remain unresolved. After decades of effort by whiz kids, few organizations can provide basic information technology services.
As one Reddit thread made clear, most information technology professionals use Google to find a fix or read the manual. See Reddit and search for “secrets about work business”.
A useful comment about the inability to tap data appears in “Improving business intelligence and analytics the top tech priority, say Government CIOs.” Here’s the statement:
IT contracts expert Iain Monaghan of Pinsent Masons added: “Most suppliers want to sell new technology because this is likely to be where most of their profit will come from in future. However, they will have heavily invested in older technology and it will usually be cheaper for them to supply services using those products. Buyers need to balance the cost they are prepared to pay for IT with the benefits that new technology can deliver,” he said. “Suppliers are less resistant to renegotiating existing contracts if buyers can show that there is a reason for change and that the change offers a new business opportunity to the supplier. This is why constant engagement with suppliers is important. The contract is meant to embody a relationship with the supplier.”
Let me step back, way back. Last year my team and I prepared a report to tackle this question, “Why is there little or no progress in information access and content processing?”
We waded through the consultant chopped liver, the marketing baloney, and the mindless prose of thought leaders. Our finding was really simple. In fact, it was so basic we were uncertain about a way to present it without coming across like a stand up comedian at the Laugh House. To wit:
Computational capabilities are improving but the volume of content to be processed is growing rapidly. Software which could cope with basic indexing and statistical chores bottlenecks in widely used systems. As a result, the gap between what infrastructure and software can process and the amount of data to be imported, normalized, analyzed, and output is growing. Despite recent advances, most organizations are unable to keep pace with new content and changes to current content. Legacy content is in most cases not processed. Costs, time, and tools seem to be an intractable problem.
Flash forward to the problem of legacy information. Why not “sample” the data and use that? Sounds good. The problem is that even sampling is fraught with problems. Most introductory statistics courses explain the pitfalls of flawed sampling.
How prevalent is use of flawed sampling? Some interesting examples from “everywhere” appear on the American Association for Public Opinion Research. For me, I just need to reflect on the meetings in which I have participated in the last week or two, Examples:
- Zero revenue because no one matched the “product” to what the prospects wanted to buy
- Bad hires because no one double checked references. The excuse was, “Too busy” and “the system was down.”
- Client did not pay because “contracts person could not find a key document.”
Legacy data? Another problem of flawed business and technology practices. Will azure chip consultants and “motivated” MBAs solve the problem? Nah.Will flashy smart software be licensed and deployed? Absolutely. Will the list of challenges be narrowed in 2014? Good question.
Stephen E Arnold, August 16, 2013
Sponsored by Xenky
Useful Suggestion For The Magic Bullet Hadoop
August 13, 2013
Downloading Hadoop and expecting it to solve all your problems is dumb way to use the software. Silicon Angle has some suggestions on how to use Hadoop in, “To Succeed With Hadoop: Find Specific Problem Areas And Solve Them.” The advice comes from Datameer CEO/Founder Stefan Groschupf at the recent Hadoop Summit 2013.
Groschupf acknowledged that Hadoop is another tool in the big data toolbox and the real power of a company does not come from just its tools, build its customer base, quality products with an edge that no one else has, and to stay in the black. Most importantly is to find a problem no one else has resolved and do it yourself.
That seems to be the only advice the article offers. The rest is an advertisement for Datameer 3.0, which is the newest tool for big data analytics:
“Datameer 3.0 adds new Smart Analytic functions. With a single click, it automatically identifies patterns, relationships, and recommendations based on data stored in Hadoop. For the first time, four advanced machine learning techniques become self-service and accessible for data-driven business users: Clustering, Decision Trees, Column Dependencies and Recommendations. Until now, these advanced analytics required highly specialized data scientists to build custom functionality, which was a costly and time-consuming process.”
So get a gimmick kids! Once you have that you will succeed. It worked for Bill Gates, Walt Disney, Steve Jobs, and Lady Gaga.
Whitney Grace, August 13, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
IBM at Wimbledon
August 5, 2013
The realms of tennis and technology intersect at Wimbledon, Britain’s NewStatesman reminds us in, “IBM and Wimbledon: The Tech that Takes You Closer to the Tennis. Brought to You by Wimbledon Insights.” The prestigious tournament, held at the All England Club in Wimbledon, London, since 1877, has relied on IBM tech since 1994. We wonder, does IBM’s involvement boost Wimbledon’s ratings in the US?
The article takes us through the history of that partnership, listing developments like the 2000 debut of the Wimbledon Information System and the Match Analysis DVDs distributed to singles players beginning in 2007. See the article for more, but, personally, I am most interested in the incorporation of predictive analytics via SlamTracker. (Ah, but did Watson predict Serena’s recent loss? The article does not say.) We learn from the write-up:
“Introduced last year, IBM SlamTracker was enhanced with a ‘Keys to the match’ feature. Using over eight years of Grand Slam tennis data and 41 million data points, IBM is able to find the patterns and styles of play for particular head-to-head matches (or between players of similar styles if the players in question have not met before).
“In the run-up to a match, the data for one player is compared to that of his or her opponent, along with players of a similar style to determine the ‘keys to the match’: the three targets that player has to hit if they want to enhance their chance of winning. These keys are selected by analysing 45 potential match dynamics – 19 offensive, 9 defensive, 9 endurance and 8 style – to identify the ones that will be vital to each player in this specific match.”
My, how sports have changed since 1877! It almost makes one long for simpler times.
Cynthia Murrell, August 05, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Spotter Makes its Name with Sarcasm
August 5, 2013
While we are generally cheerleaders for all things big data and analytics, we are not blind to its weaknesses. One major weakness of most big data platforms would give it a devil of a time parsing much from, say, an episode of Seinfeld. That’s right, we’re talking about its inability to detect sarcasm. However, Slashdot thinks it might have the answer, according to the recent article: “Tech Companies Looking into Sarcasm Detection.”
According to the story:
Spotter’s platform scans social media and other sources to create reputation reports for clients such as the EU Commission and Air France. As with most analytics packages that determine popular sentiment, the software parses semantics, heuristics and linguistics. However, automated data-analytics systems often have a difficult time with some of the more nuanced elements of human speech, such as sarcasm and irony — an issue that Spotter has apparently overcome to some degree, although company executives admit that their solution isn’t perfect. (Duh.)
Spotter is really making a name for itself. We fell in love with the company a long while ago, after an Arnold IT interview set the tone. This is a sharp company and if their sarcasm detection comes through, they’ll be industry leaders.
Patrick Roland, August 05, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Crowdsourcing Helps Keep Big Data Companies Straight
August 4, 2013
As big data analytics begins picking up steam, we are seeing more and more interesting outlets to learn about different platforms to choose from. Not just catalogs and boastful corporation sites, but insightful criticism. One such recent stop was when we came about the “About” story of Bamboo DIRT.
According to the site:
Bamboo DiRT is a tool, service, and collection registry of digital research tools for scholarly use. Developed by Project Bamboo, Bamboo DiRT is an evolution of Lisa Spiro’s DiRT wiki and makes it easy for digital humanists and others conducting digital research to find and compare resources ranging from content management systems to music OCR, statistical analysis packages to mindmapping software.
One look at its tips for analyzing data and we were sold. Here we were turned on to such intriguing companies as 140kit and Dataverse. The user-supported recommendations were the best. About Dataverse, it said: “Researchers and data authors get credit, publishers and distributors get credit, affiliated institutions get credit.” Concise and giving all the needed vitals, this type of crowdsourcing recommendation site could really catch on as the world of big data analytics keeps growing beyond most users’ capacity to keep up.
Patrick Roland, August 04, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Healthcare Analytics Always Changing
July 30, 2013
The medical field is always evolving with new advances. The same can be said about the medical technology field, especially in mobile data analytics. Today’s hot trend is a relic faster than in almost any field, so we try hard to keep tabs, such as an illuminating article in CMS Wire, “Temis Acquires i3 Analytics to Boost Text + Data Mining.”
According to the story:
“While we don’t know how much Temis paid out in this deal, we know doctor’s love iPads. This tells us pretty much all we need to know about this deal. i3 Analytics specializes in what it calls biopharma, what most of us know as pharmaceutical research or biotechnology.”
Advances in biotech and biopharma mean more data for doctors and drug companies to rummage through, something a company like i3 Analytics is more than happy to help them with.
This is an interesting story of healthcare analytics. Frankly, nothing surprises us anymore. Heck, we recently heard that Kansas City is the new boomtown for healthcare analytics. We think if things like this are possible, there’s no way this dynamic industry will stop changing anytime soon.
Patrick Roland, July 30, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Selfless Analytic Partnerships Score Big
July 29, 2013
One would think the big data analytics field would be cutthroat and bloody with competition. Oddly, though, it is actually a mostly friendly affair. It seems like we hear more and more about disparate firms partnering up to give customers a better product. Such was the case with this recent Business Wire story, “JackBe Unveils Presto Real-Time Analytics Add On with Terracotta Big Memory.”
According to the story:
“JackBe®, the leading provider of Real-Time Actionable Intelligence software, today released its Presto Real-Time Analytics Add-On With Terracotta BigMemory (RTA Add-On), which bundles Terracotta’s enterprise-grade BigMemory in-memory data management platform for seamless high performance in-memory analytics. This supercharged combination of in-memory and analytics allows Presto to mash Big Data with live and transactional enterprise data into actionable dashboards in a fraction of the time required by traditional methods, putting real-time Big Data analytics at the fingertips of decision-makers.”
We like the way these two companies are playing to their strengths. When selfless analytics minds do that, the customers score big. It is reminiscent of the IBM big data partnership. In these cases we are seeing savvy companies partnering to do something extraordinary.
Patrick Roland, July 29, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
-
- Subscribe to Beyond Search
Feature archive
News archive
-
