Bye-Bye Enterprise Storage
October 19, 2015
Storage is a main component of the enterprise system. Silos store data and eventually the entire structure transforms into a legacy system, but BusinessWire says in “MapR Extends Support For SAS To Deliver Big Data Storage Independence” it is time to say good-bye to old enterprise storage. MapR is trying to make enterprise storage obsolete with its new extended service support for SAS, a provider of business software and services. The new partnership between allows advanced analytics with easy data preparation and integration in legacy systems, improved security, data compliance, and assurance of service level agreements.
The entire goal is to allow SAS and MapR clients to have better flexibility for advanced analytics within Hadoop as well as to help customers harvest the most usefulness our of their data.
Here is a rundown of the partnership between SAS and MapR:
“The collaboration makes available the full scope of technologies in the SAS portfolio, including SAS® LASR™ Analytic Server, SAS Visual Analytics, SAS High-Performance Analytics, and SAS Data Loader for Hadoop. Complete MapR integration delivers security and full POSIX compliance for use in “share everything architectures,” as well as enables SAS Visual Analytics to easily and securely access all data. With SAS Data Loader for Hadoop, users can prepare, cleanse and integrate data inside MapR for improved performance and then load that data in-memory into SAS LASR for visualization or analysis, all without writing code.”
Breaking away from legacy systems with old onsite storage is one of the new trends for enterprise systems. Legacy systems are clunky, don’t necessary comply with new technology, and have slow information retrieval. A new enterprise system using SAS and MapR’s software will last for some time, until the new trend buzzes through town.
Whitney Grace, October 19, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
CounterTack Partners with ManTech Cyber Solutions for a More Comprehensive Platform
August 13, 2015
A new acquisition by CounterTack brings predictive capability to that company’s security offerings, we learn from “CounterTack Acquires ManTech Cyber Solutions” at eWeek. Specifically, it is a division of ManTech International, dubbed ManTech Cyber Solutions International (MCSI), that has been snapped up under undisclosed terms by the private security firm.
CounterTack president and CEO Neal Chreighton says the beauty of the deal lies in the lack of overlap between their tech and what MCSI brings to the table; while their existing products can tell users what is happening or has already happened, MCSI’s can tell them what to watch out for going forward. Writer Sean Michael Kerner elaborates:
“MCSI’s technology provides a lot of predictive capabilities around malware that can help enterprises determine how dangerous a malicious payload might be, Creighton said. Organizations often use the MCSI Responder Pro product after an attack has occurred to figure out what has happened. In contrast, the MCSI Active Defense product looks at issues in real time to make predictions, he said. A big area of concern for many security vendors is the risk of false positives for security alerts. With the Digital DNA technology, CounterTack will now have a predictive capability to be able to better determine the risk with a given malicious payload. The ability to understand the potential capabilities of a piece of malware will enable organizations to properly provide a risk score for a security event. With a risk score in place, organizations can then prioritize malware events to organize resources to handle remediation, he said.”
Incorporation of the open-source Hadoop means CounterTack can scale to fit any organization, and the products can be deployed on-premises or in the cloud. Cleighton notes his company’s primary competitor is security vendor CrowdStrike; we’ll be keeping an eye on both these promising firms.
Based in Waltham, Massachusetts, CounterTack was founded in 2007. The company declares their Sentinel platform to be the only in-progress attack intelligence and response solution on the market (for now.) Founded way back in 1968, ManTech International develops and manages solutions for cyber security, C4ISR, systems engineering, and global logistics from their headquarters in Washington, DC. Both companies are currently hiring; click here for opportunities at CounterTack, and here for ManTech’s careers page.
Cynthia Murrell, August 13, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Hadoop Rounds Up Open Source Goodies
July 17, 2015
Summer time is here and what better way to celebrate the warm weather and fun in the sun than with some fantastic open source tools. Okay, so you probably will not take your computer to the beach, but if you have a vacation planned one of these tools might help you complete your work faster so you can get closer to that umbrella and cocktail. Datamation has a great listicle focused on “Hadoop And Big Data: 60 Top Open Source Tools.”
Hadoop is one of the most adopted open source tool to provide big data solutions. The Hadoop market is expected to be worth $1 billion by 2020 and IBM has dedicated 3,500 employees to develop Apache Spark, part of the Hadoop ecosystem.
As open source is a huge part of the Hadoop landscape, Datamation’s list provides invaluable information on tools that could mean the difference between a successful project and failed one. Also they could save some extra cash on the IT budget.
“This area has a seen a lot of activity recently, with the launch of many new projects. Many of the most noteworthy projects are managed by the Apache Foundation and are closely related to Hadoop.”
Datamation has maintained this list for a while and they update it from time to time as the industry changes. The list isn’t sorted on a comparison scale, one being the best, rather they tools are grouped into categories and a short description is given to explain what the tool does. The categories include: Hadoop-related tools, big data analysis platforms and tools, databases and data warehouses, business intelligence, data mining, big data search, programming languages, query engines, and in-memory technology. There is a tool for nearly every sort of problem that could come up in a Hadoop environment, so the listicle is definitely worth a glance.
Whitney Grace, July 17, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Digital Reasoning a Self-Described Cognitive Computing Company
June 26, 2015
The article titled Spy Tools Come to the Cloud on Enterprise Tech shows how Amazon’s work with analytics companies on behalf of the government have realized platforms like “GovCloud”, with increased security. The presumed reason for such platforms being the gathering of intelligence and threat analysis on the big data scale. The article explains,
“The Digital Reasoning cognitive computing tool is designed to generate “knowledge graphs of connected objects” gleaned from structured and unstructured data. These “nodes” (profiles of persons or things of interest) and “edges” (the relationships between them) are graphed, “and then being able to take this and put it into time and space,” explained Bill DiPietro, vice president of product management at Digital Reasoning. The partners noted that the elastic computing capability… is allowing customers to bring together much larger datasets.”
For former CIA staff officer DiPietro it logically follows that bigger questions can be answered by the data with tools like the AWS GovCloud and subsequent Hadoop ecosystems. He cites the ability to quickly spotlight and identify someone on a watch list out of the haystack of people as the challenge set to overcome. They call it “cluster on demand,” the process that allows them to manage and bring together data.
Chelsea Kerwin, June 26, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
New Analysis Tool for Hadoop Data from Oracle
June 23, 2015
Oracle offers new ways to analyze Hadoop data, we learn from the brief write-up, “Oracle Zeroes in on Hadoop Data with New Analytics Tool” at PCWorld. Use of the Hadoop open-source distributed file system continues to grow among businesses and other organizations, so it is no surprise to see enterprise software giant Oracle developing such tools. This new software is dubbed Oracle Big Data Spatial and Graph. Writer Katherine Noyes reports:
“Users of Oracle’s database have long had access to spatial and graph analytics tools, which are used to uncover relationships and analyze data sets involving location. Aiming to tackle more diverse data sets and minimize the need for data movement, Oracle created the product to be able to process data natively on Hadoop and in parallel using MapReduce or in-memory structures.
“There are two main components. One is a distributed property graph with more than 35 high-performance, parallel, in-memory analytic functions. The other is a collection of spatial-analysis functions and services to evaluate data based on how near or far something is, whether it falls within a boundary or region, or to process and visualize geospatial data and imagery.”
The write-up notes that such analysis can reveal connections for organizations to capitalize upon, like relationships between customers or assets. The software is, of course, compatible with Oracle’s own Big Data Appliance platform, but can be deployed on other Hadoop and NoSQL systems, as well.
Cynthia Murrell, June 23, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Data Darkness
May 28, 2015
According to Datameer, organizations do not use a large chunk of their data and it is commonly referred to “dark data.” “Shine Light On Dark Data” explains that organizations are trying to dig out the dark data and use it for business intelligence or in more recent terms big data. Dark data is created from back end business processes as well as from regular business activities. It is usually stored on storage silo in a closet and only kept for compliance audits.
Dark data has a lot of hidden potential:
“Research firm IDC estimates that 90 percent of digital data is dark. This dark data may come in the form of machine or sensor logs that when analyzed help predict vacated real estate or customer time zones that may help businesses pinpoint when customers in a specific region prefer to engage with brands. While the value of these insights are very significant, setting foot into the world of dark data that is unstructured, untagged and untapped is daunting for both IT and business users.”
The article suggests making a plan to harness the dark data and it does not offer much in the way of approaching a project other than making it specifically for dark data, such as identifying sources, use Hadoop to mine it, and tests results against other data sets.
This article is really a puff piece highlighting dark data without going into much detail about it. They are forgetting that the biggest movement in IT from the past three years: big data!
Whitney Grace, May 28, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Hadoop Has Accessories
May 25, 2015
ZDNet’s article, “Why Hadoop Is Hard, And How To Make It Easier” alludes that Hadoop was going to disappear at some point. We don’t know about you, but the open source big data platform has a huge support community and hundreds have adopted it, if not thousands of companies, have deployed Hadoop. The article argues otherwise, citing that a recent Gartner survey found that only 26 percent of the corporate world is actively using it.
One of the biggest roadblocks for Hadoop is that it is designed for specialist to tinker with and it is not an enterprise tool. That might change when Microsoft releases its new SQL Server 2016. With the new server, Microsoft will add Polybase that bridges Hadoop to the server. Microsoft is still the most popular OS for enterprise systems and when this upgrade becomes available Hadoop will be a more viable enterprise option.
What is the counterpoint?
“It’s also a counterpoint to the interpretation of Gartner’s survey that says Hadoop is somehow languishing. What’s languishing is the Enterprise’s willingness to invest in a new, premium skill set, and the low productivity involved in working with Hadoop through its motley crew of command-line shells and scripting languages. A good data engine should work behind the scenes and under the covers, not in the spotlight.”
So once more enterprise systems need to be updated, which is comparable to how Hadoop needs to be augmented with add-on features to make it more accessible, such as mature analytics tools, DBMS abstraction layers and Hadoop-as-a-Service cloud offerings.
Whitney Grace, May 25, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Popular and Problematic Hadoop
May 15, 2015
We love open source on principle, and Hadoop is indeed an open-source powerhouse. However, any organization considering a Hadoop system must understand how tricky implementation can be, despite the hype. A pair of writers at GCN asks and answers the question, “What’s Holding Back Hadoop?” The brief article reports on a recent survey of data management pros by data-researcher TDWI. Reporters Troy K. Schneider and Jonathan Lutton explain:
“Hadoop — the open-source, distributed programming framework that relies on parallel processing to store and analyze both structured and unstructured data — has been the talk of big data for several years now. And while a recent survey of IT, business intelligence and data warehousing leaders found that 60 percent will Hadoop in production by 2016, deployment remains a daunting task. TDWI — which, like GCN, is owned by 1105 Media — polled data management professionals in both the public and private sector, who reported that staff expertise and the lack of a clear business case topped their list of barriers to implementation.”
The write-up supplies a couple bar graphs of survey results, including the top obstacles to implementation and the primary benefits of going to the trouble. Strikingly, only six percent or respondents say there’s no Hadoop in their organizations’ foreseeable future. Though not covered in the GCN write-up, the full, 43-page report includes word on best practices and implementation trends; it can be downloaded here (registration required).
Cynthia Murrell, May 15, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
AI Technology Poised to Spread Far and Wide
April 3, 2015
Artificial intelligence is having a moment; the second half of last year saw about half a billion dollars invested in the AI industry. Wired asks and answers, “The AI Resurgence: Why Now?” Writer Babak Hodjat observes that advances in hardware and cloud services have allowed more contenders to afford to enter the arena. Open source tools like Hadoop also help. Then there’s public perception; with the proliferation of Siri and her ilk, people are more comfortable with the whole concept of AI (Steve Wozniak aside, apparently). It seems to help that these natural-language personal assistants have a sense of humor. Hodjat continues:
“But there’s more substance to this resurgence than the impression of intelligence that Siri’s jocularity gives its users. The recent advances in Machine Learning are truly groundbreaking. Artificial Neural Networks (deep learning computer systems that mimic the human brain) are now scaled to several tens of hidden layer nodes, increasing their abstraction power. They can be trained on tens of thousands of cores, speeding up the process of developing generalizing learning models. Other mainstream classification approaches, such as Random Forest classification, have been scaled to run on very large numbers of compute nodes, enabling the tackling of ever more ambitious problems on larger and larger data-sets (e.g., Wise.io).”
The investment boom has produced a surge of start-ups offering AI solutions to companies in a wide range of industries. Organizations in fields as diverse as medicine and oil production seem eager to incorporate these tools; it remains to be seen whether the tech is a good investment for every type of enterprise. For his part, Hodjat has high hopes for its use in fraud detection, medical diagnostics, and online commerce. And for ever-improving personal assistants, of course.
Cynthia Murrell, April 3, 2015
Stephen E Arnold, Publisher of CyberOSINT at www.xenky.com
EBay Develops Open Source Pulsar for Real Time Data Analysis
April 2, 2015
A new large-scale, real-time analytics platform has been launched in response to one huge company’s huge data needs. VentureBeat reports, “EBay Launches Pulsar, an Open-Source Tool for Quickly Taming Big Data.” EBay has made the code available under an open-source license. It seems traditional batch processing systems, like that found in the widely used open-source Hadoop, just won’t cut it for eBay. That puts them in good company; Google, Microsoft, Twitter, and LinkedIn have each also created their own stream-processing systems.
Shortly before the launch, eBay released a whitepaper on the project, “Pulsar—Real-time Analytics at Scale.” It describes the what and why behind Pulsar’s design; check it out for the technical details. The whitepaper summarizes itself:
“In this paper we have described the data and processing model for a class of problems related to user behavior analytics in real time. We describe some of the design considerations for Pulsar. Pulsar has been in production in the eBay cloud for over a year. We process hundreds of thousands of events/sec with a steady state loss of less than 0.01%. Our pipeline end to end latency is less than a hundred milliseconds measured at the 95th percentile. We have successfully operated the pipeline over this time at 99.99% availability. Several teams within eBay have successfully built solutions leveraging our platform, solving problems like in-session personalization, advertising, internet marketing, billing, business monitoring and many more.”
For updated information on Pulsar, monitor their official website at gopulsar.io.
Cynthia Murrell, April 2, 2015
Stephen E Arnold, Publisher of CyberOSINT at www.xenky.com
-
- Subscribe to Beyond Search
Feature archive
News archive
-
