Big Data Bridges Diverse Fields
March 3, 2012
Solving the big data problem and what we are going to do with the immense amount of unstructured information that is just sitting in cyber space, is on the forefront of many minds.
The New York Times technology blog Bits recently reported on this issue in the article “IBM: Big Data, Bigger Patterns.”
According to the article, the recent explosion of information available on the Internet paired with inexpensive computer hardware has made it possible for enterprises to store huge amounts of unstructured data. Now that we know it is possible, the goal is to do it cost effectively.
In order to remain cost sensitive, many companies are trying to find overlapping interests and commonalities between different fields.
The article states:
“The trend of looking for commonalities and overlapping interests is emerging in many parts of both academia and business. At the ultra small nanoscale examination of a cell, researchers say, the disciplines of biology, chemistry and physics begin to collapse in on each other. In a broader search for patterns, students of the statistical computing language known as R have used methods of counting algae blooms to prove patterns of genocide against native peoples in Central America.”
While the cross pollination of various business interests is very exciting, we’re interested to see if it leads to complications down the road.
Jasmine Ashton, March 3, 2012
Sponsored by Pandia.com
Hadoop Technology: Calling All Mathematicians!
February 26, 2012
Scalability and big data solutions are not simply buzzwords thrown around the search industry. These are both key items in assessing value of platforms, and are both key reasons users are drawn to Hadoop technology.
However, the fact that Hadoop is picking up steam poses a major problem to those attempting to find talent to work the technology. People experienced in Hadoop are hard to come by. Cloudera, IBM, Hortonworks, and MapR are all investing in Hadoop training programs, choosing to invest in internal candidates rather than trying to hire new talent. A related article on CIO.in, “Hadoop Wins Over Enterprise IT, Spurs Talent Crunch” asserts on the topic:
‘We originally thought we needed to find a hardcore Java developer,’ Return Path’s Sautins says. But in reality, the talent that’s best suited for working with Hadoop isn’t necessarily a Java engineer. ‘It’s somebody who can understand what’s going on in the cluster, is interested in picking up some of these tools and figuring out how they work together, and can deal with the fact that pretty much everything in the Hadoop ecosystem is not even a 1.0 release yet,’ Sautins says. ‘That’s a real skill set.’
The problem of finding talent could eventually limit the continued adoption of Hadoop technology. Search analytics is now opening doors for those with deep math skills and backgrounds in statistics and science. People with this basic skills can be taught how to use these tools, and will be very valuable to a great number of companies adopting this technology.
Andrea Hayden, February 26, 2012
Sponsored by Pandia.com
Database Design Directions
February 23, 2012
We came across a quite useful checklist every database architect should keep on hand. Java Code Geeks give us “20 Database Design Best Practices.” The list covers everything from the commonsense:
“Use well defined and consistent names for tables and columns (e.g. School, StudentCourse, CourseID …).”
To the more advanced:
“Normalization must be used as required, to optimize the performance. Under-normalization will cause excessive repetition of data, over-normalization will cause excessive joins across too many tables. Both of them will get worse performance.”
With a little strong opinion mixed in:
“Lack of database documentation is evil.”
If you design (or oversee those who design) databases, do yourself a favor and check it out.
Most people think of search as providing access to unstructured information. Examples of unstructured information include email, Word documents, and Excel. Our extensive work in enterprise search has spanned structured data; that is, information in a database.
Search Technologies can handle difficult content acquisition tasks when needed information is held within Microsoft SQL Server, IBM DB2, Oracle, or a similar data management system. In addition, Search Technologies can set up automated processes to handle extraction, transformation, and loading of data or subsets of data.
For more information about our capabilities to make structured and unstructured data more findable, navigate to www.searchtechnologies.com.
Iain Fletcher, February 23, 2012
Sponsored by Pandia.com
A Look at DynamoDB from Amazon
February 20, 2012
We’ve found a useful write up with useful links in Paperplanes’ “A Tour of Amazon’s DynamoDB.”
DynamoDB is Amazon’s distributed data store. This is a distributed database. It lives in the cloud and appears to leap frog Google and Microsoft in a key solution space.
Billed as fast, reliable, andcost-effective, DynamoDB is a NoSQL database service is tailored tofit Internet-scale applications. It was built on lessons Amazonlearned from Dynamo, the scalable storage system it created in 2007 tomeet its own needs. Writer Mathias Meyer spells out the purpose ofhis article:
Time to take a good look at what it has to offer, how that works outin code, and to make some wild guesses as to what’s happening underthe covers. . . .My goal is not to outline the entire API and its full set of options,but to dig into the bits most interesting to me and to show someexamples. . . . I’m focusing on the effects DynamoDB has on you, theuser. We’ll look at API, general usage, data model and what DynamoDB’sfeature generally entails.
Meyer goes on to examine those points and much more. Even if thearticle weren’t full of useful information, it would be worth a peekif only for the links at the end, under Resources. Check it out.
Cynthia Murrell, February 20, 2012
Sponsored by Pandia.com
BPM and Big Data
February 8, 2012
Search and business process management: a shotgun marriage. The two can’t help but come together, as IT Business Edge reports in “Two Examples of BPM’s Role in Data Integration.” Writer Loraine Lawson writes that Talend intends to integrate BonitaSoft’s business process management solution with its Unified Platform.
Lawson is pretty sure Talend is the first data integration specialist vendor to venture into this area. She asked Talend’s VP of marketing, Yves de Montcheuil, how the use of BPM contributes to integration. The write up states:
Data governance. To use a master data hub as a system of record, you’ll need to load it from multiple sources, which will have conflicting data. The MDM and data quality tool will resolve many of these conflicts automatically through matching, but for more complicated conflicts, you’ll need a workflow. BPM can drive this workflow, sending the data to business users who can resolve the conflict by validating the correct data.
That does sound more efficient. Another way Talend expects BPM to help is to manage and automate data integration and data quality services.
Talend provides both open source and SaaS big data solutions to organizations around the world. BonitaSoft also offers both open source and paid solutions, but its realm is business process management. Best wishes to the happy couple!
Cynthia Murrell, February 8, 2012
Sponsored by Pandia.com
New Oracle E-Book
February 3, 2012
Oracle brews a data latté with cinnamon you may enjoy if you if you suck down this new book. The Salvatoreyc blog presents for download, “Oracle XSQL: Combining SQL, Oracle Text, XSLT and Java to Publish Dynamic Web Content.” The product description states:
Welcome to the exciting world of eXtended Structured Query Language (XSQL). ‘Oracle XSQL: Combining SQL, Oracle Text, XSLT and Java to Publish Dynamic Web Content’ presents a complete approach to building XML Web applications and Web services with XSQL, Oracle Text, SQL, XSLT, and Java from data found in Oracle databases. Companion Web site contains the code examples in the book as well.
Sounds helpful. The download is only available to Media Search members, and the download page takes you right to their signup sheet. Naturally, there is a fee.
Cynthia Murrell, February 3, 2012
Sponsored by Pandia.com
RightNow Takes the Oracle Money
January 29, 2012
Good idea in our opinion.
RightNow joins Endeca as a billion dollar baby, as revealed in TechCrunch’s “RightNow Stockholders Approve $1.5 Billion Merger with Oracle.” The vote was nearly unanimous; the $1.5 billion deal represents a per-share value of $43.
What will the customer experience management company bring to Oracle? What Writer Robin Wauters wrote:
RightNow’s solutions help companies handle customer interactions across a multitude of channels, including call and contact centers, the Web and social networks. Its products are used by nearly 2,000 organizations across the globe, the company says. With the acquisition, which is still subject to regulatory approval, Oracle will thus be adding a robust cloud-based customer service offering to its own Public Cloud solution.
RightNow has been around since 1997, and is driven to bring order, efficiency, and good will to customer interactions. For its part, Oracle needs to get its revenues back on track. Maybe, with Oracle magic, customer support and search will do the trick? However, Oracle faces legal set backs with regard to its Oracle litigation, open source threats, and the challenge of making those big search investments pay off in real money.
Cynthia Murrell, Janaury 29, 2012
Sponsored by Pandia.com
SAP HANA Spreads Its Wings
January 19, 2012
We learn more about SAP HANA from PR Newswire’s “SAP HANA™ Opens Up to Wider Industry and Partner Adoption.” Leading software vendors are tapping the speed and power of SAP HANA, a storage and data management system, for both existing products and for new applications. The press release boasts:
SAP also showcased the latest customers to experience the power of SAP HANA across industries, giving them instant access to massive amounts of information and the ability to make real-time decisions, in the process achieving breakthrough performance improvement by factors of thousands, even tens of thousands, compared to existing systems in their landscapes.
The article lists the vendors and describes how SAP HANA will help each. The companies represent fields from telecom to business visualization to healthcare. Examples include UFIDA, TIBCO, Tableau Software, and T-Mobile.
SAP serves up business management software to organizations of all sizes around the world, allowing workers to collaborate efficiently and to gain an edge on the competition. The company, founded back in 1972, is headquartered in Walldorf, Germany. Will HANA triumph in the fierce storage and data management wars? Uptake, according to our research, has been plugging along.
Cynthia Murrell, January 19, 2012
Sponsored by Pandia.com
Amazon: Will DynamoDB Electrocute the Big Boys?
January 18, 2012
I want to capture a few business related observations about Amazon’s now public DynamoDB. The blog post by Amazon’s chief technical officer provides a good overview of the home grown NoSQL data management service. Navigate to “Amazon DynamoDB–A Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications.” For a run down of some of the features, point your browser at “Notes About Amazon DynamoDB.” The basic idea is that Amazon has created its own NoSQL database, matched it to the Amazon cloud environment, and packaged it with taxi meter pricing.
Why didn’t Amazon use Hadoop or some other NoSQL, open source, Codd free systems? My hunch is that Amazon sees big money in a ready-to-role, automatic sharding, solid state disc base data management solution. Rolling its own solution gives Amazon control. In fact, Amazon is cranking up the dial on its Controlometer.
The issue that interests me is the business angle of the DynamoDB. Here are several preliminary thoughts.
First, Amazon is getting frisky but slowly. My sources report that work on the DynamoDB system began several years ago. Microsoft picked up wind of the project and was unable to respond. Right now, Amazon’s an engineering magnet, attracting talent from outfits once considered the best in the soggy city. With higher quality engineering horsepower, the dowdy retailer is shifting from a horse and wagon to a far more capable vehicle.
Second, MarkLogic had the idea that it could impinge on Oracle. Well, we know how that turned out with AtomicPR (the content fallout kids), management change, and wild and crazy marketing. Now Amazon is on the path to make life tough for Oracle. Amazon had Oracle as a steady date, but senior year is coming. Amazon may be marrying the DynamoDB, leaving Oracle without a homecoming date. If Amazon pulls off this new hitch up, Amazon may be ready to go for the enterprise gold. I think this is better than a 50-50 deal but I may change my mind.
Third, Amazon has demonstrated the value of a “Google Legacy.” Google plunged forward, diffused its resources, and ended up with its lovely self snared in legal and social thorns. Amazon, on the other hand, has avoided some of the traps into which Google threw itself. In the process itself, Amazon used Android to move its branded hardware forward. There is nothing like a friend who plans on evicting you from your home. Amazon is, once again, going beyond Google.
I have a number of other thoughts, but the goose’s liver needs a rest. Oh, oh, here comes a scowling nurse. Will she rescue the electrocuted big boys of database? I doubt it.
Stephen E Arnold, January 18, 2012
Sponsored by Pandia.com
Oracle Faces EnterpriseDB Surge
January 18, 2012
Oracle has ruled the information processing market for years now with fear as the dominating marketing tool. The article, EnterpriseDB Is Giving Oracle a Run For Its Money, on Forbes, explains how that has changed (with Oracle stock prices serving as proof) thanks to a declining economy and EnterpriseDB.
For years now Oracle has been on fire doing the once believed impossible – making money on open source technology. Controlling over fifty percent of the information processing market with their MYSQL Oracle thought they were scot-free. They were until the economy tanked and some smooth talking competition stepped in.
With IT departments’ budgets drastically cut across all industries some creative outside-the-box thinking has been going on. Oracle has kept their customers in line by telling scary stories of what will happen if they switch to another provider. With EnterpriseDB offering the same services as Oracle but at only ten percent of the price, most IT departments are willing to take their chances.
The article speaks of EnterpriseDB’s growth by reporting,
One of the reasons that it’s grown so fast is that EnterpriseDB has lowered customer switching costs. How so? Along with its support of the open source database, PostgreSQL, EnterpriseDB has added features that let customers port the applications they’ve developed for Oracle and other databases over to PostgreSQL.
Oracle might have overcome the open-source hurdle a while back and come out ahead, but we wonder about how they will fare with new competition like EnterpriseDB. Obviously, we are not alone as their stock shares have taken a dramatic hit.
Catherine Lamsfuss, January 18, 2012
Sponsored by Pandia.com
-
- Subscribe to Beyond Search
Feature archive
News archive
-
