Amazon AWS Jungle Snares Some Elasticsearch Functions
July 1, 2016
Elastic’s Elasticsearch has become one of the go to open source search and retrieval solutions. Based on Lucene, the system has put the heat on some of the other open source centric search vendors. However, search is a tricky beastie.
Navigate to “AWS Elasticsearch Service Woes” to get a glimpse of some of the snags which can poke holes in one’s rip stop hiking garb. The problems are not surprising. One does not know what issues will arise until a search system is deployed and the lucky users are banging away with their queries or a happy administrator discovers that Button A no longer works.
The write up states:
We kept coming across OOM issues due the JVMMemoryPresure spiking and inturn the ES service kept crapping out. Aside from some optimization work, we’d more than likely have to add more boxes/resources to the cluster which then means more things to manage. This is when we thought, “Hey, AWS have a service for this right? Let’s give that a crack?!”. As great as having it as a service is, it certainly comes with some fairly irritating pitfalls which then causes you to approach the situation from a different angle.
One approach is to use templates to deal with the implementation of shard management in AWS Elasticsearch. Sample templates are provided in the write up. The fix does not address some issues. The article provides a link to a reindexing tool called es-tool.
The most interesting comment in the article in my opinion is:
In hindsight I think it may have been worth potentially sticking with and fleshing out the old implementation of Elasticsearch, instead of having to fudge various things with the AWS ES service. On the other hand it has relieved some of the operational overhead, and in terms of scaling I am literally a couple of clicks away. If you have large amounts of data you pump into Elasticsearch and you require granular control, AWS ES is not the solution for you. However if you need a quick and simple Elasticsearch and Kibana solution, then look no further.
My takeaway is to do some thinking about the strengths and weaknesses of the Amazon AWS before chopping through the Bezos cloud jungle.
Stephen E Arnold, July 1, 2016
IBM Cloud Powers Comic-Con Channel
June 30, 2016
The San Diego Comic-Con is the biggest geek and pop culture convention in the country and it needs to be experienced to be believed. Every year the San Diego Comic-Con gets bigger and more complex as attendees and the guests demand more from the purveyors. If you are at Comic-Con, then you need to think big. Thinking big requires thinking differently, which is why it would seem “IBM And Comic-Con HQ Make Strange Bedfellows” says Fortune.
IBM announced that they have teamed with Lionsgate to run a Comic-Con HQ video channeled powered by IBM’s cloud. The on-demand channel will premiere during 2016’s Comic-Con. Comic-con attendees and those unfortunate not to purchase a ticket have demanded video streaming services for years, practically ever since it became possible. Due to copyright as well as how to charge attendees for the service have kept video on-demand on the back burner, but now it is going to happen and it is going to be a challenge.
Video streaming is:
“Video is a demanding application for cloud computing. Storing and shipping massive video files, often shot in ultra-high-definition 4k format, is a useful testbed to show off cloud services.”
Anything new related to Comic-Con always proves to be a hassle and troublesome. One of the cases in point is when the SDCC launched its digital waiting room to purchase tickets and had way more traffic than their servers could handle. The end result was a lot of angry fans unable to buy tickets. Another challenge was handling the massive crowds that started flocking to the convention halls around the mid-2000s (attendance swelled around 2011 with the Twilight movies).
Anything that will improve the Comic-Con experience and even allow non-attendees a taste of the magical July event would be welcome.
Whitney Grace, June 30, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
CloudFlare Claims Most Activity from Tor Is Malicious
June 28, 2016
Different sources suggest varying levels of malicious activity on Tor. Tech Insider shared an article responding to recent claims about Tor made by CloudFlare. The article, entitled, Google Search has a secret feature that shouts animal noises at you, offers information about CloudFlare’s perspective and that of the Tor Project. CloudFlare reports most requests from Tor, 94 percent, are “malicious” and the Tor Project has responded by requesting evidence to justify the claim. Those involved in the Tor Project have a hunch the 94 percent figure stems from CloudFlare attributing the label of “malicious” to any IP address that has ever sent spam. The article continues,
“We’re interested in hearing CloudFlare’s explanation of how they arrived at the 94% figure and why they choose to block so much legitimate Tor traffic. While we wait to hear from CloudFlare, here’s what we know: 1) CloudFlare uses an IP reputation system to assign scores to IP addresses that generate malicious traffic. In their blog post, they mentioned obtaining data from Project Honey Pot, in addition to their own systems. Project Honey Pot has an IP reputation system that causes IP addresses to be labeled as “malicious” if they ever send spam to a select set of diagnostic machines that are not normally in use. CloudFlare has not described the nature of the IP reputation systems they use in any detail.”
This article raises some interesting points, but also alludes to more universal problems with making sense of any information published online. An epistemology about technology, and many areas of study, is like chasing a moving target. Knowledge about technology is complicated by the relationship between technology and information dissemination. The important questions are what does one know about Tor and how does one know about it?
Megan Feil, June 28, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
SLI Systems Hopeful as Losses Narrow and Revenue Grows
June 14, 2016
The article titled SLI Systems Narrows First-Half Loss on Scoop reports revenue growth and plans to mitigate losses. SLI Systems is a New Zealand-based software as a service (SaaS) business that provides cloud-based search resources to online retailers. Founded in 2001, SLI Systems has already weathered a great deal of storms in the form of the dot-com crash that threatened to stall the core technology (developed at GlobalBrain.) According to a statement from the company, last year’s loss of $502K was an improvement from the loss of $4.1M in 2014. The article states,
“SLI shares have dropped 18 percent in the past 12 months, to trade recently at 76 cents, about half the level of the 2013 initial public offering price of $1.50. The software developer missed its sales forecast for the second half of the 2015 year but is optimistic new chief executive Chris Brennan and Martin Onofrio as chief revenue officer, both Silicon Valley veterans, can drive growth in revenue and earnings.”
The SLI of SLI stands for Search, Learn and (appropriately) Improve. The company hopes to achieve sustainable growth without raising additional capital by continuing to focus on innovation and customer retention rates, which slipped from 90% to 87% recently. Major clients include Lenovo, David Jones, Harvey Norman, and Paul Smith.
Chelsea Kerwin, June 14, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Axcelerate Focuses on Control and Visibility
June 13, 2016
The article on CMSWire titled Recommind Adds Muscle to Cloud e-Discovery relates the upgrades to the Axcelerate e-Discovery platform from Recommind. The muscle referred to in the article title is the new Efficiency Scoring feature offered to increase e-discovery review process transparency by tracking efficiency and facilitating a consistent assessment. The article explains,
“Axcelerate Cloud is built on Recommind’s interactive business intelligence layer to give legal professionals a depth of insight into the e-discovery process that Recommind says they have previously lacked. Behind all the talk of agility and visibility, there is one goal here: control. The company hopes this release allays the fears of legal firms, who traditionally have been reluctant to use cloud-based software for fear of compromising data.”
Hal Marcus, Director of Product Marketing at Recommind, suggested that in spite of early hesitancy by legal professional to embrace the cloud, current legal teams are more open to the possibilities available through consolidation of discovery requirements in the cloud. According to research, there are no enterprise legal departments without cloud-based legal resources related to contract management, billing, or e-discovery. Axcelerate Cloud aims to promote visibility into discovery practices to address the major concern among legal professionals: insufficient insight and transparency.
Chelsea Kerwin, June 13, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
The Unknown Future of Google Cloud Platform
June 10, 2016
While many may have the perception Google dominates in many business sectors, a recent graph published shows a different story when it comes to cloud computing. Datamation released a story, Why Google Will Dominate Cloud Computing, which shows Google’s position in fourth. Amazon, Microsoft and IBM are above the search giant in cloud infrastructure services when looking at the fourth quarter market share and revenue growth for 2015. The article explains why Google appears to be struggling,
“Yet as impressive as its tech prowess is, GCP’s ability to cater to the prosaic needs of enterprise cloud customers has been limited, even fumbling. Google has always focused more on selling its own services rather than hosting legacy applications, but these legacy apps are the engine that drives business. Remarkably, GCP customers don’t get support for Oracle software, as they do on Amazon Web Services. Alas, catering to the needs of enterprise clients isn’t about deep genius – it’s about working with others. GCP has been like the high school student with straight A’s and perfect SAT scores that somehow doesn’t have too many friends.”
Despite the current situation, the article hypothesizes Google Cloud Platform may have an edge in the long-term. This is quite a bold prediction. We wonder if Datamation may approach the goog to sell some ads. Probably not, as real journalists do not seek money, right?
Megan Feil, June 10, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Amazon Web Services: Crushing the Competition?
March 23, 2016
I read “Attack! Run. WTF? A Decade of Enterprise Class Fear and Uncertainty with AWS.” I am not sure if Amazon’s Web Services’ business is being praised or criticized. Nevertheless, the write up has some interesting factoids. I highlighted these statements:
IBM’s Cloud Services
- IBM, … was so flabbergasted [when Amazon won a US government contract] that the Blue Shirts of Armonk decided on the old-school route to victory and filed a legal complaint asking the government to re-evaluate IBM’s deal against that of Amazon, which Big Blue later withdrew.
- Famed for re-inventing itself around software in the 1990s under Lou Gerstner, the majority of IBM’s focus for the 2000s was devoted to unloading the PC and the server businesses on China. The firm is now trapped in a maelstrom of transition, restructuring and layoffs. Like Microsoft, IBM seems to have believed AWS couldn’t happen to it, that what the world needed was the same server software and services. It was nearly seven years after AWS that IBM realized something was afoot – probably when it lost both the CIA deal and got slapped about its attempts to make the CIA love it – that Big Blue said it would spend $2bn buying computing player SoftLayer and in 2014 throw $1.2bn into a massive data centre expansion to host your data and compute.
Microsoft Cloud Services
- Azure succumbed to classic innovator’s dilemma: how to sell a new platform as a package and at a price to maximize revenue without cannibalizing the company’s actual main money-makers – PC and server software. After delayed starts under Ray Ozzie and Bob Muglia, the technology roadmap only really clicked under new CEO Satya Nadella and executive software nerd Scott Guthrie. One brought the CEO-level commitment, the other made Azure work for developers.
- Gartner today regards Azure as number two, behind AWS, and yet… According to Gartner’s incumbent Cloud Queen Lydia Leong, Azure lacks the polish of AWS.
Oracle Cloud Services
- Oracle, which bought Sun, preferred to play a Game of Thrones that was corporate M&A to hold onto its position in IT. Sadly, it chose wrong; Oracle spent $8.5bn on Sun but ultimately discontinued the company’s fledgling utility computing service. Hardware and Java was what Oracle wanted.
- Today, Oracle’s resultant hardware business makes just half the revenue of AWS and is is shrinking – falling 13 per cent to $1.1bn – versus AWS’s 69 per cent growth last quarter to $2.4bn. That past complacency of Oracle’s CEO on cloud has put Oracle firmly in a pack of also rans behind AWS on platform cloud, with Oracle now throwing PR at a problem to convince Wall St it is credible as a provider of IT as a service.
And what about Amazon? The write up points out:
- AWS is still attacking – growing at a phenomenal rate, 71 per cent in its recent quarter to $2.4bn and 69 per cent for the year to $7.88bn. The appetite among enterprises for AWS’s style of technology and model of delivery clearly hasn’t yet been satiated.
- …the truth is AWS now has its fences across so much of the cloud, removing them isn’t an option. The big question then for AWS at the age of 10 is this: when will the old men of IT regain their wind? How big will be their counter-attack and will it be concerted? Will it pose a tangible threat and how would AWS respond?
I noted that Apple has shifted some of its cloud business to the Google from AWS. I assume the Board of Directors’ excitement is now behind the kids from Cupertino. What’s clear is that IBM and Oracle seem to face an uphill slog if I understand the write up. Read the original and decide for yourself. I love the WTF. Some stakeholders may be asking this question too.
Stephen E Arnold, March 23, 2016
Recommind Enables Easier Oversight into E-Discovery for Legal Industry
February 19, 2016
A recent article, entitled Recommind Adds Muscle to Cloud e-Discovery from CMS Wire, highlights an upgrade to Recommind’s Axcelerate e-discovery platform. This information intelligence and governance provider for the legal industry has upped their offering by adding a new efficiency scoring feature to enable “extensive visibility into the overall e-discovery review process.” Recommind make the updated based on polling their clients and finding 80 percent do not have oversight in regards to the technological competency of their outside counsel:
“Citing the same survey, he added that 72 percent of respondents pointed to insufficient visibility into the discovery practices of their outside counsel — legal professionals working with them but outside the firm — as a major concern. Axcelerate Cloud also eliminates the cost unpredictability that arises with traditional hosting charges with cloud-based e-discovery tools providers and the infrastructure maintenance required for on-premises solutions.”
When insights from big data is what a company is after, stronger cloud-based functionality is often the first step. Reminds us of enterprise search firm Autonomy which was eventually sold to HP. What will be next for Recommind?
Megan Feil, February 19, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Coveo Lauds Itself for Growth, Innovation, and Industry Awards
February 16, 2016
The article on EIN News titled Coveo Achieves Another Record-Breaking Quarter and Calendar Year of Rapid Growth discusses the search companies growth and recognition in a nakedly self-congratulating post. In 2015, Coveo released both Coveo Cloud, a streamlined search-as-a-service, and Coveo Reveal, a self-learning search service aimed at understanding intent to ensure improved accuracy and relevance in search results. The article states,
“The company expanded its SI ecosystem with several leading CRM and Customer Community system integrators, including Appirio, Bluewolf, Cloud Sherpas, Etherios, NTT Data Cloud Services and Vertiba. Exiting 2015, Coveo had in excess of 100 certified SI partners… Coveo for Sitecore was named as a 2015 CUSTOMER Magazine Product of the Year Award winner, marking the fourth consecutive year that Coveo has won this award (In January of 2015 Coveo received its fifth consecutive CUSTOMER Magazine product of the year award…)”
So just how big was that fish Coveo caught? The private company reports a “record breaking quarter” lists any number of current projects and industry recognitions. According to the article, the company now has a total amount of financing of $75 million. 2015 was clearly a very good year, particularly in the financial services market. What company can resist patting itself on the back?
Chelsea Kerwin, February 16, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Feeding the Google AI Beast and Keeping in Mind, You Are What You Eat
January 13, 2016
The article titled We are All SkyNet in the Googlesphere on Disinformation refers to the Terminator’s controlling A.I., SkyNet, who determines the beginning of a machine age in the movie, and the conspiracy that Google is taking on that role in reality. Is it easy to understand the fear of Google’s reach, it does sometimes seem like a gigantic arm with a thousand hands groping about in cyberspace, and collecting little pieces of information that on their own seem largely harmless. The article discusses cloud computing and its relationship to the conspiracy,
“When you need your bits of info, your computer gathers them from the cloud again. The cloud is SkyNet’s greatest line of defense, as you can’t kill what is spread out over an entire network. Since the magnificent expose of the NSA and their ability to (at least) access every keystroke, file or phone call and Google’s (at minimum) complicity in managing the data, that is to say, nearly all data being collected, it’s hard to imagine the limitations to what any such Google AI program could learn.”
The article ends philosophically with the suggestion that the nature of a modern day SkyNet will depend on the data that it gathers from us, that we will create the monster in our likeness. This may not be where we expected the article to go, but it does make sense. Google as a company will not determine it, at least if literature has taught us anything.
Chelsea Kerwin, January 13, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
-
- Subscribe to Beyond Search
Feature archive
News archive
-
