LucidWorks Bet on Spark. Now What?
August 8, 2016
Many clear night ago, Lucid Imagination offered an open source enterprise search solution. Presidents came. Presidents went. Lucid Imagination morphed into LucidWorks. I promptly referred to the company in this way: Lucid works, really?
The firm embraced Spark and did a not-unexpected pirouette into a Big Data outfit. I know. I know. Lucid Imagination is a company anchored in key word search, but this is the 21st century. Pirouettes are better than mere pivots, so Big Data it is.
I read “Big Data Brawlers: 4 Challengers to Spark” and the write up triggered some thoughts about LucidWorks. Really.
The point of the story is to identify four open source solutions which do what Spark allegedly does so darned well. Each of these challengers:
- Scales
- Handles Big Data (whatever that means)
- Exploits cheap memory so there are no slug like disc writes
- Does the old school batch processing thing.
What are the “challengers” to Spark? Here are the contenders:
- Apache Apex. Once proprietary, now open source, the software does micro batching for almost, sort of real time functions
- Heron. Another real time solution with spouts and bolts. Excited?
- Apache Flink. This is an open source library with a one two punch: It does the Flink stuff and the Spark stuff.
- Onyx. This is a distributed computation system which will appeal to the Java folks.
What do these Spark alternatives have to do with LucidWorks, really? I think there is going to be one major impact. LucidWorks will have to spend or invest in supporting whatever becomes the next big thing. Recommind hit a glass ceiling with its business model. LucidWorks may be bumping into the open source sky light. Instead of being stopped, LucidWorks has to keep investing to keep pace with what the community driven folks generate with little thought to the impact on companies trying to earn a living with open source.
Stephen E Arnold, August 8, 2016
Hewlett Packard: A Future of Uncertainty
August 7, 2016
I read “Private Equity Ponders Hewlett Packard Enterprise Buyout.” I think this is called a tap out in millennial lingo or quitting in less zippy language. I recognize that absolutely everything I read on the Internet is true. Especially Information.
Hewlett Packard has created some of its problems (Board of Directors’ issues, the exciting Autonomy matter, and the great mitosis which saw ink go one way and enterprise services another.) Other problems are external. Who imagined Amazon, the digital Wal-Mart, becoming the big dog in cloud computing. The economy? Well, let’s leave that to the political and economic wizards with MBAs, CPAs, and lawyers, lots of lawyers.
The write up, which I assume is spot on, informed me with information:
Several private equity firms including KKR, Apollo Global Management and Carlyle Group are sniffing around Hewlett Packard Enterprise, contemplating a buyout of the firm, said a person who has had talks with representatives of the firms. Such a deal would be worth more than $40 billion.
What else does the write up assert? I don’t know because after the “exclusive” and the fetching factoid, I have to pony up dough to learn more. My hunch is that the green eyeshade crowd believes that the individual chunks of HPE are worth more than the company in its present form.
Fortune Magazine, once a unit of America Online, knows of what it speaks in “These Private Equity Firms Could Be Looking to Buy Hewlett Packard Enterprise.” No one is exactly sure what’s afoot. I highlighted this passage:
Regardless of the conflicting reports, it seems that HPE is undergoing another significant transformation again.
I like the “transformation” angle. It reminds me of Dr. Daphne Swartz’s lecture in Biology 101 about the caterpillar to butterfly thing. You know a crawly worm with fur becomes a winged creature with nifty scales. These scales impart the color. Otherwise, the butterfly’s wings, like the emperor without clothes, are not much to look at.
What about Autonomy I ask myself? Well, it seems that it is a race to see who will sell the software first. Hewlett Packard seems to be shopping the unit. But if those fun loving green hued folks get their first, some other pavement pounders will get in on the act.
What’s this mean for licensees of Autonomy? The sunk costs can be fascinating. Unlike the colors of the butterfly’s wings, the costs of training, tuning, and maintaining the system can be spectacular.
The turmoil swirling around HPE is chaotic, maybe disruptive. To some, that’s a good thing. To those who just want to find information, the future looks like a furry caterpillar: Ugly and pesticides infuse the stuff the worm eats.
Stephen E Arnold, August 6, 2016
Government IT Procurement Wobble
July 5, 2016
I read “IT Showdown: Tech Giants Face Off against 18F.” What’s an 18F? If you do work for the US government, you associate 18 F with the address of the General Services Administration. The name now evokes some annoyance among established US government contractors. The term 18F refers to a group set up to reduce the time, cost, and hassle of getting IT “done”.
In the good old days, there were people in the US government who did things. Over the years, US government professionals rely on contractors to do certain types of work. In the information technology world, the things range from talking about how one might do something to actually setting up a system to deliver certain outputs.
Along the way, commercial enterprises provided hardware, software, and services. The hardware and software were, for many years, proprietary or custom crafted to meet the needs of a particular government entity. These statements of work made life difficult for a vendor who used what were often perceived as expensive solutions. License agreements made it tricky for a government entity to get another commercial outfit to modify or work around limitations of certain commercial systems.
According to the write up, some of the established vendors are grousing. I learned:
At a House subcommittee hearing on June 10, lobbyists from the IT Alliance for Public Sector (ITAPS) and the Software & Information Industry Association (SIIA) alleged that 18F is hindering profits by acting as both a procurement policymaker and as a tech competitor inside the General Services Administration (GSA). The two groups assert a conflict of interest, and in testimony, have submitted a list of grievances and recommendations intended to curtail 18F’s authority. The hearing was conducted jointly by the House Subcommittees of Government Operations and Information Technology to assess the effectiveness of 18F and the U.S. Digital Service (USDS) — a sister tech consultancy within the White House.
The industry group perceives the 18F outfit as a bit of a threat. Blanket purchase agreements, open source solutions, and giving certain contracts for small coding jobs to non traditional outfits are not what the established information technology vendors want to happen.
I find the dust up amusing. The revenues of established information technology vendors are not likely to suffer sharp declines overnight. The 18F initiative is an example of the US government trying to find a solution to escalating costs for information technology and the gap between the commercial solutions available and actual solutions deployed in a government entity.
Will 18F reduce the gap? One thing is certain. Some vendors associate the term “18F” with some different connotations. Imagine a government professional using a mobile phone app to perform a task for personal work and then using a mainframe act to perform a similar task in a government agency. Exciting.
Stephen E Arnold, July 5, 2016
Palantir Technologies: A Valuation Factoid
July 5, 2016
I read “Palantir Buyback Plan Shows Need for New Silicon Valley Pay System.” (You may have to view this write up. Don’t email me. I don’t think about “real” journalists.) Tucked into the somewhat humorous write up was a factoid. I want to capture it because “real” reporters and “real” information can be tough to track down using an online search system.
Here’s the factoid:
It [Palantir] is offering $7.40 a share to buy back up to 12.5 percent of an employee’s shares…Morgan Stanley recently marked down the value of Palantir’s shares to $5.92.
That $1.48 just hangs there. Too bad the write up did not answer this question:
What were the valuations Morgan Stanley assigned when Palantir Technologies had a valuation of $20 billion. I assume that rainbows, unicorns, and other “real” artifacts, one must assume that Palantir is zipping right along the information superhighway.
Stephen E Arnold, July 5, 2016
More Palantir Spotting
June 27, 2016
Trainspotting is a collection of short stories or a novel presented as a series of short stories by Irvine Welsh. The fun lovers in the fiction embrace avocations which seem to be addictive. The thrill is the thing. Now I think I have identified Palantir spotting.
Navigate to “Palantir Seeks to Muzzle Former Employees.” I am not too interested in the allegations in the write up. What is interesting is that the article is one of what appears to be of series of stories about Palantir Technologies enriched with non public documents.
The Thingverse muzzle might be just the ticket for employees who want to chatter about proprietary information. I assume the muzzle is sanitary and durable, comes in various sizes, and adapts to the jaw movement of the lucky dog wearing the gizmo.
Why use the phrase “Palantir spotting.” It seems to me that making an outfit which provides services and software to government entities is an unusual hobby. I, for example, lecture about the Dark Web, how to recognize recycled analytics algorithms and their assorted “foibles,” and how to find information in the new, super helpful Google Web search system.
Poking the innards of an outfit with interesting software and some wizards who might be a bit testy is okay if done with some Onion type or Colbert like humor. Doing what one of my old employers did in the 1970s to help ensure that company policies remain inside the company is old hat to me.
In the write up, I noted:
The Silicon Valley data-analysis company, which recently said it would buy up to $225 million of its own common stock from current and former staff, has attached some serious strings to the offer. It is requiring former employees who want to sell their shares to renew their non-disclosure agreements, agree not to poach Palantir employees for 12 months, and promise not to sue the company or its executives, a confidential contract reviewed by BuzzFeed News shows. The terms also dictate how former staff can talk to the press. If they get any inquiries about Palantir from reporters, the contract says, they must immediately notify Palantir and then email the company a copy of the inquiry within three business days. These provisions, which haven’t previously been reported, show one way Palantir stands to benefit from the stock purchase offer, known as a “liquidity event.”
Okay, manage information flow. In my experience, money often comes with some caveats. At one time I had lots and lots of @Home goodies which disappeared in a Sillycon Valley minute. The fine print for the deal covered the disappearance. Sigh. That’s life with techno-financial wizards. It seems life has not changed too much since the @Home affair decades ago.
I expect that there will be more Palantir centric stories. I will try to note these when they hit my steam powered radar detector in Harrod’s Creek. My thought is that like the protagonists in Trainspotting, Palantir spotting might have some after effects.
I keep asking myself this question:
How do company confidential documents escape the gravitational field of a comparatively secretive company?
The Palantir spotters are great data gatherers or those with access to the documents are making the material available. No answers yet. Just that question about “how”.
Stephen E Arnold, June 27, 2016
Peak Unicorn: Hooves of Doom
June 23, 2016
I loved the phrase “peak unicorn.” The co9mbination of mixed metaphors and a mythical horned equine is delicious. Navigate to “The Unicorn Godmother Dishes on Silicon Valley.” I find the addition of a “godmother” a bit like a 1958 Chevrolet Impala with additional chrome bolted on by an ambitious retro rod shop. Unicorns, peaks, and godmothers!
The main point of the write up in my opinion is not fruit salad metaphors. Here’s the passage I highlighted in passion fruit reddish purple:
I think we’re in a valuation-adjustment period where we’ve basically had very bullish markets both in the private and the public sectors for tech stocks over the past three to five years, and valuation multiples just got out of whack. There was too much money pouring into tech; and a perception developed that the only way to win was to offer a higher price. You know if there’s one house in a neighborhood that everybody wants, generally the way to get the house is to offer a higher price.
Yes, real estate. The “value” of a house in Holmby Hills compared to the value of a home in Pig, Kentucky.
The write up makes clear that some folks in Sillycon Valley may be getting nervous. Time to cash in and enjoy the good life. Unicorn farming in Pig, Kentucky? Search and content processing vendors are welcome too. A quick trip via flying car I hear.
Stephen E Arnold, June 23, 2016
Digital Currencies: More Excitement
June 21, 2016
An “attacker” explains the legal perception he has. You can read this argument at this link. I do not have a horse in this race. In my recent lecture at a security conference in Myrtle Beach, SC, I pointed out that digital currencies work reasonably well for what I call small scale transactions. Putting one’s life savings into a digital currency is a step some bad actors are reluctant to take. Traditional non digital money laundering and tax evasion methods will slowly yield to Fancy Dan types of “money.” But if you are adventurous, have a go.
Stephen E Arnold, June 21, 2106
The Job Duties of a Security Analyst
June 15, 2016
The Dark Web is a mysterious void that the average user will never venture into, much less understand than the nefarious reputation the media crafts for it. For certain individuals, however, not only do they make a lively hood by surfing the Dark Web, but they also monitor potential threats to our personal safety. The New York Times had the luck to interview one Dark Web security analyst and shared some insights into her job with the article, “Scouring The Dark Web To Keep Tabs On Terrorists.”
Flashpoint security analyst Alex Kassirer was interviewed and she described that she spent her days tracking jihadists, terrorist group propaganda, and specific individuals. Kassirer said that terrorists are engaging more in cybercrimes and hacking in lieu/addition of their usual physical aggressions. Her educational background is very impressive with a bachelor’s from George Washington University with a focus on conflict and security, a minor in religious studies, and she also learned some Arabic. She earned her master’s in global affairs at New York University and interned at Interpol, the Afghan Embassy, and Flashpoint.
She handles a lot of information, but she provides:
“I supply information about threats as they develop, new tactics terrorists are planning and targets they’re discussing. We’ve also uncovered people’s personal information that terrorists may have stolen. If I believe that the information might mean that someone is in physical danger, we notify the client. If the information points to financial fraud, I work with the cybercrime unit here.”
While Kassirer does experience anxiety over the information she collects, she knows that she is equipped with the tools and works with a team of people who are capable of disrupting terroristic plots.
Whitney Grace, June 15, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Ransomware as a Service Deals in Bitcoins of Course
June 14, 2016
Countless “as-a-service” models exist online. A piece from SCMagazine, Dark web forums found offering Cerber ‘ransomware as a service’, reveals more information about one such service called ransomware-as-a-service (RaaS), which we’ve heard about now for quite some time. Ransomware injects a virus onto a machine that encrypts the user’s files where they remain inaccessible until the victim pays for a key. Apparently, an Eastern European ransomware, Cerber, has been offering RaaS on Russian Dark Web forums. According to a cyber intelligence firm Sensecy, this ransomware was setup to include “blacklisted” countries so the malware does not execute on computers in certain locations. The article shares,
“Malwarebytes Labs senior security researcher Jerome Segura said the blacklisted geographies – most of which are Eastern European countries – provide “an indication of where the malware originated.” However, he said Malwarebytes Labs has not seen an indication that the ransomware is connected to the famed APT28 group, which is widely believed to be tied to the Russian government. The recent attacks demonstrate a proliferation of ransomware attacks targeting institutions in the U.S. and Western nations, as recent reports have warned. Last week, the Institute for Critical Infrastructure Technology (ICIT) released a study that predicted previously exploited vulnerabilities will soon be utilized to extract ransom.”
Another interesting bit of information to note from this piece is the going ransom is one bitcoin. Segura mentions the value ransomers ask for may be changing as he has seen some cases where the ransomer works to identify whether the user may be able to pay more. Regardless of the location of a RaaS provider, these technological feats are nothing new. The interesting piece is the supposedly untraceable ransom medium supplanting cash.
Megan Feil, June 14, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Alphabet Google Factoid: Media Spend Control
May 31, 2016
I noted “Google Now Controls 12 Percent of All Global Media Spend.” My immediate reaction was, “Just 12 percent.” I assumed that the Alphabet Google thing had cornered much more of the media spend. I learned:
Alphabet controls 12 percent of all global media spend, which primarily comes from Google and YouTube’s ad sales. The company collects $60 billion in U.S. ad spend—a figure 166 percent larger than No. 2 ranking The Walt Disney Company. To compare, Google’s ad revenue was 136 percent larger than Walt Disney last year. Alphabet’s overall ad revenue is up 17 percent year-over-year.
Google is not without competition. I love “competition” in the online digital world. The write up points out:
Facebook in particular continues to become an advertising juggernaut. The social network jumped from No. 10 in 2015 to No. 5 this year, making it the fastest-growing company on Zenith’s list with 65 percent year-over-year growth. Chinese Internet company Baidu is the second fastest-growing company, with ad revenues up 52 percent.
I am not an ad expert. I certainly don’t know anything about media spend. After 15 years of slogging, the 12 percent figure strikes me as interesting. It seems that in a shorter time period, Facebook has been the hot item. Search or social media? Which is the “winner”? Both? Who are the losers?
Traditional media. Another surprise?
Stephen E Arnold, May 31, 2016
-
- Subscribe to Beyond Search
Feature archive
News archive
-
