Microsoft: The Joy of Figuring Out What Code Can Do
October 26, 2020
DarkCyber finds Microsoft in an interesting spot. On one hand, Microsoft wants to be open sourcey. The idea of community created and community supported software provides a useful source of ready-to-microwave code nuggets, hints about whom to hire, and an opportunity to reduce the maintenance cost of certain components.
On the other hand, monitoring what’s on GitHub and, more importantly, how code can be used is a sticky wicket.
“RIAA Blitz Takes Down 18 GitHub Projects Used for Downloading YouTube Videos” explains:
Microsoft-owned GitHub has removed today 18 projects from its code-hosting portal following a legal request filed by the Recording Industry Association of America (RIAA)….In a letter sent to GitHub, RIAA argued that the “clear purpose of this source code [the youtube-dl library]” was to “circumvent the technological protection measures used by authorized streaming services such as YouTube” and to allow users to “reproduce and distribute music videos and sound recordings […] without authorization.”
The issue is likely to be a thorny one. Code can be used for many things:
- To perform a function
- A way to learn how to do a task
- Create software unrelated to the GitHub offering.
Microsoft has removed the “offending” software. But the problem could become the seed of a giant junk maple in the main Redmond campus green space. The article makes this point, and it is an important one:
RIAA isn’t alleging the library infringed on its rights, but that the library is illegal in itself.
Just as Microsoft wants to get open sourcey and more social, it finds itself in an interesting spot. Who or what will fertilize and water this tiny take down seed? Exactly what can code do? Exactly to what purposes can code be put? What about software which includes code which can do something a third-party defines as illegal? So many questions for the JEDI knights.
Stephen E Arnold, October 26, 2020
Open Source: A New Slogan Emerges. No Poster Art Yet
October 23, 2020
I read “Huawei’s Open Source Innovation Inspired by Of All, By All, for All.” Interesting. Microsoft is interested in open source. Amazon is semi interested in open source. Google is probably still interested in open source unless the team working on open source lost interest. But Huawei? Huawei is interested in open source. The write up reports:
Huawei has acknowledged the importance of open source and the role it plays in accelerating innovation within the software industry, stating that ecosystems such as openEuler, openGauss, openLooKeng, and MindSpore have created an ecosystem of open source basic software projects….The openEuler, openGauss, openLooKeng and MindSpore open source communities are all ‘led’ by Huawei as the company seeks to lay the groundwork for full-stack hardware and software collaboration.
Does Huawei’s support of open source fit into the strategic plan for Chinese technology?
The article provides a partial answer:
Huawei Cloud & AI Open Source business general manager Du Junping says that open source enables organizations to create innovation and value in an environment that is ‘open, fair, transparent, and secure’. Huawei says it is inspired by the mindset of fostering a sustainable, open source basic software ecosystem ‘Of All, By All, For All’.
Catchy: Of all, by all, for all. Very egalitarian and kumbaya-ish. Is it similar to “Smash the gang of four” or “Have fewer children, raise more pigs”? No, of course not.
Stephen E Arnold, October 23, 2020
Comparison of Elasticsearch, Solr, and Sphinx
October 8, 2020
Search and retrieval underpins most policeware and intelware systems. Open source search software has made life more challenging for vendors of proprietary enterprise search solutions. There are versions of an “in depth” enterprise search analysis like this available for thousands of dollars from marketers like https://www.adroitmarketresearch.com sporting this title:
Enterprise Search Market Demand Analysis and Projected huge Growth by 2025| IBM Corp, Coveo Corp., Polyspot & Sinequa Inc., Expert System Inc., HP Autonomy, Lucidworks, Esker Software Corp., Dassault Systemes Inc., Perceptive Software Inc., and Marklogic Inc.
Notice that none of the search vendors in “Elasticsearch vs. Solr vs. Sphinx: Best Open Source Search Platform Comparison” appears in the Adroit Market Research report. That’s important for one reason: Open source search has driven vendors of proprietary systems into a corner. What’s even more intriguing is that some vendors of enterprise search like Attivio and IBM Corp. use open source search technology but take pains to avoid revealing the plumbing under the house trailer.
The comparison is, for now, available without charge online, courtesy of Greenice. This firm, based in Ukraine, is what I would describe as a DevOps consulting and services company. It’s a mash up of advisory, coding, and technical deliverables.
The comparison contains some useful information; for example:
- Inclusion of examples of the search systems’ visualization capabilities
- Examples of organizations using each of the three systems compared
- Presentation of the analyst’s perception of strengths and weaknesses of each system
- References to machine learning in the context of the three systems.
What caught my attention is the disconnect between the expensive and somewhat over enthusiastic for fee study about search and this free analysis.
Many of the problems in search are a result of what may be described as “over enthusiastic marketing.” This approach to jazzing up what can be accomplished by information retrieval technology has resulted in at least one jail sentence for an enterprise search entrepreneur and may be followed by jail time for other companies’ executives who practice razzmataz sales techniques.
The principal value of the free comparison is that it does a good job of walking through basic information without the Madison Avenue hucksterism. Net net: A free write up with some helpful information.
Stephen E Arnold, October 8, 2020
Google and Its User Privacy: Happy Hunting
October 4, 2020
DarkCyber spotted an open source intelligence tool called GHunt. By the time an open source software becomes publicly available, DarkCyber believes that hardened systems and methods are integrated into specialized policeware and intelware systems. If you want to try to learn more about a particular Google email user, for instance, you may want to take a look at GHunt. There are screenshots and basic information available on Github. Google appears to be taking steps to address some of the “features” which the GHunt software taps. Some interesting open source software becomes available and then disappears; for example, DARPA Memex tools have evidenced this type of behavior. If you want this tool, DarkCyber suggests you move along in a sprightly manner.
Stephen E Arnold, October 4, 2020
Elastic: Making Improvements
August 27, 2020
Elasticsearch is one of the most popular open-source enterprise search platforms. While Elasticsearch is free for developers to download, Elastic offers subscriptions for customer support and enhanced software. Now the company offers some new capabilities and features, HostReview reveals in, “Elastic Announces a Single, Unified Agent and New Integrations to Bring Speed, Scale, and Simplicity to Users Everywhere.” The press release tells us:
“With this launch, portions of Elastic Workplace Search, part of the Elastic Enterprise Search solution, have been made available as part of the free Basic distribution tier, enabling organizations to build an intuitive internal search experience without impacting their bottom line. Customers can access additional enterprise features, such as single sign-on capabilities and enhanced support, through a paid subscription tier, or can deploy as a managed service on Elastic Cloud. This launch also marks the first major beta milestone for Elastic in delivering comprehensive endpoint security fully integrated into the Elastic Stack, under a unified agent. This includes malware prevention that is provided under the free distribution tier. Elastic users gain third-party validated malware prevention on-premises or in the cloud, on Windows and macOS systems, centrally managed and enabled with one click.”
The upgrades are available across the company’s enterprise search, observability, and security solutions as well as Elastic Stack and Elastic Cloud. (We noted Elastic’s welcome new emphasis on security last year.) See the write-up for the specific updates and features in each area. Elasticsearch underpins operations in thousands of organizations around the world, including the likes of Microsoft, the Mayo Clinic, NASA, and Wikipedia. Founded in 2012, Elastic is based in Silicon Valley. They also happen to be hiring for many locations as of this writing, with quite a few remote (“distributed”) positions available.
Cynthia Murrell, August 27, 2020
Twitch: Semantic Search Stream to Lure Gamers, Trolls, and Gals?
July 31, 2020
Amazon Twitch may be more versatile than providing the young at heart with hours of sophisticated content. There are electronic games, trolls (lots of trolls armed with weird icons), and what appear to be females.
Now Twitch will be moving along the content spectrum with the addition of a stream about semgrep. If you are not on a first name basis, semgrep is a semantic search thing. You can join in for free, no waiting rooms, and no big technical hurdles. I suppose one could create a lecture about semantic methods in TikTok 30-second videos which might be a first for the non-invasive, controversial app. Nah, go for Twitch. Skip YouTube and Facebook. Go Bezos bulldozer.
Navigate to https://twitch.tv and go to the jeanqasaur stream. The time on July 31, 2020? The show begins at 4 pm US Eastern time.
The program is definitely perceived by some as super important. A motivated semantic wizard posted a message on the TweetedTimes.com semantic page. Here’s what the message looks like:
DarkCyber’s suggestions:
- Do not become distracted by Raj recruiting, Bad Bunny, or Celestial Fitness. Keep your eye on the grep as it were.
- Sign up because Amazon wants you to be part of the family. Prime members may receive extra Bezos bucks somewhere down the line
- Exercise good grammar, be respectful, and keep your clothes on. Twitch banned SweetSaltyPeach who reinvented herself as RachelKay, Web developer, fashion model, and gamer icon. You may have to reincarnate yourself too.
- Avoid the lure of Animal Crossing Arabia II.
Stephen E Arnold, July 31, 2020
Messaging: Pushing the Envelope
July 31, 2020
In my lectures for the 2020 National Cyber Crime Conference, I discussed messaging as a rapidly evolving mechanism. Simple text has morphed into a viable alternative to a traditional Dark Web site. Via encrypted messaging services, individuals can join groups, locate products and services, and pay for them often with bitcoin or other digital currency. Although it is possible to compromise encrypted messages, the volume poses a significant problem for law enforcement. I pointed out that the developers of Telegram reached an agreement with Russia in order to prevent their messaging service from being blocked.
Another messaging service warrants some attention. The service is called Element. Element was formerly known as Riot and Vector, according to some individuals. The system is based on Matrix; that is, an open source protocol for real time communication. Element, like other modern messaging systems, encrypts data.
In an email from an individual who wishes to remain anonymous, the Element messaging service can interact with with other services, including the aforementioned Telegram. Is Element an alternative to Slack and similar programs like Microsoft Teams?
The answer is, “Could be.”
Slack and Teams are widely known and engaged in what may become an interesting legal tussle. Facebook, however, continues to push toward a unified messaging platform, offering features that make finding, buying, selling, and communicating a mostly one click process.
Element has the potential to become an open source alternative to encrypted messaging solutions from vendors like Facebook and Telegram.
In light of the capabilities of the US National Security Agency and the continuing efforts of the European Union to force providers to allow instream decryption, the resolution is likely to be political.
Until users of encrypted messaging services demand government respect for privacy, which is a Fourth Amendment issue in the US, governments will continue to pressure and possibly resort to what some may characterize as blackmail. The pressure may be unconstitutional in some countries and unwarranted in others.
Encrypted messaging has become the “new” Dark Web if the DarkCyber research team’s analysis is accurate. The issue is yet another one to add to the pile of contentious services for ubiquitous mobile devices.
For more information about the chat service, navigate to the Element information page.
Stephen E Arnold, July 31, 2020
Funding Open Source: Saddle Up, Don Quixotes
July 30, 2020
I read “A New Funding Model for Open Source Software.” The main idea is that the current approach to financial “support” of open source software is broken. I agree, particularly if one looks at the problem from the developer or developers in the “community.”
The fix, according to the write up, is “sponsor pools.” Here are the details:
Every month, you donate some amount into a “wallet”. Your funds are then distributed to the projects in your “sponsor pool”. Your sponsor pool is just the set of open-source projects you want to support. Adding new projects to your pool should require one click — as easy as starring the repo on GitHub. That’s it. It’s hardly ingenious, which is why it’s surprising that no major player in OSS has implemented it for facilitating open source donations.
The comments to the post at this link are interesting and raise a number of points, both pro and con.
I noticed that none of the comments pointed out that open source has become the hunting ground for certain large technology companies. Github is owned by Microsoft; Amazon is ferrying open source code into its proprietary AWS walled garden; Google is “contributing to the community” and then using the community as a recruiting supply line. Other techniques are in play as well.
Also, open source is more attractive to large established companies. These firms have the staff and financial resources to make chunks of open source play nicely together. The goal is to eliminate dependence on proprietary solutions, restrictive license agreements, and those necessary maintenance and engineering services deals. Smaller outfits often find Microsoft a convenient way to solve a database problem. Why? It’s available and semi-reliable. Keep in mind that Microsoft bought Github for control and revenue opportunities.
Finally, a number of the comments suggest, “Let Github do it.” Yeah, I really think Microsoft has open source software love as a business motivation. But that’s just my view.
My view is that open source, like other nifty things associated with the “old days” of the Internet may be facing some challenges and not just from Rona.
Stephen E Arnold, July 30, 2020
Linux Developer Is Unhappy with Amazon
June 17, 2020
Who doesn’t love Amazon? Maybe the person credited with developing Linux? That would be Linus Torvalds, developer of note.
No one pays attention to insults on the Internet unless someone with clout says them. The IT community definitely paid attention to the head of the Linux kernelLinus Torvalds when he said, “Linus Torvalds Rejects ‘Beyond Stupid’ AWS-Made Linux Patch For Intel CPU Snoop Attack” reports ZDNet.
In early 2020, Snoop launched attacks on Intel andCore CPUs and AWS discovered it. The attack causes CPUs to leak data from its L1D cache via bus snooping—a cache-updating operation that happens when the L1D modifies data. AWS developed a patch for the Linux kernel that would allow applications to opt in to flush the L1D cache when a task is switched out. Torvalds thinks the patch would degrade performance in other applications. Torvalds said:
“ ‘Because it looks to me like this basically exports cache flushing instructions to user space, and gives processes a way to just say ‘slow down anybody else I schedule with too’…‘In other words, from what I can tell, this takes the crazy ‘Intel ships buggy CPU’s and it causes problems for virtualization’ code (which I didn’t much care about), and turns it into ‘anybody can opt in to this disease, and now it affects even people and CPU’s that don’t need it and configurations where it’s completely pointless’.
‘I don’t want some application to go ‘Oh, I’m _soo_ special and pretty and such a delicate flower, that I want to flush the L1D on every task switch, regardless of what CPU I am on, and regardless of whether there are errata or not. Because that app isn’t just slowing down itself, it’s slowing down others too.’’
Torvalds also think the patch is crazy because a hack could inhabit another core within the CPU and attack the L1 cache before its flushed. Another fun word he used was pseudo-security.
Usually “pseudo” is reserved for science, but this works too.
Whitney Grace, June 17, 2020
Degoogling: Yes, It Is a Thing
June 8, 2020
Can free and open source software “degoogle”? Probably not easily or quickly. Nevertheless Reddit user TheEvilSkely wants to try. You can dig through the details, explore the GitHub information, and follow the links at this link. The challenge is that most of the whippersnappers are just okay with the Google. Like Amazon, the service is just wonderful. Why make a change if everything is A-Okay?
News flash: Open source is not just tangled with Googzilla’s feathers (dinosaurs, according to modern research by thumbtypers, had feathers, bright ones too). The Redmond contingent is into open source. Imagine Linux in the really easy to update Windows environment. Amazon is driving its Bezos bulldozer through the thicket of new growth saplings like Elastic as I type this post. If you listen closely, you can hear the bulldozer shift into a lower gear to push annoying old growth software into the dirt. One doesn’t need to have an oracle to understand the earth moving concept.
Open source is a target for these reasons:
- Community software lowers certain coding costs and has enough bugs to make proprietary fixes a money maker
- Young developers learn open sourcey ways in college and arrive ready to earn and burn in their virtual frat and sorority duds when they become WFHers and on prem contractors
- Big companies love open source because they can devote resources to tweaking the software and have enough money to pay legions of certified advisers help out, license open source optimized cloud environments, and pay for proprietary widgets that don’t change the “no handcuffs” idea of non proprietary plumbing.
Worth monitoring, of course.
Stephen E Arnold, June 8, 2020
-
- Subscribe to Beyond Search
Feature archive
News archive
-
