Big Data Needs to Go Public
December 16, 2016
Big Data touches every part of our lives and we are unaware. Have you ever noticed when you listen to the news, read an article, or watch a YouTube video that people say items such as: “experts claim, “science says,” etc.” In the past, these statements relied on less than trustworthy sources, but now they can use Big Data to back up their claims. However, popular opinion and puff pieces still need to back up their big data with hard fact. Nature.com says that transparency is a big deal for Big Data and algorithm designers need to work on it in the article, “More Accountability For Big-Data Algorithms.”
One of the hopes is that big data will be used to bridge the divide between one bias and another, except that he opposite can happen. In other words, Big Data algorithms can be designed with a bias:
There are many sources of bias in algorithms. One is the hard-coding of rules and use of data sets that already reflect common societal spin. Put bias in and get bias out. Spurious or dubious correlations are another pitfall. A widely cited example is the way in which hiring algorithms can give a person with a longer commute time a negative score, because data suggest that long commutes correlate with high staff turnover.
Even worse is that people and organizations can design an algorithm to support science or facts they want to pass off as the truth. There is a growing demand for “algorithm accountability,” mostly in academia. The demands are that data sets fed into the algorithms are made public. There also plans to make algorithms that monitor algorithms for bias.
Big Data is here to say, but relying too much on algorithms can distort the facts. This is why the human element is still needed to distinguish between fact and fiction. Minority Report is closer to being our present than ever before.
Whitney Grace, December 16, 2016
Costs of the Cloud
December 15, 2016
The cloud was supposed to save organizations a bundle on servers, but now we learn from Datamation that “Enterprises Struggle with Managing Cloud Costs.” The article cites a recent report from Dimensional Research and cloud-financial-management firm Cloud Cruiser, which tells us, for one thing, that 92 percent of organizations surveyed now use the cloud. Researchers polled 189 IT pros at Amazon Web Services (AWS) Global Summit in Chicago this past April, where they also found that 95 percent of respondents expect their cloud usage to expand over the next year.
However, organizations may wish to pause and reconsider their approach before throwing more money at cloud systems. Writer Pedro Hernandez reports:
Most organizations are suffering from a massive blind spot when it comes to budgeting for their public cloud services and making certain they are getting their money’s worth. Nearly a third of respondents said that they aren’t proactively managing cloud spend and usage, the study found. A whopping 82 percent said they encountered difficulties reconciling bills for cloud services with their finance departments.
The top challenge with the continuously growing public cloud resource is the ability to manage allocation usage and costs,’ stated the report. ‘IT and Finance continue to have difficulty working together to ascertain and allocate public cloud usage, and IT continues to struggle with technologies that will gather and track public cloud usage information.’ …
David Gehringer, principal at Dimensional Research, believes it’s time for enterprises to quit treating the cloud differently and adopt IT monitoring and cost-control measures similar to those used in their own data centers.
The report also found that top priorities for respondents included cost and reporting at 54 percent, performance management at 46 percent, and resource optimization at 45 percent. It also found that cloudy demand is driven by application development and testing, at 59 percent, and big data/ analytics at 31 percent.
The cloud is no longer a shiny new invention, but rather an integral part of most organizations. We would do well to approach its management and funding as we would other resource. The original report is available, with registration, here.
Cynthia Murrell, December 15, 2016
On the Hunt for Thesauri
December 15, 2016
How do you create a taxonomy? These curated lists do not just write themselves, although they seem to do that these days. Companies that specialize in file management and organization develop taxonomies. Usually they offer customers an out-of-the-box option that can be individualized with additional words, categories, etc. Taxonomies can be generalized lists, think of a one size fits all deal. Certain industries, however, need specialized taxonomies that include words, phrases, and other jargon particular to that field. Similar to the generalized taxonomies, there are canned industry specific taxonomies, except the more specialized the industry the less likely there is a canned list.
This is where the taxonomy lists needed to be created from scratch. Where do the taxonomy writers get the content for their lists? They turn to the tried, true resources that have aided researchers for generations: dictionaries, encyclopedias, technical manuals, and thesauri are perhaps one of the most important tools for taxonomy writers, because they include not only words and their meanings, but also synonyms and antonyms words within a field.
If you need to write a taxonomy and are at a lost, check out MultiTes. It is a Web site that includes tools and other resources to get your taxonomy job done. Multisystems built MultiTes and they:
…developed our first computer program for Thesaurus Management on PC’s in 1983, using dBase II under CPM, predecessor of the DOS operating system. Today, more than three decades later, our products are as easy to install and use. In addition, with MultiTes Online all that is needed is a web connected device with a modern web browser.
In other words, they have experience and know their taxonomies.
Whitney Grace, December 15, 2016
The Robots Are Not Taking over Libraries
December 14, 2016
I once watched a Japanese anime that featured a robot working in a library. The robot shelved, straightened, and maintained order of the books by running on a track that circumnavigated all the shelves in the building. The anime took place in a near-future Japan, when all paper documents were rendered obsolete. While we are a long way off from having robots in public libraries (budget constraints and cuts), there is a common belief that libraries are obsolete as well.
Libraries are the furthest thing from being obsolete, but robots have apparently gained enough artificial intelligence to find lost books, however. Popsci shares the story in “Robo Librarian Tracks Down Misplaced Book.” It explains a situation that librarians hate to deal with: people misplacing books on shelves instead of letting the experts put them back. Libraries rely on books being in precise order and if they are in the wrong place, they are as good as lost. Fancy libraries, like a research library at the University of Chicago, have automated the process, but it is too expensive and unrealistic to deploy. There is another option:
A*STAR roboticists have created an autonomous shelf-scanning robot called AuRoSS that can tell which books are missing or out of place. Many libraries have already begun putting RFID tags on books, but these typically must be scanned with hand-held devices. AuRoSS uses a robotic arm and RFID scanner to catalogue book locations, and uses laser-guided navigation to wheel around unfamiliar bookshelves. AuRoSS can be programmed to scan the library shelves at night and instruct librarians how to get the books back in order when they arrive in the morning.
Manual labor is still needed to put the books in order after the robot does its work at night. But what happens when someone needs help with research, finding an obscure citation, evaluating information, and even using the Internet correctly? Yes, librarians are still needed. Who else is going to interpret data, guide research, guard humanity’s knowledge?
Whitney Grace, December 14, 2016
DuckDuckGo Makes Search Enhancements by Leveraging Yahoo Partnership
December 13, 2016
The article on Duck.co titled New Features from a Stronger Yahoo Partnership relates the continuation of the relationship between DuckDuckGo and Yahoo. DuckDuckGo has gained fame for its unique privacy policy of not tracking its users, which of course flies in the face of the Google Goliath, which is built on learning about its users by monitoring their habits and improving the search engine using that data. Instead, DuckDuckGo insists on forgetting its users and letting them search without fear of it being recorded somewhere. The article conveys some of the ways that Yahoo is mingled with the David of search engines,
In addition to the existing technology we’ve been using, DuckDuckGo now has access to features you’ve been requesting for years: Date filters let you filter results from the last day, week and month. Site links help you quickly get to subsections of sites. Of course our privacy policy remains the same: we don’t track you. In addition, we’re happy to announce that Yahoo has published a privacy statement to the same effect.
Paranoid internet users and people with weird secretive fetishes alike, rejoice! DuckDuckGo will soon be vastly improved. The article does not state an exact date for this new functionality to be revealed, but it is coming soon.
Chelsea Kerwin, December 13, 2016
Ten Search Engines That Are Not Google
December 13, 2016
Business-design firm Vandelay Design shares their 10 favorite alternatives to Google Search in their blog post titled, “Alternative Search Engines for Designers and Developers.” Naturally, writer Jake Rocheleau views these resources from a designer’s point of view, but don’t let that stop you from checking out the list. The article states:
New intriguing search engines frequently pop up as a replacement to the juggernaut that is Google. But it’s tough to find alternative search engines that actually work and provide real value to your workflow. I’d like to cover a handful of alternatives that work well for designers and developers. These aren’t all web crawler search engines because I did throw in a few obscure choices for design resources too. But the sites in this list may be better replacements for Google no matter what you’re searching for. …
All 10 of these search engines are viable choices to add into your workflow, or even replace existing sites you already use. Designers are always looking for new tools and I think these sites fit the bill.
Rocheleau describes his selections and gives tips for getting the most out of each. He leads with DuckDuckGo—come for the privacy, stay for the easter eggs. StartPage also promises privacy as it pulls results from other search engines. Designers will like Instant Logo Search, for locating SVG vector logos, and Vecteezy for free vector designs. Similarly, Iconfinder and DryIcons both offer collections of free icons.
For something a little different, try The WayBack Machine at the Internet Archive, where you can comb the archives for any previously existing domain. Rocheleau suggests designers use it to research competitors and gain inspiration, but surely anyone can find interesting artifacts here.
We are reminded that one can get a lot from WolframAlpha if one bothers learning to use it. Then there is Ecosia, which uses ad revenue to plant trees across the globe. (They have planted over four million trees since the site launched in December of 2009.) The final entry is Qwant, another engine that promises privacy, but also offers individual search features for categories like news, social-media channels, and shopping. For anyone tired of Google and Bing, even non-designers, this list points the way to several good alternatives.
Cynthia Murrell, December 13, 2016
Add Free Search to the Free Tibet Slogan
December 13, 2016
China is notorious for censoring its people’s access to the Internet. I have heard and made more than one pun about the Great Firewall of China. There is search engine in China, but it will not be in Chinese, says Quartz: “How Censored Is China;s First Tibetan Language Search Engine? It Omits The Dalai Lama’s Web Site.”
Yongzin is the first Tibetan language search engine. It is supposed to act as a unified portal for all the major Tibetan language Web sites in China. There are seven million Tibetan people in China, but the two big Chinese search engines: Baidu and Sogou do not include the Tibetan language. Google is banned in China.
Yongzin rips off Google in colors and function. The Chinese government has dealt with tense issues related to the country of Tibet for decades:
The Chinese government wants the service to act as a propaganda tool too. In the future, Yongzin will provide data for the government to guide public opinion across Tibet, and monitor information in Tibetan online for “information security” purposes, Tselo, who’s in charge of Yongzin’s development, told state media (link in Chinese) at Monday’s (Aug. 22) launch event.
When people search Yongzin with Tibet related keywords, such as Dalai Lama and Tibetan tea, China’s censorship shows itself at work. Nothing related to the Dalai Lama is shown, not even his Web site, and an article about illegal publications.
China wants to position itself as guardian of the Tibetan culture, but instead they proffer a Chinese-washed version of Tibet rather than the true thing. It is another reason why the Free Tibet campaign is still important.
Whitney Grace, December 13, 2016
Algorithmic Selling on Amazon Spells Buyer Beware
December 12, 2016
The article on Science Daily titled Amazon Might Not Always Be Pitching You the Best Prices, Researchers Find unveils the stacked deck that Amazon has created for sellers. Amazon rewards sellers who use automated algorithmic pricing by more often featuring those seller’s items in the buy box, the more prominent and visible display. So what is algorithmic pricing, exactly? The article explains,
For a fee, any one of Amazon’s more than 2 million third-party sellers can easily subscribe to an automated pricing service…They then set up a pricing strategy by choosing from a menu of options like these: Find the lowest price offered and go above it (or below it) by X dollars or Y percentage, find Amazon’s own price for the item and adjust up or down relative to it, and so on. The service does the rest.
For the consumer, this means that searching on Amazon won’t necessarily produce the best value (at first click, anyway.) It may be a mere dollar difference, but it could also be a more significant price increase between $20 and $60. What is really startling is that even though less than 10% of “algo sellers,” these sellers account for close to a third of the best-selling products. If you take anything away from this article, let it be that what Amazon is showing you first might not be the best price, so always do your research!
Chelsea Kerwin, December 12, 2016
Torrent Anonymously with AlphaReign
December 12, 2016
Peer-to-peer file sharing gets a boost with AlphaReign, a new torrent sharing site that enables registered users to share files anonymously using Distributed Hash Table.
TorrentFreak in an article titled Alphareign: DHT Search Engine Takes Public Torrents Private says:
AlphaReign.se is a new site that allows users to find torrents gathered from BitTorrent’s ‘trackerless’ Distributed Hash Table, or DHT for short. While we have seen DHT search engines before, this one requires an account to gain access.
The biggest issue with most torrent sites is The Digital Millennium Copyright Act (DMCA), which prohibits the sites (if possible) and the search engines from displaying search results on the search engine result page. As content or torrent indexes on AlphaReign are accessible only to registered users, seeders and leechers are free to share files without risking themselves.
Though most files shared through torrents are copyrighted materials like movies, music, software and books, torrents are also used by people who want to share large files without being spied upon.
AlphaReign also manages to address a persistent issue faced by torrent sites:
AlphaReign with new software allows users to search the DHT network on their own devices, with help from peers. Such a system would remain online, even if the website itself goes down.
In the past, popular torrent search engines like YTS, KickAssTorrents, The Pirate Bay, Torrentz among many others have been shut down owing to pressure from law enforcement agencies. However, if AlphaReign manages to do what it claims to, torrent users are going to be the delighted.
Vishal Ingole, December 12, 2016
At Last an Academic Search, but How Much Does It Cost?
December 9, 2016
I love Google. You love Google. Everyone loves Google so much that it has become a verb in practically every language. Google does present many problems, however, especially in the inclusion of paid ads in search results and Google searches are not academically credible. Researchers love the ease of use with Google, but there a search engine does not exist that returns results that answer a simple question based on a few keywords, NLP, and citations (those are extremely important).
It is possible that a search engine designed for academia could exist, especially if it can be subject specific and allows full-text access to all results. The biggest problem and barrier in the way of a complete academic search engine is that scholarly research is protected by copyright and most research is behind pay walls belonging to academic publishers, like Elsevier.
Elsevier is a notorious academic publisher because it provides great publication and it is also expensive to subscribe to it digitally. The Mendeley Blog shares that Elsevier has answered the academic search engine cry: “Introducing Elsevier DataSearch.” The Elsevier DataSearch promises to search through reputable information repositories and help researchers accelerate their work.
DataSearch is still in the infant stage and there is an open call for beta testers:
DataSearch offers a new and innovative approach. Most search engines don’t actively involve their users in making them better; we invite you, the user, to join our User Panel and advise how we can improve the results. We are looking for users in a variety of fields, no technical expertise is required (though welcomed). In order to join us, visit https://datasearch.elsevier.com and click on the button marked ‘Join Our User Panel’.”
This is the right step forward for any academic publisher! There is one thing I am worried about and that is: how much is the DataSearch engine going to cost users? I respect copyright and the need to make a profit, but I wish there was one all-encompassing academic database that was free or had a low-cost subscription plan.
Whitney Grace, December 9, 2016
-
- Subscribe to Beyond Search
Feature archive
News archive
-
