How-To Overview of Building a Data Platform to Handle Real-Time Datasets
March 11, 2016
The article on Insight Data Engineering titled Building a Streaming Search Platform offers a glimpse into the Fellows Program wherein grad students and software engineers alike build data platforms and learn cutting-edge open source technologies. The article delves into the components of the platform, which enables close to real-time search of a streaming text data source, with Twitter as an example. It also explores the usefulness of such a platform,
On average, Twitter users worldwide generate about 6,000 tweets per second. Obviously, there is much interest in extracting real-time signal from this rich but noisy stream of data. More generally, there are many open and interesting problems in using high-velocity streaming text sources to track real-time events. … Such a platform can have many applications far beyond monitoring Twitter…All code for the platform I describe here can be found on my github repository Straw.”
Ryan Walker, a Casetext Data Engineer, describes how these products might deliver major results in the hands of a skilled developer. He uses the example of a speech to text monitor being able to transcribe radio or TV feeds and send the transcriptions to the platform. The platform would then seek key phrases and even be set up to respond with real-time event management. There are many industries that will find this capability very intriguing due to their dependence on real-time information processing, including finance and marketing.
Chelsea Kerwin, March 11, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
A Hefty Guide to Setting up SharePoint 2013 Enterprise Search Center
March 8, 2016
The how-to guide titled Customizing SharePoint 2013 Search Center on Code Project provides a lengthy, detailed explanation (with pictures) of the new features of SharePoint 2013, an integration of the 2010 version and Microsoft FAST search. The article offers insights into certain concepts of the program such as crawled properties and managed properties before introducing step-by-step navigation for customizing the result page and Display template, as well as other areas of Sharepoint. The article includes such tips as this,
“Query rules allow you to modify the users keyword search based on a condition. Let’s say when the user types Developer, we want to retrieve only the books which have BookCategory as Developer and if they type ‘IT Pro’, we only want to retrieve the Administrator related books.”
Nine steps later, you have a neat little result block with the matching items. The article outlines similar processes for Customizing the Search Center, Modifying the Search Center, Adding the Results Page to the Navigation, and Creating the Result Source. This leads us to ask, Shouldn’t this be easier by now? Customizing a program so that it looks and acts the way we expect seems like pretty basic setup, so why does it take 100+ steps to tailor SharePoint 2013?
Chelsea Kerwin, March 8, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Search Dually Conceals and Retrieves for an Audience
March 8, 2016
There are many ways to trace a digital footprint, but Google is expanding European users’ ability to cover their tracks. An article entitled, In Europe, Google will now remove ‘right to be forgotten’ search results from all its sites, from The Verge tells us the story. Basically, European users who request links to be removed protect those links from being crawled by Google.fr in addition to all their other homepages. The write-up explains,
“So, for instance, if someone in France had previously requested that a link be hidden from search results, Google would just remove it from its European homepages, including google.fr. But a savvy searcher could have just used google.com to dig up all those hidden results. Now, however, the company will scrub its US homepage results, too, but only for European users. The company didn’t provide specifics on how it’ll detect that a user is in Europe, but it’s likely going off IP addresses, so in theory, someone could use a VPN to subvert those results.”
As the article mentions, European privacy regulators are happy about this but would still prefer contested links not appear, even if the searcher is in the U.S. or elsewhere. Between the existence of the Dark Web and the “right to be forgotten” protections, more and more links are hidden making search increasingly difficult.
Megan Feil, March 8, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
A Place to Express Yourself on the Dark Web
March 7, 2016
For evidence that the dark web is not all about drugs and cybercrime, check out this article at Motherboard: “The Dark Web Now Has a Literary Journal.” As it turns out, anonymity is also good for people who wish to freely explore their creativity and private thoughts.
The new journal, the Torist, was just launched by a professor at the University of Utah, Robert W. Ghel, and a person known simply as GMH. Inspired by the free discussions on their dark-web-based social network, Galaxy, they have seized their chance to create something unexpected. The journal’s preface asks:
“If a magazine publishes itself via a Tor hidden service, what does the creative output look like? How might it contrast itself with its clearweb counterparts? Who indeed will gravitate towards a dark web literary magazine?”
So, why is one of the Torist’s creators anonymous while the other is putting himself out there? Writer Joseph Cox tells us:
Gehl, after being pitched the idea of The Torist by GMH, decided to strip away his pseudonym, and work on the project under his own name. “I thought about that for a while,” Gehl said. “I thought that because GMH is anonymous/pseudonymous, and he’s running the servers, I could be a sort of ‘clear’ liason.”
So while Gehl used his name, and added legitimacy to the project in that way, GMH could continue to work with the freedom the anonymity awards. “I guess it’s easier to explore ideas and not worry as much how it turns out,” said GMH, who described himself as someone with a past studying the humanities, and playing with technology in his spare time.
Gehl and GMH say part of their reasoning behind the journal is to show people that anonymity and encryption can be forces for good. Privacy furthers discussion of controversial, personal, and difficult topics and, according to GMH, should be the default setting for all communications, especially online.
Submissions are currently being accepted, so go ahead and submit that poem or essay if you have something to get off your chest, anonymously. If you dare to venture into the dark web, that is.
Cynthia Murrell, March 7, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
The FBI Uses Its Hacking Powers for Good
March 4, 2016
In a victory for basic human decency, Engadget informs us, the “FBI Hacked the Dark Web to Bust 1,500 Pedophiles.” Citing an article at Vice Motherboard, writer Jessica Conditt describes how the feds identified their suspects through a site called (brace yourself) “Playpen,” which was launched in August 2014. We learn:
Motherboard broke down the FBI’s hacking process as follows: The bureau seized the server running Playpen in February 2015, but didn’t shut it down immediately. Instead, the FBI took “unprecedented” measures and ran the site via its own servers from February 20th to March 4th, at the same time deploying a hacking tool known internally as a network investigative technique. The NIT identified at least 1,300 IP addresses belonging to visitors of the site.
“Basically, if you visited the homepage and started to sign up for a membership, or started to log in, the warrant authorized deployment of the NIT,” a public defender for one of the accused told Motherboard. He said he expected at least 1,500 court cases to stem from this one investigation, and called the operation an “extraordinary expansion of government surveillance and its use of illegal search methods on a massive scale,” Motherboard reported.
Check out this article at Wired to learn more about the “network investigative technique” (NIT). This is more evidence that, if motivated, the FBI is perfectly capable of leveraging the Dark Web to its advantage. Good to know.
Cynthia Murrell, March 4, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Stolen Online Account Info Now More Valuable than Stolen Credit Card Details
March 2, 2016
You should be aware that criminals are now less interested in your credit cards and other “personally identifiable information” and more keen on exploiting your online accounts. As security firm Tripwire informs us in their State of Security blog, “Stolen Uber, PayPal Accounts More Coveted than Credit Cards on the Dark Web.” Writer Maritza Santillan explains:
“The price of these stolen identifiers on the underground marketplace, or ‘the Dark Web,’ shows the value of credit cards has declined in the last year, according to security firm Trend Micro. Last week, stolen Uber account information could be found on underground marketplaces for an average of $3.78 per account, while personally identifiable information, such as Social Security Numbers or dates of birth, ranged from $1 to $3.30 on average – down from $4 per record in 2014, reported CNBC. Furthermore, PayPal accounts – with a guaranteed balance of $500 –were found to have an average selling price of $6.43. Facebook logins sold for an average of $3.02, while Netflix credentials sold for about 76 cents. By contrast, U.S.-issued credit card information, which is sold in bundles, was listed for no more than 22 cents each, said CNBC.”
The article goes on to describe a few ways criminals can leverage these accounts, like booking Uber “ghost rides,” or assembling personal details for a very thorough identity theft. Pros say the trend means service providers to pay closer attention to usage patterns, and to beef up their authentication processes. Specifically, says Forrester’s Andras Cser, it is time to move beyond passwords; instead, he proposes, companies should look for changes in biometric data, like phone position and finger pressure, which would be communicated back to them by our mobile devices. So we’re about to be even more closely monitored by the companies we give our money to. All for our own good, of course.
Cynthia Murrell, March 2, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
IBM Continued to Brag About Watson, with Decreasing Transparency
February 29, 2016
A totally objective article sponsored by IBM on Your Story is titled How Cognitive Systems Like IBM Watson Are Changing the Way We Solve Problems. The article basically functions to promote all of the cognitive computing capabilities that most of us are already keenly aware that Watson possesses, and to raise awareness for the Hackathon event taking place in Bengaluru, India. The “article” endorses the event,
“Participants will have an unprecedented opportunity to collaborate, co-create and exchange ideas with one another and the world’s most forward-thinking cognitive experts. This half-day event will focus on sharing real-world applications of cognitive technologies, and allow attendees access to the next wave of innovations and applications through an interactive experience. The program will also include panel discussions and fireside chats between senior IBM executives and businesses that are already working with Watson.”
Since 2015, the “Watson for Oncology” program has involved Manipal Hospitals in Bengaluru, India. The program is the result of a partnership between IBM and Memorial Sloan Kettering Cancer Center in New York. Watson has now consumed almost 15 million pages of medical content from textbooks and journals in the hopes of providing rapid-fire support to hospital staffers when it comes to patient records and diagnosis. Perhaps if IBM put all of their efforts into Watson’s projects instead of creating inane web content to promote him as some sort of missionary, he could have already cured cancer. Or not.
Chelsea Kerwin, February 29, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
More Hacked US Voter Data Appears on the Dark Web
February 25, 2016
From HackRead comes a piece called More US Voters Data Circulating on the Dark Net, which points to the lack of protection surrounding data on US voters. This data was leaked on the site The Hell on Dark Web. No reports yet suggest how this data was hacked. While no social security numbers or highly sensitive information was released, records include name, date of birth, voter registration dates, voting records, political affiliation and address. Continuing the explanation of implications, the article’s author writes,
“However, it provides any professional hacker substantial information to initiate and plan a phishing attack in the next election which takes place in the US. Recent discoveries, news and speculations have exposed the role of nation-state actors and cyber criminals in planning, instigating and initiating hacking attacks aimed at maligning the upcoming US elections. While social media has emerged as one of the leading platforms adopted by politicians when they wish to spread a certain message or image, cyber criminals and non-state actors are also utilizing the online platform to plan and initiate their hacking attacks on the US election.”
As the article reminds us, this is the not first instance of voter records leaking. Such leaks call into question how this keeps happening and makes us wonder about any preventative measures. The last thing needed surrounding public perception of voting is that it puts one at risk for cyber attacks. Aren’t there already enough barriers in place to keep individuals from voting?
Megan Feil, February 25, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
monograph
CyberSpark Billed as New Cybersecurity Capital for Israel
February 24, 2016
Beersheba, a city in Israel with a population of about 200,000 has become the site of several connected academic and technological influences, led by government and industry, which may position it to be the cyber capital of the country. The article Israel’s Cyber Sector Blooms in the Desert article from Security Week covers Beersheba’s industrial park, CyberSpark. A project leader for the Israeli National Cyber Bureau is quoted explaining how this area is primed to become a leader in cyber security. The report describes CyberSpark’s projected growth,
“Two more complexes comprising 27 buildings are to be added, and the municipality expects the population to grow by 100,000 in the next 10 years. About 30,000 soldiers, including 7,000 career officers, will move in the coming years to bases and a technology campus to be built on 100 hectares (250 acres) near CyberSpark and around Beersheba. As a lure from the bustle of cosmopolitan Tel Aviv, the government plans a bonus of $18,000 for single officers and $50,000 for families who spend at least five years in Beersheba.”
More often than not, we hear about cybercriminals taking the initiative while law enforcement, intelligence and others attempt to catch up. While the article frames CyberSpark as a case of proactive collaboration with necessary partners for the sake of forwarding the cyber security industry and protecting citizens, we are not sure it can be called proactive. Let’s not forget, as the article mentions, Israel may be the most heavily targeted country in the world with reports suggesting as many as a thousand web attacks per minute.
Megan Feil, February 24, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
No Evidence That Terrorists Are Using Bitcoin
February 23, 2016
If you were concerned virtual currencies like Bitcoin are making things easier for Islamic State (aka IS, ISIS, ISIL, or Daesh), you can rest easy, at least for now. The International Business Times reports, “Isis: Bitcoin Not Used by Daesh.” That is the conclusion reached by a Europol investigation performed after last November’s attacks in Paris. Though some had suggested the terrorists were being funded with cyber money, investigators found no evidence of it.
On the other hand, the organization’s communication networks are thriving online through the Dark Web and a variety of apps. Writer Alistair Charlton tells us:
Better known by European law enforcement is how terrorists like IS use social media to communicate. The report says: “The internet and social media are used for communication and the acquisition of goods (weapons, fake IDs) and services, made relatively safe for terrorists with the availability of secure and inherently encrypted appliances, such as WhatsApp, Skype and Viber. In Facebook, VKA and Twitter they join closed and hidden groups that can be accessed by invitation only, and use coded language.”
se of Tor, the anonymising browser used to access the dark web where sites are hidden from search engines like Google, is also acknowledged by Europol. “The use of encryption and anonymising tools prevent conventional observation by security authorities. There is evidence of a level of technical knowledge available to religiously inspired terrorist groups, allowing them to make their use of the internet and social media invisible to intelligence and law enforcement agencies.”
Of course, like any valuable technology, anonymizing apps can be used for weal or woe; they benefit marginalized peoples trying to make their voices heard as much as they do terrorists. Besides, there is no going back to a disconnected world now. My question is whether terrorists have taken the suggestion, and are now working on a Bitcoin initiative. I suppose we will see, eventually.
Cynthia Murrell, February 23, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
-
- Subscribe to Beyond Search
Feature archive
News archive
-
