Full Text Search Gets Explained
October 6, 2015
Full text search is a one of the primary functions of most search platform. If a search platform cannot get full text search right, then it is useless and should be tossed in the recycle bin. Full text search is such a basic function these days that most people do not know how to explain what it is. So what is full text?
According to the Xojo article, “Full Text Search With SQLite” provides a thorough definition:
“What is full text searching? It is a fast way to look for specific words in text columns of a database table. Without full text searching, you would typically search a text column using the LIKE command. For example, you might use this command to find all books that have “cat” in the description…But this select actually finds row that has the letters “cat” in it, even if it is in another word, such as “cater”. Also, using LIKE does not make use of any indexing on the table. The table has to be scanned row by row to see if it contains the value, which can be slow for large tables.”
After the definition, the article turns into advertising piece for SQLite and how it improves the quality of full text search. It offers some more basic explanation, which are not understood by someone unless they have a coding background. It is a very brief with some detailed information, but could explain more about what SQLite is and how it improves full text search.
Whitney Grace, October 6, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Learn One of These Programming Languages, Crunch Big Data. Easy, Right?
October 3, 2015
I read a listicle called “Ten Top Languages for Crunching Big Data.” The list is interesting but the underlying assumption about the languages and “crunching” Big Data was remarkable.
The core of the write up is a list of 10 programming languages which make it possible (maybe semi easy) to “generate insights.” The list has some old familiar programming languages; for example, SQL or structured query language. There’s the graduate student in psychology fave SAS. Some might argue that SPSS Clem is the way to chop Big Data down to size. There is a toolkit in the list. Remember Matlab, which for a “student” is only $49. For the sportier crowd, I would add Mathematica to the list, but I don’t want to melt the listicle.
Also on the list are Python and R. Both get quite a bit of love from some interesting cyber OSINT outfits.
For fans of Java, the list points to Scala. The open source fan can use HiveQL, Julia, or Pig Latin.
The listicle includes a tip of the hat to Alphabet Google. According to the write up:
Go has been developed by Google and released under an open source license. Its syntax is based on C, meaning many programmers will be familiar with it, which has aided its adoption. Although not specifically designed for statistical computing, its speed and familiarity, along with the fact it can call routines written in other languages (such as Python) to handle functions it can’t cope with itself, means it is growing in popularity for data programming.
Yep, a goodie from the GOOG spells Big Data magic. For how long? Well, I don’t know.
However, the assumption from which the listicle hangs is that a programming language allows Big Data to be crunched.
Whoa, Nellie.
There may be a couple of “trivial” intermediary steps required. Let me mention one. The Big Data cruncher has to code up something to get useful outputs. Now that “code up” step may require some other bothersome tasks; for example, dealing with messy data to ensure that the garbage in, garbage out problem does not arise. The mathematically inclined may suggest that the coded up “script” actually work within available computer time and memory resources. Wow, that might make a script to crunch Big Data either not work or output results which are dead wrong. What if the script implements algorithmic bias?
Whoa, whoa, Nellie.
I know that programming languages are important. But some other tasks deserve attention in my experience.
Stephen E Arnold, October 3, 2015
Legacy Servers: Upgrade Excitement
October 2, 2015
Enterprise management systems (ECM) were supposed to provide an end all solution for storing and organizing digital data. Data needs to be stored for several purposes: taxes, historical record, research, and audits. Government agencies deployed ECM solutions to manage their huge data loads, but the old information silos are not performing up to modern standards. GCN discusses government agencies face upgrading their systems in “Migrating Your Legacy ECM Solution.”
When ECMs first came online, information was stored in silos programmed to support even older legacy solutions with niche applications. The repositories are so convoluted that users cannot find any information and do not even mention upgrading the beasts:
“Aging ECM systems are incapable of fitting into the new world of consumer-friendly software that both employees and citizens expect. Yet, modernizing legacy systems raises issues of security, cost, governance and complexity of business rules — all obstacles to a smooth transition. Further, legacy systems simply cannot keep up with the demands of today’s dynamic workforce.”
Two solutions present themselves: data can be moved from an old legacy system to a new one or simply moving the content from the silo. The barriers are cost and time, but the users will reap the benefits of upgrades, especially connectivity, cloud, mobile, and social features. There is the possibility of leaving the content in place using interoperability standards or cloud-based management to make the data searchable and accessible.
The biggest problem is actually convincing people to upgrade. Why fix what is not broken? Then there is the justification of using taxpayers’ money for the upgrade when the money can be used elsewhere. Round and round the argument goes.
Whitney Grace, October 2, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Share by from StoryCloud Reigns in Control of Online Content by Content Creator
October 1, 2015
The article titled Permission Based Publishing Lets Users Keep Control of Content on Beta News describes an innovative approach to allowing online content publishers a tighter grip on how their content is disbursed. StoryCloud, the permission-based publishing provider of Share By, explains the myriad potential uses for their platform, from teachers measuring a class’s understanding of the homework assignment to a musical group sharing a song with specific subscribers. The article explains how the platform functions,
“By using permission-based technology that is tightly integrated with social networking, analytics and ecommerce, Share By allows content providers to easily determine who sees their content, when, and from what location. Other permissions include duration, view or download limits and scheduling time periods for sharing and the devices that are permitted. Once content providers upload content to StoryCloud and determine permissions, they receive a unique URL which can be shared with any online audience, including Facebook and Twitter.”
Beyond the privacy and control aspects of Share By, there is also the ability to graphically analyze the content they have released online. For most individuals, this might just mean checking in on who really spent time consuming the content, but for companies it means monetization. They can charge per viewing and offer subscriptions without worrying about people getting the content without consent.
Chelsea Kerwin, October 01, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
National Geographic Sells Out
September 30, 2015
The National Geographic Society is one of the most respected institutes in regards to science and journalism related to nature. For 127 years, National Geographic managed itself as a non-profit organization. Buzzfeed reports that 21st Century Fox purchased National Geographic in the article, “Rupert Murdoch Is Buying National Geographic.” Before you start getting upset that National Geographic has “sold out” in the same manner that Sesame Street has a new partnership with HBO, be aware that 21st Century Fox already owned and operated a joint-venture partnership with the company.
The bulk of National Geographic’s properties are being turned over to 21st Century Fox, who will manage them and allow the National Geographic Society to focus on:
“The National Geographic Society said the deal will let the foundation invest more money in sponsoring explorers and scientists. ‘The value generated by this transaction, including the consistent and attractive revenue stream that National Geographic Partners will deliver, ensures that we will have greater resources for this work, which includes our grant making programs,’ said CEO Gary Knell, in a statement.”
While National Geographic is still popular, it faces stiff competition from other news outlets that generate similar if not more content. National Geographic wants to have better, modern storytelling “so that we may all know more of the world upon which we live.”
Hopefully this will free up more monies for scientific research, endeavors to protect endangered species, educational programs, and better ways to educate people on the natural world.
Whitney Grace, September 30, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
The Many Applications of Predictive Analytics
September 29, 2015
The article on Computer World titled Technology that Predicts Your Next Security Fail confers the current explosion in predictive analytics, the application of past occurrences to predict future occurrences. The article cites the example of the Kentucky Department of Revenue (DOR), which used predictive analytics to catch fraud. By providing SAS with six years of data the DOR received a batch of new insights into fraud indicators such as similar filings from the same IP address. The article imparts words of wisdom from SANS Institute instructor Phil Hagen,
“Even the most sophisticated predictive analytics software requires human talent, though. For instance, once the Kentucky DOR tools (either the existing checklist or the SAS tool) suspect fraud, the tax return is forwarded to a human examiner for review. “Predictive analytics is only as good as the forethought you put into it and the questions you ask of it,” Hagen warns…. Also It’s imperative that data scientists, not security teams, drive the predictive analytics project.”
In addition to helping the IRS avoid major fails like the 2013 fraudulent refunds totaling $5.8 billion, predictive analytics has other applications. Perhaps most interesting is its use protecting human assets in regions where kidnappings are common by detecting unrest and alerting organizations to lock up their doors. But it is hard to see limitations for technology that so accurately reads the future.
Chelsea Kerwin, September 29, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
NTENT Has a New CEO
September 29, 2015
NTENT is a leading natural language processing and semantic search company, that owns the Convera technology, and according to Business Wire Dan Stickel was hired as the new CEO, says “NTENT Appoints Dan Stickel As New CEO.” NTENT is focused on expanding the company with AltaVista and Google. Using Stickel’s experience, NTENT has big plans and is sure that Stickel will lead the company to success.
“CEO, Stickel’s first objective will be to prioritize NTENT’s planned expansion along geographic, market and technology dimensions. ‘After spending significant time with NTENT’s Board, management team and front-line employees, I’m excited by the company’s opportunities and by the foundation that’s already been laid in both traditional web and newer mobile capabilities. NTENT has clearly built some world-class technology, and is now scaling that out with customers and partners.’”
In his past positions as CEO at Metaforic and Webtrends s well as head of the enterprise business at AltaVista and software business at Macrovision, Stickel has transitioned companies to become the leaders in their respective industries.
The demand for natural language processing software and incorporating it into semantic search is one of the biggest IT trends at the moment. The field is demanding innovation and NTENT believes Stickel will guide them.
Whitney Grace, September 29, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
A Modest Proposal: Universal Internet Access and a Chief Digital Officer in Every Organization
September 28, 2015
Facebook supports universal Internet access. Support is not enough from one or two outfits. Facebook wants the United Nations to make universal Internet access a priority.
Navigate to “Mark Zuckerberg Addresses the UN, Declaring Universal Internet Access a Global Priority.” I wonder if peace keeping, food, education, and other priorities of the United Nation will be down prioritized or de-prioritized. If I were hoping for UN food assistance, I would definitely want to be able to check my Facebook. Maslow’s hierarchy of needs is obviously wrong. At the top, Facebook.
I also noted this article, “IT Still Doesn’t Understand Its Role in Society.” The author is a self described “IT leader.” The point is, I think:
It struck me, when I opened this September’s edition, just how much things have moved on. This month gives much more space to the changing role of IT and its part in business leadership. That of course lies at the heart of the debate about CIOs and CDOs – the former seeming inextricably constrained by operational IT matters, whether insourced or outsourced, and the latter filling the vacuum created by misalignment between IT activity and business priority. Everyone seems agreed that the role of a CDO is not about the technology. It is about people and process. But it cannot operate without a fundamental understanding about the opportunity that technology offers, and therefore CDOs must work closely with IT professionals.
The word choice is well matched with the imperative to make technology become the source for wild and crazy assertions. I like the use of the acronyms CIO and CDO. I am not sure what a CDO is, but that is not important. The precision of insourced and outsourced, where the outsourcing thing fills “the vacuum created by misalignment between IT activity and business priority.”
Okay, the folks running the business are not exactly sure what’s up with IT. If a senior manager tuned in to Facebook’s remarks about universal Internet access, there might be some furrowed brows.
What’s the fix?
The answer is a new job position at companies. The CDO. (My hunch is that this acronym means “chief digital officer.”) When revenues are stressed, most senior managers will gladly add to the organization’s headcount to get a CDO on the team.
I highlighted this passage:
So we need clever technical specialists, but we also need IT professionals who can bridge the gap between technology opportunity and the benefits that technology can bring society. That is why the goal of BCS – the Chartered Institute for IT – is “to make IT good for society”. That should be the role of IT professionals. This means that IT professionals need to understand the impact of technology, positive and negative, in the way systems and IT tools are designed. It means IT has a significant part to play in the debate about privacy and trust emerging from IT changes. It means we have a part to play in the way systems are designed to benefit society, not just to make profit. And it means IT is a creative, human discipline, not just a scientific and engineering profession.
Okay. But what about the Facebook suggestion to make Internet access universal. Will checking the Facebook obviate hunger and disease? Will information technology move beyond profit to benefiting society.
What’s at stake here? My hunch the focus for Facebook thing and the self appointed expert’s CDO recommendation has more to do with money and boosting the notion of the importance of technology in the modern world.
Was Maslow incorrect? Is Facebook connectivity more important than food? Are companies in need of more headcount in order to make headway in the datasphere?
Nope. Why not sit back and let the Alphabet Google thing do the job? Some big thinkers want governments to be more like Google. No Facebook. No information technology intermediaries. Why search for information when a commercial enterprise and self appointed experts know best what folks like me want?
Stephen E Arnold, September 28, 2015
Accidental and On-Purpose Insider Threats in Federal Agencies Still Raging
September 28, 2015
The article on Eweek titled Insider Threats a Major Security Issue for Federal Agencies looks at the recent results of a MeriTalk survey investigating federal response to insider threats through interviewing federal IT managers. The results are shocking, with almost 30% of agencies acknowledging data lost to an insider threat in the last year and half of respondents claiming that unauthorized personnel commonly fail to observe protocols. Even worse, most agencies have no tracking in place to recognize what a staffer may have seen or shared, making them virtually incapable of following up on risky behavior in their employees. The article says,
“The most startling finding from the survey is the fact that 45 percent of agencies say they’ve been a target of an attack – malicious or unintentional – yet 50 percent still say employees do not follow all the protocols in place,” Steve O’Keeffe, founder of MeriTalk…”There is also a lack of agreement on the best solution. Frequent, hands-on employee training is the key to preventing these incidents, as well as accountability. However, we are all human and people make mistakes.”
O’Keefe recommends the immediate and comprehensive adoption of better encryption and two-factor authentication to address the issue. But perhaps equally important is continuously updated training, and ongoing training, to avoid the common accidental insider threats.
Chelsea Kerwin, September 28, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Help Wanted: Chief Marketing Technology Officer
September 28, 2015
A new, indispensable position for companies is the chief technology officer or the chief information officer. Their primary responsibilities are to manage the IT department, implement new ways to manage information, and/or develop software as needed. There is a new position that companies will be creating in the future and the title is chief marketing technology officer, says Live Mint in “Make Way CIOS, CMOS: Here Comes The CMTO.”
Formerly the marketing and IT departments never mixed, except for the occasional social media collaboration. Marketers are increasing their reliance on technology to understand their customers and it goes far beyond social media. Marketers need to be aware of the growing trends in mobile shopping and search, digital analytics, gamification, online communities, and the power of user-generated content.
“The CMO’s role will graduate to CMTO, a marketer with considerable knowledge of technology. The CMTO, according to Nasscom, will not only conceptualize but also build solutions and lay down the technical and commercial specifications while working alongside the IT team on vendor selection.”
It is not enough to know how to market a product or promote an organization. Marketers need to be able to engage with technology and understand how to implement to attract modern customers and increase sales. In other words, evolving the current marketing position with a new buzzword.
Whitney Grace, September 28, 2015
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
-
- Subscribe to Beyond Search
Feature archive
News archive
-
