Software: Its Dark Side and Search
September 10, 2013
I read “The Dark Side of Software Development That No One Talks About.” Well, excuuuuse me — as Steve Martin the comedian once said — my goslings and I have yammered about “bad” developers and the rise of the nerd kings for years.
A couple of the goslings are/were nerd kings until they discovered one or more of these areas of interests: People of the opposite sex or just interspecies interaction in general, money, power, clients who cannot log in to an email account, automobile racing, weapons. I think I have covered some of the interests. If I omitted some triggers, excuuuuse me.
Well, snacks are available at many high-tech firms. How about an Odwalla and some trail mix? A happy quack to Family Wings for the image: http://familywings.org/from-star-wars-to-heart-wars/come-to-the-dark-side-cookies/
The write up points out, “Software developers are jerks.” The write up explains the jerkiness reasonably well. Among the characteristics mentioned are:
Not everyone has your best interests at heart
Vileness (great word, by the way)
Bloated egos (I thought every CEO needed a jet plain and a trophy nerd)
The “cure”, quite surprisingly, is almost Jungian:
there needs to be more of us—especially those of us who’ve been in the field for awhile—who tell people why they can do it and how easy it really is.
My view is less psychy.
First, since most folks can click a mouse, this expertise immediately converts to technical expertise. Consequently the likelihood that a development project will move forward as if on Teflon is zero. Everyone has an informed opinion, and the folks with “power” dictate what happens. Just look at interfaces which are unusable or odd ball iPad apps which are almost unusable without mindless tapping for evidence.
Second, the split between those who can code and those who cannot is another type of one percent societal split. Talking about a method is different from implementing the method in software. Since it is more fun to talk than work, the process of creating an innovation effectively stops. Does the world need another wearable device like a smart watch? I suppose so, but what are the fancy watchmakers doing? Those throwbacks keep making watches some of which cost more than an Autonomy, Endeca, or SharePoint Search implementation. Nissan is doing a smart watch. Great idea.
Third, the need for software solutions is rising rapidly. I just heard a podcast which argued that Microsoft needs to focus only on apps. That’s a great idea. My wife has an iPhone and I can’t figure out which colorful icon does what. We need more apps. I think that top developers can meet this need whether I understand the solution or not.
Stepping back, let me ask a question. Does any of the enterprise software work without legions of technical experts applying bandages 24×7? Of the hundreds of thousands of apps, how many are in for the long haul? Of the thousands of open source programs, how many will survive the loss of a couple of key committers?
Software and the type of experts described in the article have helped make modern life brittle. With expertise concentrated and the constructs on the edge of functionality, will the software infrastructure support an increasingly complex future?
And what about search? Findability is not making much progress. I am not sure the developers are to blame. Computational boundaries, the complexity of language, and the “needs” of the busy worker contribute to systems which are increasingly alike. The enterprise search systems are equally useful and equally disappointing. Good business for consultants and technical wellness staff comes from the present situation.
Stephen E Arnold, September 10, 2013
Enterprise Search: Finding Flounders Floundering
September 9, 2013
A founder is a flat fish. In today’s whiz kid world, “flounder” does not make one drool for a fish stew. “Flounder” means to the Free Dictionary, “to move or act clumsily and in confusion.” I made the connection to search as a result of a seemingly innocuous discussion on LinkedIn about improving search.
A flounder not yet out of water.
I am not sure about the rules for linking to LinkedIn content. I have to watch my Ps and Qs because two of the goslings and I learned on Friday, September 6, 2013, that some of the queries I launched from my research computer were not processed by Slideshare. Was this a glitch or some intentional action? I don’t know. To be on the safe side, I will not link to the thread called “How to manage queries having no relevant answers but still matching some terms.” If you are a LinkedIn customer, you can log in and locate the discussion using the LinkedIn “finding” system. How well with that work out for you? Well, that’s another search topic.
To recap the thread, a LinkedIn customer is responsible for an Intranet search system. When its users run a query, the system produces a results list which do not answer some users’ questions. There is term matching, but the content is not on point. I no longer like to beat the drum for precision and recall. We are now in the era of good enough search. Few take the time to create a vetted content inventory. When the search system is rolled out, no one really knows what’s “in” the index. The point that a query contains terms which match some content but makes users grouse is not new.
Caught by an unhappy user who happens to be the CFO figuring out why so much money was spent for a search system that did not work.
The fix, of course, is like trying to refuel an old fashioned propeller driven aircraft with a somewhat more modern jet powered tanker. The job is going to be tricky and may end with some excitement. Jets and prop driven aircraft like enterprise search and quick mixes might not be a happy combination like peanut butter and jelly.
In 2004, then Googler Dave Girouard said in eCommerce Times:
“The funny part is it’s easier to find box scores from the 1957 World Series than it is to find last quarter’s sales presentation in the enterprise. While Web search has gotten really good, enterprise search has stagnated, and that’s why we really believe it’s a problem that needs to be solved and that Google has a unique set of capabilities to solve it.”
Well, Mr. Girouard has moved on and Google is advertising on LinkedIn for yet another wizard to work on enterprise search. If Google cannot knock the ball out of the park, who can? Is HP Autonomy the go-to system? What about a low-cost option like dtSearch? Why not download Elasticsearch, Constellio, of one of the other open source solutions? Maybe a company should embrace a predictive solution from Agilex or Palantir?
Once Again SharePoint Search Needs Augmentation
September 9, 2013
Collaborative content management programs are a solution to share information and connect users across multiple devices. An important function of the platform is search. SharePoint is a favorite out-of-the-box content software of many organizations, but like any Microsoft product it needs to be tweaked to make it perform at its best. Search is one of its functions that needs a lot of tweaking and Mentor Mate has a SharePoint Tips blog to help administrators improve search. One of its latest posts, “Pitfalls Configuring A SharePoint 2010 Search Service Application” tries to answer how to set up restricted access:
“In the following, first, blog we talk about restricted access for Search Service Application on a clean SharePoint farm even for a farm administrator account. So, we have a clean SharePoint 2010 install, we try to access the Search Service Application (under Application Management ? Manage Service Applications) and… we get an unexpected access denied. “
The author, Kiril Illiev, follows through with how he resolved the situation and the steps he took to get there. The post reads as an instructional article on how to fix SharePoint’s search application, complete with screenshots. SharePoint search can be enhanced with tips like this.
Whitney Grace, September 09, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
A New Search Tool Kicks Its Heels
September 6, 2013
Apache Lucene/Solr is already the most popular open source search software, but there are other options available including GigaBlast and Elasticsearch. Another search project wishes to join the ranks of open source search. Visit the main Web site and read the synopsis entitled, “Introducing Searchkick” to gain a basic understanding of the new technology. Searchkick is touted as helping developers “bootstrap their search.” It is an out-of-the-box solution that can stem, use special characters, ignore extra white space, handle misspellings, and is smart enough for custom synonyms.
Interestingly the new search tool is an intelligent search function:
“Searchkick uses conversion data to learn what your users are looking for. If a user searches for “ice cream” and adds Ben & Jerry’s Chunky Monkey to the cart (the conversion metric), that item gets a little more weight for similar searches. Results get better as more people search.”
Searchkick can also reindex without any downtime, uses SQL, and can be deployed to any language. The search software is available via Github for download and development. We see a lot of small search projects starting off using open source software. The main problem is that they are often abandoned. Searchkick appears to be off to a better start than others. Let us see if the spark sticks.
Whitney Grace, September 06, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
Power Search for Open Source Developers
September 6, 2013
Open source is cutting across the world as solution revolution. It is making technology cheaper and more widely available. It could have positive far reaching consequences in education and aerospace technology, but all revolutions need a little help getting off the ground.
“Open source projects need all the help they can get. If not with funding, then with volunteers contributing to open source programming and free tools they can brandish. Search engines tuned with algorithms to find source code for programming projects are among the tools for the kit bag. While reusing code is a much debated topic in higher circles, they could be of help to beginner programmers and those trying to work their way through a coding logjam by cross-referencing their code.”
Makeuseof.com points to the article, “Open Source Matters: 6 Source Code Search Engines You Can Use For Programming Projects” that lists code search engines to help developers out in their projects. Ohloh Code is one of the largest code search engines with over ten billion code lines in its system. It allows users to search by different code classes, but currently it does not support regular expressions. SearchCode searches through open source communities such as Github, SourceForge, and CodePlex. Amazingly, a single person maintains it. For those who have code with special symbols, Google and other engines cannot cut it. That is where Symbol Hound sniffs around the Net for odd character.
There are a few more code search engines described in the article, but head on over to read it on your own. Code search engines are indicative of the open source mentality-share and spread the wealth.
Whitney Grace, September 06, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
The New Yahoo: A Logo and Pressure on an Executive
September 5, 2013
Yahooooo. I read two stories about the grandma of Web sites. The first was “Introducing Our New Logo!.” I like the exclamation point. The logo is okay, but it seems to be cosmetic. When I was in Portugal in August, Yahoo would not render 70 percent of the time. Why? I am no rocket scientist, so I suppose I could blame it on the hapless Portuguese connectivity providers. But Gmail worked about 90 percent of the time, so maybe the problem is Yahoo’s. Will a new logo address the time outs? One hopes.
Then I read “Pressure Mounts on Yahoo’s De Castro.” No exclamation point after Yahoo, however. The main point of the write up in my opinion was:
Sources close to Yahoo say that De Castro is feeling increasing pressure to deliver better ad results, as the blustery exec has found himself on the outs with CEO Marissa Mayer. There even has been talk that De Castro could be gone by the end of the year, according to numerous sources. The big knock against De Castro is, despite Mayer’s string of mobile acquisitions, lots of positive press and the massive Tumblr deal, the company’s ad business has languished in a marketplace that is enjoying robust growth. Particularly alarming is that Yahoo’s display business is getting hit on both the branding front and programmatic, which would theoretically be a De Castro strength, given his Google background.
My thought is that a new logo and creating discomfort for senior managers adds a different octave to the Yahoo yodel. Do I hear a screech? No, no. The sound is what I hear when one of the goslings tries to:
- Figure out which page will display when accessing Yahoo.com
- Looking at search results which have modest relevance to the query
- Scanning a shopping search result.
I hope that the new logo and excellent management will make the Yahoo yodel more melodious for the fellow in Big Bear, California.
Stephen E Arnold, September 5, 2013
Sponsored by Xenky. Oh, wait. I am Xenky.
History of Web Indexing: BBC Style
September 4, 2013
I read “Jonathon Fletcher: Forgotten Father of the Search Engine.” I have no quibble with the claims that the first Web crawler was an invention spawned in the United Kingdom.
I did find several interesting factoids in the write up; for example:
- Google has indexed more than one trillion pages. On the surface, this sounds just super. However, what is the cost of maintaining the index of the alleged one trillion pages? Is Google cutting corners in its indexing to reduce costs? Perhaps the BBC will expand on this statement. A trillion is a big number and I wonder what percentage of those “pages” are indexed on a daily basis to keep the index fresh.
- “Because websites were added to the list manually, there was nothing to track changes to their content. Consequently, many of the links were quickly out-of-date or wrongly labeled.” Is this true today?
- “By June of 1994, JumpStation had indexed 275,000 pages. Space constraints forced Mr Fletcher to only index titles and headers of web pages, and not the entire content of the page, but even with this compromise, JumpStation started to struggle under the load.” Decades ago the black hole of Web indexing was visible. Now that Big Data have arrived, won’t indexing costs rise in lock step? What cost savings are available? Perhaps indexing less content and changing the index refresh cycles are expedient actions? Have Bing, Google, and Yandex gone down this path? Perhaps the BBC will follow up on this issue?
- “But in my [Fletcher’s] opinion, the Web isn’t going to last forever. But the problem of finding information is.” Has progress been made in Web search?
One interesting aspect of the write up is the conflation of Web search with other types of search. The confusion persists I believe.
Perhaps the BBC will look into the contributions to search of Dr. Martin Porter, the inventor of the Porter Stemmer. Dr. Porter’s Muscat search technology was important, arguably more important than Mr. Fletcher’s.
Stephen E Arnold, September 4, 2013
Sponsored by Xenky
Attivio Teams up with Capax Global
September 4, 2013
Attivio has signed up another partner, this time a leader in search. PR Newswire reveals, “Capax Global and Attivio Announce Strategic Reseller Partnership.” The move will help Capax Global’s customers smoothly shift from conventional enterprise search to the more comprehensive unified information access (UIA) approach. The press release quotes Capax Global CEO and managing director John Baiocco:
“We have seen a natural shift towards UIA as our enterprise search customers contend with massive volumes of information, coming from multiple sources, in different formats. Traditional approaches are no longer adequate in dealing with the scale and complexity of enterprise information. Attivio leads the industry in addressing the demands of big data volume, variety, and velocity that our customers face.”
David Schubmehl, research director at analysis firm IDC, also weighs in on the importance of UIA:
“Unified information access is the next logical progression beyond enterprise search as companies face unprecedented volumes of disparate information, of which 85 percent or more is unstructured. Because UIA platforms can integrate large volumes of information across disconnected silos, technologies like AIE have become a key enabler for big data analytics and decision support.”
Founded in 2007 and headquartered in Massachusetts, Attivio also has offices in other U.S. states, the U.K., Germany, and Israel. The company’s award-winning Active Intelligence Engine integrates structured and unstructured data, making it easier to translate information assets into useful business insights.
Capax Global celebrates its 20th birthday this year, making it a veteran in the search field. The privately-held company, based in New York, offers consulting services, custom implementations, and cloud-hosting services. An emphasis on its clients’ unique business objectives is no doubt part of its appeal for its many customers, which include Fortune 500 companies and major organizations around the world.
Cynthia Murrell, September 04, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Cape Town WordPress Event Promises More Developments in Autocomplete and Search
September 4, 2013
The article titled ElasticSearch Soon Available for Websites on HumanIPO discusses LightSpeed’s developments in WordPress. The program will soon be emulating Google’s site search, and utilizes autocomplete based on Google entries. This development is scheduled for August. The article states,
“Also keen to improve the WordPress search functionality, the coding expert explained the key role of data in promoting the tool to scalability.
“Hopefully one day the datascale in WordPress is going to scale better because they are looking at reworking the post-type database infrastructure,” Shaw said, relating his knowledge of future plans for the popular open source platform.
The launch of a hostage server environment is also anticipated in August with a new SSL offloading system similar to Google’s recently released SPDY. “
Ashley Shaw, the founder of Lightspeed, spoke at the WordPress event in Cape Town. It is the hope of Shaw and others at WordPress that such events will lead to an African community akin to the WordCamp Europe. Kenya is particularly attractive, having taken mobile technology further. In the meantime, WordPress hopes to lower the cost of its new add-on service, which is currently set at US$2,500 for setup and an additional US$1,119 monthly subscription charge.
Chelsea Kerwin, September 04, 2013
Sponsored by ArnoldIT.com, developer of Augmentext
Commerce Search Gets A Tad Bit Better With Enhancements
August 31, 2013
What is the difference between an enhancement and an upgrade? An upgrade indicates a whole new system and solutions for bugs. An enhancement implies that the current piece of software works well, but it is only being made better. Exorbyte, a commerce search application, announced its “Enhancements and Optimizations In July 2013.” The enhancements and optimizations for Exorbyte come in the form of two new features. The first feature is face normalizations that allows users to map different spellings and variations under a single facet value. Another cool thing about this feature is that users can specify the number of times an individual facet value appears in a search. This can push rarer data into search results and limit alternate forms of a query.
Here is the neatest new feature for query-based ranking:
“It is now possible to influence the ranking based on the query itself, allowing for even higher result relevancy and hence conversions. In a global ranking rule search terms can be defined that trigger this ranking rule to come into effect. For example, you can specify that the ranking rule “boost the category toy” is only activated when the query contains the word “ball”. If the query term was not used as a restriction, the category “toys” would always be boosted. For example if the query term was “golf” toy golf products would be placed on top, although users might expect professional equipment, so that this rule should not apply. So the query-based ranking allows you to tune the relevancy in cases.”
Putting the intelligence in intelligent search. It also reminds me of using an auto-tuner to pick up the proper frequency. Features like these help normalize search and make the results useful to the user. Exorbyte asserts it can eliminate the need for facet normalization with its software.
Whitney Grace, August 31, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search
-
- Subscribe to Beyond Search
Feature archive
News archive
-
