A Wrinkle in the Government Procurement Envelope
March 3, 2009
Government agencies buy quite a bit of hardware, storage, and systems to deal with digital information. I avoid Washington, DC. I went to grade school there. I fought traffic on I 270 when I worked in the city for a decade after getting booted from a third tier university. I then did the SDF to BWI run on Southwest for five or six years when I was hooked up with a government-centric services firm. I don’t know that much about procurements, but I do know when what looks like a trivial event could signal a larger shift. You can take a look at the ComputerWorld story “DOJ Accuses EMC of Improper Pricing” here. If I were writing the headline, I would have slapped in an “allegedly”. Keep in mind I am reporting second hand news and offering a comment. I am not sure how accurate or how much oomph this DOJ (Department of Justice) matter has. The thrust of the story is that DOJ is sniffing into payments and tie ups. Now most folks in Harrods Creek, Kentucky don’t pay much attention to the nuances of Federal acquisition regulations. Let’s assume that this is little more than a clerical error. But in my opinion this single matter signals a tougher line on how companies that manufacture or create hardware and software deal with the government. Some organizations sell direct to the government and others take the lead and turn it over to partners. The relationships among the manufacturers and the partners and the government is a wonderland of interesting activities. Why is this important? Search vendors operate in different ways and some systems trigger significant hardware acquisitions. With a massive Federal deficit, I wonder, “Is this single alleged action a harbinger of closer scrutiny of some very high profile companies’ business dealings?” My hunch is, “Yep.” Some companies will want to tidy their business processes. When rocks get flipped over, some interesting things can be spotted. One major search vendor does not sell directly to the US government. The vendor deals through partners. Some partners are loved more than others. My thought is that if I were investigating these tie ups, I would prefer to see partners treated in an equitable way with documentation that backs up the compensation, limits, and responsibilities with regard to the US government and the source of the hardware or software. If the system is “informal”, I would dig a little deeper to make sure that US government procurement guidelines were followed to the letter. Just my opinion. I might come out of retirement to do some of the old time procurement fact finding when spring comes.
Stephen Arnold
Mysteries of Online 9: Time
March 3, 2009
Electronic information has an interesting property: time distortion. The distortion has a significant effect on how users of electronic information participate in various knowledge processes. Information carries humans along much as a stream whisks a twig in the direction of the flow. Information, unlike water, moves in multiple directions, often colliding, sometimes reinforcing, and at others in paradoxical ways that leave a knowledge worked dazed, confused, and conflicted. The analogy of information as a tidal wave connotes only a partial truth. Waves come and go. Information flow for many people and systems is constant. Calm is tough to locate.
Vector fields. Source: http://www.theverymany.net/uploaded_images/070110_VectorField_test012_a-789439.jpg
In the good old days of cuneiform tablets, writing down the amount of wheat Eknar owed the king required specific steps. First, you had to have access to suitable clay, water, and a clay kneading specialist. Second, you needed to have a stylus of wood, bone, or maybe the fibula of an enemy removed in a timely manner. Third, you had to have your data ducks in a row. Dallying meant that the clay tablet would harden and make life more miserable than it already was. Once the document was created, the sun or kiln had to cooperate. Once the clay tablet was firm enough to handle without deleting a mark for a specified amount of wheat, the tablet was stacked in a pile inside a hut. Forth, the access the information, the knowledge worker had to locate the correct hut, find the right pile, and then inspect the tablets without breaking one, a potentially bad move if the king had a short temper or needed money for a war or a new wife.
In the scriptorium in the 9th century, information flow wasn’t much better. The clay tablets had been replaced with organic materials like plant matter or for really important documents, the scraped skin of sheep. Keep in mind that other animals were used. Yep, human skin worked too. Again time intensive processes were required to create the material on which a person would copy or scribe information. The cost of the materials made it possible to get patrons to spit out additional money to illustrate or illuminate the pages. Literacy was not widespread in the 9th century and there were a number of incentives to get sufficient person power to convert foul papers to fair copies and then to compendia. Not just anyone could afford a book. Buying a book or similar document did not mean the owner could read. The time required to produce hand copies was somewhat better than the clay tablet method or the chiseled inscriptions or brass castings used by various monarchs.
Yep, I will have it done in 11 months, our special rush service.
With the invention of printing in Europe, the world rediscovered what the Chinese had known for 800, maybe a thousand years. No matter. The time required to create information remained the same. What changed was that once a master set of printing plates had been created. A printer with enough capital to buy paper (cheaper than the skin and more long lasting than untreated plant fiber and less ink hungry than linen based materials) could manufacture multiple copies of a manuscript. The out of work scribes had to find a new future, but the impact of printing was significant. Everyone knows about the benefits of literacy, books, and knowledge. What’s overlooked is that the existence of books altered the time required to move information from point A to point B. Once time barriers fell, distance compressed as well. The world became smaller if one were educated. Ideas migrated. Information moved around and had impact, which I discussed in another Mysteries of Online essay. Revolutions followed after a couple hundred years, but the mindless history classes usually ignore the impact of information on time.
If we flash forward to the telegraph, time accelerated. Information no longer required a horse back ride, walk, or train ride from New York to Baltimore to close a real estate transaction. Once the new fangled electricity fell in love with information, the speed of information increased with each new innovation. In fact, more change in information speed has occurred since the telegraph than in previous human history. The telephone gave birth to the modem. The modem morphed into a wireless USB 727 device along with other gizmos that make possible real time information creation and distribution.
Time Earns Money
I dug out notes I made to myself sometime in the 1982 – 1983 time period. The implications of time and electronic information caught my attention for one reason. I noted that the revenue derived from a database with weekly updates was roughly 30 percent greater than information derived from the same database on a monthly update cycle. So, four updates yielded a $1.30, not $1.00. I wrote down, “Daily updates will generate an equal or greater increase.” I did not believe that the increase was infinite. The rough math I did 25 years ago suggested that with daily updates the database would yield about 1.6 percent more revenue than the same database with a monthly update cycle. In 1982 it was difficult to update a commercial database more than once a day. The cost of data transmission and service charges would gobble up the extra money, leaving none for my bonus.
In the financial information world, speed and churn are mutually reinforcing. New information makes it possible to generate commissions.
Time, therefore, not only accelerated the flow of information. Time could accelerate earnings from online information. Simply by u9pdating a database, the database would generate more money. Update the database less frequently, the database would generate less money. Time had value to the users.
I found this an interesting learning, and I jotted it down in my notebook. Each of the commercial database in which I played a role were designed for daily updates and later multiple updates throughout the day. To this day, the Web log in which this old information appears is updated on a daily basis and several times a week, it is updated multiple times during the day. Each update carries and explicit time stamp. This is not for you, gentle and patient reader. The time stamp is for me. I want to know when I had an idea. Time marks are important as the speed of information increases.
Implications
The implications of my probably third-hand insight included:
- The speed up in dissemination means that information impact is broader, wider, and deeper with each acceleration.
- Going faster translates to value for some users who are willing and eager to pay for speed. The idea is that knowing something (anything) first is an advantage.
- Fast is not enough. Customers addicted to information speed want to know what’s coming. The inclusion of predictive data adds another layer of value to online services.
- Individuals who understand the value of information speed have a difficult time understanding why more online systems and services cannot deliver what is needed; that is, data about what will happen with a probability attached to the prediction. Knowing that something has a 70 chance of taking place is useful in information sensitive contexts.
Let me close with one example of the problem speed presents. The Federal government has a number of specialized information systems for law enforcement and criminal justice professionals. These systems have some powerful, albeit complex, functions. The problem is that when a violation or crime occurs, the law enforcement professionals have to act quickly. The longer the reaction time, the greater the chance that the bad egg will tougher to apprehend increases. Delay is harmful. The systems, however, require that an individual enter a query, retrieve information, process it and then use another two or three systems in order to get the reasonably complete picture of the available information related to the matter under investigation.
The systems have a bottleneck. The human. Law enforcement personnel, on the other hand, have to move quickly. As a result, the fancy online systems operate in one time environment and the law enforcement professionals operate in another. The opportunity to create systems that bring both time universes together is significant. Giving a law enforcement team mobile comms for real time talk is good, but without the same speedy and fluid access to the data in the larger information systems, the time problem becomes a barrier.
Opportunity in online and search, therefore, is significant. Vendors who pitch another fancy search algorithm are missing the train in law enforcement, financial services, competitive intelligence, and medical research. Going fast is no longer a way to add value. Merging different time frameworks is a more interesting area to me.
Stephen Arnold, February 26, 2009
Microsoft Trumps Google, Dismisses Its Enterprise Services
March 2, 2009
Microsoft seems to be returning to its glory days as vanquisher of the weak and destroyer of the newcomers. Phil Wainewright wrote “Microsoft Pumps Cloud, Trumps Google with GSK.” I must admit the GSK threw me. It is the insiders way to refer to Glaxo SmithKline, a pharmaceutical giant. The comment that stuck in my beak was:
Not only that. Ron Markezich, corporate VP of Microsoft Online Services, was scathing of Google’s efforts to make headway in the enterprise market. “Google we really do not feel is ready for the enterprise,” he said in a call briefing bloggers on the announcement an hour ago. “They’re offering three nines SLA and they’ve missed three of the last six months,” he added, referring to last week’s Gmail outage and earlier incidents. In a sideswipe at Google’s offer of a 15-day credit for last week’s outage, he went on to add that Microsoft maintains its services at four-nines availability, while backing up its three-nines SLA with financial penalties: “We don’t just give service credits, we give hard dollars if we miss an SLA.” [Emphasis added]
My take on this announcement includes these thoughts:
- Looks like each Google announcement will trigger an aggressive response from Microsoft
- Microsoft is sending a signal to Google and probably to any other company that it intends to protect its customer base. Good cheer and happiness may not be part of the response
- Google must have landed a kaisho (open hand strike). Microsoft’s statement (cited above) suggests to me that Google is not an annoyance; Google is a threat.
More from the battle front as reports arrive.
Stephen Arnold, March 2, 2009
YAGG Update: PageRank Tweak or Bug
March 2, 2009
If you are mesmerized by things Google, you will want to navigate to Search Engine Roundtable and read “Google March 2009 PageRank Update or Glitch?” here. The article provides links to a couple of posts that identify what may be a potential glitch or goof as in “yet another Google goof” or YAGG. I know the acronym annoys Alex, a potential Googlephile. The article quotes a Googler who uses the phrase “some kind of glitch”, which may be old news if you were bitten by the Gfail issue a few days ago.
Stephen Arnold, March 2, 2009
Another YAGG: Picasa Privacy
March 2, 2009
Philipp Lenssen at Googleblogoscoped wrote “Picasa Privacy Oddity” here. If the information is accurate, the Google has another YAGG (yet another Google glitch) to resolve. Mr. Lenssen wrote:
this goes to show that not password-protecting a sign-in locked album’s image URLs themselves is still not as utterly-security-obsessive as could be (which is noteworthy considering Picasa Web Album’s mixed privacy history of the past).
Alex, once a reader, is not too keen of my YAGG coinage or my pointing out the feet of clay that Googzilla may have. Worth watching I suppose.
Stephen Arnold, March 2, 2009
Potential Trouble for LexisNexis and Westlaw
March 2, 2009
Most online surfers don’t click to Reed Elsevier’s LexisNexis or Thomson Reuters Westlaw. The reason? These commercial services charge money–quite a lot of money–to access legal documents. Executives at both firms can deliver compelling elevator pitches about the added value each company brings to legal documents. In the pre-crash era, legal indexing was a manual process. Then the cost crunch arrived so both outfits are trying to slap software against the thorny problem of making sense of court documents, rulings, and assorted effluvia of America’s legal factories. I may write about how these two quasi US outfits have monopolized for fee legal information about American law for lawyers, government agencies. Both Reed and Thomson then turn around and sell access to these documents to the agencies that created them in the first place. I wonder if the good senator is aware of this aspect of commercial online services’ busness practices?
What’s the trouble? I bet you thought I was going to mention Google. Wrong. Google is on the edge of indexing legal information in a more comprehensive way. But the right now trouble is Senator Joe Lieberman. Wired reported that the good senator wondered by public documents are not available without a charge. You can read the story “Lieberman Asks, Why Are Court Docs Still Behind Paid Firewall?” here. Senator Lieberman’s question may lead to a hearing. The process could, in my opinion, start a chain reaction that further erodes the revenue Reed Elsevier and Thomson Reuters derive from public documents. Somewhere in the chain, the Google will beef up the legal content in its Uncle Sam service here.
At their core, Reed Elsevier and Thomson Reuters are traditional publishing and information companies. As such, their business model is fragile. Within the present financial pressure cooker, the Lieberman question could blow the lid off these two organization’s for fee legal business. If government agencies shift to a service provided by Google, Microsoft, or Yahoo, I think these two dead tree outfits will crash to the forest floor.
What the likelihood of this downside scenario. I would put it at better than 60 percent. Have another view? Share it, please. Set the addled goose straight.
Stephen Arnold, March 2, 2009
Ask.com Frames in the Picture
March 2, 2009
Frames and iframes are nifty. Over the years, their use has aroused some controversy. At one time, Google took a dim view of iframes. I have had reports that Google itself uses iframes. Other vendors have employed the technology to allow users to visit sites that are not what they seem. You navigate to another site and then discover that you are not where you want to be. Over the years, I have stumbled across patent documents that include variations of the iframe technology. Some uses are for the purpose of tracking user behavior. Others allow a Web site operator to inject content around the user’s intended destination. I lose interest in this type of cleverness, having lost my enthusiasm for tilting at windmills. There are quite a few clever and tricky folks who find ways to warp a naïf’s Internet experience.
Pandia.com, a news service that I like quite a bit, reported on some frame use at Ask.com, the also-participated Web search vendor. Ask.com for me is a good example of what happens when someone who is good at one thing tries to extend that expertise to another domain unrelated to the first. The outcome of this type of master-of-the-universe thinking is a service like Ask.com. It’s not bad; it’s not good. It’s one thing today; it will be another thing tomorrow. I recall a dinner two years ago when an azure chip consultant told me that Ask.com was on the move. I thought, “This fellow is getting paid to advise publishers about online partners?” Now Ask.com is the search engine of NASCAR. I wonder if any of the Ask.com executive team hangs out with Kentucky’s NASCAR fans? I have. I am not sure this demographic is where the action is for search.
Search Engine Roundtable followed up with its February 27, 2009, story, “Ask.com Crosses The Line: Frames Search Results.” This is a useful write up, and it includes a screenshot. For me, the most interesting comment was:
Searchers are not happy about this at WebmasterWorld. Robzilla said, “this annoys me as both a user and a webmaster, and overall just seems a little desperate.” Senior member, skipfactor, accurately points out that the search ads are not framed in.
What’s my take? Behavior that tricks users or actions that are designed to pump up revenue are part of the present culture norms. When it is a banker paying himself / herself a bonus for losing money or an insurance company refusing to honor a claim, I see behavior that makes me uncomfortable in many places. Why should anyone be surprised that online companies caught in a cash crunch would push into such murky areas. As more people use the Internet, there are more opportunities to snooker users.
The Internet is no longer something new, accessible only to scientists, engineers, and researchers. The Internet is like the Kentucky State Fair. As long as you can get on the grounds, you’re good to go. Last time I checked, the Kentucky State Fair was a mirror of the best and worst in the bluegrass state. I think it is useful to alert users of certain methods, but I don’t think most users know or care about Ask.com. Those who do may be quite happy with whatever Ask.com provides.
Stephen Arnold, March 2, 2009
Harry Collier, Infonortics, Exclusive Interview
March 2, 2009
Editor’s Note: I spoke with Harry Collier on February 27, 2009, about the Boston Search Engine Meeting. The conference, more than a decade into in-depth explorations of search and content processing, is one of the most substantive search and content processing programs. The speakers have come from a range of information retrieval disciplines. The conference organizing committee has attracted speakers from the commercial and research sectors. Sales pitches and recycled product reviews are discouraged. Substantive presentations remain the backbone of the program. Conferences about search, search engine optimization, and Intranet search have proliferated in the last decade. Some of these shows focus on the “soft” topics in search and wrap the talks with golf outings and buzzwords. The attendee learns about “platinum sponsors” and can choose from sales pitches disguised as substantive presentations. The Infonortics search conference has remained sharply focused and content centric. One attendee told me last year, “I have to think about what I have learned. A number of speakers were quite happy to include equations in their talks.” Yep, equations. Facts. Thought provoking presentations. I still recall the tough questions posed to Larry Page (Google) after his talk in at the 1999 conference. He argued that truncation was not necessary and several in attendance did not agree with him. Google has since implemented truncation. Financial pressures have forced some organizers to cancel some of their 2009 information centric shows; for example, Gartner, Magazine Publishers Association., and Newspaper Publishers Association. to name three. Infonortics continues to thrive with its reputation for delivering content plus an opportunity to meet some of the most influential individuals in the information retrieval business. You can learn more about Infonortics here. The full text of the interview with Mr. Collier, who resides in the Cotswolds with an office in Tetbury, Glou., appears below:
Why did you start the Search Engine Meeting? How does it different from other search and SEO conferences?
The Search Engine Meeting grew out of a successful ASIDIC meeting held in Albuquerque in March 1994. The program was organized by Everett Brenner and, to everyone’s surprise, that meeting attracted record numbers of attendees. Ev was enthusiastic about continuing the meeting idea, and when Ev was enthusiastic he soon had you on board. So Infonortics agreed to take up the Search Engine Meeting concept and we did two meetings in Bath in England in 1997 and 1998, then moved thereafter to Boston (with an excursion to San Francisco in 2002 and to The Netherlands in 2004). Ev set the tone of the meetings: we wanted serious talks on serious search domain challenges. The first meeting in Bath already featured top speakers from organizations such as WebCrawler, Lycos, InfoSeek, IBM, PLS, Autonomy, Semio, Excalibur, NIST/TREC and Claritech. And ever since we have tried to avoid areas such as SEO and product puffs and to keep to the path of meaty, research talks for either search engine developers, or those in an enterprise environment charged with implementing search technology. The meetings tread a line between academic research meetings (lots of equations) and popular search engine optimization meetings (lots of commercial exhibits).
Pictured from the left: Anne Girard, Harry Collier, and Joan Brenner, wife of Ev Brenner. Each year the best presentation at the conference is recognized with the Evvie, an award named in honor of her husband, and chair of the first conference in 1997.
There’s a great deal of confusion about the meaning of the word “search”, what’s the scope of the definition for this year’s program?
Yes, “Search” is a meaty term. When you step back, searching, looking for things, seeking, hoping to find, hunting, etc are basic activities for human beings — be it seeking peace, searching for true love, trying to find an appropriate carburetor for an old vehicle, or whatever. We tend now to have a fairly catholic definition of what we include in a Search Engine Meeting. Search — and the problems of search — remains central, but we are also interested in areas such as data or text mining (extracting sense from masses of data) as well as visualization and analysis (making search results understandable and useful). We feel the center of attention is moving away from “can I retrieve all the data?” to that of “how can I find help in making sense out of all the data I am retrieving?”
Over the years, your conference has featured big companies like Autonomy, start ups like Google in 1999, and experts from very specialized fields such as Dr. David Evans and Dr. Liz Liddy. What pulls speakers to this conference?
We tend to get some of the good speakers, and most past and current luminaries have mounted the speakers’ podium of the Search Engine Meeting at one time or another. These people see us as a serious meeting where they will meet high quality professional search people. It’s a meeting without too much razzmatazz; we only have a small, informal exhibition, no real sponsorship, and we try to downplay the commercialized side of the search world. So we attract a certain class of person, and these people like finding each other at a smaller, more boutique-type meeting. We select good-quality venues (which is one reason we have stayed with the Fairmont Copley Plaza in Boston for many years), we finance and offer good lunches and a mixer cocktail, and we select meeting rooms that are ideal for an event of 150 or so people. It all helps networking and making contacts.
What people should attend this conference? Is it for scientists, entrepreneurs, marketing people?
Our attendees usually break down into around 50% people working in the search engine field, and 50 percent those charged with implementing enterprise search. Because of Infonortics international background, we have a pretty high international attendance compared with most meetings in the United States: many Europeans, Koreans and Asians. I’ve already used the word “serious”, but this is how I would characterize our typical attendee. They take lots of notes; they listen; they ask interesting questions. We don’t get many academics; Ev Brenner was always scandalized that not one person from MIT had ever attended the meeting in Boston. (That has not changed up until now).
You have the reputation for delivering a content rich program. Who assisted you with the program this year? What are the credentials of these advisor colleagues?
I like to work with people I know, with people who have a good track record. So ever since the first Infonortics Search Engine Meeting in 1997 we have relied upon the advice of people such as you, David Evans (who spoke at the very first Bath meeting), Liz Liddy (Syracuse University) and Susan Feldman (IDC). And over the past nine years or so my close associate, Anne Girard, has provided non-stop research and intelligence as to what is topical, who is up-and-coming, who can talk on what.These five people are steeped in the past, present and future of the whole world of search and information retrieval and bring a welcome sense of perspective to what we do. And, until his much lamented death in January 2006, Ev Brenner was a pillar of strength, tough-minded and with a 45 year track record in the information retrieval area.
Where can readers get more information about the conference?
The Infonortics Web site (www.infonortics.eu) provides one-click access to the Search Engine Meeting section, with details of the current program, access to pdf versions of presentations from previous years, conference booking form and details, the hotel booking form, etc.
Stephen Arnold, March 2, 2009
Google a Twittering
March 1, 2009
On March 1, 2009, another story about a possible tie up between Google and Twitter surfaced. The source? Jennifer Bosavage and CRNCanada. You can read the story “Wedding bells for Google and Twitter?” here. For me, the most interesting comment in the article was:
Could Google be eyeing Twitter as an acquisition? That possibility’s got the blogosphere all “a-twitter,” pardon the pun. Earlier this week, Google activated its Twitter account and all Tweets broke loose. As of Friday morning, Google had more than 26,000 followers. The speculation is that, in a move similar to its purchase of YouTube, Google is interested in buying Twitter.
Google has been somewhat clumsy in the real time news space. Maybe Ms. Bosavage and CRNCanada have an inside track on this alleged tie up.
Stephen Arnold, March 2,, 2009
InOrder Conceptual Search
March 1, 2009
Update: link updated, March 2, 2009
A happy quack to the reader who sent me a link to the conceptual search engine InOrder.org here. Here’s the description of the system from the organization’s Web site:
InOrder is a collaborative conceptual search interface. It is being developed by Garrett Camp at the EIS Lab at the University of Calgary. It’s design premise is that search engines such as Google already find relevant results for well-formed queries, but do not efficiently elicit these search needs from users. InOrder solves this issue by creating an interactive environment for collective group search. InOrder acquires domain knowledge of semantic relevance within a given search context. Mediated sets of “topics” and “terms” guide search exploration by collective intuition, reusing search strategies utilized by ones peers. Incremental and explicit elicitation of these collective strategies enables participants to make better-informed search decisions. In terms of existing web media InOrder may be viewed as a structured weblog of the semantic interactions of those with similar search goals.
We ran several test queries and found the system interesting. Here’s a screen shot of the result for our query “enterprise search”:
We will do some more testing.
Stephen Arnold, February 28, 2009
-
- Subscribe to Beyond Search
Feature archive
News archive
-
