Tag Archives: big data

Escalating Ignorance in the Information Age

Oxymoronic?  Perhaps, but true.  The more ‘information’ that we have produced in the past forty years of networked information systems and the internet, the less we seem to know or trust. We are in an era of information entropy in which more is less.

I remember six years ago when an acquaintance of mine mentioned that she did not have cable t.v.  I wondered how she could possibly keep informed of current events.  Two years later, I dropped it myself, never regretting my ‘loss’.  Subsequently, I have become progressively more selective in my reading, particularly on the web, finding  that much of what I have consumed provides less insight.

The information age has provided a wealth of data, but not a corresponding wealth of insight. Why is that? Let’s review.

  1.  Reality is changing at warp speed. Yesterday’s facts and truisms are being rapidly  rendered obsolete. This ain’t your granddaddy’s nothin’!
  2.  We are producing mountains of data, but proportionately less ‘information’ (remember: data and information are not the same) .
  3.  The information that we do produce from the data  is often without meaningful context or perspective, and therefore of limited utility, relevance or reliability in a world where context can change as quickly as facts, and perspectives proliferate.
  4.  The institutions and information intermediaries (the press, government, academia, science, professions , unprofessional organizations such as Facebook and Google) that we depend upon for reliable and trustworthy information have almost all been diminished by scandals as they have become ‘monetized’, or otherwise compromised directly or indirectly by economic forces which have bent their values to serve other objectives.
  5. Concerted efforts to distort or undermine or repudiate otherwise valid information have been refined and deployed with devastating effectiveness.
  6. We have become conditioned, if not programmed, to suspend, if not avoid, critical thinking in preference to simple or comforting dogmas, also known as ‘thought on auto-pilot’.  We have willingly become prisoners of our own illusions, or those which too many are willing to sell us, in a world where there are now too many factoids to make sense of very much for very long.

One of the interesting consequences of all this is that in many subtle ways we take more time to do things that once seemed so simple, or to make decisions that are now more difficult in an increasingly complex world. I remember standing in the soap isle of the local supermarket gazing at the various offerings of dishwasher detergent.  There before me was New and Improved, Extra New, Super Improved, and You Won’t Believe Your Eyes, all in similar but different containers by the same manufacturer, all at nearly the same price. Along came a lady who engaged in the same exercise as I.  After a few minutes, we looked at each other and asked ‘What’s the difference?’.  I could just grab one off the shelf and be done, but I’ve been programed to optimize; best value for the price. Ultimately, I just grabbed one off the shelf.  Now multiply this simple example across the plethora of shopping transactions. Recognize that this phenomenon applies to information as much as dishwasher soap. The default for decision gridlock is snap judgment which often leads to the unintended consequence of buyer’s remorse, and the oft resultant lament: ‘What was/were I/they thinking?’

At a higher level of consequence, business and governmental decisions become similarly captive of a world that is devolving from long assumed perceptions of homogeneity to ever more complex and finite sub-groups, sub-cultures, sub-markets, sub divisions; each with their peculiarities and potential risks to the unwary, and few of which we really understand.  Middle east peace? Climate Change? Healthcare policies? Renewable energy strategies? Transportation strategy? Tax reform? Nuclear energy?

So here we are at the pinnacle of the data-pile at which our economic elite, blessed with all the raw data and algorithms they possess, are risk averse to investing their parked trillions for fear of risks they cannot effectively define, and therefore cannot effectively hedge.

And our ‘intelligence services’ with their army of server farms cannot pro-act with reliability; only react once the threat has manifested itself.  You don’t need big data to set up a sting for the witless.  But all their data isn’t helping them to preempt the wily.

And government, which is more reactive than proactive by nature, works on old and fragmented systems evolved from  the vastness of its enterprise and the granularity of its operations as defined by ever more complex regulations; systems which are too big, too complex to upgrade, but too critical to let die.  This also applies to large corporations, which are bureaucratically not too far removed from government.

I do not consider myself an information Luddite.  By virtue of the very nature of my profession, I love good data; I crave good data; I pine for good data; but I also distrust all data until its reliability can be proven.  More is not necessarily better.

Our data and its infrastructure is steadily holding us captive while we perpetuate the delusion that it is setting us free.  Unwilling to accept this possibility, we double down on our bet on artificial intelligence (AI) as the means to master the data-pile and set us free. No doubt, AI will bring many advantages.

But it also holds the risk that in seeking to outsource our thinking and judgements to so-called sentient machines, we are inviting a concentration of power (think Amazon, Facebook and Google) and a potential for manipulation that enslaves rather than liberates us.  Given our own individual and collective imperfections as citizens, professionals and societies, is it reasonably plausible that we can create AI that transcends our manifest imperfections and biases, but is vastly more capable of the harm we can already do without AI’s assistance? Stated more simply, can imperfect humans create perfect machines, or merely machines more capable of leveraging our imperfections to greater consequence?

We need not look far to preview the risks. Darkness is descending as the Trump administration seizes the reins of power and systematically draws the shades on the windows of government.  Today it seeks to withhold information; to render us less informed. Today, as it has for the past two years, it perpetrates blatant lies, increasingly devoid of any subtlety, to propagate its world view.  Imagine what it might do once it has implanted its partisans where all the levers of information creation or influence are located.  Consider a modification of the adage: ‘To err is human; to really screw up takes a computer’.

The possibilities are exponential.

Happy Presidents’ Day.  Better ones are coming. Hopefully.

Onward

20170220

Advertisements

The Grid: News of its Death Is Premature

The latest claptrap ricocheting off the walls of the business and environmental media echo chambers is word of the inevitable decline and death of the electric grid…or not.  It is more likely to succumb to neglect, along with the rest of our infrastructure, than to technological displacement by renewable technologies any time in the foreseeable future.

Nonetheless, the subject is a tug-of-war between two rival camps of idiocy.  One is comprised of the fossil fuel feudalists and their various front organizations who are fighting the growth of renewables at any scale as a perceived threat to the inevitable decline of cheap and easy carbon.  The other camp  is a combination of visionaries and eco-huckster capitalists who believe in a Disney World (both capitalist and fantasy) future but ignore the social and economic inertia that impedes reasonable progress at best, and their wildest dreams, to be sure.  The utilities themselves in varying degrees are more inclined to be on the side of the Feudalists  than the Fantasy folks.

The following quote is indicative of the kind of siloed, echo-chamber consensus that propagates the illusion of inevitability:

“Utilities are afraid that solar power will be to the electrical grid what PCs were to mainframes, or e-mail to the Postal Service: a technology that will simply kill its predecessors. Coal and nuclear power are both doomed, and the profit-making power grid with it. That’s all to our benefit.”

The writer may not realize this, but personal computers have not dispatched the mainframe; they have merely augmented it.  The Mainframe of yore has morphed into the server farm of now; Big Iron still lives. If it didn’t, there wouldn’t be The Cloud and Big Data.  And, while information is more widely distributed than in the ’70s through the internet, ironically it is every bit as centralized and concentrated as previously.  Just ask your government, or Jeff Bezos, or Eric Schmidt or Mark Zuckerberg.

And email has assuredly impacted the Post Office, but it has not rendered it to the dust bin of history yet, nor is it likely to in the near future.  Congress has reserved that privilege to itself.  The Post Office can still evolve, adapting its network to new and evolving needs, if allowed. But the same sclerotic forces that would kill renewables are doing their best to slay the postal service and any other institutions of social and economic unification.

The electric grid is most definitely in transition, but it is by no means a ‘dead man walking’ for the next half century for three very simple reasons.

1) The technology to replace it with renewable energy is not there, will not be sufficiently developed by the end of this decade, and will take three to four decades or more to propagate to scale.

2) Much of the generation of renewables will be on industrial scale wind, wave  and solar farms (the equivalent of mainframes, if you will) in places of best advantage, for transport to places of greatest need.  Hence, the grid, on some scale, in some configuration, lives. And some power consumers will never have the means to deploy stand-alone renewables without augmentation from the grid.

3) The ultimate system will be a hybrid of distributed and industrial scale generation integrated in a truly intelligent grid because neither option by itself will meet a variety of circumstances than can compromise each.

The Grid is The Net, It will evolve technologically, but it is unlikely to be replaced.  The fact that phone companies are seeking to dis-own their land lines as subscription declines is not the death of the telephone network; it is simply moving to wireless technology, but it is still a centrally managed and financed network.

What is disconcerting in this nonsensical rumination about the inevitable death of The Grid is that it distracts us from the real issues affecting the grid and its constituent utilities:  an orderly technological,  financial and regulatory transition to a differently configured and operational reality.

The notion of ‘going off the grid’ for most of us is more a fantasy than a future.  For those of you with solar panels on your house, ask yourselves: if your system was wiped out by a hurricane, along with many others in your town, how quickly would you be able to replace your panels and get back to business?  Alternatively, how long would it take the utility to re-string lines to your house?  Aside from storms, how many office buildings are likely to meet their own needs from home-grown renewable generation any time in the intermediate term future?  Can we get Metro-North to run its trains on anything but industrial grade electric generation in the intermediate term?

And finally, much as many of us would welcome the demise of coal, it is notable that a growing number of thought leaders in the climate change community are becoming resigned to the need for carbon sequestration technology, given the growing consensus that coal will be with us much longer than our fondest hopes would allow, for a variety of reasons.

Among the major concerns that the utilities and everyone else should be concerned about are:

–  resurrection of the grid that is from its current state of decay due to deferred maintenance;

–  armor the grid against cyber-attack or otherwise mitigate the risks of conventional sabotage;

–  armor the grid against probable extremes and transitions in climate;

–  create a grid that is more modular in design and thereby more scalable and adaptable to the various fluctuations in energy generation, distribution and utilization technology and patterns of usage:

–  evolve a more enlightened management capable of managing a more sophisticated grid;

–  evolve a more sophisticated citizenry willing to accept that this is an infrastructure that we will all depend on in varying degrees, and therefore we must all support.

The Grid is not dead or at death’s doorstep.

But we must not let it linger in agony, or its agony will be our own.

Onward.

20140427

 

 

Data Tyranny – Through the Looking Glass

At the end of a long week last week, I needed a spark of humor to get me over the Friday afternoon finish line.  And here it is:

It was reported that Mark Zuckerberg has taken President Obama to task over governmental spying and subversion of privacy on the internet. Either the lad’s brain is hermetically sealed in his own fantasy world of ‘specialness’, or he has single-handedly recalibrated the dimensions of chutzpah.  Frankly, I think that he is such an exceptional personage that he is fully capable of both. As are so many of his peers in the Enchanted Valley.

Honestly, would you buy a used car from Mark, or Eric Schmidt over at the other Data Death Star?  Or a new car, for that matter? Think about Google’s track record of ‘experimenting’ with new services only to drop them in an instant once some child genius gets bored.  Imagine yourself tooling down the boulevard in your auto-piloted Google-mobile, having an otherwise delightful conversation with Siri (or not, from what I’m told), when all of a sudden the Google mother-ship decides to do an app tweak on the autopilot.  Ooooops.

But I digress.  The real focus of this rant is the issue of data and personal privacy in the age of Mad Max and the self-styled buccaneers of free enterprise.

Am I worried, as a citizen, about the NSA?  Yeah.

Do I fear institution of a totalitarian regime by my government?  Not tomorrow morning, and probably not next week; but its excesses could conceivably lead to such, if not for its own apparent incompetence.  An organization with such loose internal security is a diminished threat in its capacity to effectively project AND sustain tyranny.

Does the NSA require greater transparency? No, stupid! It’s a spy agency!

Does the NSA require greater oversight and checks and balances?  Absolutely!

And how about Facebook and Google?  Yeah, them too. And let’s throw in Amazon and Apple and Yahoo while we’re at it.  Because these various data pimps and the galaxy of private sector John’s they service (insurance, finance, employment and recruiting, retail) can and do support a subtle tyranny of manipulation and exclusion that is perhaps more immediate and pervasive and subversively disruptive to the greater society than anything that is likely to emanate from Fort Meade any time in the foreseeable future.

But what oversight do we have on them? What oversight SHOULD we have on them?  If Google and Facebook are the data-mart of choice for our esteemed NSA for ‘shoplifting’, what does it offer its paying customers, whose profit driven endorphins seek the same data for more predatory purposes? Let’s have that conversation too, Mr. Mark.

Big Data and its twin, Analytics, most definitely have a place in our evolving society. We have a lot to learn about ourselves and our world for positive outcome. But the Googles and Facebooks are not being driven by the search of insight and wisdom.  They are being driven by the search for profits, which is not, in itself, a bad thing.  But at what expense and whose expense do they derive their profits and to what end?

If Facebook and Google, et al, are data pimps, and the various buyers are Johns, who are the prostitutes in this analogy?  You guessed it.  You and me.  We aren’t even selling anything.  We’re givin’ it away for free.  For free information, free music, free video, free books, free porn, soft and hard, and not thinking twice about who’s looking back at us from behind the screen, and with what intent.

The truly amusing part of this is that we have no concept of the potential consequences of data tyranny, although we should by now.  We simplistically think ‘I know everything there is to know about me. I’ve got nothing to hide’.  But that’s not how it will work.

Big data may store and regurgitate the bits and bytes of your digital life on demand for the select few (who are willing to pay) to see.  But then the fun begins with the ANALytics.

Analytics are tools that theoretically enable us to compile big data into big insights. They can assemble huge amounts of data never before possible, and present it in various dimensions and perspectives. They can slice and dice the mountain in ways we might not achieve in a couple of lifetimes.

But Analytics are not simply about compiling data.  Analysis is ultimately about perspective (how you choose to approach it) and judgment (what you deem relevant and important).  Perspective and importance are ultimately the system designer’s choice; not the machine’s. Therein lies the potential for good and evil.  It’s not merely the factoids of our lives; its the perspective and value judgements applied to them by people or institutions in positions of power.

Analytics become ANALytics when we use these tools to substitute for critical thinking, to ‘cut to the chase’ as we love to say in business, to bang through the ‘clutter’ and get to the market’s ‘sweet spot’, and ya-di-ya-di-ya. ANALytics are when lazy people in positions of influence let the machine do the thinking for them. There are an incredible number of highly paid, lazy people in positions of influence.

I’ve watched ANALytics at work on a modest scale in the medical field as a casual observer.  In three family situations over three decades, I’ve watched medical science throw barrages of tests at patients, only to come up empty of useful insights. In many cases, it took an exceptional doctor to look beyond what the test data was telling her/him for what it was NOT telling hem, and then make an informed judgment (guess) as to what course of action might be productive.  In the most troubling of instances, a doctor, operating on the ‘big data’ of his time, made a judgment that the life in question was probably not worth the surgery that was necessary. Fortunately, other doctors who stuck their necks and their careers out as a matter of principle and humanity, chose not to write off that life based on ‘the odds’.  There were too many other unknowns to be so glib and callous.

Thirty years later, the same profession contemplates investing in a simple surgery with potentially devastating consequences and costs to prolong a life that by all reasonable measures was nearing its inevitable conclusion. What was lacking was not data, but perspective. Ultimately, judgment was exercised which required more perspective than data, and the patient was spared one week of agony to pass in relative peace to a place that was destined; a place that too may medical folks fear to accept.  Data does not always = Reality.

But these are matters of life and death.  What relevance to the everyday mundane world of marketing, employment, credit, access to services, access to get back in the country?

If the NSA and Homeland Security can access the Facebook and Google factoid factories and draw erroneous conclusions from data viewed under a distorted lens of dubious values, why can’t all other more anonymous, but no less consequential, johns in the information data-mart do the same with similar consequences?  What prevents the trolls in the political arena from cherry picking a wealth of factoids and mis-representing them into another swift-boating assault on an otherwise worthy candidate for public office.  The original Swift-boaters were a bunch of unsophisticated hacks.  Imagine what you could do with the benefit of DATA and a dossier built on factoids carefully cherry-picked and cooked to perfection.

And what might stop a high-powered executive from digitally knee-capping a competitor for the corner suite with the same dis-information campaign?  Or an academic for that prized Chair. (As Kissinger correctly noted, few can fight so hard for so little as academics.)

Whereas Eric Schmidt and Mark Zuckenberg and Princess Marissa would like you to believe that their messianic quest to order the world in their vision will be a force for good, it is in fact far more dangerous than the NSA. In their universes, there is no presumption of democratic rule and oversight.  Just their whim of the moment, and the opportunities of their market place, in which you and I ARE NOT the buyers or sellers.

The age of information tyranny, institutional and freelance, has arrived.  And its implementation will be so subtle, the redneck devotees of the NRA won’t know who to shoot.

Onward

20140321

Surveillance: A Dollar for your Thoughts

You’ve probably heard more than you want by now about the government’s various nefarious programs to stockpile all the information they might ever need to know in case any of us choose to go to the Dark Side.

You’ve also likely heard that the conduits of much of their information are the various information intermediaries that are becoming ever more entwined in our daily lives in great detail: Google, Facebook, Yahoo, Microsoft, Apple, Amazon, banks, the various phone companies, the beleaguered Postal Service, etc.etc.etc.

If you’re anywhere on the political spectrum between liberal and libertarian, you’re probably concerned about this invasion of privacy which really hasn’t been revealed to us suddenly. It’s been progressing for twelve years before our eyes and under our nose, but we’ve remained mute, or indifferent.  Mostly indifferent, I believe, because we do not yet feel personally threatened by this growing government power.

Now the pieces of the puzzle are coalescing into a more coherent picture, and we’re having an OMG moment.

I would be less concerned about this growing agglomeration of government power if I believed that there is a system of checks and balances in place that assures that its application is effectively overseen.  But there is no rational basis for such belief, and therefore every reason for concern.

I am less concerned about the data than I am about Guantanamo, and renditions, and water-boarding and the Citizens United Supreme Court decision and private for-profit prisons, which say something profound about our political value system and commitment to justice and due process; and none of which require Big Data to perpetrate tyranny.  The data is just icing on the cake.

The risk of Government tyranny is real.  If perpetrated, it could be great in scope, but most likely will be more narrowly targeted to real or perceived security threats.  It was a greater concern to me during the Bush/Cheney regime which specialized in bypassing due process and checks and balances.  But it is disturbing that the Obama administration has done so little to reverse those excesses, and has apparently added to the infrastructure.

The greatest safety against the threat of broad scope government tyranny is the government’s own apparent ineptitude to conduct large-scale security operations (border security) and intelligence operations to effect.  I imagine that if the government really tried to institute large-scale repression in this country, it would find itself confronted with various levels of resistance that would make Egypt look like an Easter Egg hunt.  Further, in a highly complex society such as ours, a government that would turn against its people would depend on an infrastructure far more vulnerable than its supply chain in Af-Pakistan.  And I would like to believe that there are still people in government who are smart enough to understand this and tread lightly.

But here’s my bigger concern in the realm of surveillance.  My bigger concern is with the intermediaries the Government relies so heavily on for so much of their data: the Googles and Facebooks, and Amazons and Apples and banks and telcos.  The Government is presumably focused on maintaining power and security.  But these intermediaries have a much different motive in collecting all the data they do in their own impressive data farms…Profit.  The quest for profit, as we should know by now, has no bounds. Certainly no ethical or moral bounds.

We have at least the illusion of a system of checks and balances and legal limits relating to government operations.  But in the US, what checks and balances exist over these private sector intermediaries in selling whatever of our relinquished data to whomever for whatever purposes. If these intermediaries are unwilling servants of the government in the security realm, they are willing pushers to other economic parasites in the private sector for clear economic gain.  We barely get insight into these relationships, and have little sense of our vulnerabilities and avenues of recourse.

For example, we know that businesses use algorithms to determine which customers can be better fleeced with price adjustments. What data sources feed those algorithms? We know that some insurance policies rate you on more than your driving record.  We don’t know how much information, from what sources, and with what degree of accuracy, may be gathered by a prospective or current employer in evaluating your suitability for employment, promotion, or retention.  What freedom-of-information processes apply? What due process exists for wrongful denial of service or employment or other opportunity apply. There is, in this realm of Private Dark Data, a potential tyranny far more immediate and impactful on personal liberty and well-being than the more likely scenarios of  NSA, CIA and FBI drones and data mining. Private Sector Surveillance is the more immediate and pervasive threat to the average citizen.

A dollar for your thoughts.

Oh, but I forgot to mention. you don’t get the dollar.

Every virtue has within it the potential of deception and tyranny. And every clever tyranny gains ground with the appearance of virtue.

Take the opportunity to kill the power on your cellphone and tablet every so often while in transit.  Drop off the Radar for a day. Make ’em wonder what you’re up to.

It’ll freak ’em, Man!

Onward.

20130714

Appropriate Technology

During last month’s International Festival of Arts and Ideas in New Haven, CT, a participant in a panel discussion on innovation shared an interesting anecdote.  He explained how his company was pondering how to provide a lower cost incubator warming system for premature babies in lesser developed countries.  The high-tech model sold in the US was approximately $23,000 per unit; too expensive and too complex for less developed countries.  After collaborating with an innovations center in its India affiliate, it hit upon an ingenious solution: stripping down the highly automated US unit to a low-tech but just as effective manually controlled unit at a fraction of the price.

That got me to wondering.  If the low tech, low-cost unit works for Indian kids, why isn’t it good enough for US kids?  How much does our obsessive fetish with automated this and high-tech that drive up the cost of service in the US with little or no discernible benefit? To what degree does this mindset account for the fact that we pay a premium for mediocre medical results?

I remember the commencement speaker at my brother-in-law’s graduation from dental school in 1979.  He spoke of how hospitals with state of the art cardiac treatment centers were experiencing higher than anticipated mortality, and longer recoveries to discharge, contrary to their projections.  They ultimately concluded that automating the monitoring of patients to reduce the human contact was having an adverse affect on patients psychologically. Human interaction was as important a component of the healing process as monitoring vital signs.

Then I remembered a video I saw of a Soviet fighter jet that was ranked competitive with ours.  Its maneuvers and performance in flight were impressive.  But equally impressive was the observation by experts that the aircraft could be maintained in the field with a much smaller support complement, and could operate from less developed facilities, making the system easier to deploy to a broader array of combat theaters.

Which brings us to Iraq-istan.  We built ever bigger vehicles to protect our forces from ever more potent, but comparatively primitive, lethal devices, modified on the run by people not prepped at The Point. Why are we trying to build indigenous armies to look and operate and equip like us, when our tactics didn’t win the wars, and they are unlikely to be able to sustain an army in a format we can no longer afford ourselves?

Then there’s the Navy’s Littoral Combat Ship, which was recently reported to be of questionable combat capability. At $670 million a pop, that’s not reassuring, but for the DOD, that’s not unusual.

Then there’s ‘Big Data’, which I have spoken to previously. There are appropriate places for Big Data. But is it a need, a niche, or a rage? Is it a security blanket to cover lazy analysis and a paucity of insight with an impressive pile of data that reveals little?

No doubt, there are marketing opportunities for big data to identify targets of opportunity with the precision of an electron microscope, but how much is the supporting investment worth in aggregate in sucking blood from the bloodless of the broader economy?

A marketing professor once noted a survey of CEOs revealed that most doubted the value of their advertising in affecting sales, but they nonetheless felt compelled to make the investment in it.  Faith based management.  The constant struggle of the data parasites to gain more ‘insight’ into the market and attract more web advertising seems more like a churning of possibilities for would-be advertisers than achievement of measurable results.

In my own professional pursuits, I have often been dismayed to observe how little we use of the technological capabilities we have.  For example, most professional business people use Excel as little more than an electronic piece of paper.  They are substantially ignorant of most of its functionality to make them more productive, and not highly motivated to learn.  Do we need a better Excel, with more options on the ribbon? A different format? More intuitive? A dab of artificial intelligence to anticipate our needs and fill in the logic as well as the blanks?  No. We need to learn to use what we have to better effect.  More managers are adept at using PowerPoint than at using Excel or various database applications that create the content for the presentation. More focus is placed on the image of presentation than the substance of what is being presented. Is that where the focus should be?

And apparently nobody does Big Data like NSA? Does it make us safer? I doubt it. Sweeping everything into a gargantuan pile gives a delusion of control; being able to drill quickly and reliably to the critical core is what matters.  If our intelligence services are so capable, and so well equipped with the latest data crunching technology, why is it that we are continually blind-sided by international events that they are so meticulously scanning?  Wrong data? Wrong systems? Wrong management perspective? Clearly, we are not getting enough ‘bang for the bytes’.

Eventually, our pursuit of technology becomes an unsustainable defense of technology’s promise more than an attainment of its productivity. The perpetual upgrade cycle is a hook, not a destination. As true improvement becomes tougher to achieve, superficial refinements mask mediocrity as progress. Marketing supplants R&D. PT Barnum (“There’s a sucker born every minute, and two to take him”) becomes the management guru of choice.

We are now witnessing a victim of this cycle.  Apple has become a mature company.  It is now more a competitor than an innovator.  It has critical mass, momentum.  Like Microsoft. It can cruise for a long, long time, doing its thing. But it may have passed its moment as ‘disruptor’. Now, as it moves through the tech jungle, bloated with its success, and financially insulated from the immediate consequences of lethargy, myopia and bad decisions, it must wonder what small microbe could bring it down.

*  *  *

By now I hope you get my drift. I’m no Luddite (or so I’d like to think). I love data and respect technology, but I question the pay-back in many instances. Big Data does what only Big Data can do in gene sequencing and climate change modeling.  The Mars rovers and Hubble space telescope are truly remarkable machines, evidence of our best capabilities put to appropriate use. I hope that someday, some large-scale particle collider will uncover mysteries of energy that will make desk-top fusion and unlimited cheap, clean energy a reality.

But I question the wisdom and benefit of constant turnover for the sake of turnover. I question disruption for the sake of profit without progress.  I don’t suggest that we return to some imagined ‘golden age of simplicity’. But we do need to put the brakes on neurotic ‘innovation’ for its own sake.

Or, putting it another way, we now have over $4 trillion in infrastructure investment deficit in the US to upgrade sewage, water treatment and distribution, roads, public transportation, bridges, power generation and distribution, reliable and modern communication to areas lacking it, basic healthcare to those with none, basic nutrition to those with little,…..

Do we really need a better iPad?

Onward

20130704

Big Data, Quantitative Arrogance, and Other Delusions

Don’t get me wrong, I love data.  But I’m a little bemused by the fadishness of the recent obsession over ‘big data’.  Actually, it has been growing for a while in the IT mills, but it now seems to be erupting into the general business press like a new discovery. The new silver bullet of management.  Heard that before?

Big Data, for those who have been away at an ashram for the past five years is the idea of corralling all of an organization’s operational and business environmental data into an information structure that makes it readily accessible and insightful in ways never before possible with that other popular buzz-word: Analytics.

In fairness to both concepts, there is genuine need for both of them.  But, as with so many things in business, the current rush to embrace them suggests the all too typical tendency of management to ‘run with the herd’ without thinking too deeply about where they’re going.  It has the same plug-and-play fantasy of ERP platforms, the be-all/do-all applications that would handle all aspects of the business seamlessly, but often proved devilishly difficult to master and costly to maintain.

This notion was given reinforcement by an excellent book review by Floyd Norris of “The Physics of Wall Street“, a book about the application of mathematical modeling by physicists to economics and investing.  It is bad enough that economists can get the economy so wrong so often, but to really blow it, you need a first class model that only a quant-head can devise.  And where better than the discipline of physics. After all, aren’t business cycles kind of like lunar cycles?

The arrogance of quants and of the business executives they serve, is that everything can be reduced to numbers, and with enough data a pattern can be discerned that a strategy can exploit. As an accountant, that logic has superficial appeal to me. As an auditor, I remain highly skeptical.

My career has spanned the better part of the information age as of this point in time. I remember pegboards and punch card programming when that was wiz-bang. The first accounting office I was in celebrated the arrival in the staff room of our first Wang digital calculator. It was a big money investment at $1,000 in 1970, and we shared it.

Yet for all the advancements in technology, and they have been considerable, I look back over time and sense that while our data has grown exponentially, our information, and our understanding of it have severely declined. Too often we look to technology to provide ‘fire and forget’ insight.  We forget that some of the most important insights that good information can provide are highly nuanced, lost in the regression analysis, or normalized off the radar screen.

I remember reading a Wall Street Journal article, sometime in the late ’80s, which announced that Wall Street firms had begun hiring mathematicians and physicists to do the work previously done by finance MBAs in designing derivative investment products.  Having been a chemistry major before becoming an accounting major, that news struck be as being suspect.  The business environment does not have the precision and predictability of the natural environment that scientific disciplines address.  To assume to take scientific tools out of the Ivory Tower and embed them in the corporate glass tower without a heavy dose of management oversight by business savvy people seemed a tad risky.  I have rarely seen a business plan land with the precision of a Mars rover.  But we have indeed seen some pretty fancy investment products crash with the impact of a meteor.

Scott Brave’s recent article tackles the dilemma of Big Data from a more technical perspective, but makes a similar point. We become seduced by technological complexity, and too often miss the point of it–to run a business better.  In many instances, less can be more.

Most entrepreneurs, even today, kick-start their enterprises without the benefit of Big Data and progress far without it before their management successors elect  to become burdened with it. Their secret?  Not Big Data, but Right Data. And what is Right Data?  Ah! That is the ultimate secret, and there’s no app or algorithm for that yet.  That’s still very much in the human domain.