by Steve Gordon
Early in my career in data processing (yes, that’s what we called it then), I was working in the real estate department at a fast-food chain as part of my graduate work requirements. My job was to develop a site location model to assist the corporation with finding new, successful sites, to compete and grow in the marketplace.
The role I had meant I had to use a time-sharing system from Dun and Bradstreet/NCSS, one of the leading commercial time-sharing services of the era. I became familiar with the tools that were available on the NCSS system and started my work to develop a statistical model that would take all the variables collected by the real estate department, such as site access, parking, demographics of the neighborhood and other potential predictors I felt were useful in building a model. NCSS had many of the statistical tools I needed as well as a database system, Nomad, that made loading this data easy.
One day, once I had loaded as much data as I could for the state of Utah and the sites that were under consideration, I let my model run and noticed a series of $$$$$$ start to appear on my terminal. After about 30 minutes of constant dollar signs, the model came back with an answer, and everyone was satisfied.
About a week later, I got tapped on the shoulder by my boss who asked me if I could come into his office for a quick discussion. He brought out a piece of paper from NCSS and showed me that my one model run had cost $1,400. He explained that the ARU or Application Resource Unit, which was a mystery resource calculation that allowed NCSS to bill the company, was off the charts in how it was used. Fortunately, my boss didn’t fire me for this rogue calculation as the model was effective, but he suggested that I look closely at my code to determine if there was a way in which it could be modified to “cost less”.
As time-sharing costs rose, corporations began moving to on-premises, in-house systems — and that shift eventually killed off time-sharing altogether (NCSS is a mere memory now). Over the past decade, corporations have been spending millions of dollars to reduce costs and move as much in-house processing as possible to the cloud, where they can pay a version of consumption pricing. Basically, paying only for what they use rather than having flat fee data costs.
Can someone then tell me what the difference is between the way my graduate school model worked and how it would work if I ran it today in the cloud (other than costs)? Time-sharing was no more than a consumption pricing model with a host of tools and applications you could call to run jobs – no different than the current way things work with Google, Azure or AWS. The only difference that I see is that there is a huge middle level of fast and secure communications complexities and you can load as much data as you want (and pay for it of course). I was limited back then in the amount of data I could load as the cost was prohibitive to load even a gigabyte of data. Now, there are no limits to how much data or where your data comes from as costs are substantially less and you can’t even buy a hard drive small enough to hold just a gigabyte.
The lessons learned in the time-sharing era (cost governance, resource efficiency, unit economics) are just as relevant today and largely being relearned the hard way by cloud adopters. Did we make progress or have we gone full circle?
About the author:
Steve Gordon is Managing Director of AlignBiz Consulting. He holds a B.S. in Geography and an
M.S. in Planning, disciplines that inform his belief that the most powerful thing you can do with a
complex system is map it accurately. With 35+ years of technology leadership experience spanning
IBM, Teradata, and AWS, he brings deep fluency in the enterprise analytics ecosystem and the
vendor, tool, and buyer dynamics that shape mid-market and enterprise technology decisions. He is
a TDWI speaker and the creator of Technology Cartography™.
www.align-biz.com