Thinking sensibly about lock-in

Welcome to the 2nd half of 2021: particularly to those of you who found me via Peter Taylor’s Pottyville. I knew Peter drew but never had any idea of his scope. It’s great to see his creativity given a suitable canvas. Thanks for the shout out Peter.

So lock-in: in every role I ever worked we struggled with how to manage it. Start with a definition: lock-in is the condition of being unable to (easily) migrate away from a product or technology. One is ’locked in’ to the choice and subject to the whims of the owner. There is always a way out, but the costs of doing that might seem unbearable until, like a boiled frog, one is already cooked.

A classical example: you’re using the Oracle relational database for your application. You are, to some degree, committed to that product. If Oracle double the price of support on renewal, you have to decide if you want to pay the support, run without cover or incur the cost of swapping the database out.

A more modern example: your data is all stored in AWS and you do all of your analytics using Amazon Athena. You’re not getting the performance you’d like but there is a switching cost to move to something else.

Vendors have inherent incentive to lock you in: especially with subscription based pricing, the vendor doesn’t want you to churn away when renewal comes up. The more you use their secret sauce features the more locked in you are and less likely you’ll walk.

It’s not just commercial software though: when the team builds on an open source package there is some degree of lock-in. The costs are different (will the maintainers keep maintaining? can you get talent that wants to work with the technology?) but some form of lock-in remains.

Which leads us to: The no lock-in fallacy

One thing that we need to be clear about: there is always lock-in in your stack. Short of designing your own silicon and working upwards, you’ve already made a set of choices that commit you. Be wary of architects or teams that strive for no lock-in.

So what do to? In my mind, there are three complementary strategies that can be deployed:

Abstract

To abstract is to build a layer that you manage over the artifact you wish to avoid being locked in to. For example, classically, one would build a data abstraction layer over the relational database and use ODBC or JDBC to abstract the database implementation underneath. The idea being that you can then ‘swap out’ the database. If you’re building a product that customers are going to deploy this is great strategy: you’re letting them choose which database to use.

A more modern equivalent is packaging using Kubernetes: a containerized application can, in theory, run on any cloud provider that supports containers.

There are downsides. Fundamentally to avoid one layer of lock-in you’ve locked into an abstraction layer on top. Not only does that layer add complexity and latency but it ‘feature masks’: key features of a particular product may not be exposed to you through the abstraction layer.

Embrace

If abstract is all about insulating yourself from a technology then the opposite is to fully embrace. The most powerful features in a product are often the newest and therefore the least likely to be standardized. By embracing them you gain the maximum value from the technology and quickest time to market. Further, you place gentle pressure on competitors (‘we’d love to look at your product but it doesn’t do x’) to up their game, giving you future options and negotiating leverage.

The downside is that you’re all in. Use every piece of Azure Data Factory? It’s not going to be simple or quick to switch to AWS Glue.

Insulate

The final approach is to insulate your choices to minimize blast radius when something needs to change. Clear well defined interfaces between major components in the enterprise stack give you the ability to change incrementally without breaking everything else. The insulation approach will vary depending on where you look in the enterprise: an API strategy built on well worn open standards such as JSON Schema and REST, or SOAP, or Linked Data. A data lake consisting of parquet files in a clearly defined folder hierarchy.

Fundamentally there is no ‘forever’ solution in technology. It moves too fast and improvements are impossible to ignore. Change is therefore constant, our role as technology leaders is to make smart decisions that don’t corner the enterprise yet balance the need to deliver. Good luck!

Featured image by Sam Cox, some rights reserved. Source Flickr