We all use(d) third party software, privately or business-wise. Whether for taxes, HR, private banking or accountancy, nothing escapes digitalisation. Our smartphones and cars are loaded with software to which we entrust nothing less than our lives. Tomorrow, AI will connect the missing dots for our governments, our businesses and our private lives. Software are literally the default position. And if it goes really wrong, what recourse do we have against it? Your car braked in the middle of the street.. you must have done something wrong. We like to pretend that these software are flawless, infallible. Because the alternative would be too scary.
It is scary, unless we reverse the way software is currently operated and fielded.
The core of the UK Post Office scandal was the wrongful prosecution of individuals based on a faulty EPOS, cashier, tool, named Horizon. Without the presumption that a software is immanently correct, the lack of any kind of proof would never have led to prosecution.
The blatant lack of empathy, the recklessness, the incompetence could be easily solved by appointing a better management team. But at the heart of it, the outcome of the legal investigations would have been the exact same without a chance insider, a whistleblower.
That in itself cannot be the solution. And it will happen again somewhere else, and probably already does. The media circus is moving in on social media as the problems hit very public figures: Taylor Swift, Elon Musk,… But for the rest of the technologies, there is much less star power.
Much of our lives, personal or corporate, are software driven. We must accept its characteristics, implications and seriously address this.
This article is the last in the trilogy covering the UK Post Office scandal. You may wish to read the first two as well: The Post Office Scandal: Legacy Software, Legacy Management and The Post Office: Management Smoke and Consultancy Mirrors.
Any IT system is in constant flux
IT is forever the future, and so it is never future proof. Whichever system you use, you sign up for continuous maintenance for as long as it runs. Hardware and software specifications constantly change – just for storage, we moved in 20 years from physical files and binders to zip disk, to hard drives, off to the Cloud.
This also means that we lost much of our information: 80% of the scientific data from the 90s is gone. Data format, storage formats, underlying database software… all of that has actually made our data infrastructure incredibly fragile. Faster, bigger but also ephemeral.
What type of raw information will be available to research tomorrow? What records from 20 years back would be admissible?
Technology means constant adaptation, which leads to an increase in the chances of errors, systemic failures, and nothing short of a cultural risk.
You need to have a structural process to review constantly your technology architecture. The fact that the Post Office ran the same EPOS software for 20 years without anyone hitting the panic button is staggering, but not unheard of.
Any system is a transitory solution and cannot be seen or understood as an immanent information source. Software failures cannot be a post-it on a scrum management white board.
Statistically, any system will fail
No product ever has a 0% failure rate. Not even NASA spaceships in the 70s. And neither are software.
Basically, something that can never happen will happen. Your technological management must start with this assumption, and build in a fail-safe (for example keeping hard copies of legally critical documents) plus a constant critical review processes.
Software can fail technically. For example, your software may track transitions wrongly, as in the EPOS Horizon software of the UK Post Office. But it can also fail in its mission, for example in tracking a particular event or function.
Commercial team management software will streamline your team’s workload, avoid “redundant” commercial reports, and improve your and your team’s work-life balance. That was the pitch I was given to roll out an “industry gold standard” piece of software. People in the affidavits and on the stock pictures were smiling while filling sales reports. As you probably experienced as well, not only did it add 15% pure reporting workload, but it also created unusable reports – anyone with half a neuron had quickly understood how to “manage” the reports.
Even if the software would have been 100% accurate – which it was not – the data collected and the reporting looked quasi-legal (this is your monthly financial report), but will remaining entirely dependent on discrete human data entry.
Beyond the pure accuracy of the software, it is always necessary to have a healthy pragmatic look at how likely it is to be accurate or usable, and derive the necessary fail-safe and required management.
Failure is not a challenge, it is a problem
Technology often presents itself as a kind of prototype. It is the spirit of the shed, the start-up attitude, the lone genius entrepreneur… the hilarious Blue Screen of Death, batteries overheating, graphic cards melting, … all IT myths and legends.
They are fireside stories by now, and no-one expects them to be acceptable. The implications should be taken onboard.
Problems, bugs, errors will happen. At its heart, a software is humans writing instructions for a machine to execute. Both can fail. To see what actually runs in the background of your computer or smartphone, just hold Ctrl + Shift + I (capital i) next time you use Google. But it cannot be the default position.
And yet the overall industry still seems to take for granted that they can get away with releasing faulty software.
Look at video game releases for high profile proxy. Bugs at release seem natural, part of the process. Until “unsinkable” blockbusters just fail to reach their audience, and the once-unbeatable video game industry slowly shrinks. A game revenue peaks in its first week. Combine this with a highly engaged social-media audience, and you would expect that the industry would react. Not so sure.
So, despite the corporate meme, you can NOT embrace failure in your software or your software rollout. You will lose both external and internal customers. Tech sits today at the heart of your company and your private life.
Even if the software is “industry gold standard”, it has to match your life, your privacy, your company mission and give you the ability to intervene and fix it when it fails. As it will.
A software is only something you manage at arm’s length if you devolve responsibility. Just looking at the Post Office case, expecting that it will not turn on you as an easier fix is irrational.
You cannot offload responsibility to the system; you need system redundancy
Over the past 20 years, how many companies moved to IT systems for HR, finance, sales team management? Many? All? For most companies, they are de facto the management decision process at scale, alerting, highlighting, focusing.
As an individual, your bank accounts, your health, your taxes, your diplomas, your pay, your pension, depends on software, apps, websites – all coded by humans. For companies, it gives the illusion of a real-time company management, business-sim like. To the individual at the receiving end, they seem very much like, “The Truth”.
And yet, any and every reporting system should only ever be taken as a hint, a presumption, but not evidence. The presumption should be as well that the records could possibly be inaccurate. Regardless of vendor, state entity.
Is it an old software?
What is the share of legacy software?
The world-standard German railways run their systems on MS-DOS and Win 3.11 and 19 US States unemployment records run on COBOL. The remaining coders for both, if still alive, have to be found in retirement homes. How many other tax systems use Excel files, Access? Because it is rare to re-develop from scratch.
And then there is the physical maintenance. Belgium has no less than 4 web portals for part, or all, of my health information. Each runs on its own structure and infrastructure. I am still a French expat in the UK, in Singapore and or in Belgium, probably because the files do not transfer to a central repository. Who would have guessed that expats change country. But in India, failing to pay your taxes online to the right federal portal means that you cannot leave the country.
The answer? Often PICNIC. Problem In Chair, Not In Computer. Whether hardware or software, IT is never infallible.
The First of Azimov’s Laws states: “A robot may not injure a human being or, through inaction, allow a human being to come to harm”. – Neither should a software.
Processing personal or legal data must always be optional and informed
How many people do read the End-user license agreement (EULA)? No-one.
We all know that these are contractual engagements, but we also presume that, somehow, magically, “Justice will prevail”.
Ultimately, the one thing we should always be able to do while using a software, is not selling our soul. Basically, any EULA must actively seek your approval to use personal data. Any EULA should also give you the option to use the software without agreeing to share any personal information or terminal information, even in some marginally downgraded capabilities, and however much you are assured that your personal data will be “aggregated”.
The next generation of AI is already here. My bank nicely let me know today that I spent more money on food and less on restaurants this month. They can help me with that. They can even give me good prices on a list of friendly restaurants. Maybe they could process my tickets next and analyse what recycling strategy I should use? Or if my carbon footprint to procure food is compliant.
Despite anything that was said, China actually was first to step into the future with “social credits”. Europe is clearly trying its damned best to catch up. For the Greater Good.
So, we do need to have a firm opt-out option of any data collection and processing in the same way that we need the right to defend against directed self-incrimination.
We need to review the burden of proof
As societies, hard physical evidence used to be the proof, the evidence. Often counter-checked through witnesses: signatures, stamps, seals were the basis for centuries. With the electronic society, we moved to presumption of evidence, with emails, signed PDFs, etc.
All of these are admissible, but they don’t have the absolute value of a signed and stamped contract.
We saw that technology is by definition transient. There will come a day when no-one can read a PDF, in the same way that Betamax, laser Disks and V-CDs can’t be.
Already today, when using electronic data, such as emails, we have to make more efforts to prove our claims. It means that we accept the relativity of the evidence we have: emails, records, screenshots, any piece of electronic data can be manipulated. Yet, instinctively, we still trust that “justice will prevail” and decide to trust the given evidence to facilitate the transaction.
We will not reverse digitalisation as the benefits outstrip the risks by far, but we must then adapt our management and technology culture and practises to the same legal and cultural shift that happened in the legal circles: evidence, even collected by supposedly watertight IT systems, are only one part of the evidence we have to collect.
In the case of the Post Office, using at least a second accountancy system and possibly tracing the sums supposedly stolen would have both cleared the sub postmasters and strengthened the financial reporting.
Reverse the burden of proof
That all sounds very technical and very legalese. And yet it is very basic: you have to prove to me that I was wrong, not just by showing me some standalone electronic record, but a fully-fledged investigation.
If not avoiding mistakes, the weight of it should create sufficient traction to force an institutional reaction. The company or the government have to establish strong multi-sourced cases to claim. In its current application, you as an individual have to prove that the system went wrong, and this is practically mostly impossible.
In the case of the Post Office, the only evidence was the Horizon records, the deus ex machina solution, a whistle-blower, and it took years to revert the convictions.
You cannot hide behind the systems’ results. You are liable for making your tracking system.
“That is what the experts told us” should not be able to stand – not in Nuremberg, not in Bracknell.
To remedy this standard status in the XXIth century, we need to REVERSE the burden of proof to reflect the merging of physical and digital worlds into a fully merged reality.
This article is part of a trilogy covering the UK Post Office Scandal. You may wish to read part 1 and part 2 as well:
The Post Office Scandal: Legacy Software, Legacy Management
The Post Office: Management Smoke and Consultancy Mirrors.
Get the latest stories of Pascal Bollon in you Inbox
You can follow Pascal Bollon on Reddit, Facebook, LinkedIn, Spotify, Whatsapp and Instagram. Do leave your comments.
The post Don’t let the Post Office Case Happen to You – Fireproof Your Business appeared first on MNOI.