Utopia 1.0 – Dealing With Deletes

This is an extract from a larger article by the same author on how to define a dream system

With my varied project management experience in the nuclear, oil & gas, utility, and
manufacturing sectors, I have come to a definitive conclusion about deletes.

Don’t do it!

Now when talking about deletes and I say don’t do it, I mean do not delete off the face of the earth ANY and ALL morsels or scraps of information relating to content. Also – don’t delete the document(s) itself/themselves either.
Let me explain my theory in more detail but first let’s put this into context.

In the early/earlier days of computing, floppy and stiffy disks (remember them?) and even HDD’s, (which means Hard Disk Drive for those who have forgotten or are too young to know what it is I am rabbiting on about,) were all of small capacity so space was at a premium. (My first PC, a Compaq ‘Luggable’*, had a 5Mb HDD and a TWIN 250k disk drives!) * Compaq Portable for sale to a good home. Museum piece. Offers? Contact the author who is also a museum piece!

Today, we almost do not care about drive space, or at least we don’t have to worry about it at work as it is one of the IT Department’s functions to carry that burden. Alternatively, if it is your home PC you just fill up the drive until Windows grinds to a halt or complains about lack of disk space.
Managing drives is almost a full-time job for somebody in IT. For you there are a huge number of tools available to deal with the huge quantities files likely to be held. Each scenario has its own set of problems.
The common issues with files are ….

• Duplication through use: Where a file or file set has been created using ‘Save As’ by a user. This does not mean all the files are simple duplicates, as some may well be subject to ongoing change, (possibly at project design or production level,) so a copy of the original, or first, or best, etc., files are made as working copies for the users.
• Replication: This is where an entire collection of files has been copied verbatim, by a user – not
  necessarily automated, (the process, not the user,) perhaps for use as a backup for some purpose such as an imminent project handover, or to prevent further change from ever taking place. (e.g. Redundant process or equipment.)
• Backup: Always an IT function at work, this is the best-practice workflow where a copy or copies of a list of pre-defined folders and their files are made. This is usually an iterative process over an agreed period of time and will follow very strict (internal) company rules.
• DR Replication: The frequency and depth of the Disaster Recovery process will be defined by the IT Department, and will take into account a risk assessment of the organisation’s need for immediacy of access in a DR scenario. e.g. Because of a disaster event such as a fire, evacuation, computer virus strike, DOS attack, EM pulse strike, Alien invasion, etc.

Leave IT to deal with all the above and take comfort from the fact that when designing your Utopia system, you and/or your analyst(s), should have put in place a corporate-led, corporate-approved and adequately communicated to the masses dictate which states something like:

“… if a document, OR ANY RELATED FILE forms ANY part of a controlled revision or version-led change process – it MUST be kept in a secure and fit-for-purpose document management system.”  question is no longer visible to general users. All references and the document itself remain intact. The status of that flag can be reversed only after authorisation and activation by a suitable qualified knowledgeable person.

Please bear in mind that when I say ‘fit for purpose’ I am not talking about chucking a few files into some ‘vaguely-configured-by-Jim-the-geeky-chap-in-accounts-who-knows-how-to-do-it’ SharePoint ‘bucket’, and then letting users and Microsoft loose on it, to perform what MS laughingly call document, content, project, and security ‘management’. MS SharePoint OOTB (Out of the Box) is NOT such a document system. It is instead a (useful) collaborative system for other non-critical things. Yes, MS SP has its uses, but it is a bag of blunt tools where you need to get somebody in to sharpen the bits required to get it to function correctly for what we in the trade call ‘proper document management’!

All of which means that when it comes down to looking after your critical content, managing your change management, and ultimately looking after what is essentially a crucial part of your IP (Intellectual Property), – SharePoint is about as useful as a fireguard made by Lindt. [Rant over.]

So, you have a system, a good system, and let’s assume it is called Utopia 1.0. The thorny subject of Deletes will, (should/must/d**m well better,) come up in the initial Functional Specification.

Consider the following carefully. To me, if thinking about ‘what kind of delete you want to implement’, my view is to consider only the one option – and leave it at that. Lets’ call it Virtual Delete.

But what is a ‘virtual delete’? Well, it is exactly what it says in the text – a delete which is virtual by its very nature. In actuality, the way it would work is this … If an authorised user, not just any old user but an authenticated key-user or super-user, IS allowed to delete, all that happens in the background is that a simple database flag is set so that the content in

Yes, I can almost hear you saying, ‘but there are other types of delete’. I agree. However, this is crucial data we are talking about here, so I would stop at virtual. Look no further. Using another form of delete can leave you and your organisation open to all manner of potential nastiness.

Let’s now look at what these alternatives are, and why they are, or can be – that nasty ….
Delete of file – retaining attribute: Why would you? Simple question, complicated answer? There is no reason I can find where you would delete the file and retain the database attribute(s). Once the file is gone from the system it is gone from the system. OK you can possibly undelete from the actual HDD, but this is not that kind of system, and you will likely have no access to the HDD anyway.

I use my own acronym – DRAAT. (Discovery, Responsibility, Accountability, Auditability, Traceability) and over the years since I coined it, I have added another T at the end – T for Transparency. With no DRAATT protocol in place you might well be litigated for contractual or corporate reasons, or, if somebody is injured or worst case scenario dies because of a delete, you are in for a world of real and highly expensive pain.

Delete of file – and attributes: In a sense – this makes more sense. If you are going to get rid, then get rid completely, but again I raise the question – why would you? Again I say that you and your
organisation are opening a can of worms. There is no DRAATT. A delete of a file and the attributes is corporate irresponsibility within what should be a highly controlled environment.
Delete of history: (Oldies will get this) Deleting the entire history of a document and all database
entries is akin to Colonel Oliver North standing shredding documents to protect the Contra Rebels.

The history is part of DRAATT – so don’t delete it.

I could go on (and on, and on) but if you feel that strongly about what I have said, positive or
negative, – email me. I love a good debate.

About Utopia 1.0
A fictitious system dreamt up for a series of articles in IDMi Magazine in 2020. The idea was to
propose best-practice and even sometimes improbable functionality, to spec out the ideal document
management system.