When validation doesn’t rule

Every once in a while, somebody comes up with a great idea that is really dangerous. Here’s some ammunition you can use to counter one we’ve encountered.

Comments

Data-driven applications like CRM and ERP typically have a mechanism to validate data entries, in order to prevent entries that are out of bounds (such as negative inventories, customer counts in the billions, or pointers to nowhere). A big system may have hundreds of them scattered across the UI landscape. While these validation rules aren’t as universally enforced as true database constraints, they still irritate users because they prevent the record from being saved. The careless user may just move on, thinking that some of their data has been saved when in reality it’s been lost and has to be re-entered later.

Worse, validation rules can cause errors in application code, prevent integrations from properly inserting data, and throw countless errors in test code. So every time somebody asks to create another validation rule (which involves no coding), all the coders have to be called in to test their modules and deploy the updates before that new “nothing” rule can be deployed.

Annoying, yes. And so somebody will say, “we can have a work-around that makes it so validation rules are skipped when the data entry is coming from code, rather than from people.” The idea is floated that next time a user is in a record s/he can amend it so it conforms to the new validation rules. Sounds logical enough, particularly when it comes from an executive.

Fly into the danger zone

So let’s go for it! With new validation rules, we’ll put in logic to check a field indicating “last time this record was updated by code.” If that time is essentially “now,” we skip the validation rule. If not, this is a human entry and the rule is enforced.

Unfortunately, we’ve just volunteered for the Silent Killer.

The first thing to think about is that only newer validation rules will be skipped (because nobody will go back and fix every single rule across the system), so the data quality is strictly enforced in some areas and not in others. And of course, none of this is documented or done in a thorough way, so some data conditions that should be impossible will start to creep in unnoticed.

The second thing to think about is that some bits of your software will know to update that “last time this record was updated by code” field, but others won’t. You can also bet that even within modules that update that field…not every data manipulation will update it (simply because laziness knows no bounds). So this means that some of your components will be able to skip the validations, and others won’t. Processing will continue alright, but outcomes and data conditions will start to get flaky in more and more subtle ways.

At first, nobody will notice much. Typically, the only thing that becomes obvious is that older, un-modified records will go farther and farther out of data compliance. So when a user edits one of those old records there may be a half-dozen validation rule violations to fix before the data can be saved – even if the user’s changes were innocent. And sometimes those things-to-be-fixed are in records that the user isn’t working on (and may not even have access to). Those always generate AYFKM reactions (Google it).

The next thing people might notice is that reports that compare historical records to current ones (such as “this quarter’s pipeline vs last year’s”) will start to yield results that are superficially OK, but misleading. Roll-ups and quantities will be there, but may not “foot.” Data segmentation and “buckets” may look silly, and the data semantics will get blurred for sure. Only careful scrutiny by a diligent data analyst will see it, but management decisions based on historical comparisons will increasingly be lost in the fog of war.

The result, of course, is that confidence in the system insidiously goes down. Nobody blames the users or the sloppy thinking. They just blame the app, and wonder why they can’t really trust the data.

Bad data is never a good thing, but the real danger here comes from code processing errors. Because of erratic/inconsistent enforcement of validation rules, data will be processed in ways never contemplated by the developers. Logic will be inconsistently applied. Code branches that are selected based on ranges of calculation results won’t be followed. All too often, this stuff is happening without the code throwing an obvious error … which means that illogical or impossible outcomes may not be noticed for a long time. But those outcomes are being propagated across the data nonetheless. The more validation rules you have in the system, the worse the scope of these issues.

Eventually, some module will start throwing errors that prevent some part of a business process to complete. Now that it’s way too late, you bring your coders in to investigate the “problem in their damned code.” But if it’s been months since they last looked at it, they’ll face three levels of issues:

Learning curves to remember how their own code works
Learning curves about the validation rules that have been added and the resulting data pollution
The inability to deploy any fix until they have troubleshot and repaired their code, their test code, and the data pollution.

And guess what: these learning curves and delays aren’t additive – they’re multiplicative. Throw in some extra stress because of a screaming boss, and you create a nice little vortex. The bug fixes from yesterday become the #1 cause of new bugs in the system.

This all goes double for integration code that may propagate the bad data into other systems that are expecting things to be well-behaved.

This is about as far away from agile as it gets.

How to avoid a heart attack

Instead, how about reducing the use of validation rules in the first place? Every time there’s an addition or modification to a validation rule, see if you can remove it and replace it with “nagware.” Nagware lets the user and the code always save the record, but it notifies the user that the data is out of spec every time they look at that record. In addition, the nagware sends an email every couple of days to the owner of any record that’s out of spec. They can choose to delay the data update, but in our experience they won’t do so for long. In practice, this relaxed validation approach works in a fairly safe way. Of course it’s not iron-clad, but it contains the scope of any data pollution. This mess all started because the users didn’t want the handcuffs presented by unconditional enforcement of the validation rule, and we give them the equivalent of a mild shock-collar to give them the illusion of freedom.