This data will kill you

Let’s face it, nobody wants to hear about risk and problems.  But it’s only when they are ignored that they get really dangerous.  So let’s take a little tour through your CRM’s data to understand the risks that live in the dark corners of the system. 

The good news is that most of the system’s data is not dangerous, and the better news is that if you manage the system well you can be vaccinated against the most important problems.  

The data that will cost you

The dirty little secret of IT is that data is never free, even if you are able to obtain it for free.  IOW, all data will cost you.  Problem is, you rarely know how much.  Data almost always comes with externalities, initially invisible costs that become important over the long run.  The clearest example of this is email, where any number of defendants can tell you “more is not always better.” 

Whenever you’re evaluating a new data source, quantify the 3 or 5 year TCO of the data.  For example, in a CRM system a Lead might cost $1 or $100 or more per record to obtain.  Cleansing, deduping, and updating that Lead might cost you another $5 a month, but if you don’t do that work the Lead’s value goes to less than $0.  Why?  Because users don’t know the data is rubbish, they depend on it’s being correct, and they end up wasting time.  The externality here?  System credibility gets diminished as users no longer trust any of the data and executives think the reports are garbage. 

[Related: What price CRM data quality?] 

Nobody can afford to be perfectionistic with data quality.  But you ignore data quality at your professional peril. 

Best practices:

• Don’t add new data sources to a CRM until you’ve characterized their inherent data quality, cost of ownership, and long-run value.
• Understand that data maintenance is a budget line-item that doesn’t go away until the data does.  It’s just like the dentist says, “only floss the teeth you want to keep.”
• Prune data records that are obsolete — but that is usually calibrated in years.  This means you have to develop policy and governance around each table in the system, and an expunging/archiving process that is documented, reliable, and funded.  You’re probably going to need a data steward.

The data that could have saved your butt

It never ceases to amaze me how few CRM systems have adequate backup, audit trail, archiving, and expunging processes.  To be adequate means “good enough to support you in a legal or regulatory enforcement action.”  Even if your business is lucky enough to be in an unregulated industry, law suits can come from a lot of different angles…and when they do come, the things you need to be able to prove are typically years in the past. 

Think your backups are just fine?  Check out this video about what actually happened to Pixar. 

So, what are the key data to maintain for later discovery?

• Wrongful termination:  user login history, system administration audit trail, Attachments, Contact/Account/Opportunity/Contract/Activity audit trails, etc.
• Improper sales tactics:  Opportunity history, call history, case history, attachments.
• Intellectual Property Theft:  Lead, Contact, Account, Opportunity, Quote, Contract, Case, Attachment and Activity records; report run history; data download history; user login history
• FTC / Spamming suits:  Lead, Contact, Activity, Campaign, and email blaster records.
• Product Liability:  Opportunity, Quote, Pricebook, Case, Attachment, and Activity records
• Investor suits:  Opportunity, Forecast, Case, and Attachment records 

Best practices:

• Create a written backup, audit trail, archival, and expunging policy for each table in the system.  Make sure you take into account the system’s internal “data horizon” for each relevant table, so you capture data before it is auto-deleted. Review the policy document with your company’s attorney, and make sure they understand that the system contains company and customer emails (unless you’ve explicitly excluded that from the system). In the interests of data provenance, your backup/archive media should be write-once.
• Have a budgetary line item included your annual funding exercise…and make sure the process actually does get funded. 
• The execution of that policy should be a measured MBO on one or more person’s job description.  

The data that can kill your project

When developing a spec for a project or an Agile Sprint, the reflex is to focus on features and functionality.  The problem comes when you ignore the invisible F—the foundation, which can be rotten.  

[Related: CRM backups or audit trails? Yes, please] 

What’s going to kill you?  Here are some examples: 

• Always:  data quality and meaning.  Will dupes mess up your code or processes?  If your project’s logic depends heavily on filtering and grouping, are the field values accurate and consistent?  Do different departments ascribe different meanings or criteria to pick-list values?  If so – and this is very often the case – are you planning to code around the variances?
• With integrated systems:  Accounts, Opportunities, and Contracts.  Almost without exception, Accounts (aka “Companies” or “Customers”) are the most problematic.  It’s amazing how difficult and expensive it is to get a solid Customer Master.  What will happen to your project schedule if you can’t depend upon a clean, meaningful Account table?
• With hardware and SaaS systems (such as the phone, call director, or external application):  every one of the tables they touch in the CRM.  These external systems are inherently loosely coupled and may even be asynchronous.  If there’s any flakiness in your data (like, “Viet Nam” vs “Vietnam” vs “VIET NAM”) or unpredictability in their data flows (such as partial records or occasional dupes) your project is going to spend some very painful time in detecting and troubleshooting issues.  I’ve seen this kind of issue totally dominate a project, even though it was never even contemplated when the SOW was written.
• With configure, price, quote, accounting, billing, and contract applications (even when “native”):  the CRM price book.  This is one of the smallest tables in the CRM system, typically with only a few thousand entries.  But just getting all the right people to agree on product lines, product names, price points, discounting schemes,  international prices, bundles, and bills of materials (BOMs) can take you longer than all the technical implementation work.  Even after they agree in principle, a bunch of people have to sign off in agreement.  This trivial little table causes non-trivial delays. 

Best practices:

• Identify the data dependencies for your project before you commit to anything.  Identify the three data problems that could do the most damage to your estimates, and build some mitigation strategies into your project plan.  Data issues can easily double the cost and time of any deliverable.
• Make sure that there is a solid owner for fixing data problems and identifying what’s the “right answer.”  Typically, this will be a non-technical person on the business side.  Make sure that that person’s boss has assigned the tasks to that person, and it’s a measured MBO (rather than some sort of “nights and weekends” task).
• Put in disclaimer verbiage in the project/task definition indicating that (1) the task schedule/budget is contingent upon data being correct, complete, and meaningful, (2) unresolved data issues are the obligation of the client organization(s) to fix, and (3) unresolved data issues will result in at least a day-by-day slip and possible budgetary overages. 

Data issues are often the long pole in the tent, the ultimate constraint that makes budgets and schedules unachievable.  So make sure to shine a light on that pole before you even start work on the project.