I cringe when I hear “metadata is data about data”. It is true at a high level, but the reality is that metadata is much more compressive than that. It is hard.
In my last two posts, I spoke about hard data management and hard data modeling. These posts highlight the challenges that surround these disciplines. Both might be difficult, but the surrounding challenges tend to be the hard stuff that tends to steer folks off course. Today, I am continuing the series with hard metadata management.
- Those who know don’t know
It’s common for a data architect to be working on a project team where their fellow teammates do not understand the purpose and value of metadata. That extends from developers to business subject matter experts to project managers. Without their buy-in on collecting metadata, it is relegated to the “do it if you have time” state. Data architects need to be forceful and demonstrate the value that metadata brings to the enterprise beyond the project. - The tale of two definitions
Data architects and our business partners are definers. Data models become more valuable when objects are defined and defined well. Whether it is a business systems analyst, data architect, application developer or data governance team member, they all face the challenge of reconciling different versions and views of a definition. Each of these roles often steps into the role of mediator to resolve differences in definitions. Data architects need to hone their communication and mediation skills to ease this pain. - More holes than Swiss cheese
Metadata management starts with defining data artifacts. The added value is the relationships that a metadata artifact has with other artifacts. Relationships cross tools and platforms tending to be more difficult to manage; often manual. End-to-end source to target mapping is like Swiss cheese with holes of missing transformations and movements. The metadata manager must maximize the use of tools that automate the connections of metadata artifacts. They must perform a cost-benefit analysis of manual connections that maximizes expanding the metadata collection while respecting resource budgets and constraints. - Can’t keep up with the pack
“It can’t take you that long to define that column.” This is the same problem that data architects have with data modeling. Project managers and team members tend to time box metadata artifact creation based on their view of metadata. Good definitions are rich in context, examples, formulas and detail. This usually involves collaboration with other resources. It takes time; time that is unseen by the project team. The data architect must be agile in metadata management while working with project teams to widen their view of metadata management and its value. - Makes no sense to me
Imagine you have the perfect metadata management tool that cataloged all your data assets, relationships, and transformations. Sounds like the perfect world. The problem is now how to present this wealth of information to a varied audience in an intuitive and friendly manner. The metadata manager needs to invest adequate time in understanding their audience expectations and abilities. The devil is in the details applies to metadata deployment strategies that encompass good user interface design, adequate training, communication, and support. If it makes no sense, it will not be used.
Tom Bilcze
