5 steps to begin collecting the value of your data
It’s not new that data is called ‘the new oil’. Organisations that are able to collect, organise and combine data effectively are in a good position to start creating new value. The question is when you can collect the value of data and how to get there.
The Economist
If you are seeking to leverage the value data for your business, you’ll need to start managing your metadata. Metadata is the key enabler to help you optimise your processes, gain and stay in control of risks and enable new value creation. This blog gives a quick introduction into why you need metadata and where to start.
The value of your data is unlocked through metadata
Enable discovery and sharing of data shortening search times
Protect your investment in data due to staff turnover and enable reuse
Improve understanding and decision making with high quality data
Mitigate your risks and limit liability 
Control what your data is used for and where it goes
Increase effectiveness & efficiency in collaboration 
Reduce costs and create new value with faster development & innovation
These examples of value all sound nice, but aren’t easily achieved. Therefore we’ll go into more detail on what we need to organise to build the foundation of this value.
5 steps for managing your metadata
Working with large organizations with complex IT landscapes and data exchange, we’ve found there are couple of generic steps to take. When achieving this in a sustainable manner, you’ll be able to collect value from your data more rapidly:
1. Find the data your organisation needs
2. Understand that data
3. Know who is responsible for that data
4. Able to follow to source and end users of that data
5. Trust the data so they can use it without hesitation
So here is a way to approach these steps in practice:
1. Make data findable
As a start you’ll need to set priorities. Determine which data sources are part of your core operational & information processes. Focus on on indexing these first. This can be on a high level and doesn’t necessarily mean you need to index all attributes of all data objects. Just start by adding descriptions for data sets and add business tags to make them more easy to find. Further, people looking for data need to know they have a complete view, or at least know through which sources they are browsing.
2. Add descriptions & definitions
Next step is that they need to understand what they are looking at. For the most important data objects and key attributes you’ll need to add descriptions.
3. Make transparent who to contact
When people have found data that they think they can use, they’ll first need to get in touch with the owner to get access to the data. Governance metadata is therefore an essential part in creating value out of data.
4. Document data logistics to gain control
When people start using data, they’ll become dependent on changes in the data production flows. Changes need to be managed on their impact and thus it is necessary to follow where data comes from and who is consuming it.
5. Analyse data quality to make data usable
Finally this all adds up to trusting the data. Data quality measurements will allow people to assess whether they can use this data for their processes.
What metadata to collect
Now to make this work, we’ll need to collect metadata that help each of these steps. It will help discussion when you can categorize different metadata elements into groups. Lean Data suggests following 4 categories:
- Technical Metadata
- Business Metadata
- Operational Metadata
- Social Metadata
The technical metadata is the foundation. You’ll need to gather and organise this before you can make sense of it (see below for examples). The next step is adding the descriptions and governance metadata as business metadata for understanding and control. Now you can go to work and improve the data processing by capturing and monitoring the operational metadata. Now we have reached the stage where we can really start building value from data. Up to this point we have only enabled our ability to monitor and control our day to day business operations. The real value of data can be measured and improved when you start collecting the social metadata. Social metadata will tell you the actual use. Taking control of that data will help you developing your data as a business asset.
The real value of data can be measured through the use by its consumers
Lean Data
The actual use of data is a clear indication of whether the data is valuable. Measuring this social metadata helps you put focus on this data. Furthermore other valuable data sets that are not effectively used can be given attention to improve its use.
Metadata types within the 4 metadata categories
To get started, below is an overview of metadata types with examples that you could start organising. Start with a (data) process analysis and use your business objectives to determine your metadata needs:
Technical Metadata types
Connectivity metadata
– Source application name, location
Technical metadata
– Technical table & field name
– Data format (e.g. text, SPSS, Stata, Excel, tiff, mpeg, 3D, Java, FITS, CIF)- Compression or encoding algorithms
– Encryption and decryption keys
– Software (including release number) used to create or update the data
– Hardware on which the data were created
– Operating systems in which the data were created
– Application software in which the data were created
Structural metadata
– File relationships (e.g. child, parent & datasetgrouping)
Preservation metadata
– File format (e.g. .txt, .pdf, .doc, .rtf, .xls, .xml, .spv, .jpg, .fits)
– Significant properties
– Technical environment
– Fixity information
Business Metadata types
Business initiative metadata
– Business case (reference, contacts)
– Request purpose
Governance metadata
– Owner of the data
– Data purpose limitations
– Business rules & data retention
– Data classification (AIC & Privacy)
Descriptive metadata
– Name of creator of data set
– Name of author of the data
– Title of document/ data
– Data (as)set name & description
– Object name, description & definition
– Attribute functional name, definition & description
– Location of data
– Size of data
Administrative metadata
– Information about data creation
– Information about subsequent updates, transformation, versioning, summarization
– Descriptions of migration and replication
– Information about other events that have affected the files
– Access rights metadata
Operational Metadata types
Execution metadata
– Whether the process run failed or had warnings
– Which database tables or files were read from, written to, or referenced
– How many rows were read, written to, or referenced
– When the process started and finished
– Which stages and links were used
– The application that executed the process
– Any runtime parameters that were used by the process
– The events that occurred during the run of the process, including the number of rows written and read on the links of the process.
– The invocation ID of the job
– Any notes about running the process
Monitoring metadata
– Actual status of a data processing job (in progress, error, paused)
– Current runtime & estimated end time
– Completeness flag & percentage
– Data Disposal verification
Social Metadata types
Use metadata
– Circulation records
– Physical and digital exhibition records
– Content reuse and multiversioning information
– Search logs & parameters
– Data search results, filters and clicks
– Use and user tracking
– Data tags
– Excerpt / summary
– URL
– Number of users & viewing time
– User review & ranking of data
Controlling metadata
– Data access users & time
– Frequency of data access
– Time between data access attempts
Thank you for reading! If you would like support for your organisation, feel free to get in touch!
In another blog I’ll address what solutions and capabilities you’ll need in your architecture.