Metadata publishing

Metadata publishing is the process of making metadata data elements available to external users, both people and machines using a formal review process and a commitment to change control processes.

Metadata publishing is the foundation upon which advanced distributed computing functions are being built. But like building foundations, care must be taken in metadata publishing systems to ensure the structural integrity of the systems built on top of them.

Definition of metadata publishing

Published metadata has the following characteristics:

Metadata structures available to the general public on a public web site or by a download
There is a documented review and approval process for adding or updating data elements to the system
New releases are made available without disturbing prior versions
A publishing organization that makes a commitment to change control process

Benefits of metadata publishing

When classifying benefits of metadata publishing two groups are usually considered. External parties are usually consumers of information that are not part of the publishing organization. Internal parties are usually the various business units or departments within an organization.

Benefits to external parties

Allows external systems (both people and agents) to have a clear understanding of the semantics of data elements in a system
Allows third parties to build semantic maps between data models and import and export data between systems
Promotes service oriented architectures and allow horizontal sharing of information between traditional information silos
Allows systems to participate in accurately indexed and federated search processes

Benefits to internal parties

allows parties from diverse business units to agree on shared data definitions and separate department or function specific definitions
makes Extract, transform, load (ETL) operations more precise for data warehousing
allows user interface designers to access a common pool of screen and report header labels
promotion of model-driven architecture

Objections to metadata publishing

Organizations that publish their metadata could make it easier for unauthorized people to find sensitive data if they breach an organization's firewall
Vendors that publish their metadata risk customers creating tools that could allow their customers to export their data from computer systems, therefore making it easier to migrate off of a vendor's system

Core process in metadata publishing

The following are some of the core processes in metadata publishing

Gathering of metadata requirements
Selection of metadata registry and metadata publishing tools
Training of metadata concepts to project participants
Stakeholder group formation
Metadata harvesting
Glossary consolidation
Initial upper ontology construction (abstract data elements)
Draft data element loading
Data element review process
Publishing approved metadata elements in a variety of output formats (see below)
Creation and maintenance of versions and depreciation of unused or redundant data elements

File format metadata publishing

Organizations that create applications that store data in file systems can also publish metadata definitions. One common way to perform this is to store application data in a compressed XML file format. The XML files can be uncompressed and validated against an external XML Schema. An example of this is done by the Open Source FreeMind tool.