Many use the term "metadata" to refer solely to digital objects, making it seem a different undertaking from "cataloging." The creation of metadata is not a conceptually new activity; the cataloging of physical holdings is the creation of metadata. There are additional needs in terms of metadata that document the technology associated with the objects that are used to manage large, disparate, and possibly distributed collections of digital files, but much of what is needed is much the same as for physical objects - authoritative description of the title, creator(s), subject(s), and geographical coverage, and documentation of holdings, location, and status.

Metadata may be either embedded in the digital object, or exist externally to the digital object. Metadata is generally separated into four broad categories:

As an example, for a digital image, one might have the following metadata recorded:

Descriptive:

Administrative:

Technical:

Structural: (documents the location to access the various files that make up the object)

Encoding standards are codifications of the practice of organizing data. This ranges from data dictionaries that describe local fields or elements and the standards for their use, to international standards for the creation of shareable metadata. Best practices for encoding range from a consistent record structure, to the consistent use of the data structure in a record, to the consistent use of controlled vocabularies in those records, and down to the consistent encoding of the characters in those records.

Examples of encoding standards:

Content data for some elements, such as the subject element, may be selected from a "controlled vocabulary," a limited set of consistently used and carefully defined terms. Using terminology from a controlled vocabulary ensures consistency and can improve the quality of search results, and may also reduce the likelihood of spelling errors when recording metadata. The description of each element indicates whether content should be selected from a controlled vocabulary, if possible.

Examples of controlled vocabularies:

The long-term utility of digital media files remains a major unknown. As data files are created and collected, their creators and stewards must take pains to ensure that the content will be accessible for as long as possible. Using standard file formats and documenting their use is the best guarantee of that longevity. Research the de facto standards in use in the community, decide what formats are most appropriate for your collections, use the formats consistently, document your content creation practices, and include information about the digitization process in the technical metadata that accompanies your digital objects. Consistent and well-documented practices will mean that future migrations to new formats (and preservation migration of formats is a certainty) are more likely to be successful.

There is much discussion in the community of the importance of using non-proprietary file formats. It is crucial to the long-term survival of digital content that it be created using file formats that can be migrated into new formats when necessary. Proprietary formats can often be migrated, but files in proprietary formats may be inaccessible once the software that created them has disappeared from the market. Migrations from non-proprietary or open standard formats can more likely be carried out without the cooperation of a software vendor, since the formats are publicly defined. This is not to say that you must avoid all content in proprietary formats or never use in it your operations - formats like Word or PDF (until fully replaced by PDF/A) are often unavoidable when accepting deposited content or necessary to ensure broad usability of files. When proprietary formats are unavoidable, documenting their use and avoiding proprietary features that complicate data migration are the best practices to follow. This article will not endorse or recommend any particular formats.

The following sites offer discussions and recommendations of technical formats for different types of digital content:

Interoperability is the ability of two or more information systems to exchange and use information. Metadata interoperability is highly dependent upon the ability to map identical or similar elements of data structures. The simplest strategy for interoperability is for each system to employ similar data structures and similar or identical encoding semantics and controlled vocabularies, as with MARC records, AACR2, and controlled vocabularies in the creation of library records. In the reality of the digital library world, metadata is created by a myriad of repositories using a myriad of data structures, semantics, and vocabularies and taxonomies (community-based or local), all valid for their context and environment.

The most difficult task in metadata interoperability is semantic and taxonomic compatibility. Full metadata interoperability across diverse systems with diverse content is impossible. Achieving even limited interoperability between systems requires coordination, most often achieved through a common mapping between the fields or elements that each system uses to organize metadata. A "map" documents the correspondences between similar data fields or elements in different systems or standards. When exchanging information, each system can recognize the mapping between fields or elements and translate between them.

Best practices demand consistent utilization of encoding rules and practices (e.g. AACR2 or CCO [Cataloging Cultural Objects]), and standardized vocabularies (e.g., the Getty Art & Architecture Thesaurus, Library of Congress Subject Headings, or those developed by specialized communities), as well as documentation of the vocabularies in use for each field or element. Digital library programs should develop and maintain "data dictionaries" that document all fields or elements in all local systems and the semantics, taxonomies, and vocabularies used in all instances. Also vital is documentation between local practice and other standards and systems called a "crosswalk," documenting the mappings between local practice and MARC, Dublin Core, VRA Core (Visual Resources Association), Text Encoding Initiative (TEI), and Encoded Archival Description (EAD).

There are interoperability concerns not just for metadata, but for the files that make up the content of the digital objects. Some institutions use TIFF, some use JPEG 2000, and some use Mr. Sid. TEI has passed through a number of stages of development, and some institutions use TEIxLite with others use TEI P4. Formats change over time. TIFF has had many versions. There is a new TEI profile under development. JPEG 2000 is a relatively new standard. PDF/A is a developing open version of PDF. There is no right or wrong; each institution picks what is appropriate for its needs at the time of collection building.

There are many best practices to keep in mind for format interoperability:

While these practices are essential in any local digital collection building initiative, the need is magnified in the sharing or aggregation of metadata and collections. Many institutions make the metadata representing their digital collections available for harvesting and aggregation through the Open Archives Initiative Protocol for Metadata Harvesting (http://www.openarchives.org/). When records from disparate institutions are aggregated in services such as OAIster (http://oaister.umdl.umich.edu/o/oaister/) or the Collaborative Digitization Project (http://www.cdpheritage.org/), it is a boon to the user to have so many resources available through a single search interface. It is also a challenge to the user because some institutions use MARC and some use ContentDM and some use EAD and some use TEI, and so on. How have those institutions recorded their metadata, and how have they mapped it? Will a search for "ocean" find all the applicable content? While every institution should use the standards that are most appropriate for their collections, all institutions should pay attention to community practices in the use of record formats, encoding practices, controlled vocabularies, and mapping and crosswalking between standards to better ensure that their collections are shareable. One truth has become self-evident in this area: the richer the metadata - descriptive, administrative, technical and structural - and the more standard the formats used, the better the chances of interoperability.

The following sites link to professional organizations, projects, and reports that, in addition to those in the section above, provide a wealth of experience and information for any institution planning projects to create, manage, and deliver collections of digital content.