Zthes: a Z39.50 Profile for Thesaurus Navigation, version 0.5

Version history is now available at zthes.z3950.org/profile
Changes between v0.4a and this version are at zthes.z3950.org/profile/changes-05.html

1. Introduction

1.1. Overview

1.2. Status

All feedback is very welcome, and should be emailed to the author at <mike@zthes.z3950.org>

1.3. Scope

It is intended that the abstract model described here is sufficiently general that it can also be implemented by protocols and data formats other than Z39.50. As an example, an appendix defines an XML DTD for thesaurus terms based on the model, and includes an example XML document using that DTD.

Because the model is abstract, it will generally have little or no effect on the data model used by a server's local representation of its thesaurus. In particular, pre-existing thesaurus databases may well represent a more complete model of a term than what's required by this profile's abstract model. That's fine: the additional information is not be representable in the Zthes model, but may be used in any number of other ways beyond the scope of this profile.

This profile does not mandate any relationship between a thesaurus and any other database. The model is that terms from any thesaurus database may be used to search any other database (called a target database).

1.4. Acknowledgements

• Mike Taylor <mike@zthes.z3950.org>, TECC Ltd.
• Thomas Place <t.w.place@kub.nl>, Tilburg University.
• Denis Lynch <Dlynch@ACM.org>, Silver Platter.
• Sebastian Hammer <quinn@indexdata.dk>, Index Data ApS.
• Ralph LeVan <levan@oclc.org>, OCLC.
• Pat Stevens <Pat_Stevens@oclc.org>, OCLC.

2. Abstract Model

2.1. Overview

Each individual term in a thesaurus is represented by a record in the database. In the interests of simplicity and orthogonality, even non-preferred terms must be represented by their own records.

Term records consist of an initial part describing the term itself (with information such as its unique identifier, scope note, etc.), together with sub-records (that is, named sections within the main record) briefly describing related terms. The primary means of navigation from one term to another is by searching for the unique identifiers of the terms related to the first one.

2.2. Schema

The top level term record is composed of the following elements:

It is recognised that in many thesauri there is no explicit unique identifier field, and the term itself, perhaps in combination with the qualifier, uniquely identifies a record. Thesauri such as these must nevertheless provide a termID field, which may be automatically generated simply by combining the term and qualifier.

The termType element may take the following values:

``PT''

Preferred term (also known as a descriptor)

``ND''

Non-descriptor: that is, a non-preferred term.

``NL''

Node label: that is, a dummy term not assigned to documents when indexing, but inserted into the hierarchy to indicate the logical basis on which a category has been divided - for example, by function. Also known as a guide term or a facet indicator.

Servers may return other values of termType at their discretion. It is recommended that such extension values begin with the string ``X-''.

Each postings sub-record is composed of the following elements:

If a server wishes to communicate separate postings counts for a term in more than one field, then multiple postings sub-records with the same value of sourceDb should be used.

Each relation sub-record is composed of the following elements:

The relationType element may take the following values:

``NT''

Narrower term: that is, the related term is more specific than the current one.

``BT''

Broader term: that is, the related term is more general than the current one.

``USE''

Use instead: that is, the related term should be used in preference to the current one.

``UF''

Use for: that is, the current term should be used in preference to the related one

``RT''

Related term.

``LE''

Linguistic equivalent: the current term and the related term are preferred terms representing the same concept - or ``sufficiently close'' concepts - in different languages.

Servers may return other values of relationType at their discretion. It is recommended that such extension values begin with the string ``X-''.

With a single exception, this profile deliberately restricts its set of supported relations to those discussed in ISO 2788 [2], in the belief that it is better for a small set of relations to be used interoperably than for a larger set to be specified, with different servers and clients in practice using different subsets.

That sole exception is the addition to the standard relation types of ``LE'', introduced to model the multilingual links described in ISO 5964 [7].

The ``NT'' and ``BT'' relationships are reciprocal; so are ``USE'' and ``UF''; and ``RT'' and ``LE'' are reflexive. That is, when any term T1 points to another T2 using the relation ``NT'', T2 should point back to T1 using ``BT'' and vice versa; when T1 points to T2 using the relation ``USE'', T2 should point back to T1 using ``UF'' and vice versa; and when T1 points to T2 using the relation ``RT'' or ``LE'', T2 should point back to T1 using the same relation.

The termType element in a relation sub-record may take the same values as in the top-level record.

2.3. Searching

• A search which finds the single record with a specified termId at the top level (i.e. not within a relation sub-record.)
• A search which finds records with a specified termName at the top level.
• A search which finds records with a specified termQualifier at the top level. (This is useful primarily in conjunction with the termName search.)
• A search which finds words occurring anywhere in the term record.

Support for additional searches, including the following, may be useful.

• A search which locates all the records considered suitable as starting points for browsing.
• A search which finds all the records in a specified relation to the term with a specified termId. (This is useful for finding the narrower terms of a very broad term without needing to retrieve a very large record.)
• A search which finds a special record describing not a term but the thesaurus as a whole - authorship, version information, etc.
• A search for terms in a specified language.
• A search for terms whose termNote contains specified words.
• A search for terms of a specified type.
• Searches for terms created and/or modified at a specified time or by a specified person.

3. Z39.50 Specification

3.1. Overview

As such, it requires servers and clients to support version 3 of the Z39.50 protocol.

The intention of this profile is that it is used in conjunction with other profiles. It is envisaged that an application will use the Zthes profile to navigate a thesaurus and thereby obtain terms suitable for searching in a target database; and use a second, domain-specific profile such as GILS or CIMI to search in and retrieve from that database.

The Z39.50 objects defined by this profile have the following OIDs.

3.2. tagSet-Zthes

3.3. Schema

The abstract schema described in section 2.2 is represented in Z39.50 by a GRS-1 record encoded with the tag-paths specified in the following table. Where possible, standard tags from tagSet-M and tagSet-G are re-used; in these cases, the generic names of the tags are listed in the right-hand column.

The termLanguage element is expressed as one of the standard codes described in RFC 1766 [8] and ISO 639 [9] - for example, ``en'' for English, ``fr'' for French and ``de'' for German.

The administrative date fields should be returned in the ASN.1 GeneralizedTime format. (The working group considered the Z39.50 ASN.1 date/time definition [11], but reached the conclusion that the benefits would be outweighed by the barrier raised to implementation.)

The person-name elements, termCreatedBy and termModifiedBy, may be returned in whatever format is convenient for the server: this profile does not attempt to address the interpretation of such administrative information across multiple databases.

The sourceDb element should be returned in the form of a z39.50s URL as described in RFC 2056 [10]. For example, if the related term is in the database called ``aat'' on the server running on port 3950 on the host foo.bar.org, then the sourceDb element should have the value z39.50s://foo.bar.org:3950/aat.

Servers may, at their discretion, include additional tagSet-M, tagSet-G and string-tagged (type 3) elements in the records they return; they may include such additional elements at the top level, within relation sub-records, or both. In particular, records may begin with a schemaIdentifier (1,1) element, in accordance with standard ``good practice''. Clients may display any such additional elements as they see fit, or may ignore them.

3.4. Element Sets

3.4.1. Element Set ``f''

3.4.2. Element Set ``b''

This element set may be useful when constructing a summary of several records found by a search for initial entry points to a thesaurus; it is unlikely to be useful when navigating from term to term.

3.5. The Zthes-1 Attribute Set

The thesAdmin access point must be used with one of a small set of well-known strings as the search term. Servers may support the following values:

The Zthes-1 attribute set conforms to attribute set class 1 as described in the Z39.50 Attribute Architecture. However, it prescribes no rules to resolve conflict between its own semantics and those of another attribute set in the case where attributes for both are used in a single search term of a type-1 query and the top-level attribute-set of that query is Zthes-1.

3.6. Searching

Note: Since the publication of version 0.2b of this profile, the numbering of the utility attribute set has changed, so that the all elements search now uses access point 11 rather than 10. Servers are encouraged to support searches from old clients which use the old attribute value 10 as well as the new one.

For the purpose of searches on the local control number access point, values of the termID function as opaque ``magic cookies''. Therefore, such search terms should not include any contentAuthority attribute, even if it happens that for the specific thesaurus in question, the termID identifiers are taken from a well-known source.

The following additional access points may optionally be supported:

Note: Since the publication of version 0.2b of this profile, functional qualifiers for use with the Record Date/Time and Record Agent access points have been added to the utility attribute set. Accordingly, we now use the official functional qualifiers ``creation'' and ``modification''. Servers are encouraged to support searches from old clients which use the old functional qualifiers ``date/time created'', ``creator'', ``date/time last modified'' and ``last modifier'' as well as the new one.

3.7. Explain

3.7.1. Overview

A client can gain two kinds of information about Zthes databases from Explain: the fact that a particular database is a Zthes database, and the fact that a Zthes database is relevant to a particular TermList.

These two uses are specified in the next two sections.

3.7.2. Identifying a Zthes Database

Once a client has observed the Zthes schema in the schemas for a database, it may presume that the server observes the behaviour described in this profile. (The client may still need other information from Explain, for example additional record syntaxes that may be available.)

3.7.3. Locating a Relevant Zthes Database

AuthorityFileInfo ::= SEQUENCE {
    name        [1] IMPLICIT HumanString,  -- for display
    database    [2] IMPLICIT InternationalString,
                     -- z39.50s URL to the authority database.
                     -- Simplifies to a database name if on the same server.
    exclusive   [3] IMPLICIT NULL OPTIONAL
                     -- If present, all terms in the term
                     -- list come from this authority file.
                     -- If absent, other terms may or may not
                     -- be present in the term list.
}

Note: It may be desirable to include an additional item to indicate the kind of authority file being referenced. If it is desirable, an appropriate identification scheme will be required.

4. Future Directions

• A specification for the schema of the optional whole-thesaurus descriptive record that can be found by searching for thesAdmin=whole.
• A way for clients to request servers to include postings sub-records only for a specified set of ``interesting'' target databases.
• Use of the Scan service on termName to derive alphabetic displays.
• Support for version numbers of thesauri and/or terms.
• Post-coordination: for example: coal mining USE COAL + MINING

5. Appendix A. References

6. Appendix B. The Zthes Abstract Model in XML

This appendix has been removed, since more up to date information about the XML representation of the Zthes data model can now be found at zthes.z3950.org/xml

6.1. The Zthes DTD for XML

See zthes.z3950.org/xml

6.2. Sample Zthes-in-XML Document

See zthes.z3950.org/xml

7. Appendix C. Implementations

This appendix has been removed, since more up to date information about implementations can now be found at zthes.z3950.org/impl

8. Appendix D. Applications

This appendix has been removed, since more up to date information about applications can now be found at zthes.z3950.org/apps