In short. tdmrep.json is a file at /.well-known/tdmrep.json that maps web resource paths to text and data mining reservation values and optional policy links.

Definition: tdmrep.json is the TDM Reservation Protocol's site-level JSON file for declaring whether a rights holder reserves text and data mining rights over web resources. It lives at /.well-known/tdmrep.json and associates URL path patterns with reservation values and optional licensing-policy links.

Text and data mining (TDM) means automated analysis of digital text and data to produce information such as patterns or correlations. The TDM Reservation Protocol report, dated 10 May 2024, specifies the file mechanism. Its publisher is the Text and Data Mining Reservation Protocol Community Group; the report has Community Group status and is outside the W3C Standards Track.

Key facts

The report's file specification requires a JSON array of rules, each containing location and tdm-reservation; tdm-policy is optional.
The TDMRep semantic vocabulary defines reservation value 1 as rights reserved and 0 as rights unreserved.
RFC 8615 places well-known locations at the top of the URL path hierarchy, so the file belongs under /.well-known/ at the origin's root.

What does the file contain?

The report's site-wide example reserves TDM rights for every file on a web server with this JSON:

[
  {
    "location": "/",
    "tdm-reservation": 1
  }
]

The outer array remains mandatory when there is only one rule. Here, location supplies the path pattern, and tdm-reservation supplies the rights holder's choice. The example omits a licensing policy.

A TDM policy describes the kinds of mining licences a rights holder offers. A rule can add tdm-policy with a URL pointing to that separate policy. The same report illustrates different choices for /directory-a/, /directory-b/html/ and /directory-b/images/*.jpg, including reserved rights with a policy link and unreserved rights for JPEG images.

How do path rules apply?

Under the report's matching rules, a TDM agent, meaning software accessing resources for mining, compares a resource URL with the paths inferred from each pattern. Matching should be case sensitive. The report requires the most specific match and says that this match is first in sequence.

The supported special characters are *, which matches zero or more instances of any character, and $, which marks the end of a pattern. Without a matching rule, the reservation value for that URL is unset. Other reservation values besides 0 and 1 are protocol errors and also produce unset.

How does the file interact with page-level signals?

The report's processing priority rules require a TDM agent to check the server's file before scraping content. HTTP response headers can then override values obtained from the file. Embedded HTML metadata overrides earlier values; embedded EPUB or PDF metadata likewise overrides previous values.

At each step, absent reservation or policy properties leave the current values in place. The report recommends that rights holders use a single signalling technique. When several techniques coexist, checking only tdmrep.json gives an incomplete view of the protocol's effective values for a resource.

FAQ

How can you check that the file is served correctly?

Request /.well-known/tdmrep.json and parse the response as JSON. The report's server requirements allow a successful response containing the representation or redirects leading to it; check that every rule contains the mandatory properties.

Must you publish a licensing-policy link?

The report makes tdm-policy optional. Its site-wide example uses only location and tdm-reservation, so a reservation can stand without a policy link.

Sources

Check how a page reads to answer engines. Open the GEO score analyzer