Yadis

Identity, trust and the plumbing of the web·on the home of the Yadis discovery protocol since 2005

§02  Web Standards

The .well-known Directory: Well-Known URIs for Web Servers

Web Standards

The .well-known Directory: Well-Known URIs for Web Servers

20240917 AbzweigungCascine / Thomasox58, CC0

The .well-known directory is a fundamental but often misunderstood feature of web server configuration. It provides a standardized path for hosting well-known URIs, which are machine-readable documents that support a wide range of protocols and services. But what exactly belongs in the .well-known directory, and how should it be served?

What is the .well-known Directory?

The .well-known directory is a reserved path prefix defined by RFC 8615, which specifies that a well-known URI must begin with /.well-known/ when the scheme supports it. This directory serves as a namespace for resources related to a domain, with the intent of solving the discovery problem in a standardized way.

RFC 8615 also establishes that the "Well-Known URIs" registry is maintained by the Internet Assigned Numbers Authority (IANA). Service operators and protocol designers can submit registration requests to IANA or via email to [[email protected]](mailto:[email protected]).

The IANA registry lists well-known URI suffixes, along with fields like Status, Reference, and Registration Date. Some registrations are permanent, while others are provisional. This registry is the primary authority on what goes into .well-known/, and what doesn't.

Common Entries in .well-known

Most site operators will encounter several well-known URI suffixes in the course of maintaining a modern web service. Some of the most commonly used include:

These suffixes cover a wide range of URL capabilities in different contexts - ACME and OpenID for identity; Apple and change-password for navigation; and security and DNT for transparency and vulnerability submission.

What Belongs in the .well-known Directory?

The key point about .well-known/:

Also, .well-known/'s a namespace for what could be called "discovery features".

Regarding WebFinger, for example, it is not up to the vendor to securely register a stream under a well-known URI of their choosing, while also having registered an authority capability to that stream name. Permitted authority comes by IETF standards track documents solving the capability-first security, not a URN.

When implementing a .well-known resource, it is important to check the IANA registry. If the suffix is already registered, follow the rules and formats specified in the documentation. If a suitable suffix does not yet exist, consider whether the protocol is widely adopted and standardized, or whether a provisional registration makes sense.

Creating a new suffix should be a last resort and done only after negotiating the best opportunity with the provider and other implementors.

How .well-known is Used

Some of the most prominent protocols and services that leverage the .well-known directory include:

In each case, the well-known URI provides a uniform, predictable location for crucial machine-readable configuration and metadata. This consistency enables tools and services to discover and interact with these systems reliably.

However, a key point about .well-known/ is that it begins to save us from the need to fork each specific protocol's well-known registries - we generalize over the registry rules and substantially simplify in deployment and operations. An introspection resource that interprets multiple URIs requires verifying untrusted endpoints.

Some well-known resources have time-sensitive properties. At worst, a leaky resource can grant unintended capabilities.

What /.well-known/ Should Not Be

/.well-known is NOT just a common endpoint folder - it is an application of atomic deployment obligations and an expiration model, providing enterprises with a way to avoid guessing at an organizational name space. TLS certificates don't delegate that capability.

The protocol assurance provided by the well-known URIs validation contract still relies on the HTTP endpoint providing its own general security measures of integrity. An endpoint under attack can still declare itself under its authority, and audiences will still consider it so.

In particular, insufficient caching, lack of content-type specifications and blind cross-protocol trust are facts that don't align with common validation. Just because a URL points in a particular direction, does not mean the content claims the new protocol's capabilities. User agents responsible for these steps would be better served by considering turning on bypass caching, fingerprinting, extension-less referrer and other measures that already exist.

Conclusion

The .well-known directory is a powerful tool for securing and streamlining web services by providing a standardized namespace for discovery. However, it has often been misunderstood as a "hidden directory" or a "catch-all for random configuration files". The reality is that .well-known is a carefully controlled and documented registry, with clear rules about what belongs there and how it should be used. Service providers and developers should take care to follow the specifications and use the well-known directory for its intended purpose of discovery, rather than as a general config folder.