§02 Web Standards

The Evolution of Web Service Discovery: From XRDS to WebFinger

The process for modern web applications to discover each other and the services they offer on behalf of a user has evolved from specialized XRDS identifiers into the standardized, well-known URIs underpinning implementations of the WebFinger protocol in use today. RFC 5785 defined the /.well-known/ path prefix in 2010 as a URI scheme prefix for locus providers and consumers. At the same time, RFC 7033 calls out /.well-known/webfinger as the endpoint specifically used by its defined webfinger protocol for discovering a user's web services by email, Twitter handle, or other identifier.

The pre-well-known

well-known era of web service discovery centered on XRDS, an XML document format describing the services associated with an XRI (Extensible Resource Identifier) or URL. The Yadis specification defined an XRDS document as containing as the root element, with required namespaces including xri://$xrds and xri://$xrd($v2.0). An XRDS document told a client which service providers were associated with a given identifier and should be contacted, often for the purpose of authentication. The protocol was initially used by the Yadis identity framework before being adopted by OpenID 1.1.

The Yadis specification was circulated directly by XRI Technical Committee members in October 2005, before being published as a formal specification by Yadis.org in March 2006. The protocol was initially designed as a way of discovering services by looking up an XRI, with a fallback resolution by URL. Later, The XRI Resolution protocol added a URL-based method of XRDS discovery starting in November 2007. URL-based XRDS eventually became part of the OpenID 2.0 Authentication protocol.

However, there was a shift from the XRDS/identifier-based service discovery of the early 2000s to the URL path-based discovery now reflected in the .well-known/webfinger endpoint. This shift was connected to the creation of the /.well-known/ URI path prefix, a standardized scheme for locating the location of services provided by a domain or person. RFC 5785 defined the .well-known "root" URI for registration of well-known URIs and their associated web services, with options for HTTP, HTTPS, and other URI schemes as specified. Well-known URIs are defined as starting with the Unicode character / and following with /.well-known/, followed by the URI itself.

The shared location also allows mitigation of a broader attack surface: the resource document is always at a known and specific location. In the case of WebFinger, resource discovery using an identifier proves the existence of the .well-known/webfinger URI, while an "out of band" discovery client is no longer required. This prevents exploitation if the well-known URI does not exist.

At the same time, the WebFinger protocol standardized how a client could query a WebFinger-capable server for the services provided for a user with either a basic registration point in /.well-known/webfinger named in RFC 7033. Entering a user's identifier, such as an email address, into a browser with well-known/webfinger appended allows access to the server's WebFinger endpoint. Registering a URI path under /.well-known/ ensures standardized discovery.

In practice, the transition from older XRDS discovery to WebFinger via well-known discovery is most apparent within OpenID, which originally adopted Yadis as its discovery mechanism in OpenID 1.1. As mentioned, URL-based XRDS discovery was later incorporated in OpenID 2.0, and the newer OpenID Connect standard cites RFC 5785 directly for its discovery approach in OpenID Connect Discovery 1.0.

While the older XRDS approach used an XML document to describe services by identifier, today's simpler "you type this in the browser and this is the mapping document" approach is the result of standardization by RFC 5785 and RFC 7033. There was never a specific reason to transition from XRDS to well-known, only a natural progression of technology. Adopting well-known for various protocols maintains the original application of service discovery and reduces the number of strings a user must remember during account setup for third-party services.

Today, this shift is reflected in how OpenID standardized URL-based XRDS as its discovery mechanism, whereas WebFinger uses its own standardized mapping of identifiers to services under /.well-known/. The shifts demonstrate the historical relationship but lack of identity between XRDS and WebFinger. XRDS, used for discovery in OpenID 1.1 and 2.0, and WebFinger, adopted by the OpenID Connect specification, fill the function in the modern space.

In conclusion, while web service discovery as practiced today is based on /.well-known/ and WebFinger, it has roots in the Yadis framework that defined discovery via XRDS documents with URL fallback. This history is visible in OpenID, which has shifted entirely to well-known URIs as its standardized discovery approach, while WebFinger and others have adopted the same URI scheme for service discovery. Accessing the specified URI for a WebFinger-enabled domain makes the discovery process user-friendly, without the need to remember specific strings for each service.