So You Want to Define a Well-Known URI
Hi, I’m Mark Nottingham. I write about the Web, protocol design, HTTP, Internet governance, and more. This is a personal blog, it does not represent anyone else. Find out more .
Comments? Let's talk on Mastodon. @mnot@techpolicy.social
As one of the authors of the Well-Known URI specification and current Designated Expert for the registry , I field a lot of questions about how they should be used, and end up coaching a lot of folks on how to best use them. Below, I’ve summarised how I think about them. Note that these aren’t all requirements for registration – just what I consider good practice.
Well-known locations work best when the client – whether it’s a browser, bot, or other software – knows the site 1 and needs to discover something about the whole site in an efficient way.
robots.txt is the perfect example – it pre-dated the RFC so it doesn’t use a well-known location, but was a major part of the reason we reserved a space for them. A crawler needs to know what the access policies for the site are, and putting it in one central place for the site avoids the need to check headers and content on every response (which would defeat many of the purposes of having such a policy).
A well-known location doesn’t have to contain policy, though. Any mechanism where the client already knows the site but needs to learn something about or interact with it as a whole is a candidate for becoming a well-known location. For example, the change-password well-known location allows clients to change their password for a site.