Hannu Hartikainen hannu.hartikainen+gemini at gmail.com
Mon Jun 8 10:18:06 BST 2020
- - - - - - - - - - - - - - - - - - -
On Sun, Jun 07, 2020 at 10:47:41PM +0100, Luke Emmet wrote:
If a client must not make subsequent network requests when interpretinga
page, does this mean that search engines and crawlers are nownon-compliant
clients? This seems to go much too far.
The spec says
Clients can present links to users in whatever fashion the client authorwishes, however clients MUST NOT automatically make any network connectionsas part of displaying links whose scheme corresponds to a network protocol(e.g. gemini://, gopher://, https://, ftp://, etc.).
I find this reasonable: a crawler does not make any extra networkconnections *when interpreting a page* or *as part of displaying links*.Rather, it fetches single pages per spec, while building a graph of allknown pages (which it then fetches, still as single pages in a waycompatible with the spec). A crawler need not fetch any other pages inorder to add a single page to its index. If a search engine startedsupporting inlining content from links it would be breaking the spec.
My two cents.
-Hannu-------------- next part --------------An HTML attachment was scrubbed...URL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20200608/089033a4/attachment.htm>