💾 Archived View for auragem.letz.dev › techlog › 20240525.gmi captured on 2024-07-09 at 00:09:38. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2024-05-26)
-=-=-=-=-=-=-
In this article I want to briefly discuss my plans and ideas for my projects, but I want to first announce that the first two server hosts, outside of my own servers, for the Scroll Protocol have been added to the project's homepage!
You can create your own with scroll's reference server, which can be downloaded via Gemini here:
There are a couple of things I would like to do for AuraGem Search, and a couple of experimental ideas I want to try.
Firstly, I am starting work to add more searching options for other languages outside of English. This involves setting up an FTS (Full-Text Search) index for each langauge. I have already done this for Russian, and other languages are easy to do. I'll then need a way for users to access this. One method is to auto-detect the language of the search query and use that to search the FTS for that detected langauge. Another option is to add different search engine sub-capsules for different languages. Lastly, over the Scroll Protocol I can easily get the requested language(s) and use that. Whether search results should just search based on the language or should filter out for just that language remains to be decided.
I also want to add preformatted block and code searching. This will involve caching preformatted blocks and their alt text in gemtext and markdown, code blocks in scrolltext, and code in code files. I will also need a different FTS indexing method (called analyzers in Lucene) that is tailored for programming languages.
I also want to cache different link types, like tor links, email and misfin addresses, etc.
For the experimental ideas, I want to look into categorizing terms to search for in documents as keywords (a form of keyword extraction) and to use in searches. Terms fit into a couple of categories: a term as a broad category, a term as a sub-category, an alias to a category or sub-category, and a proper name. For example, the word "Python" is a programming language. Documents that use the term "python" should be searchable via the terms "python" and "programming language" and aliases like "pypi" and "code". This is an extension on the idea that many search engines already use when they search for the user's query along with any synonyms of the terms in the query.
I have also started compiling a list of common search terms, and so far the list includes:
The general methodology of searching in AuraGem Search is that specific queries should give specific pages/sites, and broad queries should give broad results. Searching "Antenna" should give Antenna. Searching "aggregator" should give a list of aggregators.
I want to look into expanding and clarifying the spec to include footnotes, a better citations format, and a few other things. I'm also thinking about whether I should add some way of getting a document's available languages. I would also like to start working on more tools, including libraries for golang, rust, and python.
I would also like to work on designing a wiki system over the Scroll Protocol. The addition of the modification date is quite useful for this because it allows the client to know which version of a document the user is modifying - it's a way of versioning edits. When a user uploads an edit in whichever method is available, the client will then be able to tell what version the user has edited and this can be used for merging simultaneous edits or rejecting edits to old versions.
I want to quickly note that lethe made a tool similar to curl but for the scroll protocol, written in Python:
An important tool that I will want very soon is a website that proxies from the Scroll protocol. Ideally it should be able to proxy spartan and gemini too.
Lastly, I want to write more on the Scroll Protocol, especially comparing it to HTML, Gemini, and Gopher.
I had this idea a while ago: a media server, similar to plex/emby/jellyfin, but over Gemini (and maybe Scroll). This idea came after I launched AuraGem's Music and Public Radio services. I wanted to essentially detach AuraGem's Music stuff into its own server for others to be able to host their own radios and music hosting. Soon after I decided that extending this to videos and books might be useful.
I want to write more about non-tech things, particularly at my Scholastic Diversity capsule: