💾 Archived View for agnos.is › projects › ai-game captured on 2024-03-21 at 15:20:57. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

AI-Driven Text Adventure Game

AI Game Git Repository

Status: In Development
Language: Rust

My obsession of the moment is creating a text-based adventure game driven by a large language model (LLM). The game is being written in Rust, backed by KoboldCPP and the Mistral 7b Instruct model. Documentation for getting the project to run is in the linked repository.

This page will document my journey and things I've learned throughout this project.

Development Journal (Newest First)

March 2024: gbnf_limit and Constrained LLM Output

As of early March 2024, I have finally finished the implementation of the GBNF limiting feature, required for better coherence in the LLM, and essentially a blocker for any further development of useful/fun things in the game. The game is now capable of limiting individual fields of the LLM's JSON response to specific values, which is extremely useful when we want the LLM to pick from a list of IDs (of people, items, exits, etc in a scene). This is the basis of all command processing. Combined with the existing coherence code, I think the game will be able to advance at a much faster pace now.

The rest of this month will focus on:

Integrating the new feature into the codebase.
Squashing any further bugs with the limiting (e.g. at time of writing on March 5th, deserializing of LLM responses is broken due to mismatch between serde field name and the generated response).

Some tweaking of the event responses themselves might also be necessary. Right now, there are two string parameters: `applies_to` and `parameter`. The `parameter` field is often used for an ID (e.g. what exit to pick), but it can also be the amount of damage taken, or something else. This can confuse the LLM, so it might be best to either remove one of these fields, or rearchitect the event responses so that they are less confusing.

I think a one-of function might be the next big issue to deal with in the GBNF derive macro. Enabling conversion of Rust enums to GBNF rules would give more flexibility to the LLM to generate proper event responses. That way we could have strongly-typed events in JSON, where semantic meaning is clear. Combined with value limiting, we could have very expressive GBNF rules.

February 2024: derive(Gbnf) and More Coherence!

I am currently focusing on creating a derive macro to automatically generate GBNF grammar rules from Rust types. This has two main benefits:

No more brittle maintenance of hardcoded GBNF rules by hand.
The ability to create hyper-specific limits to further constrain LLM output.

The necessity of limiting LLM output was the main driving factor behind the creation of this derive macro. By forcing the LLM to, for example, output only specific UUIDs or database IDs in response to a prompt (e.g. "Select the exit the player should take"), the accuracy of its responses should be much, much higher. Without these dynamic GBNF rules, the LLM can still sometimes pick an ID that does not exist, or fill the response with a nonsense value.

I will likely spin this GBNF derive macro out into a separate crate for use by the wider Rust community.

January 2024: Persistent World and Coherence

In the first month of 2024, I put a massive focus into creating a procedurally-generated persistent game world that the player can navigate. The other main focus was the coherence of the LLM, and reorganizing the code to make it worthy of presenting to the world (and making feature implementation easier, of course).

This month saw the addition of numerous coherence checks and systems around the output of the LLM, as well as proper implementation of the ability to "continue" prompts with the LLM, if it needed to generate more data than could be delivered in one reply.

December 2023: Beginnings of Something Playable

By December, the simple comand parser had turned into a proper "thing with a game loop." It was still very raw and basic, but a long vacation over Christmas 2023 allowed me to implement almost all of the core concepts needed to get the game working. This is when I discovered GBNF grammars to constrain the LLM's output, as well as dealing with a number of challenges related to how the LLM actually outputs data:

Coherence: By switching to Mistral 7b Instruct, the LLM is actually very good at outputting the right data ... usually.
Creating Non-Existing Things: The LLM could still create strange information or add things to the game world that should not be there.
GBNF challenges: Despite constraining the LLM output with a GBNF, it could still output nonsense within those constraints.

November 2023: LLM Command "Parsing"

The project began by writing a simple Rust application that calls KoboldCPP's API to "parse" commands given as if the user was playing a text-based adventure game. The results were coherent, but not regular enough to be of any use to a machine attempting to decode what the LLM means. This was the beginning of my research into how LLMs work and the challenges associated with prompt engineering, coherence, and sanitizing their output.

‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗‗

⤴️ [projects]

🏠 Home