💾 Archived View for blog.schmidhuberj.de › 2023 › 01 › 21 › lisp-is-spawning-programs-weirdly captured on 2023-05-24 at 17:40:14. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

Lisp is Spawning Programs Weirdly

Posted on 2023-01-21

Recently I was learning a little bit of Lisp for something I may reveal in a few months. But a few days ago I hit a major roadblock, Lisp is running programs weirdly.

Basics of Running Programs

To my knowledge, the following runs programs with an input and gets its output as a string:

(setq output-string
    (with-output-to-string (stream)
        (uiop:run-program 
          '("rev")
          :output stream
          :input '("Hello World")
        )
    )
)

This is working fine (tested with the sbcl-compiler).

It can also run curses programs fine with interactive stdin and stdout:

(uiop:run-program 
  '("vim")
  :output :interactive
  :input :interactive
)

The Problem

The problem is combining both, piping input into stdin, interacting with the program and getting the output from stdout. There are some programs that require this, e.g. fzf takes a list of newline-separater strings and lets the user choose from these to return to the output. The dynamics of how programs can do this is a little bit complicated, but basically it requires reading the entire input, re-opening the input from the TTY, opening the TTY as the output, running curses (or any similar library), resetting the output and finally returning the result. How I would run such programs in Lisp is the following:

(setq output-string
    (with-output-to-string (stream)
        (uiop:run-program 
          '("fzf")
          :output stream
          :input '("Hello World Hello")
        )
    )
)

But this does not work and blocks the entire application. Replacing fzf with a little program written by myself shows that the call to curses initscr blocks. Maybe, I thought, I actually did something that was not possible in any programming language, so I re-implemented it in Rust, and it works fine. Here is the equivalent Rust code:

let mut out = Command::new("fzf")
    .stdout(Stdio::piped())
    .stdin(Stdio::piped())
    .spawn()?;
let stdin = out.stdin.as_mut().unwrap();
stdin.write_all(b"Hello\nWorld")?;
drop(stdin);
let output = out.wait_with_output()?;
let string = String::from_utf8(output.stdout)?;

Furthermore, it also obviously works in any normal shell, simply with:

echo "Hello\nWorld" | fzf

So something is weird with how Lisp (or uiop) runs programs.

The Solution

Honestly, I don't know. There is something weird going on I don't understand. Either this is a bug in uiop or I am missing some argument. Maybe there are some Lisp-magicians here to whom this is obvious, but I am stuck. Either I will find a resolution to this issue or will just rewrite this (small) Lisp program in another programming language.

Return to home