I’m currently interested in a tool that will help me set the mood for a role-playing game session. Basically something that will help me generate a movie soundtrack on the fly. The soundtrack can be simple but it has to vary over time, per scene. And switches have to be easy.
So I’m thinking that perhaps *procedurally* generated soundscapes would be perfect for a simple user interface containing a bunch of sliders for a person telling a story or for a group of table top role-players to adapt to the current mood, on the fly. I know that manipulating actual track lists is to complex because my brain is elsewhere, and there’s no way to gradually shift from one mood to another as context demands.
It has to be really small and simple. Apps like Ambient Mixer are already way to complicated for what I have in mind. It would probably be a web app for max usability? Do you think it would be possible to write the sound modules (generators?) attached to sliders all in Javascript or what would we use? It would be cool if other people could add more over time.
Sadly, I’m only a wannabe audio hacker. A friend of mine is into synthesizers. I just gawk at it and scratch my head. But I’d like to learn.
My limit is fooling around with the Animoog and PixiTracker apps on the iPad.
On Mastodon, I started talking to @Miredly who recommend Max (”a visual programming language for music and multimedia”), and there is a Free Software alternative called Pure Data. I have faint memories of LabVIEW. I decided to start reading the Pure Data manual. I’m stunned and overwhelmed. A whole new world of graphical programming. Do I have time for all of this? Life is too short!
Anyway, I’m still thinking about a mood music thing. What kind of user interface would it have? How about a something that generates many sounds, and they all play at the same time, and we have a 2D slider, a kind of Law/Chaos and Good/Evil alignment thing. As you move around, the generators produce anything from angelic choirs to death growls, dungeon drones, forest birds, happy flutes, marching boots, and so on. I’m still wondering: do I need to understand synthesizers and patches for this?
More thoughts about the user interface of this web application I’m thinking off. If we have a spatial arrangement of sound sources, then perhaps we can “walk” through space and experience a different mix of all the sources. For example, using the LOVE game engine, you an have sources in space, and a listener that moves through this space. But sadly, those sources are probably just audio files, not generators. This clearly requires more thought and perhaps some exploration.
All the while I’m still reading through the Pure Data documentation. The idea of the tutorial letting me build a simple synthesizer myself has appeal. I’ll have to see how entertaining that is, in the end.
@Miredly sent me a little *patch* (a visual program for Pure Data that shows how one would program it) and offered to send examples of how to start implementing an environment with sound sources in Unity, I started feeling way in over my head I just can’t add more unknown tech to the stack. I think I need to connect the thing I want to do with a lot of things I know already and move incrementally. Managing the overwhelmedness is where I’m at.
OK, another idea: assuming I didn’t know anything about writing music generating code: take a generator that takes 100 input parameters to generate 2min samples, pick a number of music pieces and categorise them manually (Hogwarts great hall, burning the Jedi temple, tears in rain, in the mood for love, etc), use competitive AI training to create similar sounding pieces, then have sliders to move around the parameter space; mapping 2-3 sliders to 100 parameters might work due to categorisation?
I hate the buzzword level in that toot. Makes me want to stand up and shout “bullshit!” But it sounds so attractive because we’re not writing the code, we’re “searching” for the code, I guess? Bullshit Bingo
Why 100 parameters? I have no idea. I figured it was probably more than 10 so I just picked the magnitude above that.
Here’s why Deep Learning is probably not for me. The blog post Generating music with deep learning mentions NSynth (”NSynth is a WaveNet-based autoencoder for synthesizing audio”) saying the following about training the net:
Generating music with deep learning
The WaveNet model takes around 10 days on 32 K40 gpus (synchronous) to converge at ~200k iterations.
The blog post then argues:
Training a model like that would cost more than $5,000 using Google Cloud resources. Of course, it is possible that a simpler model could be trained faster and cheaper, but the example still shows that some technologies are most definitely not available for everyone. We live in a time where there is great access to many technological advances, but the availability is often limited in practice, because of the scale at which the technologies need to operate.”
I agree. Should you want to look into it, here are some links:
I found Neural Nets for Generating Music to be a nice introduction full of links. It’s a good read if you’re as ignorant about the history of this topic as I am.
Neural Nets for Generating Music
Anyway, here’s another user interface idea. Assume this was a tool to generate a soundtrack for a silent movie. Two axis will not suffice. The first scene is the imperator then; the next is the Jedi temple; the next is a battle; etc. So how about each theme being the corners of a triangle, first in, first out. If the next scene is Padmé then the imperator drops off the triangle. Within the triangle, a button mixes the three themes. This would get around the two axis limitation. I really need a sound generator that takes some parameters.
The MDN docs are so cool. I started reading Basic concepts behind Web Audio API.
Basic concepts behind Web Audio API
A few years ago I was interested in Holly Herndon and her music. The blog post I wrote at the time led me to the tutorial “Generating accessible music with Clojure and Overtone (and a primer in Music theory)” at Tenor. But I didn’t pursue it, feeling that I just knew too little about music theory.
“Overtone is an open source audio environment designed to explore new musical ideas from synthesis and sampling to instrument building, live-coding and collaborative jamming. We combine the powerful SuperCollider audio engine, with Clojure, a state of-the-art lisp, to create an intoxicating interactive sonic experience.”
Sam Aaron of Overtone moved his efforts to Sonic Pi, as far as I remember. “The Live Coding Music Synth for Everyone.” It’s simple, it’s cute, it has good documentation, and seeing the videos where he’s live coding are glorious. Such enthusiasm.
Anyway, so I’m here, somewhere. I was just clicking through a set of slides which I loved. Watch out: auto plays sound, so watch the volume! Generative Music in JavaScript & Web Audio I’ve read the name Brian Eno a few times and feel like once again like I’m standing at the foot of this mountain and see that not only have people been here, but they have been here decades ago. Oh well. Still got to make my baby steps.
Generative Music in JavaScript & Web Audio
That slide set led me to the huge blog post and tutorial talking about the very thing: JavaScript Systems Music: Learning Web Audio by Recreating The Works of Steve Reich and Brian Eno. This is great. I shall try to take a longer look.
JavaScript Systems Music: Learning Web Audio by Recreating The Works of Steve Reich and Brian Eno
And now for some links to Javascript libraries, for reference.
And for the hard core there’s the original W3C recommendation, of course:
Getting Started with Web Audio API
#Music #Programming
(Please contact me if you want to remove your comment.)
⁂
Another weird idea: “P22 Music Text Composition Generator” somehow translates letters to notes, using some sort of music font. Sadly, it requires Flash. – Alex
---
I would also like a project where I get to use Fennel, a Lua Lisp Language. @technomancy wrote up a blog post about what Fennel is, how it works, and why it’s so interesting, in which a compiler gets its wings.
in which a compiler gets its wings
Perhaps I need to check List of languages that compile to JS again.
List of languages that compile to JS
– Alex
---
It’s late at night and I’m listening to more weird music, semi generated or something. Terry Riley’s In C; and I’m reading the article that promises to explain this all. I’m fascinated by “a form of music where these kinds of open ended processes are front and center”.
@drm replied: «If you like “In C” (or at least the concept behind it), i also recommend “Canto ostinato” from Simeon ten hold (available on yt) or “Drumming” (and other stuff) from Steve Reich. (Haitian) Vodou drumming also shares the concept of small, combined patterns. A bit like the unix command line 😉 There is also “In D” from Arandel, which is completely different but also very nice imho.»
– Alex 2018-03-08 06:42 UTC
---
@julienxx recently mentioned using Rust and WebAssembly. This is the starting point he used: Web Audio.
– Alex Schroeder 2019-10-27 08:44 UTC
---
Chris Beckstrom is working on a techno-generating python script:
a deterministic techno generator for tidalcycles: input an integer value as a seed and it spits out some techno ready to play in tidalcycles – Auto-Techno Generator
– Alex 2021-10-09 18:36 UTC