So I was randomly browsing and found this talk by Peter van Hardenberg, “Why Can’t We Make Simple Software?”. [1] This YouTube comment summarizes it well: «This presentation is basically the talk I have with every stakeholder at some point but most of it just gets dismissed as “nerd shit”» - JordanManfrey
In this talk, he defines the complexity of a system as the instances when different internal components of a system interact. The articulation between components allows for unpredictable behavior, as we do not live in an ideal world where independent, but connected, things always follow the happy path. This is especially true for the current model of architectures in bigger platforms: called microservices by some (those who want to be nice about it), but known as Big Ball of Mud, by the rest. [2]
He proceeds to list different kinds of complexity and immediate methods for mitigation. Starting with "Defensive Code", which refers to the measures to handle edge cases. For example, when the user gives invalid input, or tries to access restricted data. So you add validation and authorization to the endpoint, but now every endpoint needs this for validation and authorization; so you extract the function to use globally, but now you have some new endpoints which need specific handling; so you need to generalize the middleware, which leads to more interactions between components, and so on.
A potential mitigation pointed out by Peter is to delegate the complexity of unknown variables to the tools offered by the programming language, such as a strict type system, and external libraries. Although the idea of externalizing risk is quite appealing, too many people would stop here and go on with their programming, using all kinds of libraries they can find that minimize their effort. However, as a consequence of people taking this kind of advice, we have seen the rise of an endless barrage of awful libraries and frameworks, especially for web. Later in the presentation we can see a myriad of tools for cloud environments, and tools for development environments, and services for storing this and that, and software for managing all that. Suddenly, you end up with more components, more interactions, and extra complexity.
Interoperable standards are not the solution either. We have standards for browsers and the web, and yet, to make websites that work in any browser, you need a tool that translates code differently for each platform. The speaker even continues by saying that the rise of engines like Unity and Electron is due to the fact that they allow the abstraction of all the work that is done underneath. However, to maintain a sane and stable interface for the average developer, the work underneath has to be a true monster, handling all problems that any previous developer has met or may meet in the future. Later I will expand on this with the philosophy that "less is less".
Another topic introduced is scale, the crux of all non-technical project managers. I have had terrible experiences before with the, all too common, "what if more...", "what if we have 10^6 messages", "what if we have 10^9 users". When building a new tool, it is extremely hard to predict the actual loads on the system, and to write a tool for 100 or for 100 000 is two different kinds of problem. You have different priorities when catering to the specific few, or throwing together something that can work for any John Doe.
When listening to the requisites of the many you end up trying to create a multitool that fix the sink while calculating the best move in a game of Go. If you are entering in the middle of a big project, it's hard to redirect the full-speed train, but when in the early stages of a project, it is sensible to compromise and try to start with a small, robust tool: Then it can then be replicated as needed. Dedicating very early to the idea of MILLIONS, will require extra time and effort designing a flexible architecture and creating the foundation for all the weird edge cases and unknown future. By starting small and growing as needed, you get all the quick victories at first, learning about the project and growing with it, while also giving yourself space to adapt. Otherwise, development is rushed towards the idealized super-platform, which takes more time and can easily grind to a halt, stuck in the unknown without ever seeing the first end user.
I liked his way of presenting these two concepts. I will not delve too deep as he does a good explanation. It ties back to the use of dependencies to abstract complexity, because since we don't see the complexity, handled elsewhere, anymore, we believe we have more slack for adding to the project. New features are added, more complexity, which leads to a pushback and a restart of the cycle.
"[A]s a system has more demands on it, it responds by adding", otherwise it will "starve and die".
In the later stage of the talk, Peter starts introducing a new form of mitigation through destruction 💣💥 (and rebirth), similar to how a lizard might cut its tail to run off. Just as Joel Spolsky commented about the Excel team's motto: "Find the dependencies and eliminate them" [3]. Although it is a bit extreme to cut off all dependencies, they are the easiest this to remove without affecting the project's idea, as they are external tools and should not restrain the scope of the project; if they do, even more reason to gut them.
[3] Joel Spolsky about Excel team
This concept reminds me of a characteristic that I love about local and personal projects (even research projects have it), which is the fact that you are not bound by the tradition of the sunk cost fallacy. My application may be bad, but I can just `git reset --hard` to the first commit, choose better project structure, better architecture, better dependencies, and with starting over each time comes the experience of getting faster and wiser. Next time, I will know exactly how to structure my models and what frameworks to avoid. This is also one of the reasons I love rogue-like games, it's sad to see "You Lose" on the screen, but hey, you can always have another chance, and this time you will know better. There is nothing like "the forgiveness of a blank page", to forgive your previous mistakes, not to forget them, but to learn from them.
In this part Peter presents a playdate [4], a "tiny handheld game system", which also reminded me of a discussion on lemmy about tangara [5], "a portable music player". Recently we have seen more and more people against the jack-of-all-trades-applications with the use of "dumbphones", and dedicated single purpose tools, like the previous two examples. Unfortunately, I am not a die-hard fan of music, so much so to invest in getting a tangara, and also don't do much mobile console gaming.
Nonetheless, the point to be made is the fact that these are limited use devices, without many features, but are still loved by the community. By reducing the scope and the scale, the developers freed up time from expanding the device's applications and dedicated it to the effort of polishing the end result of the few features they have. When creating a product for someone else, you need to focus your scope and scale to what really matters, to the individual gains, cut the distractions.
This should not be confused with the modern trend of minimalism (which took the best brand logos from us). That is why I say "less it not more", a common slogan for this movement; in fact, less is only more in situations where we are already past the threshold of too much.
To conclude, the presentation reinforces the idea of "simplification [...] via amputation". With this he means that the main ideal to creating simple software is to focus on solving the problem with immediacy, without sidetracking, thinking about the comparisons between frameworks. In other words: listen to fewer demands and funnel the scope. This is not always straightforward: many problems are hidden behind layers and layers of abstractions and assumptions. But if we mold our assumptions about reality and explore the possibilities, we might find that our problem disappears if we solve another problem, which may be the origin of the first.
I am currently creating a tool where you can define a topology of services, users, inputs, and outputs. Initially, I wanted to allow the system admin to specify the names of each individually, which made validation and parsing of configuration a nightmare. So I funneled the scope. Now each service has a unique name, the user associated to each service has the same name, and the inputs and outputs, which don't interfere with each other, also have the same name as the service. With this the validator and parse are much simpler, and as a bonus, the sysadmin does not have so much configuration to do, as this functionality was abstracted away.
As a final conclusion I thought of this acronym: DSAP. When a project starts getting complex, with too many variables and moving parts, you start by gutting
Dependencies: if you are building around the nuances of frameworks or languages, find alternatives. The balance between complexity added or removed by a given dependency should be continually analyzed as the project develops. One key thing to remember is that you are making a tool to solve a problem, not a toy for which the only purpose is to be used. External dependencies are the easiest to remove without affecting the overall structure and objective of the project;
Scope: if a part of the project feels more like an appendix, than a main chapter, chop it; if needed, reuse it in another spinoff project. You generally see this in microservices architectures, because you have multiple independent services which hold information and do processing that is accessory to the central functioning of the product;
Architecture: once the dependencies and scope are tightly packed, and there is still complexity, you need to guarantee the consistency and coherence of the architecture. Complexity comes from articulation of components, so it can be reduced through merging and spliting of actions, such as the switch between layered monolith and modular layered monolith architectures;
Project: is the last to go. "Tabula rasa" is the term that comes to mind when it comes to this point. If all the designing, preparing, and assumptions made until now are not working, the whole project needs an overhaul. It is time to concede and accept a "New Game".
Some quotes that stayed with me:
"simplification [...] via amputation"
"the forgiveness of a blank page"
"be deliberate about how and when you take on that complexity"
"it never gets easier, you just go faster" - Greg LeMond
Next up: Alan Kay's "Power of Simplicity"