Comment by rzwitserloot on 27/01/2025 at 00:21 UTC*

49 upvotes, 1 direct replies (showing 1)

View submission: Massive Failure on the Product

Chalk this up to a pricey lesson: Death marching is **extremely dangerous, not to be undertaken lightly.**

If that's too nuanced a point and need it simplified, okay then: **Do not ever deathmarch**.

To explain it in a way that relates to your situation:

After multiple 12+ hour sessions, the state of the delivered product is, *of course it is*, in a fairly precarious, unstable state.

The usual fix is to simply not do that. Not just the 12 hour thing - work 12 hour days if you must. No, the thing that tends to make people work 12-16 hour days: Unreasonable deadlines.

The problem with those is that pretty much by definition, the 'stuff we still have to do' list is too large to fathom in a single human brain, and yet there is clearly no time to take any clarity that is gained when implementing stuff somewhere along the path to the final product and adjust the earlier stuff to take into account this clarity. After all, IF you feel it is necessary to work 12-16 hour days to deliver the stuff that still needs to be done, obviously there is no time to adjust already-done tasks.

So instead you get out your twine, tape, and spit, and you just stumble about a bit, apply a whole bunch of shortcuts and 'works for me', and move on to the next item on the endless, *endless* todolist.

And that, naturally, leads to unstable software. Which has a nasty tendency to fail exactly when it matters: devs testing the stuff they write has the nasty tendency to fail to cover 'real life', because those scenarios tend not to quite match what devs do. One trivial example for websites, as we're in `/r/webdev`: Users tend to connect to your site simultaneously. And yet devs clicking around tend not to generate concurrent situations. Concurrent situations if not written 'properly' tend to cause things to end up in invalid states: Bugs that take down signup forms until someone fixes it.

Hence, **just do not do it**. If you must, because, hey, we've all been there (or at least, I have), you *can do it*, but know a few things:

• There should be a post mortem: If there's a need to pull a 12-16 hour day, let alone a few, somebody fucked up the planning and it needs to be reviewed. This is not good for code quality and customer satisfaction, let alone your programmers' sanity. Somebody needs to apologise, figure out what went wrong, and take steps to prevent it happening again.

• There needs to be *extra downtime* afterwards to clean up the shit. All code written in the crunchtime (and there will be loads) needs to be extensively reviewed afterwards. Wipe the slates clean: No new todos for 2 to 3 weeks afterwards. These are the costs of unreasonable deadlines.

• The team cannot rest on 'release day', they need to stay on call and be ready, at a moments notice, to fix problems, because there will be problems. It sounds like you guys really messed up on this one. For web dev, often this means 24/7 coverage for 48 hours; set up a schedule!

if you want it stated in a way that is easy to convey to folks who might not really get what software dev is about, here's a parable:

• One day, you walk into the forest and meet a lumberjack who is really whaling away at a tree. They tell you, whilst continuing to chop, that they've been at it for 20 hours, are dead tired, but they have to clear this patch. You notice the axe is completely blunt, and there's lots of trees left. You offer to sharpen it, but the lumberjack says: "DID YOU NOT HEAR WHAT I SAID? NO TIME, NO TIME! MUST CONTINUE TO CHOP!".*

That lumberjack is an idiot. Don't be like that lumberjack.

Replies

Comment by Sordino54 at 27/01/2025 at 13:25 UTC

1 upvotes, 0 direct replies

This guy has worked on enterprise level systems/teams and everything they said is 100% accurate.