đŸ Archived View for dioskouroi.xyz âș thread âș 29368457 captured on 2021-11-30 at 20:18:30. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
________________________________________________________________________________
I don't accept jobs that want me to be on call unless fuck you money is involved, and so far nobody has offered it to me. The product that drives most of my employer's revenue is a gigantic pile of tech debt, and there are some people on call whose lives are kind of miserable.
What some people struggle to realize is how easily the responsibility of an individual can scale at certain companies, without their compensation improving in the slightest. If you are working on-call and not making big bucks your company is likely milking you dry.
> If you are working on-call and not making big bucks
Both of those are very relative. I've seen on-call which means daily events, and on-call which requires something once a year. There's a very different level of compensations I'd be happy with in each case.
> I don't accept jobs that want me to be on call unless fuck you money is involved, and so far nobody has offered it to me. The product that drives most of my employer's revenue is a gigantic pile of tech debt, and there are some people on call whose lives are kind of miserable.
Yeah this is the right answer. I know a lot of people who are earning more than I am in exchange for being on-call (or being de-facto on call, ahem) and it just ain't worth it.
This is on point. I think the reason people do it without adequate pay is that some people like the feeling of being needed.
Actually itâs the inability to say a simple ânoâ in most cases.
What rates are typical for being on-call for a week? (my team has a rotating weekly roster, Australian team).
In Finland the collective-agreement that covers IT work defines this. The way I've been paid has been the same across several companies:
You get paid 50% of your normal hourly salary for every hour you're "available".
You get paid 200% of your normal hourly salary for every hour you're working on an issue, incident, outage, or problem.
So being on-call is pretty lucrative, given that you'd usually do it for a seven day period. (i.e. 9am Monday to 9am the following Monday.)
Australian Team, We get $20 per day (pre tax) oncall allowance, then time and 1/2 for any work done, time calculated to when you can go back to doing what you were previously. Generally works out to a minimum of 30 minutes overtime for any calls.
If you don't get 10 hours continuous break before the next work day, you fall into fatigue leave and either get paid overtime for the next days work or are stood down till you have had a 10 hr break.
Still doesn't stop our operations team being completely snowed with both workday and oncall work, but they can make a fairly good bonus wage out of it.
Your ops team might be at my old employer. Ops team had the above benefits but no one in SaaS Engineering (devs) got anything as second level support.
Yeah, we seem to contract second level support from the lowest bidder. Tick, we have a support contract, regardless of how dubious it's value.
We get a flat ~800$ for an on call week (Thursday to Thursday) and Friday is off. Nothing extra for working night, but youâre not expected to work a full day if you had an incident the night before.
At my previous employer we got $4/hour for every hour on call outside normal business hours. So $64 per work day or $96 on a weekend or holiday or $512 for a typical week. Paid quarterly.
One of my reports pointed out that he made more from on call each year than the difference between an average and outstanding performance bonus. He wasnât wrong.
$35 per day for me. 1.5x for the first 3 hours. 2.5x for every hour after.
Minimum call out fee of 2 hours regardless of the length of the call.
Also in Australia. This only started when I joined as an SRE.
Prior to that the Devs just said no it'll stay broken till we come back to work and that was largely acceptable. The on call rates only started as our reliability became more of a business need.
I bill my hourly rate for all on-call hours regardless of if I'm called. If my employer wants any amount of control over my time (sobriety, proximity, etc) then they pay my regular hourly rate for it.
Most will not accept these terms, and that's the goal.
That's the way to go. On-call is bearable if you get paid for it. As a contractor who charged by the hour I was perfectly fine with sitting in useless meetings or fixing things at night that weren't even my fault. Doing this without extra compensation is just not OK.
And of course it should be that way: if they want your time (you not being able to do other jobs, you not being able to relax, etc) they should pay for it.
Forcing them to think in your hours can also have the nice side effect that they get their communications in order.
what would you say is big bucks?
A surgeon friend of mine gets $1000 per day that he's not working but on-call.
And what does it go to if he gets called in?
I donât know but itâs a hell of a lot more than anything I make.
levels.fyi
Nice!
FWIW the original definition of "fuck you money" was enough money to be able to say "fuck you" to anyone, which I conservatively estimate at a few billion, but most likely more than that.
It seems these days to just mean "multiple millions", which reduces the utility of the term.
A few millions are sufficient for you to be able to say "fuck you" to any employer or person you theoretically depend on, while still being able to live a comfortable live and - possibly - defend your right in court, even if it does not make economical sense to do so. That's what most people mean.
Having a few billion gives you some additional fuck you privileges - police officers, world leaders, other billionaires -, but you probably won't get on a level where you can say that to anyone and not fear repercussions. Also, if you can live happily on your own, you can already tell them that if you have a few million to your name.
In the US, for legal protection, I would say $10M is a good number based on legal costs I have seen. Maybe $5M works too, but I would say $5M protects you from healthcare costs and not being able to earn income due to health issues, but a little more is nice for paying lawyers if you are also not able to work at the same time.
There's always a bigger fish, but a few million is sufficient for almost every interaction most people will have in their lifetimes.
Unless you live in the US and get cancer.
This got downvoted, but I think is a legitimate concern. Having a terminal disease in the US can drain bank accounts quickly. Itâs a real hazard that can destroy families / relationships.
This misses the key point that on-call itself is an abusive practice regardless of how well an organization practices it.
If an organization thinks their systems should be available 24x7 they should staff people 24x7.
There is no situation where itâs ok to contact someone to do work outside of their work hours.
"There is no situation where itâs ok to contact someone to do work outside of their work hours."
Here's one. I get paged for urgent fixes maybe three times per year. To staff all three shifts with senior engineers to cover for that would be enough to make the company unprofitable and lose many jobs. I don't much like being woken in the middle of the night for such things, but it's clearly necessary and not abusive.
Yeah,my issue with being on call is that i can't ever really let loose if I'm expected to be contactable anytime 24/7...what if I'm out with friends, drunk, high, on the side of a mountain? Etc.
My hours are 9-5, after that is entirely me time and if you expect me to br contact able and in s state of contribute you're going to be disappointed.
No amount of money is worth losing that me time.
Usually, being "on call". should mean no alcoho, and have inet + computer closed by i.e. 5-15min or so.
which also means that this should be time that is paid on some hourly rate for the lower quality of live during the on-call time.
Also, there should be a rotation, with enough people to make this bearable.
Are you describing the current state of on-call? Because that is what it is.
There should be no rotation; there should be no on-call in the first place. If hiring staff to work normal hours is too expensive for the company, then the company has bigger problems than a service going down.
I'm confused. What's your plan for fixing Gmail on a Saturday night? Hope one of the devs doesn't have a social life?
Google is big enough to have teams around the world, all in their own daytime zone.
It's still the weekend wherever you are if it's Saturday in North America
If you have teams in Israel, you at least can cheat a bit on Sunday v.s Friday.
And having at least one daytime rotation agree that "well, you're expected to be on-call during business hours on Saturday" isn't nearly as crazy as "you can't relax at any time during the weekend".
Or just, like... allow people to have a Sun/Mon weekend (or Mon/Tue, or whatever) instead of Sat/Sun.
There are plenty of people who would be happy with that and plenty more who would accept that arrangement if it came with extra compensation.
Instead, particularly at big companies, people get hired with some generic comp. package and then it's luck of the draw whether you have an on-call rotation or not, and if you do whether it's onerous or not.
Employees and employers should be able to negotiate alternative weekends in that case. These alternate days are dealt with like real weekends with the real force of law and societal expectations.
> what if I'm out with friends, drunk, high, on the side of a mountain
You can plan around it, assuming a reasonable oncall schedule. If its every other week, yeah screw that. If its a week out of 4, you can plan it and still have it better than virtually every other highly paid profession there is, and many low pay ones.
> My hours are 9-5, after that is entirely me time
Then maybe an hourly job is better than a salaried one... Of course, if you're not paid the part, it's not worth it. But if you're paid 150, 200k or more? I hope you're not expecting 9-5.
> But if you're paid 150, 200k or more? I hope you're not expecting 9-5.
Speak for yourself, but I would never accept a $150k a year gig that expected me to be on-call, say, 1 week out of every 6. $150k is absolutely a "9-5 and stop thinking about work when you get home" kind of salary, if we're talking California standards
On call typically means you gotta be online and respond within 15 minutes if paged. That's abusive. If you want a person to do that, then pay them for that. As an example, on call in Amazon works the way I described, and yes it is abusive.
Unless you're working hourly or the on-call duties weren't disclosed during salary negotiations, you're already paid for it. I'll happily take a call after hours if it means I'm in a cushy job being paid 200k/year to solve puzzles. My partner manages a Safeway and makes well under half what I do for easily twice the work, and he's effectively on-call 24/7 for his store. Is either situation ideal? No, but I recognize that even with the injustice of having to ack an occasional alert at 2am, I'm in a crazy fucking comfy spot.
Yeah, "solving puzzles". If that's the case, surely it can wait - why does it need to be solved at 2 am? Anyway, if you're gonna be a slave for $200k please go ahead, its yo choice.
The question is whether it's abusive or not, not whether you personally are willing to accept being abused for X amount of money.
I would think the question of whether it's "abusive" should at some level be determined by observing labor practices elsewhere. Even if you don't, I doubt anyone is going to shed any tears for something "objectively" abusive if their own work situation is much worse.
> I would think the question of whether it's "abusive" should at some level be determined by observing labor practices elsewhere
Let's take this idea to it's logical extreme then. Surely you would agree that slavery is abusive. If we imagine a society where there are only slaves and slaveowners, your position would lead one to believe that slavery is not abusive, simply because it's the status quo.
This practice is abusive because it subverts the agreement that one will exchange their labor for pay within a defined set of hours (8 hours per day) and replaces it with the expectation (not agreement) that one will be "available" 24/7, but not actually "working" unless a pager goes off. This is plainly abusive, because it destroys your ability to use your free time to do things like drink alcohol, smoke marijuana, go hiking, go sailing, go for a run, etc. because you are required to be online with 15 minutes notice.
OK, fine, maybe that doesnât work as a universal principle. But I am compensated very well for a job where I rarely have to do anything that demanding and in exchange I canât easily make plans for one week every two months. Hardly comparable to being a helot.
> This practice is abusive because it subverts the agreement that one will exchange their labor for pay within a defined set of hours (8 hours per day)
But for salaried employees, this isn't the agreement. I sometimes work four hour days. Sometimes I work odd hours. The agreement, for salaried employees, isnt about hours, it's about work getting done.
If someone consents to that, swell! I think many companies are abusive in that they don't compensate oncall work enough, but salaried oncall positions aren't inherently more abusive then having a lawyer on retainer.
Agreed. There are way too many people in our industry who accept being treated like utter garbage, and that's a problem, but we should also be clear that at the end of the day people do need to learn to stand up to themselves and not to consent to insanity.
My previous job I was on-call 1 week out of every 6. My first on-call week I got paged 100+ times (obv many of these alerts were firing within a minute of each other, but even accounting for that I was still being woken up multiple times per night). That week was particularly bad but I was still woken up an average of >= once per week on my on-call weeks. My team - the sole SRE team - was on-call for everything, including of course the 6 other dev teams' services.
Even worse, when I would fight to get bullshit alerts removed (the alerts that don't actually indicate a problem), I would get incredible pushback from, of all people, my own manager! (who was the manager of the SRE team since like I mentioned we were the only SRE team)
So not only were half the alerts bullshit, but I was counter incentivized to actually fix the problems causing the alerts. One great indicator of how bad things were was that everyone else on the SRE team (except the manager but he somehow always had unusually light on-call weeks) had a 5 minute delay set for pages, because the vast majority of pages would resolve within 1-3 minutes.
Oh, and by the way we weren't given any extra compensation whatsoever. Indeed I was told it was "part of the job description", which effectively meant that SRE skills were less valuable than the devs, because the devs were making identical salaries with no on-call requirement whatsoever! So a more specialized and, at least at this organization, difficult skillset, was worth less.
Anyway, I quit that job, and never looked back. I still have a friend on that exact team who is still putting up with being on-call, despite the incredible impact it has on his ability to go out and do stuff (mid 20's guy). I hear him complain about it all the time. But, like most people, he doesn't have the balls to either (a) push for internal change (which in fairness I tried and failed, although half the reason I failed was because nobody else on my team was willing to stick their neck out and push for the change with me), or (b) quit and find a new job.
So...yeah. Kind of rambling but we have a huge problem in our industry with people who either "don't have lives", or kind of do yet have so little self esteem or whatnot that they can't actually say no and protect their personal lives. In most cases on-call is simply a case of someone getting a raw deal and being too afraid to admit it to themselves.
OK, that's definitely a shitty situation. But if you imagine that instead you were empowered to shut off nonsense alerts and address causes of real ones, you wouldn't have actually eliminated on-call, so I'm a little confused about how you conclude from this that "all" on-call is abusive.
What's actually happening in this thread is that people like your friend are post-hoc justifying their own abuse so that they can remain internally consistent with their past decisions. Classic cognitive dissonance.
> The question is whether it's abusive or not
The question on whether it's abusive or not very obviously depends on the details. Yes, lol, I am actually willing to accept $250k per year, unlimited time off that I use upwards of 6 weeks per year of, full benefits, working from home, in the event my boss needs to call me once a year on a Saturday.
Sweeping generalizations are not helpful, and will likely hold you back in your career. You should evaluate things on their own merits.
I interviewed at Brex, and they required getting online within 5 minutes of being paged. (I did not get the offer, but I would not have accepted the offer after learning that.) Where I currently work the response time is "ideally within 15 minutes", which is a lot more manageable, since plenty of places I like to go are within 15 minutes of my house. My job pays more than average because of on-call, it's 1 week out of 6, and there's probably an average of 1 call out of business hours per week (mode is 0).
5 minutes is absurd. You'd basically have to keep your laptop with you at all times
One argument is that you _are being paid_ for it; itâs just not broken out into two lines, but the dollars are there.
If youâre making $200K/yr, maybe $150K of that is for your job and $50K is for being on-call. That seems like a more than fair price (or at least one thatâs in the ballpark) for periodic on-call service.
In my experience, often people paid _more_ end up not having on-call duties. I know several people who have negotiated high salaries from the start of their employment along with a stipulation that they will never be on the on-call rotation.
In my own situation, I removed myself from on-call duties during a leave of absence, and never went back on after I returned to work. Since then, I've still gotten the same raises I got before.
An unfortunate reality of on call is that companies care about projects delivered, not disasters averted.
Unless your stepping in to right the ship when a company is on fire operationally, and can point to specific action/results you delivered to right the ship - getting pages 8x per week does nothing for your career.
If that was the case we could easily see it in the salaries of on-call versus not on-call positions. This is clearly not the case.
That's mostly because our industry is a bit of a mess. We clearly have different people doing completely different roles that are in the same career ladder with the same title. Forget on call, some folks with the same titles and same salaries often do easier or harder jobs. That's a separate problem.
Salaries in the industry have a lot of expectations baked in, like the expectations of being able to learn and train yourself without the company having to pay for it (aside the occasional paid book). Being on call is another. That some folks can get the same pay regardless of their responsibilities or how hard their job actually is is, indeed, a problem.
>>On call typically means you gotta be online and respond within 15 minutes if paged.
No place I have ever worked has this kind of oncall policy.
Most I have seen have a response time of 1hr, note resolution, but response.
Amazon is large enough they should have people staffed 24/7 just by staggering the timezone codes at the different offices. This is how Cisco TAC works, you call them you get what ever timezone code is "day time" at that time.
So I would agree that 15min on call is abusive, luckily this is not my experience as "typical" in the industry, hell even fully staffed 24/7 call centers typically do not have a 15min response time
Call centers and oncall are different. Oncall is "when Facebook suddenly stops working, who is responsible for figuring out why and fixing it". These systems often have SLAs (either internal or public) in minutes. Response (ack and log on) times for these can be 30 minutes for less relevant services, but some have 15, or even 1-5 minute response times for highly critical stuff.
1-5 minutes response time is not "Oncall". It's being at work.
In France you're necessarily compensated for being on call and then paid when you are paged.
Furthermore, if you do get called, you shouldn't go back to work until 11 hours after you finished handling that call.
French salaries for senior developers are less than entry level developers in the US.
I have bootcamp students with 6 months of software development experience making more than top developers in France, lol. The few extra Euros you get for being on call hardly seems worth it.
There's a reason French workers are almost always rioting. Low pay and very few opportunities.
How do salaries in France compare to the US?
I don't know about France but in my country that's also factored in the law and hiring offers except for jobs that are on managerial level (them the law says you don't have clock in but you also pretty much don't get to clock out and you mostly get paid well for that), which doesn't apply for developers and that means you get paid less if there is no "on call". Basically your base pay is the same on company A and company B (assuming they are truly equal), but since company B requires "on call" that is extra work that gets compensated based on your hourly rate or double that if it is during night time, in the same way for the same job in the same company who ever works night shifts get paid twice as much as the daily shift (before taxes).
3'000 to 4500 USD monthly for a SRE
They do pay people for that - the salaries at MAANG are inflated as is.
This sounds a bit hard to believe. Mind sharing some of the $ figures of this business?
Nevertheless, I'm with the parent. This practice is abusive. When you're not working, you're not working. Being on call is working. If a company wants you to work longer hours they should make it explicit, pay for it, and comply with the local employment laws. This sort of argument that the business has no merit if they were to hire more people or pay can be applied to many exploitative situations. Most of the companies doing this are drowning in profits.
My company pays for on-call. There is a rotation, you go for a week and they pay you more for that week by several hundred AUDs and it is basically a job requirement.
Does the fact that it is discussed during hiring process and that it is paid change you opinion about being oncall?
I think that sounds fair. The concept of being on call is not unique to software engineers. You might be an emergency doctor. You might be a fighter pilot. It's possible that some software engineers need to be on call, because they support some critical infrastructure, and that can be part of their contract.
I would say the default assumption for most software engineers should be they are not on call and the company should try to structure itself around that assumption. The problem starts it's just considered the norm that software engineers are expected to work 24/7.
I am of the mindset that a healthy team doesn't need an on call. Group chats, tight relationships, etc. usually mean someone is free just based on the odds to handle a true emergency. If you have so few people that the ship can't run, but you're not in a startup environment that justifies having so few people, that is your real problem.
An average company with engineers who don't hate their job, staffed adequately, it should be a stars align kind of scenario where you can't get someone on a weekend. It's a simple matter of probability, and if that probability looks grim, either A. you're understaffed, or B, your staff is underengaged.
I have fixed customer outages on my phone out having a good time more than once. Usually with another engineer happy to help. Not just at one company. If you need an on call rotation, your culture sucks. I'm not advocating that people live to work, I'm advocating that people work at places where people enjoy working with their peers and their product enough that an occasional blip isn't a big deal. With the right infrastructure and the right people you should have this.
> Group chats, tight relationships, etc. usually mean someone is free just based on the odds to handle a true emergency.
Congratulations, you've made you entire team on call year round.
I don't read work chats outside of core working hours if I'm not currently on-call. In general, most people shouldn't. Doing so is awful for your stress levels and work-life balance. I did it for more than a decade, burning out twice in the meantime, and things have been much better since I stopped.
I don't read group chats _inside_ working hours. I'm working at work, not group chatting.
If that means you're reading your chats later it sounds as if you're also working outside working hours.
I was on a team that handled a lot of things that way, and we made a deliberate, conscious shift to pull them into oncall rotations. We found that it was severely compromising work-life balance; engineers outside of the old guard thought they were expected to work _most_ nights and _most_ weekends, because they'd constantly see people get bonuses and accolades for some stuff they did at 6PM Saturday. I personally enjoy the "work whenever you want, play whenever you want" model, but I really don't think it works as an oncall policy.
The only thing worse than on-call is the expectation that everyone should be available at all times.
Please refer to it as "culture," not "expectations," the latter makes people uncomfortable.
Iâm on a team of six in the same time zone. None of us are going to conveniently be watching the alerts at 4 AM Sunday. One of us has to get woken up who wasnât planning to be well rested for anything that day.
Ah yes, the "always be working" method. That way you never need overtime!
no buddy your idea sounds like unlimited vacation days, it is worse than the original method
If they canât afford to staff for 24x7 operation and maintain profitability thatâs their problem. They should either reduce their service hours or charge more.
By working for free youâre subsidizing your organizationâs unsustainable business model.
Getting paid 200k+ for a job that had on-call as part of stated duties (and thus baked into the comp) prior to accepting the offer is now working for free? There is hyperbole and then there is whatever this is.
Donât like it, quit and find a company that is moving slowly and doesnât value ownership from engineers, someone else will fill in the vacant post. Sounds like a reasonably sustainable business model to me, especially since it has worked like this for over decades at some companies.
There are plenty of jobs in good companies that pay well and don't require you to be a slave to your job. Companies that can move fast and treat their employees well vs. companies that exploit their employees. I am a manager in such a company and my team can move fast and own stuff during the work day and then relax on their weekend so they can get back to moving fast and owning stuff when they're back at work.
This has totally not been this way for decades.
Now if you're a founder, or you're being paid for this extra work or own a significant part of the business and benefit from it's success, that's a little bit different.
"Getting paid 200k+ for a job that had on-call as part of stated duties (and thus baked into the comp) prior to accepting the offer is now working for free? There is hyperbole and then there is whatever this is."
The lawyers at my company go home at 5 and make way more. I know a doctor who gets paid somewhere around $1500 for every day he is on call. And when they actually call him he makes even more.
You're likely very wrong about the lawyers. They leave at 5 when nothing is urgent. But in certain crisis situations -- company being bought out, a severe injury occurred on premises, a client is suing you, etc -- those lawyers will be expected to respond 24/7 all through a full weekend or beyond.
Of course, you can still go to law school if you're really that jealous.
How many IT people on call are actually making 200+k? The general rule I have seen is people on call get paid less as people eventually negotiate very slightly lower salaries for not being on call.
I make well over $200k with bonuses and equity. I am on call right now.
I just DoorDashed lobster tail. Life is so unfair.
Amazing to see all these awful takes. This _is_ America I suppose.
You realize this isn't just a 200K programmer problem ? People getting paid 60K to do IT support (or other support for various operations systems) also get roped into the same exact on-call duty. (Often this not advertised either...) It's growing problem from penny pinching tech companies that want to tout 24/7 support without paying for 24/7 support.
Below a certain level I mostly agree with you. One company in my past didn't work this way, and one too many pages in the middle of the night for a business I wasn't personally invested in made it an easy job to walk away from.
That said, there is a level of seniority at which "their problem" is "your problem." If you'd chuck the business out the door rather than getting paged a few times a year, you're not ready for that level yet. This varies by company and organization size, but fundamentally you can't (and shouldn't try to) anticipate everything up front. Sometimes you need the knowledge, judgement, or simply signing authority (literal or metaphorical) of someone specific.
> That said, there is a level of seniority at which "their problem" is "your problem.â
Iâve long decided that unless I have the significant stock options to show for it itâs delusional. Itâs what the company wants you to think: to have all the emotional investment and none of the actual ownership benefits.
One can debate the title/comp level where this change happens but, certainly, at some executive level it's reasonable to expect to be contacted off-hours if there's some customer/PR/facility/etc. crisis. That's not to say an exec can never be out of touch; a friend told me that their former CEO had a new communications head quit because they couldn't deal with said CEO telling them that they should just deal with PR issues if he was on vacation.But, above a certain level, traveling on the weekend, being away nights, late night calls, and yes the occasional "Houston, we have a problem" moment happen.
I am quit amazed at the general reaction of this forum to this standard industry practice.
If you want a cushy 9-5 job, go work for a code factory like HCL, TCS and whatnot. That way you can build crap and wash your hands after handling it to the customer.
In a startup environment, you gotta build your shit and maintain it to. It's an environment that's not for everyone.
I've been in oncall roosters several times in the past. And it made freaking sure we wrote the best software we could.
I also have been on the management side of it, and the way I set it up is that whoever was the oncall engineer, if he had to work for more than 2 hours after normal working hours, he would get an additional PTO day. It actually worked quite well.
I think most experienced engineers realize on-call is a complete fool's game -- you will never get accolades for fixing shit at 2am. Literally never. Your only "reward" will be to become the escalation point to continue getting called at 2am. It's not only mentally demanding work, but also bottom-feeding work that gets no recognition. No one is doing their best work after being woken up in the middle of the night, it's all band-aid hacky fix shit that is being generated. Also in general the type of personality who would tolerate working 2+ hours beyond their working time is probably the type of person that isn't even going to use up all of their PTO to begin with, so I always find those kinds of rewards laughable.
Well, putting the pipe dreams people have about a global team so nobody is ever working outside regular working hours aside (hope you've considered holidays, guys), what's the alternative? One I've seen is that there's an ops, or "devops," team, that's responsible for picking up the slack instead. But if it's unfair for engineers to have to do it, it seems even less fair for someone who doesn't even have control over what code ships to do it, and to do it without a rotation.
> what's the alternative?
Let it crash. If you can't afford to pay people enough that someone /wants/ to fix it at 2am, and you can't arrange a retainer for a contractor halfway around the world to be available if needed during their worktime, and you (owner/manager) don't want to do it yourself, then it can be off until someone starts at 9am next business day. (Most products and services aren't life critical).
I think the whole point of the article and the discussion is that many people donât want to do it _even when paid enough_.
Nobody on any of the threads is arguing that you should not be paid more for being on call.
The issue with being on call constantly is, that you have to basically give up any private life you should have. Your daughters theatre play? Sad to interrupt! Your long awaited weekend trip with your girlfriend? Would be a shame if you had to cancel!
The quality of life lost by being on call is not insignificant. This is why being on call should always be the absolute exception and not a reliable, planned in, tool at your companies disposal.
If you regularily fail to organize your core business around business hours, maybe you should get someone who is good at planing projects and communicating these plans and put them in charge? Or maybe just close your company and search for a different career path if you are unable to plan your projects?
This is kind of absurd. The point of on-call is not to have regular off-hours work, but to have someone available for emergent situations. Since anybody who intends to operate a service with constant availability must always have _someone_ who can respond to emergencies, on-call is the system used by the world's most successful tech companies, not just two-bit upstarts. It's hard to imagine anybody serious about running any kind of online business who is going to respond to an outage on Friday night by saying "we'll look at on Monday at 9:00 PT." It's unfortunate, but it's really not that different from ER doctors, firefighters, utility workers, and numerous other workers working with similar constraints.
Besides that, the rotation is known, and planned for, long in advance. So your "long-awaited weekend trip" shouldn't have been scheduled during your on-call shift in the first place.
They can pay me enough that Iâll do it but not enough that Iâll want to do it.
You've probably just described most jobs.
Yeah, and this is a relatively good one, so let's have some perspective.
I'm replying to the post saying this is an "industry standard" thing--I'm not saying that there is no disaster scenario ever. Obviously not every contingency can be planned.
However, there are certainly signals that a company respects a developers' time. Things like...was there effort put in the product around things that might minimize developer on-call time, like having more user self-serve options, or spending more time on testing, or having a proper feedback loop for recurring issues to be permanently squashes in a timely fashion, etc etc. Are managers regularly a part of these off-hour on-call issues so they're affected too, thus motivated to resolve it? Or does the company just abusively unload all of the burden on the developer and force them to use up their own personal time to maintain a product? Far too often it's the latter, and I think it should be clear to all that this is abusive practice and not "normal" or "standard" nor should any of us consider it that way.
Sure, but no matter how many measures you take, on-call still means you have to be available, so itâs a drag even if itâs quiet.
I don't find your experience to be the general case, though that doesn't invalidate your point. Accolades are an easy intangible reward to hand out so you will often see it given for heroics. Extra cash would be better. PTO is somewhere in the middle. On teams that can rotate through on-call, bonus PTO is more likely to be used than on ones where one or two engineers shoulder the responsibility all year.
>_If you want a cushy 9-5 job, go work for a code factory like HCL, TCS and whatnot. That way you can build crap and wash your hands after handling it to the customer._
You got it backwards: if you want a crappy quality of life (or have no life and you only identify with the company that sees you as a replaceable cog), and crappy products and customer service from a company that doesn't care (to plan right, to hire accordingly, to treat its staff right) go on call and overwork, producing sleep-derived crap.
Craftsmen and artisans take their time and have boundaries.
"Feature factories", startups, and mass market crap companies forego 9-to-5 and indoctrinate naive employees that they do something important by doing so.
>_In a startup environment, you gotta build your shit and maintain it to. It's an environment that's not for everyone._
Yes. It's for starry-eyed naive youngsters right off the bus. The kind of people to believe they're "changing the world" by building a Facebook or Groupon.
i'd never heard of a "feature factory" before, but after reading about them, holy hell, that explains a ton of friction i had in a recent engagement.
they had a laser focus on just shipping and success theater, but little attention on actual utility of the end product. i was berated constantly for slowing things down or wanting to be more careful and do things that weren't necessarily easy but actually had potential to be useful. it's relieving to learn this is a common management deficiency.
> Craftsmen and artisans take their time and have boundaries.
Yeah but most of the people complaining only think they are Craftsmen but their day job is to write bad software to push ads
> Craftsmen and artisans take their time and have boundaries.
I've been an engineer on-call for 7 years since graduating. In the meantime I handled a handful of business critical situations, developed my ability to keep cool during crisis and put my skills to the test. When I started doing it, I wasn't even paid for on-call. Now I am. The money has no bearing on me having been exploited or not. I've done it because it's cool, and when I think it's too tiring I won't do it anymore.
Am I less of an engineer, not a true craftsman (wtv that means), because of this? There's other opinions in the world, no need to be so close minded.
When you were doing on-call without being paid, that's called wage theft. It's the single biggest kind of theft in North America:
https://www.epi.org/publication/wage-theft-bigger-problem-fo...
It's also the kind of theft that insidiously convinces you that it's really cool and that you're actually OK with it because you're learning how to 'keep cool during crisis'. You know how else your employers could have taught you that? By providing actual training during working hours, while you were being paid.
This "us vs them" concept is where I think we've lost the plot. Remember that we're the experts in complex systems that we've been selling to the business to add value and we've been happily taking their money for decades. When they ask "can the technology support this?" we, like an eager child showing off our new toy, assure them that it can and beg them to let us prove it.
We can't just chunk these complex systems over the wall and say "have fun with that" and demand more money when the solution breaks. This means we failed to properly estimate total cost of ownership when we sold them the system and now they're held hostage. This is borderline legal breach of contract.
If we told them the true sustainable cost of ownership for a follow the sun support model they likely would have never purchased the solution in the first place and we'd be out of a job.
We could have lots of annoying jobs or no jobs at all. I'll happily accept the former.
I don't know about you but I'm not 'selling' my work to the business like a black box, I am working under the direct supervision of the business and any and all work that I do is the result of a team effort and direction from management, non-engineering, and engineering staff. Let me emphasize: all the work that gets put into production is a team effort and a team responsibility.
I'm not begging anyone to let me prove my technology, I'm working with management to come up with a solution that will work for the customer. Oftentimes management will forbid or otherwise prevent me from using a better, more robust technology that would have prevented more production issues because of short-sightedness. If I pointed this out and was still told to go ahead with the short-sighted solution, and it breaks in production, whose fault is that?
I'm not suggesting the 'it's your problem now, have fun' approach, I'm suggesting the 'pay overtime for overtime work' approach. This is nowhere even near to 'breach of contract', please don't throw around lawyer-like terms when you clearly don't understand what's being discussed.
I mean this with all the respect in the world, but I think you've gotten yourself caught in an ideological bubble here. To someone who doesn't already agree with you, this comment sounds like an argument against the concept of wage theft more than an argument against oncall. After all, you can't prove something's bad just by classifying it under a nasty-sounding label!
To be fair, if they are so far off the end that they think "man, getting called up at 2am without being paid was great because I learned how to keep my cool during a crisis!" then they are probably a lost cause.
It's fine to drink of the chalice of the corporate kool aid with moderation. Still, one should never get black out drunk out of it.
I purposefully did not make an argument against on-call, although I might have easily done that and others have in this thread. All I said is that doing on-call _without pay_ is wage theft. And this is not an ideological bubble thing, as anyone will quickly understand if they ask a plumber to do on-call for their house plumbing system without pay.
I think a another analogy is that you had a plumber do some work for you a day before a leak, and in the middle of the night the pipes start leaking and you need to call him out to prevent property damage. In that case I would absolutely not expect to have to pay extra because he warranties his own work.
A better analogy would be a plumber you pay a handsome annual salary to maintain and improve your pipes--to what degree should they owe some emergency time in return is the question in this thread.
Plumbers (or companies employing plumbers) warranty their work. Do employees warranty their work for employers? I have never heard of that arrangement in the US.
I havenât heard of that here either. The closest I can think of are sanctions that can be levied on individual employees by professional standards organizations. I do know trades warranties arenât provided by employees working in that trade to their employer, just to customer individuals or organizations.
>_The money has no bearing on me having been exploited or not. I've done it because it's cool_
Convincing them of the "coolness" of it is the most common way to exploit the naive/fresh.
The same people who do so to others, wouldn't even piss if they weren't compensated for it...
(And being exploited or not is not a personal decision. If you aren't compensated for overtime, then you are exploited. It just means you're ok with it.).
There's nothing in the world you'd do for free for a while for the chance of doing it? And I'm the naive one? Is money all there is in life in this comment section?
Also: For some people these nights might have been the only times they connected to their team mates on a different level.
I had that too, but it was for art projects, films, music, my own programming projects or some other thing that I certainly did not do for money.
The one thing that differs here is that the people I did this for/with would cross the country if I told them I am in need.
>In the meantime I handled a handful of business critical situations, developed my ability to keep cool during crisis and put my skills to the test.
That's great, but you could still have done all those things if you were being compensated appropriately for the extra time you were putting in.
It's like unpaid internships. I'm sure unpaid interns often do learn useful skills and make useful connections. But the practice is still exploitative.
You do understand that most startups fail, right? Clearly, on aggregate, their practices are not really working or worthy of imitation. For the ones that succeed, there's a huge element of luck. Otherwise people would make guarantees about success.
The number of pages is only part of the story. Even if you never get paged a single time, oncall still sucks because you have some response requirement. If I _could_ get paged and I need to respond within 30m then it is no hikes for me this weekend.
FWIW, we have on-call at my work (all the developers are on PagerDuty schedule except the newest) and donât have a written or agreed response requirement. Itâs essentially best-effort, we had one instance where a page got ignored for hours by both primary and secondary responders and no one got in trouble. YMMV of course and I suspect we are laxer than most for various corporate and leadership reasons.
I do bring my computer with me when traveling out of state when on call but generally donât worry about e.g. going for a few hours hike or whatever even if I wonât have the laptop with me. If it were stricter it would definitely be more annoyingâŠ
This one is even weirder to me. If it is best effort then this means that it is okay if the work doesn't get done. If that is the case, then why can't it be done the next morning?
My team has an "oncall" shift like this where we assign a person to be responsible for triaging bugs and other non-urgent alerts just so we don't all stare at it and wait for somebody else (or inevitably me, the lead) to handle it but they are explicitly told that this responsibility does not extend beyond working hours.
Thatâs your thought and unfortunately it lines with the parroting by exploitative managers. But if Iâm on call and expected to reply to that call whenever the bell rings Iâm working all these hours and days no matter how rarely or frequently that bell rings.
On the other hand if itâs that less crucial for your company, 2-3 times a year without much monetary impact to warrant dedicated staff, Iâm sure it can wait until necessary working morning just fine.
Itâs not whether Iâm being utilised or not, itâs that Iâm being required to project my availability!
"Here's one. I get paged for urgent fixes maybe three times per year. To staff all three shifts with senior engineers to cover for that would be enough to make the company unprofitable and lose many jobs. I don't much like being woken in the middle of the night for such things, but it's clearly necessary and not abusive."
Do you have to be always available or can you go somewhere without connection?
The article explicitly states thatâs reasonable. Itâs the rotating pager duty thatâs unreasonable.
This suggests a pricing issue for your uptime guarantees.
> If an organization thinks their systems should be available 24x7 they should staff people 24x7.
Iâm a software engineer. Iâm paid well partly because things like oncall are necessary. Itâs priced into the compensation.
> There is no situation where itâs ok to contact someone to do work outside of their work hours.
It sounds like you should find a company that agrees with this opinion. I donât agree.
This attitude is very common with software engineers and I find it baffling. Is it some kind of inferiority complex? I doubt Warren Buffett worries that he is overpaid and demands to be woken up in the middle of the night for some self-flagellation. So why do so many software engineers think "I am paid well so I deserve whatever the company throws at me"? You are selling something: your skills, expertise, and time on this planet. These things are limited and valuable. Negotiate the price and terms of the sale!
Lots of other fields have on calls or out of work expectations. That's part of the job, and you're compensated highly.
It's not like on call is a shock or something, it's not "whatever they throw at me", it's just... part of the job. My sister is a dentist, she has to be on call sometimes, it's not a shocker or something you don't know when you're signing up for the job.
You're implying that people aren't negotiating, but that's baseless. Software engineers are highly compensated because of these expectations, and we all negotiate accordingly.
> Lots of other fields have on calls or out of work expectations.
Yes, and in many of them there are call-out fees and overtime. Programmers and sysadmins have convinced themselves that, as "professionals" they are not aligned with traditional working-class constructs like this.
> Yes, and in many of them there are call-out fees and overtime.
Why is that any better than just getting paid overall more? Lots of companies also have internal policies like "if you get called in on a weekend take a long weekend next week" etc in my experience.
Because then the company has a financial incentive to reduce call-outs and overtime.
>we all negotiate accordingly
This is quite a sweeping statement to make. My first engineering job salary was non-negotiable. I was told to either accept it, or they 'rapidly' move to another candidate.
I meant more collectively. But of course, yes, it's sweeping. It'll change regionally, or based on experience, or company culture, etc.
But I don't think it's unfair to say, especially in the US, that software engineers are highly compensated.
I would say its a superiority complex : thinking software engineers are paid well and hence need to be ready to sacrifice more to maintain this superior position.
And of course also because the risks of bad or interrupted sleep is not recognised widely and mostly ignored.
Which is why you can also see the exec team paged in the middle of the night, right? Because they're paid well and need to sacrifice?
Yes, and had their vacations cut short and other such things than rank and file employees would tolerate far less.
Every oncall rotation I've been in for the past five years has had C-suite executives in it. Earlier the CTO, currently the CEO.
The executive team elects to do it, and they also get a fuckload of stock incentive to do it. Not the case for lower-rung engineers.
Iâve definitely seen my exec team get paged in the middle of the night, and they didnât give any indication that it was unreasonable or uncommon. If anything they seemed to enjoy it, which to be honest I do as well. Being woken up to solve a problem (on rare occasion) can be validation that youâre an important person working on important things.
Gross
Clearly you don't think very highly of yourself. Sad.
Can you please not create accounts to post flamewar comments to HN? It's not what this site is for, and it destroys what it is for.
https://news.ycombinator.com/newsguidelines.html
The CEOs of every Fortune 500 company (and many smaller ones too) are on-call 24/7.
1. They get paid 10's of millions of