💾 Archived View for dioskouroi.xyz › thread › 24987197 captured on 2020-11-07 at 00:44:56. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
________________________________________________________________________________
I like Cloud Run a lot as an evolution of Cloud Functions. It has the ability to scale to 0 docker containers. My OpenResty API gateway takes 400ms to spin up and can serve a ton of traffic concurrently (
https://futurice.com/blog/openresty-a-swiss-army-proxy-for-s...
). This is much better and cheaper than minimum scale of 1 Fargate containers IMHO. There are a lot of legacy things you can put in Cloud Run (e.g. JVM application servers) that would not be possible in Cloud Functions. Cloud Functions are good for small custom node.js express apps. Cloud Run is awesome for heavier IO bound services as you can make use of the runtime resources when waiting for other services to complete (by serving concurrent requests).
I’m using Cloud Run too and love it in principle and, mostly, in practice. I’m running a Rails app which takes about 30 seconds to start on Run (much quicker on a standard instance or local and I’m not sure why). My other gripe is that it will just randomly restart (not after being idle for some time but literally as I’m clicking around the website). There are no server errors and nothing I can see in the logs. Coupled with the fact that restarts take around 30 seconds it makes for quite a poor initial experience for my web users.
Cold starts can be one of the biggest trade offs when adopting a Serverless platform like Cloud Run, especially with 30 second start up times. We introduced minimum instances[1], currently in beta, which will allow you to specify a minimum number of container instances to be kept warm and ready to serve requests.
There is a cost for doing so as instances kept running in this way do incur billing costs [1].
I've done some work with Ruby in the past, and based on my experience, you might not be able to take advantage of Cloud Run's best feature, concurrency[2], which is another way to reduce cold starts by routing concurrent requests to the same container instance. Ruby maybe blocking in flight requests and forcing us to fire up another instance to handle it. Once the first request has completed there is a chance the container instance processing that request will be spun down, this is something minimum instances can help with. You might need more than one minimum instance to compensate for the lack of concurrency if you really want to see improvement in overall latency.
[1]
https://cloud.google.com/run/docs/configuring/min-instances
[2]
https://cloud.google.com/run/docs/about-concurrency
This advice might not be at all helpful, but anyway.
Consider checking the following:
- Number of CPUs. The number of CPUs is not CPU cores but Kubernetes millicores. 1 vCPU on CR = 400m (4/10 of a CPU core).
- Background processes - not recommended [
https://stackoverflow.com/questions/61154349/google-cloud-ru...
]
- Container size on disk. Google Cloud is fast. However they need to cap the moving container images around as they can't use all the network capacity for moving a 400MB Docker image. If you're not already, use multi-stage builds with tiny final images.
This is not correct.
> Limits and requests for CPU resources are measured in cpu units. One cpu, in Kubernetes, is equivalent to 1 vCPU/Core for cloud providers and 1 hyperthread on bare-metal Intel processors.
Where do you get the info that 1 vCPU on CR = 400m? I would have expected 1 vCPU on CR = 1000m
Yeah I have a jvm jboss type thing that take 5 mins to startup, which is why I have that pubsub to self in the proxy so I can absorb delays in a managed queue. This of course relies on the upstream not being time sensative or needing a syncronous reply. However, nginx does start in 400ms on cloud run so it's clearly it's the cruft in the docker that's doing it and not the hosting
(code is here BTW
https://github.com/futurice/terraform-examples/tree/master/g...
)
For comparison I’ve been using Go and a distroless container image and never had any problems.
However you can set the minimum instance count now. That feature was opened up for Cloud Run managed a few weeks ago.
> There are a lot of legacy things you can put in Cloud Run (e.g. JVM application servers)
I wouldn’t call JVM application servers legacy. If you are referring to WebSphere and Weblogic, then maybe. But the open source JVM application servers like tomcat,jetty, netty etc., are widely used in modern applications
I’ve been using GCP full time since 2015 and I expect Google to deprecate AppEngine and Functions in favor of Cloud Run almost any day now.
Thanks for your comment. This is something the team keeps top of mind and influences our decisions and road maps. Please continue to keep us honest here. I'll get others to weight in on this, but here's my personal opinion on the matter.
AppEngine has been around for a long time, 12 years to be exact, and in my mind it's a completely different product, and still one of our most successful. While Cloud Run competes with AppEngine on some fronts, customers really appreciate AppEngine's deep integration with other GCP managed services. It's a full blown PaaS.
I expect Cloud Run to improve over time and gain more features that will no doubt match the requirements of many AppEngine customers, some may switch to Cloud Run, but we are not forcing them to.
Cloud Functions already shares a lot of underlying infrastructure with Cloud Run, which shares a lot of underlying infrastructure with AppEngine, and that's by design. These days I like to think of Cloud Functions as a simplified developer experience on top of Cloud Run focused on task orientated workloads backed by events and triggers.
Can we continue to support 3 somewhat overlapping Serverless platforms going forward? I believe the answer is yes, thanks to the shared infrastructure, and the fact we truly believe all three platforms offer a unique set of developer experiences worth preserving. Maybe there is future state where one product satisfies all use cases, but that's not today, so we plan to keep listening to customers, and invest across the board.
One recent example of investing across the board: you can now leverage global load balancing[1] across all three Serverless platforms.
[1]
https://cloud.google.com/load-balancing/docs/negs/serverless...
Hi Kelsey! Appreciate your take here. You obviously have deeper insight than I do. Honestly I'm in favor of the path I described. Engineers I work with spend too much time already arguing the best platform for any given service, especially when the pros and cons are very, very similar. That being said, with the recent features released for Run (load-balancing, min instances, streaming gRPC) we're basically 100% bought in to that direction. I can't think of any feature AppEngine or Functions have (esp AppEngine) that would pull me back.
Though I will say I dearly miss the days of first generation GAE standard, when things like memcache, taskqeue, datastore, auth and logging were all baked in as first-class citizens and Just Workedâ„¢.
Cloud run is attempting to enable that "Just Works" experience, but we are trying to do it by leveraging the best of open source, and native client libraries. Feels like we are getting close and I'll be sure to share this feedback with the rest of the team.
My tiny team has been using Cloud Functions, Pub/Sub, and Apache Beam(Dataflow) to process tens of millions of daily events. I'm middle of the road on them.
Pro:
- They are stupid fast to develop, deploy, and update.
- Firebase tooling emulators makes local dev much easier.
- Simple enough a developer can build, deploy and monitor;
Cost less then hiring another person to handle devops.
- Now has decent IAM restrictions
- Baked in access to almost all other GCP tools.
Con:
- Like any google product could be deprecated at any time.
- They have poor versioning compared to App Engine or Cloud Run. No rollback, A/B, or multi-version deployments
- Cold starts are slow.
- Like any google product could be deprecated at any time.
Regarding cold start time; I am luke warm on golang, but I think lambdas / functions are one area where they shine. Start time is pretty quick for a golang function, and memory usage is low. Also, from what I understand it's one of the few statically typed languages supported on gcf.
I think my team will be shifting to use java more, just out of sheer familiarity, but golang works nicely for functions.
It's all good but:
1) The pricing is listed as per invocation however it is also billed for the CPU seconds and It happens that on a low volume function usage I exceed the free quota exactly on that as the invocation numbers remain very low.
2) The functions take about 10s-15s to execute on cold start. Every execution that happens between something like a minute is a cold execution. It is also billed as 15s execution even if the actual script runs for 200ms.
3) The CPU second are calculated according to time and VM memory, so if your function happens to make a call to another API and sits idle for 5s for response to arrive, you are still billed as 5s full memory usage.
Which makes me wonder why I would't spin a DO droplet instead. Maybe it's good on very high volume but from experience I know that a 5$ NodeJS droplet can handle hundreds of concurrent request. That would probably cost a lot on Google Cloud Functions. On low volume DO droplet performs much better anyway.
> The functions take about 10s-15s to execute on cold start
This may be a bit of an exaggeration, and may vary depending on your deployment (from experience, it takes up to 5s at most), but I agree. There is a very noticeable cold start time which makes it not ideal for any business critical services. It is probably only good for things like document conversions.
This is also something that you can optimize down to a small number of seconds. Crazy things such as bundling+minifying nodejs apps can make a difference. However that may make debugging difficult.
There is some latency in the infrastructure but I have found that most of the delay is puling the image and actually starting the app. So small images with few layers and fast boot up will help a ton.
> It is also billed as 15s execution even if the actual script runs for 200ms.
Wait, you get billed for cold starts?
GCP cheat sheet:
Cloud functions: when you need a endpoint to run, don't need fast response times, don't have more than a few hundred lines of code in a contained class, that don't often run. Think image resizing, async email calls, etc.
Cloud Run: when you have a simple docker file that can run your code, don't mind paying a higher cost, can't take the bootup time of Cloud functions, light or no database connections, need specific os'es or hardware AND you consider k8s confusing. Think wordpress, microservices, or demo software that come self contained in docker,
k8s: when you are an k8 expert
vms: for anything else.
I actually find VMs harder to manage than k8 via GKE.
At least kubernetes has defined for you what the orchastration and configuration files should look like. With VMs you have to figure all that out yourself, unless you are happy just installing everything manually via SSH and apt-get.
This is what terraform is for (among other things).
Another option is to use Instance Groups.
Google Cloud Functions - CGI scripts for a lot of money.
Like CGI scripts, it scales down really well though. I have an endpoint that is very important but only needs to handle a couple requests per hour. I wrote it as a Cloud Function and it's costing about $0.12 per month. Spinning up a super cheap VM that would sit idle most of the time would cost quite a bit more (plus time to set it up, deploy code to it, manage updates, etc).
$0.12/month seems expensive for such a low requirement? Is it because of the size of the function? Does GCP lack a free tier that would otherwise be free on a different platform? Not knocking your decisions. I just legitimately want to know where the cost comes from.
Yes, it requires a reasonably large amount of memory and runs for 40-50s per invocation. $0.12/month seems pretty reasonable to me. In our case, it also interacts with other GCP APIs, so the security model where we can just grant the Cloud Function service account granular access is extra convenient and would require extra engineering to duplicate if there were a free host somewhere that we could run the same code. GCP has a free tier and if that were the only thing we ran there, it would probably be covered. I really don't know about what's included in their free tier though because the other services we run on GCP put us well past whatever it might include.
Ah ok -- thanks for the clarification regarding the memory usage!
Something you describe as "very important" seems like a strange thing to do as cheaply as possible.
Why not if it is effective as this solution likely is. Just because you can afford to spend more money isn't a great reason to do so.
If it's important then having support and an SLA is essential. You're not getting either of those for $0.12/month.
Our total monthly GCP bill is in the five figures range, so it's included there if we need it. Ultimately, a very simple function running on Cloud Function is just way, way more reliable and "safe" for us than spinning up a VM, monitoring that, installing a full web server or CGI or Django/Flask setup to it, keeping it patched, managing deploys to it, etc. There's just fewer moving parts to break.
I wouldn't advise someone to spin up a key part of their infrastructure on Cloud Functions if they're not already using GCP for other things. But once you're there, it's a very useful part of the ecosystem.
Does anyone here have good ways to monitor GCP cloud functions?
I feel the monitoring/instrumentation around errors/performance is not great, but could be.
I’ve always found GCP documentation to be absolute trash, if you can use AWS Lambda do it, our indexing service/data portal runs off of it it works very well, the integration with CloudWatch insights makes logging pretty straightforward.
Can you use Health-Checks (under the Compute Engine drop down) to monitor Cloud Run? I use them with Instance Groups for deploying docker containers.
Google Cloud Functions is awesome, except for the 10 MB file limit and lack of GPU support, making it pretty much unusable for compute-heavy ML models.
If Google could ever lift these restrictions, GCF would be perfect for auto-scaling ML models.
You got the wrong idea about serverless if you think it should have a GPU attached to run your ML dataset.
Generally speaking, you want it to do something simple and quickly
GPU can also help with inference. I think running a cloud function to evaluate an input on a pretrained model seems like a good use case.
For inference, not training. Sorry for the confusion.
Also the request timeout needs to be raised.
I have a few workloads that execute only very infrequently, which makes them suitable for functions, except when they do run they sometimes exceed the 10 minute timeout.
Cloud Run has 1 hour timeout in Beta
https://cloud.google.com/run/docs/configuring/request-timeou...
What do you mean by file limit? I made a GCF service to handle 400mb+ video files
Not OP, but perhaps he meant size of data passed directly in a request which is indeed 10MB:
https://cloud.google.com/functions/quotas
I don't think I had any issues loading larger files from Storage either.
Yes, data passed directly into a request. Files loaded from storage are exempt from the limit.
Data passed directly into a request is limited to 10 MB, but you can use storage to load larger files.
Does anyone feel like cloud functions are much more expensive than aws lambda + api gateway? I think so..
The biggest difference is the COLD START, Google offering being in a different ball park. AWS sports ˜300ms cold starts (node and python) while GCP (and Azure) take 5 ~ 10 seconds. I hope GCP catches up or surpass AWS in the near future, but the gap today is HUGE. Price is quite similar if you don't hit Cold Start invocations very frequently.
Google now also supports Cloud Native Buildpacks (
) to build and deploy your functions to Google Cloud Function:
https://github.com/GoogleCloudPlatform/buildpacks
They have support for NodeJS, Go, Java, Python, even .NET Core!