Showdown at High Noon — Cloud Functions V1 Vs V2
Google Cloud Functions offers two product versions: Cloud Functions 1st gen, the original version, and Cloud Functions 2nd gen, a new version built on Cloud Run and Eventarc to provide an enhanced feature set.
V2 is still under public preview (as of Feb 2023), here are the main changes from V1.
- Function instances can now execute more than one request at a time.
- Secure your callable and HTTP functions with a new
corssetting. - Cloud Functions is now built on Cloud Run.
- HTTP functions can now have a 1 hour timeout (up from 9 minutes previously).
I decided to conduct some load testing and compare V1 and V2.
The function that I tested was a simple Go function that converts and returns the current time as a string.
I have tested four types of deployment variation of the same function.
- V1 — Vanilla
- V2 — Vanilla
- V2 – 1 CPU and 200 concurrency
- V2–6 CPU and 100 concurrency
- V2–6 CPU and 200 concurrency (Preview)
I used K6 on a VM in GCP. The test configuration was as follows
k6 run --vus 2500 --duration 2m --rps 6000 script.js
It was a bit of a surprise to me how things turned out. I was expecting that the move from V1 to V2 would have a positive effect on performance and scalability. This seemed to be the case with the first three runs. Average response time decreased steadily from V1 with 32.08ms to 24.06ms with out of box V2 and down to 21.62 with 1 CPUT and 200 concurrent requests.
However going to 6 CPU with 100 and 200 concurrency increase the average response time to 65.9 and 81.59 respectively. On top of that the number of errors was 0.03% and 0.07% compared to zero in the first three tests. This shows that even with increased resource usage, performance improved over the initial tests, and the number of errors decreased. It looks like throwing more hardware at the problem didn’t pay off!
My key takeaway from this simple experiment is that you need to understand your application and how it will interact with the underlying infrastructure. Adding more power is not always the most effective approach. It’s important to understand the bottlenecks in your environment and how they can be addressed. For example, if the application is I/O intensive, adding more RAM may not be the most effective approach. Rather, optimizing the application to make better use of I/O resources may have a bigger impact.
So basically, don’t just throw more horsepower at the problem — use your brain instead!
I do not have a reasonable explanation for why using 6 CPUs produces performance so much less efficient.
The only thing I can speculate about is that not everything has been dialed down for concurrency on multiprocessor machines. Therefore, maybe we should wait until version 2 is out of preview. This way, it can become general availability. I’m sure version 2 will be out of preview before we know it… unless it gets stuck in a deadlock!
