Learn how to use Cloud Tasks for Ruby and Rails to efficiently manage background tasks and improve the performance of your web application. Discover the benefits of using Google Cloud Platform's task management service and how to implement it in your Ruby on Rails application.
In April last year Google released Cloud Run, a service helping companies move to serverless by providing a generic deployment methodology based on Docker.
For those not familiar with the concept, serverless computing is a runtime model where the scalability of applications is dynamically managed by the underlying infrastructure based on the actual load. Forget servers and think about dynamically spawning workers/threads based on the number of concurrent HTTP requests hitting your app at a given point in time. Beyond the scalability aspect this also means no idle servers and scale-down to zero instance, which may represent huge cost savings in a micro-service context.
Laurent Julliard (Google Engineering) wrote an excellent four-part tutorial explaining how to move a Ruby on Rails application to Cloud Run. The tutorial covers all fundamental aspects but one question remained unanswered: how to manage background jobs on Cloud Run using Ruby/Rails? If you ask yourself the same question, read on!
In a serverless context there is no active process waiting for jobs. Processes are spawned based on HTTP activity. This means that background servers like Sidekiq, Resque or Delayed::Job are not an option unless you're fine with maintaining a separate dedicated server for background jobs.
The Google Cloud response to running tasks in the cloud is Cloud Tasks. Google Cloud Tasks manages queues for you and sends webhooks to your application when a background job is due for execution.
Sounds like exactly what we need in a serverless context! But wait! Does it mean I have to re-implement Sidekiq using Cloud Tasks then? Don't worry you don't.
At Keypup we believe in Google Cloud and serverless. And because we make heavy use of background jobs for our data processing pipelines we decided to release Cloudtasker, a framework that makes it easy to plug your Ruby/Rails application to Cloud Tasks.
If you like Sidekiq for server-based job processing then you will love Cloudtasker for serverless jobs.
Alright. Here are the steps to get started with a Rails application.
Add the following to your Gemfile and run bundle install
In development it is annoying to have to connect to Cloud Tasks because you need to have a publicly accessible endpoint to receive webhooks from Cloud Tasks (e.g. using ngrok). Fortunately Cloudtasker ships with a local development server that mimics Google Cloud Tasks.
The only requirement for this local server to run is to install redis. E.g. with brew: brew install redis
Add the following initializer to your application:
Now define a Cloudtasker worker in your application:
Start your Rails application and the Cloudtasker local server:
Open a Rails console and enqueue some jobs:
Voila! You should now see your jobs getting processed by your Rails app in your logs:
In production your application will use Google Cloud Tasks instead of the Cloudtasker local development server.
Edit your Cloudtasker initializer like this. Make sure to edit the GCP related settings as well as the public endpoint your app uses:
Create the default queue in Cloud Tasks using the gcloud sdk or the built-in rake task:
That's it! You can now deploy your application to Cloud Run or App Engine*. Your application will push jobs to Cloud Tasks when enqueuing workers and receive jobs for processing via webhooks from Cloud Tasks.
That was only a peek preview of Cloudtasker. The library offers many other options and features that may be useful to you such as:
All of them are explained in the Cloudtasker documentation. Happy background-jobbing!
** Note: these extensions require Redis. Unfortunately Cloud Run does not offer a connection to Memory Store yet. Only apps deployed to App Engine may use these features. Google Cloud plans to release serverless connectors between Cloud Run and Memory Store in the near future.
Keypup's SaaS solution allows engineering teams and all software development stakeholders to gain a better understanding of their engineering efforts by combining real-time insights from their development and project management platforms. The solution integrates multiple data sources into a unified database along with a user-friendly dashboard and insights builder interface. Keypup users can customize tried-and-true templates or create their own reports, insights and dashboards to get a full picture of their development operations at a glance, tailored to their specific needs.
Code snippets hosted with ❤ by GitHub.
Join teams already using AI to make data-driven decisions faster than ever.
Your bug SLA report says 91%. Your customers experienced 65%. Learn what engineering leaders on Reddit say about P1 fix-time targets, SLA clocks that start late and bugs closed to stop the timer at enterprise scale, and how Keypup MCP measures real SLA attainment by priority and team from Jira workflow history and linked pull requests.
Dependabot and Renovate open thousands of pull requests across enterprise repositories, and most of them are never merged, merged with no review, or left open for months. Learn what engineering leaders on Reddit say about dependency bot noise, unowned major upgrades and auto-merge risk, and how Keypup MCP measures the bot backlog by version bump, ecosystem, merge path, next action and owner.
Enterprises spend millions on outsourced and offshore engineering vendors, yet most can only measure them through hours billed and status reports. Learn what engineering leaders on Reddit say about vendor black boxes, hidden review load and untraceable work, and how Keypup MCP compares vendor and in-house pull requests, review wait times, review load and Jira traceability to show what each vendor actually delivers.