Sidekiq Background Jobs for Rails on DomainIndia VPS

Q: Is Sidekiq safe for critical jobs (payments)?

Yes, with care. Set `retry_on` for transient errors, use idempotency keys (check "already processed" before calling payment API), use Sidekiq Pro's reliable fetch for crash safety.

TL;DR

Sidekiq runs Ruby background jobs using Redis as the queue — essential for any Rails app doing email sending, PDF generation, image processing, or webhooks. This guide covers installing Redis, configuring Sidekiq, systemd services, monitoring, and scaling on DomainIndia VPS.

Why Sidekiq (not Solid Queue, DelayedJob, Resque)?

Rails 8 ships with Solid Queue (DB-backed queue), which is great for simple workloads. But for anything serious, Sidekiq still wins:

Queue	Backend	Throughput	Best for
Solid Queue	DB (PostgreSQL)	~1K jobs/min	Simple Rails apps, <50 jobs/sec
Sidekiq	Redis	~10K+ jobs/min	Production Rails, high throughput
DelayedJob	DB	~500/min	Legacy apps
Resque	Redis	~2K/min	Older alternative, less maintained

Sidekiq's multi-thread model + Redis backend make it 10× faster per process than DB-backed queues. It's the standard for production Rails.

Architecture on DomainIndia VPS

code

[Rails web (Puma)] ──► enqueues job ──► [Redis]
                                         │
[Sidekiq worker 1] ◄──── picks jobs ─────┤
[Sidekiq worker 2] ◄──── picks jobs ─────┘

Rails web process (Puma) accepts HTTP, enqueues jobs
Redis holds the queue
Sidekiq worker processes pull jobs, run them
All three run as systemd services on the same VPS (or different VPS for scale)

Step 1 — Install Redis

On the VPS:

bash

# AlmaLinux
sudo dnf install -y redis
sudo systemctl enable --now redis

# Ubuntu
sudo apt install -y redis-server
sudo systemctl enable --now redis-server

redis-cli ping   # → PONG

Enable persistence so queued jobs survive server restart:

bash

sudo sed -i 's/^appendonly no/appendonly yes/' /etc/redis/redis.conf
sudo systemctl restart redis

Step 2 — Add Sidekiq to your Rails app

Gemfile:

ruby

gem 'sidekiq', '~> 7.2'

bash

bundle install

config/initializers/sidekiq.rb:

ruby

Sidekiq.configure_server do |config|
  config.redis = { url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/0') }
end

Sidekiq.configure_client do |config|
  config.redis = { url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/0') }
end

config/application.rb:

ruby

config.active_job.queue_adapter = :sidekiq

Step 3 — Write a job

app/jobs/send_welcome_email_job.rb:

ruby

class SendWelcomeEmailJob < ApplicationJob
  queue_as :default
  retry_on Net::SMTPError, wait: :exponentially_longer, attempts: 5

  def perform(user_id)
    user = User.find(user_id)
    UserMailer.welcome(user).deliver_now
  end
end

Enqueue:

ruby

SendWelcomeEmailJob.perform_later(user.id)

Step 4 — Systemd service for Sidekiq

/etc/systemd/system/sidekiq-yourapp.service:

ini

[Unit]
Description=Sidekiq for YourApp
After=network.target redis.service postgresql.service

[Service]
Type=simple
User=deploy
Group=deploy
WorkingDirectory=/home/deploy/rails_app
Environment="RAILS_ENV=production"
Environment="PATH=/home/deploy/.rbenv/shims:/usr/bin:/bin"
ExecStart=/home/deploy/.rbenv/shims/bundle exec sidekiq -e production -C /home/deploy/rails_app/config/sidekiq.yml
ExecReload=/bin/kill -TSTP $MAINPID
Restart=on-failure
RestartSec=5

# Graceful shutdown — let Sidekiq finish current jobs
TimeoutStopSec=30
KillSignal=SIGTERM

EnvironmentFile=-/home/deploy/rails_app/.env
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

config/sidekiq.yml:

yaml

:concurrency: 10            # threads per worker process
:queues:
  - [critical, 3]           # weight 3 — critical queue serviced 3× more
  - [default, 2]
  - [mailers, 1]
  - [low, 1]
:timeout: 25                 # jobs killed after 25s if still running when SIGTERM

Enable:

bash

sudo systemctl daemon-reload
sudo systemctl enable --now sidekiq-yourapp
sudo journalctl -u sidekiq-yourapp -f

Step 5 — Sidekiq web UI

Mount the dashboard for monitoring:

config/routes.rb:

ruby

require 'sidekiq/web'

Rails.application.routes.draw do
  authenticate :user, ->(u) { u.admin? } do
    mount Sidekiq::Web => '/sidekiq'
  end
  # ... other routes
end

Visit https://yourcompany.com/sidekiq — see queued, retried, scheduled, dead jobs.

Job patterns

Scheduled / delayed jobs

ruby

ReminderJob.set(wait: 1.hour).perform_later(booking_id)
ReminderJob.set(wait_until: 2.days.from_now).perform_later(booking_id)

Recurring jobs (Sidekiq-cron or whenever gem)

ruby

# Gemfile
gem 'sidekiq-cron'

# config/schedule.yml
cleanup_old_sessions:
  cron: "0 2 * * *"    # 2 AM daily
  class: "CleanupOldSessionsJob"

reindex_search:
  cron: "0 */6 * * *"  # every 6 hours
  class: "ReindexSearchJob"

Unique jobs (prevent duplicates)

ruby

# Gemfile
gem 'sidekiq-unique-jobs'

class ImportantJob < ApplicationJob
  sidekiq_options lock: :until_executed
  def perform(id)
    # Only one concurrent execution per id
  end
end

Scaling patterns

Single VPS, multiple workers:

Add more worker processes on the same VPS — 2–4 is typical before RAM becomes the limit.

code

/etc/systemd/system/[email protected]  (templated service)

Start [email protected], @2.service etc.

Multi-VPS (horizontal scale):

Put Redis on a dedicated VPS. Point all web + worker VPS at it:

code

REDIS_URL=redis://redis.internal.yourcompany.com:6379/0

For HA Redis: Sentinel (free) or Redis Cloud (managed).

Monitoring and alerts

Sidekiq stats endpoint:

ruby

Sidekiq::Stats.new.enqueued       # waiting to run
Sidekiq::Stats.new.retry_size     # failed, waiting retry
Sidekiq::DeadSet.new.size         # permanently failed

Expose as /health/sidekiq and alert when queue >1000 or dead >10.

Sidekiq Pro/Enterprise (commercial):

Batches (run many jobs, callback when all done)
Rate limiting
Quiet time, reliable fetch, encryption

$179/mo. Worth it if Sidekiq is mission-critical.

Common pitfalls

Jobs fail silently

always set retry_on for expected exceptions; jobs without retry go straight to Dead set on any error.

Memory bloat over time

worker RAM creeps up. Use MALLOC_ARENA_MAX=2 to reduce glibc fragmentation; restart workers every 24h via systemd.

Redis fills up

queue backlog grows faster than workers process. Scale up workers or increase concurrency.

Job arguments too big

Redis isn't meant for MB-sized payloads. Pass IDs, fetch in the job.

Forgot retry_on on network calls

transient errors become permanent failures.

Jobs run in the wrong timezone

Time.zone isn't set in the worker. Configure config.time_zone = 'Asia/Kolkata' in Rails + Chronic for parsing.

FAQ

Sidekiq or Solid Queue for a new Rails 8 app?

If you'll never exceed ~10 jobs/sec, Solid Queue is simpler (no Redis). For anything higher or existing Redis infrastructure, Sidekiq.

Can I run Sidekiq on shared hosting?

No — Sidekiq needs a long-running worker process. Shared hosting runs PHP/Ruby per-request only. VPS required.

How many workers on a 2 GB VPS?

Depends on your jobs. Typical Rails worker uses 200–400 MB RAM. 1 web + 1 Sidekiq process (10 threads) fits comfortably. For heavier jobs (PDF, image), 2 worker processes max on 2 GB.

What about Sidekiq + Heroku/Render?

Works the same way, but you pay per worker dyno. DomainIndia VPS is more cost-efficient — one 4 GB VPS covers what costs ₹5,000+/mo on PaaS.

Is Sidekiq safe for critical jobs (payments)?

Yes, with care. Set retry_on for transient errors, use idempotency keys (check "already processed" before calling payment API), use Sidekiq Pro's reliable fetch for crash safety.

Sidekiq + Rails + Redis runs beautifully on a DomainIndia VPS. Get started

Sidekiq Background Jobs for Rails on DomainIndia VPS

Why Sidekiq (not Solid Queue, DelayedJob, Resque)?

Architecture on DomainIndia VPS

Step 1 — Install Redis

Step 2 — Add Sidekiq to your Rails app

Step 3 — Write a job

Step 4 — Systemd service for Sidekiq

Step 5 — Sidekiq web UI

Job patterns

Scheduled / delayed jobs

Recurring jobs (Sidekiq-cron or whenever gem)

Unique jobs (prevent duplicates)

Scaling patterns

Monitoring and alerts

Common pitfalls

FAQ

Was this article helpful?

Related articles