Common mistakes with Sidekiq

ActiveRecord::Base.transaction doUser.create(user_params)
UserMailer.created(user_params).deliver_later # Rails will use Sidekiq to send email
Log.create('user_created', user_params)

raise "Let's imagine something went wrong here"

You can’t, by design. Sidekiq is designed for asynchronous processing of jobs that can be completed in isolation and independent of each other. Jobs will be popped off of Redis in the order in which they were pushed but there’s no guarantee that Job #1 will execute fully before Job #2 is started.

If you need serial execution, you should look into other systems which give those types of guarantees.

Redis offers many different topologies:

Single node — offers no fault tolerence

Redis Sentinel — offers fault tolerence, fails over to a replica in case of primary failure

Redis Cluster — multi-master keyspace spread across many instances

Cluster is designed for large-scale datasets, like caches, that can spread evenly across machines. Cluster is NOT appropriate for Sidekiq as Sidekiq has a few very hot keys which are constantly changing (aka queues). I recommend using Sentinel or use a Redis SaaS which has built-in support for failover.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store