Sidekiq

About

Sidekiq is an open source scheduler written in Ruby. Sidekiq is commonly used with Ruby on Rails to handle asynchronous tasks.

Sidekiq Config

You should see a Sidekiq.configure_client where you can configure details like what the redis options are.

Active Job

Rails 4.2 introduced Active Job, a standard interface for interacting with job runners. Active Job is a framework to declare jobs and make them run on a variety of queuing backends (including Sidekiq, Resque, etc).

https://guides.rubyonrails.org/active_job_basics.html

Active Job - Queue Adapters

Queuing backends have a variety of features. For example:

https://api.rubyonrails.org/v6.0.2.1/classes/ActiveJob/QueueAdapters.html

|                   | Async | Queues | Delayed    | Priorities | Timeout | Retries |
|-------------------|-------|--------|------------|------------|---------|---------|
| Backburner        | Yes   | Yes    | Yes        | Yes        | Job     | Global  |
| Delayed Job       | Yes   | Yes    | Yes        | Job        | Global  | Global  |
| Que               | Yes   | Yes    | Yes        | Job        | No      | Job     |
| queue_classic     | Yes   | Yes    | Yes*       | No         | No      | No      |
| Resque            | Yes   | Yes    | Yes (Gem)  | Queue      | Global  | Yes     |
| Sidekiq           | Yes   | Yes    | Yes        | Queue      | No      | Job     |
| Sneakers          | Yes   | Yes    | No         | Queue      | Queue   | No      |
| Sucker Punch      | Yes   | Yes    | Yes        | No         | No      | No      |
| Active Job Async  | Yes   | Yes    | Yes        | No         | No      | No      |
| Active Job Inline | No    | Yes    | N/A        | N/A        | N/A     | N/A     |

Active Job - Creating a job

With Active Job, you can create a job with:

rails generate job my_cleanup ( ---queue my_queue if you want to also specify the queue).

This creates a job in app/jobs that looks like:

class MyCleanupJob < ApplicationJob
  queue_as :default

  def perform(*guests)
    # Do something later
  end
end

Active Job - Enqueue the Job

You can enqueue the job a few different ways (depending on when you want it to run):

MyCleanupJob.perform_later guest  # enqueues a job to be performed as soon as the queuing system is free
MyCleanupJob.set(wait_until: Date.tomorrow.noon).perform_later(guest)  # enqueues job to run tomorrow at noon
MyCleanupJob.perform_later(guest1, guest2, filter: 'some_filter')  # 'perform_now' and 'perform_later' calls 'perform'
MyCleanupJob.set(queue: :some_queue).perform_later(record)  # set a specific queue for job to run on by using 'set'

Active Job - Queues

With Active Job, you can schedule the job to run on a specific queue.

class MyCleanupJob < ApplicationJob
  queue_as :low_priority
  #....
end

You can run some logic to determine which queue the job gets sent to with queue_as You can prefix the queue name for all your jobs under config/application.rb

Active Job - Callbacks

Active Job provides hooks to trigger logic during the life cycle of a job. You can implement the callbacks as ordinary methods and use a macro-style class method to register them as callbacks.

Available callbacks include:

before_enqueue
around_enqueue
after_enqueue
before_perform
around_perform
after_perform

These callbacks provide hooks to trigger logic during the life cycle of a job. You can implement the callbacks as ordinary methods and a use a macro-style class method to register them as callbacks. For example:

class MyCleanJob < ApplicationJob
  queue_as :default
  around_perform :around_perform

  def perform
    # Do Something later
  end

  private
    def around_cleanup
      # Do something before perform
      yield
      # Do something after perform
    end
end

Active Job - perform vs perform_async

When you run perform (e.g. MyUser.new.perform(user.id)), you execute the job immediately (i.e. runs synchronously). With perform_async, you send your job to Sidekiq to run asynchronously.

When you run perform_async, Sidekiq persists the arguments to Redis as JSON. Complex Ruby oejcts do not convert to JSON (by default will convert with to_s). Even if it did serialize correctly, your queue might back up and your arguments are stale. Summary: Don’t save state to Sidekiq, just save simple identifiers and instead look up the objects you need in your perform method.

Active Job - Redis

So if you use Redis as your queue backend, you can see what gets pushed and set.

redis-cli monitor | grep -E "(hset|lpush)"

You will see messages like:

lpush queue:default { class: ExampleSendToWorker, args: [a, b], retry: true, queue: default, jid:... }
hset .... { queue: default, payload: {class: ExampleSendToWorker... }}

Active Job - Testing your Jobs

https://guides.rubyonrails.org/testing.html#testing-jobs

###