Merged
Conversation
Allows configuring queues to run without forking per job, using an
extended YAML config syntax:
my_queue:
- workers: 4
- fork_per_job: false
This is needed for long-running worker processes where fork overhead
is undesirable (e.g. workers that maintain persistent connections or
large in-memory caches).
Methods added:
- config_get_worker_count: parses both integer and array config syntax
- fork_enabled_for_queues?: checks fork_per_job setting per queue group
Methods modified:
- worker_delta_for: uses config_get_worker_count for extended syntax
- create_worker: sets worker.fork_per_job based on config
Ported from skroutz resque-pool 0.4.0 fork, adapted for 0.8.0 base.
Tests both pure extended syntax (skroutz.yml) and mixed syntax (mixed.yml) where some queues use integer counts and others use the array-of-hashes format. Covers config parsing, worker count extraction, fork_per_job detection, and actual worker creation. Ported from skroutz resque-pool 0.4.0 fork specs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Non-forking workers (
fork_per_job) for resque-pool 0.8.0Rebases the skroutz
fork_per_jobfeature from our 0.4.0 fork onto upstream 0.8.0.What
Adds support for configuring queues to skip forking per job, using an extended YAML syntax:
Both syntaxes can be mixed in the same config file.
Changes
config_get_worker_count— parses both integer and array config formatsfork_enabled_for_queues?— checksfork_per_jobsetting per queue groupworker_delta_for— updated to useconfig_get_worker_countcreate_worker— setsworker.fork_per_jobbased on config0.8.0.skroutz.1