Secure Function: Findings, musings, how-tos, and analysis

Project release: resque-state gem

I’m posting this late as the code has been available for a bit now but I’ve published my first Ruby Gem (fork) on Github; resque-state. It adds more features to the original gem (resque-status) including more interactive-like controls to allow you to run semi-interactive jobs via Resque. The biggest addition was adding pause and revert functionality.

This project came from something I built (and hope to eventually publish) that runs automated rolling deployments to AWS. What the pause functionality gave me was the ability to let a user do a one-box or canary ahead of a full roll as well as the ability to pause a job that might be having troubles. This lets an engineer launch a deployment to an Auto Scaling Group (ASG) and initially add just a single machine. Once that instance is healthy the job then pauses and waits for the engineer to give the deployment the green light to continue. The pause/unpause functionality became one of the critical features to enable safer production releases.

Just added was a revert feature. You could accomplish something similar with on_failure but I thought that might be overloading that functionality a bit. I believe these are two different cases. If a job fails you may not want to undo it because the failure may have been fatal for the job process but not something that actually needs reverted. Maybe there was a network blip the automation didn’t handle well or perhaps you’re able to course-correct without actually pulling back what was done. Revert gives you a separate path for cases where you specifically want to pull back what was done. This can be done from the paused state (for example; a deployment one-box that is no good) as well as just while the job is running.

PRs and constructive feedback are welcome. 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *

© 2017 Secure() All Rights Reserved