Executing patch scripts

While mitigating an outage, you may run into recurring problems for which you don't have engineering resources to address or just can't automate the remediation for a variety of reasons. Engineers commonly write one-off scripts to patch a problem; these one-off solutions end up in GitHub, local text files, Slack code snippets, or in Google docs and are frequently forgotten and then recreated.

FireHydrant lets you store these scripts and track their execution, success and usefulness over time with our Runbook Execute a Script step. We'll show the raw script letting your engineers copy/paste it out into their terminal or give you a curl command to execute it and report the status back into FireHydrant. This lets you keep track of when the scripts are executed, by who and their output. 

Configuration

Add a step to your Runbook with a description and then any Bash code you'd like your engineers to execute in order to mitigate an incident.

Execution

Your engineers will be presented with the script itself along with a curl  command allowing FireHydrant to capture the error status and output. This lets you record when the step was executed, by whom and if it was successful. 

The script is available from Slack and our web UI.

If you run the script using our "auto execute" functionality, the step will automatically transition from pending to complete or errored when it's run by one of your engineers. You can see below that the error status (missing API token) was recorded by FireHydrant: