Severity Matrix

Severity Matrix in FireHydrant
Severity Matrix in FireHydrant

Have you ever been uncertain about the criticality of an incident? FireHydrant's severity matrix allows you to specify what severity levels should be automatically assigned to different combinations of fault states and impacted infrastructure. Let's take a look at how this is implemented.

Note: Auto-assigning severity only occurs when an incident is started with components marked impacted. It does not automatically change the severity if a catalog item is added mid-incident.

Prerequisites

Conditions

In FireHydrant, FireHydrant has an additional item called Conditions. These denote the state of a given function in your system, and we provide the following defaults:

  • Operational: The component is working as expected.
  • Bug: The component has a minor glitch that somewhat impacts the experience but does not block critical functionality.
  • Degraded: The component has an issue that impacts the experience and may hinder or slow down critical functionality, but a workaround is available.
  • Unavailable: The component is not working at all, or critical functionality is practically unusable.

Just like with Severities and Priorities, Conditions can be customized on FireHydrant per your needs. Most customers don't see the need to modify these, but should you want to:

Adding a condition in Severity Matrix
Adding a condition in Severity Matrix
  • To add a new condition, click the top right button to "+ Add condition" and provide a name in the modal.
  • To edit or delete a condition, click on the condition itself in the table and either change the name or click "Delete".

If you add or customize your conditions here, you'll see these available to select at various different points in the application, including from Slack and the UI, when adding an impacted component to the incident.

Example of seeing conditions to select from Slack
Example of seeing conditions to select from Slack

Configure Severity Matrix

  1. From the FireHydrant web UI, go to Settings > Severities matrix.

  2. Click "Add impact" at the bottom left of the table to add rows to the matrix. A modal will pop up where you can select the appropriate Catalog component to conditionally assign severities and impacts for. Click "Save".

  3. Once the row is inserted into the table, you can configure different severities according to the condition(s) applied to the component. Click the dropdown and select the default severity for each condition applied to the given catalog component. For example, in the following example matrix:

    Severity Matrix in FireHydrant
    Severity Matrix in FireHydrant
    • If the API is marked as Unavailable when starting an incident, FireHydrant will automatically assign a SEV1 to the incident.
    • But if the In-App Chat is marked as Unavailable when starting an incident, FireHydrant will assign a SEV2 to the incident instead.

Note: If multiple components from the matrix are marked impacted at incident start, the higher severity takes precedence.

  1. To save your severity matrix changes, click "Submit" on the bottom right of the table.

Now when anyone in your organization kicks off an incident you can be assured that it is being set to the appropriate severity based on business needs.

Next Steps

There are other powerful ways to automate things in FireHydrant via Runbooks. For example:

  • Take a look at the Add Incident Impacts runbook step to automatically mark certain components as impacted.
    • This allows you to use the numerous other conditions available to Runbooks. For example, "if component A is down, then automatically mark component B down too," etc.
  • You can also look at the Update Incident Details runbook step.
    • This step allows you to change the incident's details like Severity, Priority, etc. Once again, with the available of numerous conditions, you can configure flexible automation like, "If incident has this component down and adds a tag to the incident "cache-issue", then change the severity of the incident to SEV2.
  • You can potentially skip the Severity Matrix entirely if you want to predefine the types of incidents your organization has.
    • This simplifies the declaration process so users simply need to select a type, and everything else - from runbooks, to severity assignment, team assignment, task lists, and more - are already pre-configured.

Last updated on 9/29/2023