Restore a managed instance when it is completely gone

Assess what is deleted

Navigate to sourcegraph-managed-$CUSTOMER project, and look at existing compute instance. Does the VM still exist?

• No, the VM is gone.

  • Does the data disk default-$CURRENT_DEPLOYMENT-data-disk still exists?
    • Yes, re-create the VM with existing data disk
    • No, re-create the VM with a new data disk from disk snapshot

• Yes, follow operation guides to troubleshoot services condition. If unable to recover running services on the VM, fallback to re-create the VM with a new data disk from disk snapshot.

Re-create the VM with existing data disk

1. Open sourcegraph/deploy-sourcegraph-managed and check out to the $CUSTOMER directory
2. Run terraform apply to reconcile the infrastructure to its definition in code.
3. Follow confirm instance health

Re-create the VM with new data disk from disk snapshot

1. Run gcloud compute snapshots list --project=sourcegraph-managed-$CUSTOMER --sort-by="~creationTimestamp" --limit=5 --format="table(name,creationTimestamp)" and copy the name of the latest snapshot

2. Go to sourcegraph/deploy-sourcegraph-managed and create a new branch $CUSTOMER/restore-instance

3. Edit $CUSTOMER/terraform.tfvars. NOTES: the key could be black depending on the current active instance

   disks = {
     red = { from_snapshot = "REPLACE_ME_WITH_SNAPSHOT_NAME" }
   }
   

4. Run terraform apply to reconcile the infrastructure to its definition in code.

5. Follow confirm instance health

6. Commit your changes and open a Pull Request