Add an issue template to disable CI jobs (#97045)

Per title, I will update the runbook to point to this after the review
Pull Request resolved: https://github.com/pytorch/pytorch/pull/97045
Approved by: https://github.com/clee2000, https://github.com/ZainRizvi
diff --git a/.github/ISSUE_TEMPLATE/ci-sev.md b/.github/ISSUE_TEMPLATE/ci-sev.md
index 2b6bbfc..506c7a0 100644
--- a/.github/ISSUE_TEMPLATE/ci-sev.md
+++ b/.github/ISSUE_TEMPLATE/ci-sev.md
@@ -1,6 +1,7 @@
 ---
 name: "⚠️ CI SEV"
 about: Tracking incidents for PyTorch's CI infra.
+labels: "ci: sev"
 ---
 
 > NOTE: Remember to label this issue with "`ci: sev`"
diff --git a/.github/ISSUE_TEMPLATE/disable-ci-jobs.md b/.github/ISSUE_TEMPLATE/disable-ci-jobs.md
new file mode 100644
index 0000000..b4b078b
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/disable-ci-jobs.md
@@ -0,0 +1,20 @@
+---
+name: Disable CI jobs (PyTorch Dev Infra only)
+about: Use this template to disable CI jobs
+title: "DISABLED [WORKFLOW_NAME] / [PLATFORM_NAME] / [JOB_NAME]"
+labels: "module: ci"
+---
+
+> For example, DISABLED pull / win-vs2019-cpu-py3 / test (default). Once
+> created, the job will be disabled within 15 minutes. You can check the
+> list of disabled jobs at https://ossci-metrics.s3.amazonaws.com/disabled-jobs.json
+
+> If you need to get this out ASAP instead of waiting for 15 minutes,
+> you can manually trigger the workflow at https://github.com/pytorch/test-infra/actions/workflows/update_disabled_tests.yml
+> once the issue is created to update the above JSON list right away.
+
+> Noted: you need to have write access to PyTorch repo to disable CI
+> jobs. The issue will be rejected otherwise.
+
+## Reason
+*Provide a reason why this is needed and when this can be resolved*.