---
aliases:
  en:
  - first critical tries
  ja:
  - 最初の危険な試行
concept_id: first_critical_try
preferred:
  en: first critical try
  ja: 最初の重要な試行
relations:
  relations:
    prerequisite:
    - catastrophic_risk
status: active
---

# first critical try

The first attempt to operate or rely on an AI system in a sufficiently high-stakes regime that failure of a relevant alignment or safety assumption could cause irreversible catastrophe, leaving no opportunity to learn from that failure and then retry. In AI-alignment discourse the relevant assumption may concern the system's motives or capabilities, or the theories and engineering used to make it safe. The concept does not imply that no useful evidence can be obtained from earlier, lower-stakes systems or experiments.

## prerequisite

- [catastrophic risk](../terminology/catastrophic_risk.txt)

## Documents: about

- [アラインメントは最初の危険な試行で成功する必要がある](../documents/yudkowsky.first-critical-try.txt)

## Documents: mentions

- [Yudkowsky との相違は、危険に至る経路とそれまでに取れる対応の見通しにある](../documents/christiano.gradual-progress-and-policy-response.txt)
- [アラインメントは、失敗が致命的になる分布シフト後にも汎化しなければならない](../documents/yudkowsky.alignment-must-generalize-across-the-dangerous-distribution-shift.txt)
- [Yudkowskyによる "大規模なAI学習を世界規模で無期限に停止し、計算資源規制を国際的に執行する案"](../documents/yudkowsky.international-compute-shutdown-and-enforcement.txt)
- [現在と大きく変わらない条件で人間を上回るAIを作れば、人類絶滅が最も可能性の高い結果となる](../documents/yudkowsky.superhuman-ai-under-current-conditions-would-likely-cause-extinction.txt)

[Corpus index](../index.txt)