Просмотр исходного кода

docs(e2e): state why the oracle leg stays out of CI

The oracle area was disabled with the same one-line justification as the others, that
the project holds no credentials. For oracle that is not the whole reason, and the rest
of it does not go away when someone donates an account.

Measured against a live tenancy: an always-free account caps secrets at 150, and a
deleted secret cannot return to the pool sooner than 24 hours. That floor is enforced,
not advisory; 5m, 1h, 6h, 12h, 23h and 23h55m are all rejected as out of range. One full
run of the suite creates 15 secrets, so a free account sustains ten runs a rolling day.
Two runner settings cut that further: --flake-attempts=2 re-runs a failing spec from
BeforeEach, and -p -nodes=5 drives five concurrent create/delete streams, which the
Vaults service rate-limits.

A leg on those numbers exhausts its quota partway through a busy day and then fails for
reasons unrelated to the change under test. A leg that goes red at random is worse than
no leg, because it teaches reviewers to disregard the colour. So the position is that
oracle stays a local pre-submit tool until a tenancy without that cap exists. Worth
noting the blocker is the cap rather than the bill: OCI does not price secrets
individually.

Refs: external-secrets/external-secrets#6767
Refs: external-secrets/external-secrets#6755
Signed-off-by: Alexander Chernov <alexander@chernov.it>
Alexander Chernov 1 месяц назад
Родитель
Сommit
716d11336e
2 измененных файлов с 49 добавлено и 17 удалено
  1. 42 17
      docs/contributing/e2e-credentials.md
  2. 7 0
      e2e/matrix.yaml

+ 42 - 17
docs/contributing/e2e-credentials.md

@@ -196,23 +196,48 @@ export ORACLE_FINGERPRINT=..
 export ORACLE_KEY="$(cat ~/.oci/eso-e2e.pem)"
 ```
 
-### Quota
-
-Deleting a secret is a scheduled operation, and `timeOfDeletion` defaults to **30 days**
-out. The accepted range is 1 to 30 days. A secret pending deletion keeps both its name
-and its slot, so leaving the default in place means a suite run's secrets occupy the
-tenancy for a month. At roughly 15 to 20 secrets per run that exhausts the 150 always
-free secrets in about seven runs, after which creates start failing for a reason that
-looks nothing like a quota problem.
-
-Anything creating secrets here should schedule deletion at the 1 day minimum.
-
-Every state transition in the Vault service is also asynchronous, and the next operation
-is rejected with `409 IncorrectState` until the previous one settles. Creating then
-immediately deleting fails, because the secret is still `CREATING`; cancelling a pending
-deletion then immediately rescheduling fails, because it is still `CANCELLING_DELETION`.
-Measured latencies are a few seconds each, but they are not zero and they are not
-bounded by anything documented.
+### Quota, and why this leg is not in CI
+
+**This suite is a local pre-submit tool. It will not be enabled as a CI leg while the
+only account available is always-free.** That is a deliberate position, not a missing
+task, and the numbers behind it are below.
+
+Deleting a secret is scheduled rather than immediate. `timeOfDeletion` defaults to 30
+days out and the accepted range is 1 to 30 days. **The 24 hour floor is enforced**:
+values of 5m, 1h, 6h, 12h, 23h and 23h55m are all rejected with
+`400 InvalidParameter: ScheduledTimeOfDeletion is in invalid range`. A secret pending
+deletion keeps both its name and its slot, so there is no way to return one to the pool
+sooner than a day.
+
+An always-free tenancy gets 150 secrets. A full run of this suite creates **15**, one per
+remote secret across the twelve common cases. That is **10 runs per rolling 24 hours**,
+starting from an empty vault. Anything creating secrets here should therefore schedule
+deletion at the 1 day minimum rather than accept the 30 day default, which would cut the
+same budget to roughly one run per two days.
+
+Two runner behaviours make the ceiling lower than 10 in practice:
+
+- `entrypoint.sh` passes `--flake-attempts=2`, so a failing spec re-runs from
+  `BeforeEach` and creates its secrets again, up to 3x for a spec that never passes.
+- `entrypoint.sh` runs `-p -nodes=5`, so five specs create and delete concurrently. The
+  Vaults service rate-limits: eight schedule calls in quick succession return
+  `429 TooManyRequests`. `E2E_NODES` is read inside the pod but is not in `run.sh`'s
+  `--env` allowlist, so parallelism cannot be lowered from outside without editing
+  `run.sh`.
+
+A CI leg on those numbers would exhaust the quota partway through a busy day and then
+fail for a reason unrelated to the change under test. A leg that goes red at random is
+worse than no leg at all, because it teaches reviewers to disregard the colour. Enabling
+this in CI needs a tenancy without the always-free secret cap. Note that the blocker is
+the cap rather than the bill: OCI does not price secrets individually, charging instead
+for HSM keys and virtual private vaults, neither of which this suite needs.
+
+Separately, every state transition in the Vault service is asynchronous, and the next
+operation is rejected with `409 IncorrectState` until the previous settles. Creating then
+immediately deleting fails because the secret is still `CREATING`; cancelling a pending
+deletion then immediately rescheduling fails because it is still `CANCELLING_DELETION`.
+Measured latencies are a few seconds, but they are neither zero nor documented as
+bounded.
 
 ### Why it does not run yet
 

+ 7 - 0
e2e/matrix.yaml

@@ -209,6 +209,13 @@ areas:
       - "e2e/suites/provider/cases/gitlab/**"
     enabled: false
 
+  # Oracle stays disabled by policy, not just for want of credentials. An
+  # always-free OCI tenancy caps secrets at 150, and a deleted secret cannot
+  # return to the pool for 24h (the floor is enforced). One run of this suite
+  # costs 15 secrets, so a free account sustains ten runs a day before creates
+  # start failing for reasons unrelated to the change under test. Enabling this
+  # leg needs a tenancy without that cap. Until then the suite is a local
+  # pre-submit tool; see docs/contributing/e2e-credentials.md.
   - name: oracle
     suite: provider
     labels: "oracle && !managed"