Skip to content

Commit dbf0c3f

Browse files
sushanbsushanb
authored
feat(bigtable): add per-AFE sessionList for the two-tier session pool (#20224)
## Summary First of five PRs porting the session-pool infrastructure from `feat/bigtable-sessionz-debug` (which powers the recycle-repro fleet) upstream. Stacks on #20215 (Session lifecycle, now merged). ### What lands **New file: `session_list.go`** (~600 LOC) — the per-pool sessionList data structure that groups sessions by the AFE (Application Front End) their handshake landed on. Consumed by the two-tier picker (K-choice-over-AFEs → dequeue-idle-session) in a follow-up PR. Key types: - `AfeID` (int64) — AFE identifier from the server's PeerInfo header at session-open. 0 is the sentinel "unknown" bucket for handshakes that did not carry a peer-info header. - `AfeSnapshot` — value-typed view of an afeHandle for pickers to score without holding sl.mu (Checkout re-resolves by ID; no *afeHandle escapes). - `AfeSnapshotRow` — debug-UI row emitted by `sessionList.Snapshot()` (consumed by afez/sessionz in a later debug PR). - `afeHandle` (unexported) — per-AFE bucket: FIFO idle queue, refCount (idle + inFlight + closing), two PeakEwma trackers. - `SessionHandle` — pool bookkeeping wrapper around Session. Carries `inExpectedCount` (I5 guard against WaitServerClose retry storm) and `activated / closingRecorded / closeRecorded` dedup flags for the pool's per-session hook chain. - `sessionList` (unexported) — the state machine, guarded by `sl.mu`. The state model documents **six invariants (I1-I6)** that every method preserves: ``` I1 inExpectedCount ⇒ handleToAfe[sh] != nil I2 readyCount == count of inExpectedCount handles I3 afesWithReady == {afe : len(afe.sessions) > 0} I4 afe.refCount == count of handleToAfe entries pointing at afe I5 sh in afe.sessions ⇒ handleToAfe[sh]==afe AND inExpectedCount I6 refCount-- only on OnSessionClosed (Closing keeps slot warm) ``` Lock order: `sl.mu` ONLY. `RecordVRpcOutcome` deliberately drops `sl.mu` between the map lookup and the `PeakEwma.Update` so the hot vRPC-outcome path doesn't serialize on it. Consolidates AFE types (`AfeID` + `AfeSnapshot`) that previously lived in `afe_snapshot.go` — sessionList now owns all AFE-bucket concepts. Deletes `afe_snapshot.go`. **New file: `session_list_test.go`** (~770 LOC) — I1-I6 coverage plus per-method tests (OnSessionStarted / Checkout / ReleaseToPool / OnSessionClosing / OnSessionClosed / RecordVRpcOutcome / ReadyAfes / Snapshot / AllHandles / Prune) and a concurrency stress test covering the documented lock-drop path in RecordVRpcOutcome. **Edits to `debug_tracer.go`** — three new tag constants for sessionList bookkeeping violations (all unreachable-under-invariants, kept as belt-and-suspenders): - `tagSessionListStartedNilSession` - `tagSessionListRefcountUnderflow` - `tagSessionListReadyCountUnderflow` **Edit to `session.go`** — one-line comment retarget on `AfeID()` (type now lives in `session_list.go`, not the deleted `afe_snapshot.go`). ### Stack 1. #20211 — Session debug surface (merged) 2. #20213 — Session vRPC dispatch (merged) 3. #20215 — Session lifecycle (merged) 4. **This PR** — sessionList (PR-1 of 5) 5. Next — SessionPoolImpl core, pool_lifecycle, pool_scaling, pool_debug ## Test plan - [x] `go build ./internal/transport/` passes - [x] `go vet ./internal/transport/` clean - [x] `go test ./internal/transport/ -race -count=1 -short -timeout=120s` passes — 20+ new sessionList tests plus all pre-existing.
1 parent 42a1fa8 commit dbf0c3f

5 files changed

Lines changed: 1408 additions & 29 deletions

File tree

‎bigtable/internal/transport/afe_snapshot.go‎

Lines changed: 0 additions & 28 deletions
This file was deleted.

‎bigtable/internal/transport/debug_tracer.go‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,36 @@ const (
118118
tagSessionPoolCreateFailed = "session_pool_create_failed"
119119
tagSessionPoolPickLostRace = "session_pool_pick_lost_race"
120120

121+
// sessionList bookkeeping violations.
122+
//
123+
// tagSessionListRefcountUnderflow fires when OnSessionClosed would
124+
// decrement an afeHandle's refCount below zero. Under I4/I6 this is
125+
// unreachable — every decrement is preceded by an OnSessionStarted
126+
// increment and the handleToAfe map delete guards against a
127+
// double-close reaching the decrement. A non-zero count here means
128+
// bookkeeping drifted (missed OnSessionStarted, mis-paired hook
129+
// ordering, or a force-close bypass) and should be investigated.
130+
tagSessionListRefcountUnderflow = "session_list_refcount_underflow"
131+
132+
// tagSessionListReadyCountUnderflow fires when dropMembershipLocked
133+
// would drive sl.readyCount below zero. Under I2 this is unreachable —
134+
// inExpectedCount flips true exactly once (in OnSessionStarted) and
135+
// dropMembershipLocked is idempotent via the inExpectedCount guard.
136+
// A non-zero count here means bookkeeping drifted (an inExpectedCount
137+
// increment without the paired OnSessionStarted, or a decrement path
138+
// bypassing the guard) and should be investigated before it corrupts
139+
// scale-up decisions gated on ReadyCount().
140+
tagSessionListReadyCountUnderflow = "session_list_ready_count_underflow"
141+
142+
// tagSessionListStartedNilSession fires when OnSessionStarted is
143+
// called with a SessionHandle whose session pointer is nil.
144+
// Unreachable in production (createSession populates sh.session
145+
// synchronously before the hook fires) — the assertion exists so
146+
// that a future caller who accidentally wires a nil-session handle
147+
// (e.g. a test double promoted to production) shows up in the
148+
// debug counter instead of no-op'ing silently.
149+
tagSessionListStartedNilSession = "session_list_started_nil_session"
150+
121151
// Client configuration polling.
122152
tagClientConfigPollFailed = "client_config_poll_failed"
123153
tagClientConfigPollCtxExpired = "client_config_poll_ctx_expired"

‎bigtable/internal/transport/session.go‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -232,7 +232,8 @@ func (s *Session) PeerInfo() *spb.PeerInfo { return s.peerInfo.Load() }
232232

233233
// AfeID returns the AFE identifier, or 0 pre-Ready. Stable for the session's
234234
// lifetime — PeerInfo is populated once at StateReady. AfeID type lives in
235-
// afe_snapshot.go (same package).
235+
// session_list.go (same package) alongside the per-AFE sessionList
236+
// bookkeeping that consumes it.
236237
func (s *Session) AfeID() AfeID {
237238
if p := s.peerInfo.Load(); p != nil {
238239
return AfeID(p.GetApplicationFrontendId())

0 commit comments

Comments
 (0)