Observability

Two stacks, not one

src/instrument.ts initializes both Sentry and Google Cloud's native Error Reporting + Profiler in the same bootstrap file, imported first in src/server.ts (must stay first — Cloud Profiler needs to patch V8 before other modules lock in optimizations). An older internal note describes the GCP-native stack as a "replacement for Sentry" — that's not what happened; @sentry/node, @sentry/profiling-node, @google-cloud/error-reporting, and @google-cloud/profiler are all present and initialized today. Treat this as two independent, currently-coexisting systems rather than a completed migration, unless you're specifically told one is being sunset.

• Sentry — auto-enables whenever SENTRY_DSN is set (no separate flag). Handles tracing (SENTRY_TRACES_SAMPLE_RATE), profiling, and structured Pino log forwarding so log context shows up alongside errors in the Sentry UI.
• GCP Error Reporting + Profiler — gated by its own OBSERVABILITY_ENABLED / OBSERVABILITY_PROFILER_ENABLED flags, independent of Sentry's config. Surfaces in the Cloud Console instead of a third-party dashboard, and is what Cloud Monitoring alert policies (below) hook into.

What the GCP-native side captures

• Unhandled exceptions and unhandled promise rejections at the process level (src/server.ts).
• Unexpected (non-operational) errors reaching the global error handler with HTTP status ≥ 500 (src/shared/utils/response.util.ts).
• Errors from route handlers that catch and call errorResponse() directly, when they pass originalError and the response is ≥ 500.
• CPU and heap profiles from Cloud Profiler.

Deliberately not captured: operational AppError instances and other 4xx responses — those are expected user-facing errors, and reporting them would drown the real signal.

Reporting a caught error correctly

In most route handlers, do nothing — throw and let the global errorHandler report it. If a handler catches an error and calls errorResponse() directly, pass originalError so it still reaches Error Reporting on a 5xx:

try {
  await doWork();
} catch (err) {
  return errorResponse({
    response: res,
    message: 'We could not process your request',
    statusCode: 500,
    originalError: err, // required for 5xx paths to actually get reported
  });
}

4xx callers can skip originalError — it's ignored.

Choosing a status code so the right thing gets reported

AppError defaults to isOperational: true, which the global handler's filter treats as "expected, don't report." Anything worth seeing in Error Reporting (server-side surprises, third-party failures) needs isOperational: false explicitly:

Scenario	Throw with	Reported?
Upstream third-party API failed (Apple, Paystack, etc.)	new AppError(httpStatus.BAD_GATEWAY, '...', undefined, undefined, false)	Yes
Our own bug / unexpected internal state	new AppError(httpStatus.INTERNAL_SERVER_ERROR, '...', undefined, undefined, false)	Yes
User-fixable (invalid token, expired session)	new AppError(httpStatus.UNAUTHORIZED, '...')	No — 4xx defaults to operational
Service genuinely unavailable, retry might help	new AppError(httpStatus.SERVICE_UNAVAILABLE, '...', undefined, undefined, false)	Yes

Cloud Monitoring → Slack alerting

A Slack notification channel (pointed at the team's monitoring channel) is the target for these Cloud Monitoring alert policies, if configured for the current project:

1. 5xx error spike — a logging-based metric over severity>=ERROR, alerting on a sustained rate over a short window.
2. p95 latency regression — run.googleapis.com/request_latencies (if on Cloud Run) or the equivalent GCE metric, alerting past a threshold sustained for ~10 minutes.
3. New error group — Cloud Error Reporting's own built-in "first occurrence of a new error" notification, configured directly in Error Reporting → Settings → Notifications, separate from Cloud Monitoring policies.
4. Profiler silence — optional: alert if cloudprofiler.googleapis.com/profile/upload_count shows no data for an extended period during business hours, which usually means the agent silently stopped uploading rather than that there are simply no requests.

Verifying it's working

OBSERVABILITY_ENABLED=true \
GCP_PROJECT_ID=<project> \
GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa.json \
yarn start:dev

Hit a route that throws, then check Cloud Console → Error Reporting for the same project — the event should appear within ~30 seconds under the service name from APP_NAME. Profile uploads (if OBSERVABILITY_PROFILER_ENABLED=true) typically take longer to first appear, on the order of minutes.

Related code

• src/instrument.ts — dual bootstrap, must stay the first import in src/server.ts
• src/config/env.config.ts — sentry and observability config blocks
• src/shared/utils/response.util.ts — errorHandler, errorResponse