Troubleshooting
Problems are documented the way you meet them: by symptom. Every entry names the cause, the fix, and the change that stops it happening again.
How troubleshooting works here
Troubleshooting documentation at BuiltForProd is written from the symptom backward. You do not know the cause when the pager goes off; you know the error string, the failed step, or the plan that should have been empty. So entries are indexed by what you can see, and each one carries the same four parts.
Entries are not kept in a separate manual. They live at the bottom of the page that documents the component, so the description of how something works and the list of ways it fails sit together, and fixing one keeps the other honest.
Symptom
What you actually saw: the error string, the failed step, the pod that will not start, the plan that is not empty. Entries are indexed by symptom because that is all you have at the start.
Cause
Why the platform produced that symptom, stated as a mechanism rather than a guess. Most causes are a control doing its job: a policy denying an action, a gate waiting for a reviewer, a dependency not yet applied.
Fix
The change that resolves it, as a command or a pull request, with the account and stage it runs in. If the fix is a console action, the entry says how to get the same change back into code.
Prevention
What stops it happening again: a naming rule, a guard script, a review step, a default. An entry without a prevention line is an incomplete entry.
What is covered
Eight categories, each documented on the pages that own the component.
Deployment and pipelines
A plan that will not run, an apply blocked at the production gate, a guard script failing on a hardcoded address, a unit that plans changes in four accounts at once.
Access and identity
AccessDenied where you expected access, a pipeline that cannot assume its deployer role, a new engineer who sees no accounts, an auditor who can see a resource but not its contents.
Networking
Traffic that disappears between stages, an address range that overlaps, a private name that does not resolve, a VPN session that connects but reaches nothing.
Kubernetes and ArgoCD
An application stuck out of sync, a pod that cannot reach a data store, a cluster you cannot reach with kubectl, a node group that will not scale.
Secrets sync
A secret that will not decrypt, a sync workflow that fails on a placeholder, a workload reading a stale value, a secret committed in plaintext.
Data pipelines
A job that ran but wrote nothing, a trigger that did not fire, a partition that is missing, a table an analyst cannot query.
Drift
A drift issue opened overnight, a plan that is not empty on an account nobody touched, a console change somebody made during an incident.
Cost
A bill that jumped after a merge, a switch nobody knew was expensive, a stage sized like production, an idle cluster running all weekend.
Links marked here open documentation on docs.builtforprod.com. The concept pages are public; component pages open with the entitlement for that product. The knowledge base explains the difference.
Before you open a ticket
Read the run report, not the log
Every plan and apply run attaches a report artifact listing each unit with its result and the reason it reached it. Start there; the raw log is for confirming what the report already told you.
Check for an open drift issue
Drift detection runs nightly for critical accounts and weekly for the rest, and opens a labeled GitHub issue when a plan is not empty. Nothing is auto-reverted, so an open issue is often the whole story.
Reproduce in a lower stage
The stages are built from the same code, so a change that fails in staging usually fails in dev. If it only fails in one stage, the difference is in that stage’s values, and that narrows the search to one file.
Check what changed
The pull request history is the change log for the platform. If the symptom started today, look at what merged yesterday and at any upstream update that was pulled in.
Write down the facts as you go
Stage, account, region, repository, run link, error text, and anything already changed by hand. If you still need a ticket, you have already written it.
One exception: production is down
If customers cannot use your service, open the request first and investigate second. State the impact, the stage and the time it started, then keep adding facts to the same thread as you find them.
A worked example
One real entry, in the shape every entry takes.
A pull request plan fails in one workload account with an access denied error on a storage or key action, while the same unit plans cleanly in another stage.
The security guardrail policy in the AWS Enterprise Baseline denies actions performed without multi-factor authentication, except for service roles, single sign-on roles, roles inside your namespace, and roles carrying an exemption tag. A workload role created outside the naming convention matches none of those exceptions, so the organization policy denies it before the account policy is ever evaluated.
Rename the role so it falls inside the namespace prefix, or add the documented exemption tag to it, then re-run the plan. The denial disappears because the role now matches an exception, not because a guardrail was weakened.
Name every workload role within the namespace, which the naming convention already requires, and let the module that creates the role own the name rather than setting it by hand. The convention is enforced in review, and the symptom stops appearing.
Note what the fix is not: it is not turning off the guardrail. A control that refuses an action is doing its job, and the entry tells you which documented exception applies. That is the difference between a platform you can operate and a platform you work around.
Where troubleshooting ends and support starts
- The documentation answers it. Most symptoms are known, documented and have a prevention line. No ticket needed.
- You need a second pair of eyes. That is what BuiltForProd Managed On-Demand hours are for.
- It is an incident and you have a response commitment. Service levels sets out the escalation path and how severity is assigned.
- It is a security issue. Report it to the security address on security and ownership, not through a general support channel.
Troubleshooting questions
How is troubleshooting documentation organized?
Every troubleshooting entry follows the same four-part shape: symptom, cause, fix and prevention. Entries live on the page that documents the component that produced the symptom rather than in a separate manual, so the explanation of how something works and the list of ways it fails sit next to each other.
What should I check before opening a support ticket?
Check the run report from the failed workflow, any open drift issue on the account, and the troubleshooting section of the page covering that component. Those three sources resolve most problems, and if they do not, the facts you collected are exactly what a support request needs.
Why does the platform deny actions that used to work?
Most unexpected denials are a guardrail doing its job rather than a fault. Service control policies deny long-lived credentials, actions outside approved regions and any attempt to stop the audit trail, so an action that worked in a single unguarded account is refused here, and the troubleshooting entry for that policy explains which exception applies.
Something was changed in the console during an incident. What now?
Nothing breaks immediately, but the environment and the code now disagree, and drift detection will open an issue the next time it runs. The fix is to reproduce the change as code, plan it, and merge it, so the next apply does not silently revert the thing that saved you.
Do I need a support contract to use the troubleshooting documentation?
No. The full documentation set, including every troubleshooting entry, is delivered with your deployment and stays available to your organization. A Managed tier buys engineers, not access to the manual.
Stuck on something the documentation does not cover?
Send the run report, the stage and the error text. If it is a gap in the documentation, we fix the documentation as well as the problem.