Replacing Homegrown Environment Tooling: A Migration Playbook

The Short Version

Deciding a homegrown environment setup has outgrown the team is usually the easy call. Actually replacing it without a week of broken PR previews, confused developers, and a rollback nobody planned for, is a different problem, and it's the one this playbook addresses directly. The pattern that works reliably is four phases run in sequence, never skipped: audit what the current system actually does (not what anyone remembers it doing), run the new system in parallel against real PR traffic before anyone depends on it, cut over deliberately with a clear rollback point, and only then decommission the old scripts, on a delay, not immediately.

Phase 0: Why "Just Switch It" Fails

The tempting shortcut, replace the old CI job that calls the homegrown scripts with one that calls the new platform instead, and merge it, treats a migration like a one-line config change. It isn't, because a homegrown environment setup that has been patched over months or years usually encodes a surprising amount of undocumented behavior nobody consciously decided to depend on: a specific env var that only gets set for one legacy service, a teardown script that half-works and leaves some resources behind on purpose because someone once needed that for debugging, a naming convention a separate cost-reporting script silently parses. None of this shows up by reading the scripts once. It shows up by watching what actually happens across a real week of PR traffic.

Phase 1: Audit What Actually Exists

Before writing a single line of new template, spend genuine time cataloguing the current system's actual behavior, not its documented behavior, which are often different after enough incremental patches. The concrete output of this phase is a short document, not a perfect one, covering: every component the homegrown scripts create (and in what order), every environment variable and secret they inject and where each one's value actually comes from, what the teardown path does and does not clean up, and any hidden dependency other tooling has on this system's specific naming or output format.

1# A quick, practical audit technique: instrument the existing scripts
2# to log every resource they touch during one real PR cycle
3grep -rn "kubectl apply\|docker build\|aws \|gcloud " ./scripts/create-preview-env.sh

This phase feels slow relative to the excitement of standing up the new system, and it is the single highest-leverage step in the whole playbook, because almost every migration horror story traces back to a behavior the old system had that nobody wrote down and the new system quietly does not replicate.

Phase 2: Build the Equivalent Template

With the audit in hand, translate the current stack into a bunnyshell.yaml template, matching what the audit found rather than what an idealized version of the stack should look like. This is deliberately not the moment to also fix unrelated infrastructure debt; a migration that tries to modernize the stack and replace the tooling at the same time makes it much harder to tell, when something breaks, which change actually caused it.

1kind: Environment
2name: pr-preview
3type: ephemeral
4
5components:
6  - kind: Database
7    name: postgres
8    dockerCompose:
9      environment:
10        POSTGRES_PASSWORD: SECRET['db-pass']
11
12  - kind: Application
13    name: api
14    dependsOn: [postgres]
15    gitRepo: 'https://github.com/org/api.git'
16    hosts:
17      - hostname: 'api-{{ env.base_domain }}'
18        servicePort: 8080

For the general shape of a first working template and the common early-adoption gotchas, Migrating from Homegrown Ephemeral Environments to Bunnyshell covers the conceptual mapping from a typical DIY setup's pieces (a shell script per environment lifecycle stage, a naming convention, a teardown cron job) to their equivalents in a single declarative template.

Phase 3: Run Both Systems in Parallel

This is the phase most tempting to skip and most costly to skip. Rather than cutting the old system off the moment the new template deploys successfully once, run both systems side by side against real PR traffic for a defined window, typically one to two full weeks of normal team activity, not a single manual test.

1# CI, temporarily creating both, with the new one non-blocking
2- name: Old system (blocking)
3  run: ./scripts/create-preview-env.sh
4- name: New system (parallel, non-blocking)
5  run: |
6    bns environments create --from-path bunnyshell.yaml \
7      --name "pr-${{ github.event.number }}-new" \
8      --project <PROJECT_ID> --k8s <CLUSTER_ID> --deploy || true

The || true matters here: the new system's environment is created for comparison and team familiarization, but a failure in it during the parallel phase should not block a real PR, since the old system is still the actual source of truth for reviewers during this window. Developers get a second, informal preview link to compare against the old one, and the team accumulates real experience with the new system's failure modes (a missing dependsOn, a wrong servicePort) on low-stakes, side-by-side comparisons instead of under cutover pressure.

Phase 4: Cut Over Deliberately

Once the parallel-run phase has covered a full range of normal PR activity (a database migration, a multi-service change, a reverted PR, an unusually long-lived branch) without surprises, cutover becomes a config change rather than a leap of faith: flip the new system from non-blocking to blocking, and stop invoking the old scripts.

1- name: New system (now blocking)
2  run: |
3    bns environments create --from-path bunnyshell.yaml \
4      --name "pr-${{ github.event.number }}" \
5      --project <PROJECT_ID> --k8s <CLUSTER_ID> --deploy

Pick a specific, communicated cutover date rather than a vague "sometime this sprint," and make sure the team knows exactly which day PR previews will start coming exclusively from the new system, so anyone who hits an issue on cutover day knows immediately what changed rather than guessing.

Phase 5: Decommission on a Delay, Not Immediately

The old scripts should be archived, not deleted, in the days immediately following cutover. A clearly marked, deprecated branch or a legacy/ directory keeps them accessible without being in anyone's way, which matters because the first genuinely unusual PR after cutover (an edge case the audit missed, a service the parallel-run window happened not to exercise) is exactly the moment a fallback is worth having available. A reasonable delay before actually deleting the old scripts is two to four weeks past cutover, long enough to have seen at least one full sprint's worth of varied PR activity running exclusively on the new system.

What to Do When the Migration Surfaces a Real Gap

Occasionally the audit or the parallel-run phase surfaces something the new platform genuinely does not have a clean equivalent for: a highly custom teardown step, an unusual multi-cloud resource the old scripts provisioned by hand. The GenericComponent and Terraform component kinds exist specifically for this, letting a fully custom bash workflow or infrastructure-as-code step live inside the same declarative template rather than requiring the migration to stall on a missing built-in feature.

1- kind: GenericComponent
2  name: legacy-resource
3  runnerImage: 'ubuntu:22.04'
4  deploy:
5    - './scripts/provision-legacy-resource.sh'
6  destroy:
7    - './scripts/teardown-legacy-resource.sh'

This lets a migration bring over a genuinely unusual piece of the old system as-is, wrapped rather than rewritten, and revisit replacing it with something more native later, decoupled from the main migration's timeline.

Common Rollback Scenarios and How to Plan for Them

Even a well-run migration occasionally hits a genuine surprise after cutover, and having a specific rollback plan for the most likely scenarios, rather than a vague "we'll figure it out," is what keeps a surprise from turning into a multi-day scramble.

The new template misses an edge case the parallel-run window didn't cover. This is the most common post-cutover surprise, usually a service configuration or a rarely-hit lifecycle path (a specific database migration pattern, an unusual teardown requirement) that simply didn't come up during the parallel-run window because no PR happened to exercise it. The fix is almost always a template update, not a full rollback: since the old scripts are still archived and accessible, temporarily re-enabling them for just the affected team or service while the template gets patched is far less disruptive than reverting the whole migration.

Team velocity drops in the week right after cutover. Some temporary slowdown is normal and expected, since the team is learning a new system's specific commands and failure modes. This is worth explicitly communicating in advance, so it reads as an expected, temporary cost rather than evidence the migration was a mistake, which is a very different (and usually wrong) conclusion to reach a few days after cutover.

A cost or performance regression appears that wasn't visible during the parallel-run window. Parallel-run testing catches functional gaps well; it catches cost and performance regressions less reliably, since the new system was running at reduced, secondary-priority load during that phase. Monitoring actual resource usage and cost for the first two to three weeks post-cutover, specifically, catches this class of surprise before it compounds into a much larger bill.

Team Enablement, Not Just Tooling

The technical migration is only half the actual work. A team accustomed to a homegrown system's specific commands and mental model needs deliberate onboarding to the new one, not an assumption that documentation alone will get them there. A short, hands-on session (thirty minutes is often enough) walking through the five or six commands a developer actually uses day to day, creating an environment, checking its status, viewing logs, tearing it down, covers the bulk of daily usage and headed off the majority of "how do I even..." questions that otherwise land in a platform team's inbox individually, one at a time, in the first week after cutover.

1# The small command set that covers most day-to-day usage
2bns environments create --from-path bunnyshell.yaml --name "my-feature" --project <ID> --k8s <ID> --deploy
3bns environments show --id <ENV_ID>
4bns logs --component <COMPONENT_ID> -f
5bns environments delete --id <ENV_ID>

Pairing this session with the same short runbook of common failure signatures described in Debugging Failed Deploy Pipelines Before They Block a Merge gives the team a self-service path for the handful of issues that come up most often, rather than routing every early question through whoever led the migration.

Sequencing the Migration Across Multiple Services

A stack with several independently deployable services rarely needs to migrate all of them in one motion, and attempting to is usually where a migration's scope quietly balloons into something much riskier than intended. Sequencing by service, starting with the lowest-risk, most self-contained one, gives the team a genuine dry run on real infrastructure before the higher-stakes services follow.

1Migration order, by risk (lowest first):
21. Internal admin tool (low traffic, no customer-facing impact if something's briefly wrong)
32. Background worker service (async, tolerant of a short delay if migration hits a snag)
43. Public API (customer-facing, moderate traffic)
54. Core web application (highest traffic, most scrutiny if something goes wrong)

Each service completing its own full four-phase cycle (audit, parallel-run, cutover, decommission) before the next one starts means a problem discovered on service 2 gets fixed in the template before service 4, the highest-stakes one, ever encounters it. This is slower in aggregate calendar time than migrating everything simultaneously, and it is the sequencing that actually avoids the worst-case outcome: several services all hitting the same undiscovered template gap in the same week.

Related Reading

For the conceptual mapping from a typical DIY environment setup to a single declarative template, see Migrating from Homegrown Ephemeral Environments to Bunnyshell. For the pattern that keeps environments consistent across a team once the migration is complete, see One Staging Blueprint, Infinite Ephemeral Copies.

Measuring Whether the Migration Actually Delivered Value

It's worth closing the loop on the migration the same way any of the changes in this cluster should be closed: by measuring, not assuming, that it delivered what it promised. Comparing environment creation time, deploy failure rate, and time spent on environment-related support questions for a few weeks before and after the migration gives a concrete answer, rather than a general sense that things feel better since the switch.

1Before migration (homegrown scripts):
2  Median environment creation time: 14 minutes
3  Environment-related support questions per week: ~9
4
5After migration (managed platform):
6  Median environment creation time: 3 minutes
7  Environment-related support questions per week: ~2

A team that skips this comparison loses the ability to make the case for the migration's value later, whether to justify the time invested to leadership or simply to confirm to the team itself that the disruption was worth it, which matters for building confidence in tackling the next platform change with the same deliberate process.

Checklist

• Audited what the current homegrown system actually does, not what the team remembers it doing
• Built the new template to match audited behavior, without also trying to fix unrelated infrastructure debt at the same time
• Ran both systems in parallel against real PR traffic for at least one to two full weeks
• Cut over on a specific, communicated date, not a vague window
• Archived (not deleted) the old scripts for a delay period after cutover