From Server Room to Cloud: Hard-Won Lessons from a Production Azure Migration

What I learned migrating two enterprise applications from on-premises infrastructure to Azure — the gotchas nobody warns you about, and the patterns that saved us.

Over the past several months, I led the end-to-end migration of two enterprise applications from on-premises Windows servers and SQL Server to Microsoft Azure. Frontends, backend APIs, an integration layer, a production database, scheduled jobs, CI/CD pipelines, DNS — the whole stack. We completed production cutover with a full smoke-test pass and zero rollback.

It went well. But "went well" hides a lot of 2 AM debugging sessions. This post is everything I wish someone had told me before we started.

The Architecture (Generalized)

The target landscape looked like this:

• Edge: CDN → Azure Front Door → WAF
• Frontends: Two Node.js apps on Azure App Service
• Backends: .NET 8 APIs on a shared PremiumV3 App Service Plan
• API Gateway: Azure API Management — for integration traffic only
• Database: Azure SQL Managed Instance (General Purpose) behind a private endpoint
• Secrets: Azure Key Vault with private endpoint
• Scheduled work: SQL Agent jobs on the MI + Windows Task Scheduler tasks on a small infra VM
• CI/CD: Azure DevOps with a self-hosted agent inside the VNet
• Monitoring: Application Insights per app, shared Log Analytics workspace

Nothing exotic. And yet almost every layer had at least one surprise.

Lesson 1: Azure AD 401s Are Almost Always an Audience Mismatch

Our most painful post-cutover bug: one API returned 401s for every authenticated request, even though the identical configuration "worked" on another API.

Root cause: the JWT validation middleware was comparing the token's aud claim against the configured ClientId — a plain GUID — while the tokens actually carried the audience as a full URI (api://<guid>). One app registration happened to issue GUID-form audiences; the other issued URI-form. Same code, different token shape.

The fix was one line: set an explicit Audience key in appsettings.json instead of relying on ClientId doubling as the audience.

Takeaway: When you see 401s in an Azure AD–protected API, decode the token first. Compare aud character-for-character against what your middleware validates. Don't assume the ClientId is the audience.

Related gotcha: when a gateway app registration fronts an API app registration, tokens must be requested with the gateway's client_id, not the API's. Requesting with the wrong one produces perfectly valid-looking tokens that fail validation downstream.

Lesson 2: Silent Logging Failures Are Worse Than Loud Errors

We spent hours debugging an integration API that returned 500s with no logs at all. Two separate issues had stacked:

1. The Serilog SQL sink was configured with unsubstituted placeholder strings (think #{ServerName}# left over from a deployment token that never got replaced). The sink failed to initialize — silently — and took all error reporting down with it.
2. UseAuthorization() had been placed inside a conditional block that only ran when auth was enabled. With auth toggled off for testing, the middleware pipeline was malformed and every request 500'd before reaching a controller.

Takeaways:

• Always configure a console sink as a fallback. If your primary sink dies, you need somewhere to see it die.
• Middleware order and placement is not optional decoration. UseAuthorization() belongs unconditionally in the pipeline; if you need an "auth off" mode, use a global AllowAnonymousFilter instead of restructuring the pipeline.
• Audit your deployment token replacement. A #{Placeholder}# that survives into production config is a time bomb.

Lesson 3: Key Vault References Need Version Pinning After Rotation

We rotated the SQL admin password post-cutover, updated the secret in Key Vault, and expected App Services using Key Vault references to pick it up.

They didn't — at least not promptly. Versionless Key Vault references cache, and the refresh isn't instantaneous or predictable. During a credential rotation, "eventually consistent" is not what you want between your app and its database.

Takeaway: After rotating a secret, update App Service environment variables to a version-pinned Key Vault reference. Yes, it means touching config on every rotation. It also means you know exactly which secret version every app is using, which turns out to be worth a lot during an incident.

Bonus: watch out for special characters in connection string passwords. Semicolons, quotes, and braces need careful escaping — generate rotation passwords with a character set your connection string format tolerates.

Lesson 4: SQL Agent Jobs Don't Just Lift-and-Shift to Managed Instance

We had ~20 SQL Agent jobs to migrate. Managed Instance supports SQL Agent, but with constraints that broke our scripted-out jobs in three ways:

1. Domain-scoped job owners (DOMAIN\user) don't exist on MI. Every job owner had to be replaced with a SQL login created for the purpose.
2. sp_delete_job calls embedded in the scripted output had to be stripped.
3. Only T-SQL subsystem job steps are fully supported — anything using CmdExec, SSIS, or PowerShell subsystems needs to be re-homed (we moved those workloads to Task Scheduler on an infra VM).

Takeaway: Treat Agent job migration as a porting exercise, not a copy exercise. Script everything out, then clean systematically: owners, delete statements, subsystems.

The Task Scheduler tasks themselves migrated cleanly via XML export/import — but every task's connection config had to change from Integrated Security to SQL authentication, since there's no domain trust between an Azure VM and a Managed Instance out of the box.

Lesson 5: The Database Restore Itself Is the Easy Part

The actual data move was almost anticlimactic:

1. Take the source database offline
2. Take a final full backup and verify it (RESTORE VERIFYONLY — do not skip this)
3. Upload to blob storage with AzCopy
4. RESTORE DATABASE ... FROM URL with a SAS credential on the MI

What people forget is everything around the restore:

• Recreate SQL users and role memberships. Logins don't travel inside the database backup the way you'd hope. Have a script ready for every application login with its exact role grants (datareader, datawriter, execute, etc.).
• Run sp_updatestats post-restore. First-day performance complaints often trace back to stale statistics, not the cloud.
• Verify TDE. Depending on how your database arrives, encryption at rest may not be enabled. Check it explicitly — ALTER DATABASE ... SET ENCRYPTION ON if needed — before your security review does.

Lesson 6: Draw the Traffic Flow, Then Draw It Again With Callouts

Our architecture had one nuance that everyone misread: the frontends called their backend APIs directly, while only integration/partner traffic routed through API Management.

Every new stakeholder assumed all traffic went through APIM. Every architecture review re-litigated it. Eventually I added an explicit callout panel to every diagram: "APIM handles integration traffic only. UI traffic bypasses APIM."

Takeaway: If a routing detail gets misread twice, it will be misread forever unless the diagram itself corrects the reader. Don't rely on people reading the arrows carefully — annotate the misconception directly.

Lesson 7: Workload Isolation on Managed Instance = Resource Governor

Our integration workload could occasionally hammer the database hard enough to affect interactive users. On Azure SQL Database you'd have limited options; on Managed Instance, Resource Governor is available and it's exactly the right tool:

• A dedicated resource pool with CPU and memory caps for integration workloads
• A workload group with lower importance and a MAXDOP limit
• A classifier function routing sessions by SQL login

One caution: keep the classifier function trivial. It runs on every login, so anything slow in there becomes login latency for the entire instance.

Also budget time for the General Purpose tier's IO envelope. GP is remote-storage-backed; IO-heavy workloads that were fine on local SSDs on-prem can hit the ceiling. Query tuning, DOP limits, and alerting on IO percentage got us stable — but the honest long-term answer is often a tier or vCore bump.

Lesson 8: Load Test "Failures" Deserve Forensics Before Panic

Our first serious load test reported an 18–22% error rate. Cue alarm.

Root-cause analysis showed the application was fine: the JMeter test plan was missing Authorization headers on roughly half its samplers. The "errors" were 401s the app was correctly returning to unauthenticated requests.

Similarly, an early scare about latency dissolved on inspection: p95 around 700ms with a flat p50 around 180ms doesn't mean the system is degrading — it means occasional outliers exist. If p50 is flat under ramp, you don't have systemic pressure.

Takeaways:

• Never accept a raw error percentage from a load tool. Break errors down by status code and endpoint before concluding anything.
• Learn to read p50 vs p95 together. A scary p95 with a calm p50 is an outlier story, not a capacity story.
• How the frontend calls APIs matters as much as API speed — parallel vs sequential call patterns change perceived latency dramatically.

Lesson 9: DNS Cutover Is a Sequence, Not an Event

The cutover itself was CNAME changes pointing custom domains at the Front Door endpoint. Simple. What made it low-drama:

• Pre-stage everything at Front Door: custom domains validated, certs deployed, routes and WAF associations configured before touching DNS.
• Inventory every hostname, not just the user-facing ones. We found integration and backend references still using an old domain suffix that had to be updated separately.
• Hunt for hardcoded server names in code. We found an on-prem server name hardcoded in a repository base class. It didn't break cutover, but it was a landmine waiting for the day that server gets decommissioned. Grep your codebase for old hostnames before you celebrate.

Lesson 10: The Migration Isn't Done at Cutover

Green smoke tests on cutover day are the start of the hardening phase, not the end of the project. Our post-cutover backlog looked like:

• Availability tests against the public URLs (through Front Door, not just direct)
• Action groups and full alert rule coverage — an alert without a notification target is decoration
• A PITR fire drill: actually restore the database to a point in time and prove the runbook works
• Verifying the MI's public data endpoint is disabled — the kind of thing everyone assumes and nobody checks
• Defender for Cloud, geo-filtering at the WAF, log consolidation
• Documented upgrade path (zone-redundant compute, geo-replica for DR)

Takeaway: Write the "day 2" backlog before cutover, as a gap register with severity ratings. Otherwise cutover euphoria eats it.

The Meta-Lessons

If I compress the whole experience into four principles:

1. Distrust silence. Silent sink failures, silent cache staleness, silent config placeholders — the worst bugs made no noise. Build fallback observability everywhere.
2. Identity is the hardest layer. Compute and data moved predictably. Azure AD audiences, client IDs, SQL auth conversions, and login recreation caused more incidents than everything else combined.
3. Root-cause before you react. The load test scare, the 401 storm, the latency worry — every one of them looked like a crisis and turned out to be something narrower. The discipline of "break it down before you escalate" saved days.
4. Diagrams are living defenses against misunderstanding. The best architecture doc isn't the most complete one — it's the one that preempts the specific misreading your audience keeps making.

Migrating to the cloud isn't a technology problem. It's a hundred small correctness problems, each of which is easy in isolation and only dangerous because they arrive together. Track them in a register, kill them one at a time, and cutover day gets boring — which is exactly what you want.

Thanks for reading. If you're planning a similar migration and want to compare notes, I'm happy to talk.