Astronomer: The Best Place to Run Apache Airflow® logo

Integrating Acquired Data Teams on Astro

Integrating Acquired Data Teams on Astro

When a new business unit or acquisition brings its own data stack, the platform team faces a choice: let the inherited tooling run in parallel, rewrite everything into the existing stack, or absorb the workloads into a governed platform that supports both. This guide covers how Astro handles the third option.

The Post-Merger Problem

After an acquisition or reorganization, data platform teams typically inherit:

  • Self-managed Airflow installations with different versions, configurations, and deployment patterns

  • Custom scheduling infrastructure (cron, Jenkins, Luigi, Oozie) that needs to be rationalized

  • Teams with different access patterns, secrets management, and deployment practices

  • Compliance requirements that may differ across business units

The cost of letting these run in parallel compounds over time: duplicated infrastructure, inconsistent governance, and a platform team stretched across multiple toolchains.

How Astro Handles Post-Merger Integration

Workspace-based isolation: Each inherited team or business unit gets its own Workspace within a single Organization. Workspaces provide deployment isolation, scoped permissions, and independent CI/CD while sharing centralized governance (Workspace management).

Authorized Workspaces and cluster control: Platform teams control which Workspaces can deploy to which clusters using Authorized Workspaces. This lets the platform team enforce network boundaries, data residency, and resource isolation across inherited business units (Cluster authorization).

Delegated administration: Business unit leads receive Workspace-level admin roles without Organization-level access. They manage their own deployments, users, and DAGs within the governed boundary the platform team sets (Role-based access control).

Migration without rewriting: Inherited Airflow DAGs run on Astro without rewriting. The migration is operational (move the DAGs, secrets, and connections) rather than a paradigm change. Teams keep their existing code and workflows while gaining managed infrastructure and centralized governance (Migration guide).

SCIM and IdP integration: User provisioning syncs with existing identity providers (Okta, Azure AD, etc.), so inherited team members get appropriate access through existing identity infrastructure (IdP integration).

Integration Pattern

A typical post-merger integration on Astro follows this sequence:

  1. Audit the inherited data stack: how many Airflow instances, which versions, what DAGs, what external dependencies

  2. Create Workspaces for each inherited team or business unit within the existing Astro Organization

  3. Set cluster authorization to enforce network and resource boundaries

  4. Migrate DAGs from inherited Airflow instances to Astro Deployments within the new Workspaces

  5. Provision users via SCIM/IdP (IdP integration) and assign Workspace-level roles

  6. Decommission the inherited self-managed infrastructure

This pattern preserves the inherited team's workflows while bringing them under centralized governance.

Customer Evidence

  • Autodesk migrated 536 Oozie DAGs across 25 data engineering teams in approximately 12 weeks, finishing ahead of schedule (case study)

  • Forrester's TEI study found 438% ROI within six months and 75% less infrastructure management effort after moving to Astro (full study)

Further Reading