The Control Plane

At the heart of Ascend's data pipeline automation is a DataAware™ control plane with intelligence to detect and propagate change across your ecosystem.

Data pipeline automation platform diagram with control plane for automated change management highlighted in red.

What Is The Control Plane?

An always-on engine that manages and propagates change for you.

This Is DataAware™

The control plane always knows with certainty the exact state of every node in every pipeline, at any scale of the network. This fully autonomous engine constantly detects changes in data and code across vast networks of the most complex data pipelines, and constantly responds to those changes in real time. The control plane keeps the data pipelines synchronized without any explicit orchestration code.

Fingerprint

As soon as any code or data is introduced to the system, the control plane automatically fingerprints it.

Detect

The fingerprints are then compared to pinpoint each change and the affected downstream data sets.

Propagate

The control plane senses changes in the fingerprints, and propagates them with 100% reliability to all downstream nodes.

Fingerprint

A powerful SHA (secure hashing algorithm) mechanism in the control plane creates a unique “fingerprint” for every set of data that arrives in the system. It also fingerprints every snippet of code that users provide to define the transformation steps in the data pipelines. The control plane links them into DAGs (directed acyclic graphs) to lock in their dependencies. This results in a lightweight immutable web, enriched with unique metadata to drive the autonomous orchestration processes.

Detect

The autonomous control plane compares all the SHA fingerprints to detect change as soon as it is introduced. For each change, the control plane follows the lineage in the code to identify the dependencies in the downstream transformations and linked pipelines. It then automatically generates a plan from these dependencies to propagate the change.

Propagate

The control plane propagates all changes autonomously through the pipelines end-to-end. It does this by sending discrete workloads for each transformation step to the connected data clouds where the data is stored. If any one operation breaks because of errors in the code, the platform pauses for a fix by the user. It then resumes from the point of breakage with the latest data sets, without reprocessing any workloads. DataAware™ intelligence guarantees integrity of the change from source to sink.

Learn More About DataAware™

Innovation anchored in foundational computer science.

The control plane implements a sophisticated, massively scalable computing paradigm. Based on proven computer science concepts, the technology uniquely guarantees reliability, recoverability, and integrity of large-scale data networks.