case studies / Debezium (Red Hat)

Host-Based Pipeline Deployment for the Debezium Platform

Extending change data capture to Linux hosts and cloud VMs through host discovery, concurrent scheduling, and Quarkus CDI selection.

role: Google Summer of Code 2026 Contributor timeline: May 2026 - Sep 2026 org: Debezium (Red Hat)

metrics & scope

10m 43s Pipeline Runtime
~500 MB Image Pre-pulled
7 Sub-issues Scoped Issues
24 Unit Tests SSH Parser Tests
6 REST Endpoints Daemon Endpoints

context & constraints

Debezium is an open-source Change Data Capture (CDC) platform. The project introduced a host-based deployment path so teams can operate pipelines on Linux hosts and virtual machines in addition to Kubernetes environments.

architectural constraint:

The host-based deployment path had to ship from the exact same compiled binary image as Kubernetes mode, requiring zero breaking configuration changes for existing enterprise users. Remote Ansible deployment tasks could not block conductor threads, and concurrent host provisioning required strict deadlock prevention.

decision log

1. Quarkus CDI Selection (@LookupIfProperty)

chosen design: Leveraged Quarkus @LookupIfProperty to select deployment orchestrators at build/runtime without polluting the core engine with conditional runtime branches. This enables a unified container image with dead-code elimination.

rejected alternative: Considered runtime reflection and factory patterns; rejected due to increased startup overhead and loss of GraalVM native image compatibility.

2. Zero-Dependency OpenSSH Config Parser

chosen design: Implemented a custom OpenSSH configuration parser backed by 24 unit tests, paired with a Java NIO WatchService, debounce, and scheduled reconciliation.

rejected alternative: Avoided a new SSH library dependency because OpenSSH and Ansible already handle connection setup for the supported host workflow.

3. Deadlock-Safe Scheduling with Sorted Locks

chosen design: Eliminated database deadlocks and port collisions in high-concurrency environments by implementing sorted PESSIMISTIC_WRITE locks combined with atomic file operations and Command-pattern Ansible runners.

rejected alternative: Considered optimistic locking with retry backoff; rejected because provisioning conflicts caused wasted compute and partial disk allocations.

4. Asynchronous Host Agent Daemon

chosen design: Architected a lightweight Quarkus daemon deployed via systemd exposing 6 REST endpoints with bearer-token authorization. Endpoints return 202 Accepted immediately so slow multi-gigabyte remote deployments never block central conductor threads.

rejected alternative: Considered direct synchronous SSH command execution from the orchestrator; rejected because long-lived SSH connections dropped under network jitter.

execution flow

  1. 01

    Edit ~/.ssh/config

    Engineer declares target host IP, port, identity file, and SSH parameters.

  2. 02

    Host Discovered

    Java NIO WatchService detects file modification with debounce and triggers reconciliation.

  3. 03

    Ansible Provisions

    Isolated thread pool triggers idempotent Ansible playbook, verifies systemd daemon, and pre-pulls ~500MB image.

  4. 04

    Host Marked READY

    Health checks pass, bearer tokens exchange, and conductor marks pipeline node READY in 10m 43s.

public evidence & code