Chapter 2
Architecture and Components
Beneath Concourse CI's streamlined surface lies a rigorously engineered architecture designed for resilience, modularity, and performance at scale. Embark on a tour of the moving parts-each one purpose-built to deliver predictable automation, secure isolation, and dynamic extensibility. This chapter reveals how Concourse orchestrates containers, secrets, and pipelines into an adaptive CI/CD powerhouse.
2.1 ATC, TSA, and Worker Architecture
Concourse's architecture is founded upon three principal components: the ATC, the TSA, and the workers. Each serves a distinct yet interconnected role that enables scalable, reliable, and efficient continuous integration and delivery pipelines. Understanding these components' responsibilities, interaction models, fault domains, and resilience strategies is essential for designing and operating a robust Concourse deployment.
ATC (Air Traffic Controller) functions as the orchestration brain of Concourse. It exposes the web UI and API endpoints, manages the pipeline state, schedules builds, and coordinates resources and tasks. The ATC maintains the desired system state within a backing data store (typically PostgreSQL), continuously reconciling declared pipeline configurations with actual resource and job statuses. It tracks resource versions, handles triggers for jobs, and stores metadata about build artifacts. Internally, the ATC employs a scheduler that transforms pipeline definitions into a directed acyclic graph (DAG) of jobs and tasks, resolving dependencies and ensuring orderly execution within the defined pipeline semantics.
The ATC is implemented as a stateless service, allowing it to be horizontally scaled behind a load balancer for high availability. However, its reliance on a centralized database creates a single fault domain for pipeline state persistence. It is appropriate to protect the PostgreSQL instance with robust backup and failover mechanisms to avoid systemic failures.
The TSA (TSA is the SSH Gateway) operates as a secure tunneling layer, managing persistent SSH connections that enable worker registration and bidirectional communication between workers and the ATC. When a worker starts, it establishes an outgoing SSH connection to the TSA, which authenticates it and creates a multiplexed communication channel. This design choice offloads worker connectivity complexities from the ATC, centralizes authentication, and facilitates NAT traversal: since workers initiate connections outbound, they easily operate behind firewalls and dynamic IP addresses.
TSA's operation principle hinges on persistent connections that remain active irrespective of worker job states, enabling the ATC to issue commands or resource fetch requests asynchronously. The TSA is also stateless and can be scaled horizontally to accommodate large clusters by load balancing incoming connections. Its primary fault domain concerns availability; any outage or saturation of the TSA impacts the ability of workers to communicate with the ATC, stalling task orchestration.
Workers serve as the task executors, running the actual build and test jobs defined in pipelines. Each worker is a self-contained environment capable of provisioning containers, managing volumes, and executing tasks in isolation. Workers register themselves with the ATC via the TSA tunnel, reporting capabilities such as available resource types, container runtime versions, and platform details. Based on pipeline requirements and queue state, the ATC schedules jobs to specific workers, which then instantiate containers to perform resource fetching, task execution, or artifact uploading.
Workers provide fault isolation at the compute level. Containerized task execution ensures failures or state pollution do not propagate beyond the local environment. If a worker crashes or becomes unreachable, the ATC detects heartbeat timeouts (via TSA connection monitoring) and reschedules pending tasks onto other healthy workers. This mitigates fault domains by localizing transient failures and preserving pipeline continuity.
Communication patterns among these components form a triadic topology. The ATC initiates scheduling decisions and exposes APIs, but all direct worker interaction happens through the TSA tunnel. Workers maintain a persistent SSH session to the TSA, which multiplexes control messages, logs, and artifact streams bidirectionally. This structure allows for asynchronous, event-driven orchestration where the ATC can enqueue jobs, receive streaming logs, and monitor task states through reliable transport provided by the TSA.
Robustness in this architecture derives from explicit separation of concerns, fault domain partitioning, and scalable stateless intermediaries. Key strategies include:
- Stateless ATC deployment: Employing multiple ATC instances behind a load balancer prevents single points of failure in orchestration logic. Database high availability complements this approach.
- Horizontal scaling of TSA: Distributing TSA instances to handle large volumes of SSH tunnels reduces performance bottlenecks and enhances fault tolerance.
- Worker isolation: Each worker runs containers in isolation, preventing cross-contamination and enabling independent failure recovery.
- Health monitoring and failover: The ATC monitors worker liveness through TSA connections. Disconnects trigger automatic rerouting of jobs, minimizing impact of node failures.
- Secure communication: SSH tunnels encapsulated by the TSA ensure integrity and confidentiality. Authentication between workers and TSA uses key-based methods, establishing trust boundaries.
By leveraging these principles, Concourse achieves effective orchestration of complex CI/CD pipelines while maintaining scalability and resilience across distributed system fault domains. The triad of ATC, TSA, and workers provides a modular framework capable of operational robustness in dynamic cloud or on-premise environments.
2.2 Containerization and Execution Environments
Concourse employs containerization as a foundational principle to establish hermetic build environments, ensuring consistent task execution across diverse infrastructure and temporal contexts. This approach encapsulates each build step within an isolated container, thereby addressing challenges related to environment drift, dependency conflicts, and security vulnerabilities intrinsic to traditional build systems.
Central to Concourse's strategy are its support for two primary container runtimes: Garden and Docker. Garden is a container runtime developed by Pivotal specifically for Concourse and Cloud Foundry, designed to provide a lightweight, efficient, and highly composable environment. Docker support extends Concourse's compatibility across the ubiquitous container ecosystem, leveraging Docker images for build tasks and resource containers. Both runtimes facilitate ephemeral execution environments-containers instantiated at job runtime and disposed immediately upon completion-thereby ensuring no residual state persists between builds.
The ephemeral nature of these execution environments directly contributes to hermeticity: builds are insulated from external influences and previous runs, reinforcing reproducibility. By discarding containers and associated file systems after each task, Concourse guarantees that no side effects propagate, enabling deterministic outcomes. This is critically important for continuous integration workflows that rely on identical conditions, making regression diagnostics more reliable and build failures more explicable.
Isolation achieved through containerization manifests in multiple dimensions. System-level isolation ensures resource segregation-such as CPU, memory, and network boundaries-mitigating resource contention and enabling precise resource allocation and throttling. Namespace isolation protects against interference between concurrent builds, while user and filesystem permission boundaries enhance security postures by restricting arbitrary access to host resources. These containment guarantees help thwart vectors that malicious or compromised build scripts might exploit, elevating the overall security of CI/CD pipelines.
In practice, Garden containers are implemented using Linux kernel features like namespaces and cgroups, optimized for high-density task execution. Garden's design emphasizes fast container startup times and minimal overhead, critical for large-scale pipelines with numerous parallel tasks. Docker containers, conversely, benefit from a rich ecosystem of pre-built images and flexible tooling, allowing teams to leverage existing artifacts and streamline onboarding. Integration with Docker registries further facilitates image management and version control.
Advanced containerization strategies in Concourse cater to workloads with varied complexity and requirements. For example, task containers may be augmented with volume mounts that inject source code or cached dependencies, accelerating build throughput while preserving hermetic principles-since the volumes are ephemeral and scoped to the task life cycle. Moreover, Concourse supports network...
