Skip to content

Install Spool

Spool needs Java 21 or later. It is published on Maven Central under the group {site.latestRelease.groupId}, so no extra repository is needed in your pom.xml.

The runtime artifact starts pipelines and brings the rest of the framework with it. Add it to the dependencies of your project.

pom.xml
<dependency>
<groupId>io.github.spool-framework</groupId>
<artifactId>runtime</artifactId>
<version>1.4.0</version>
</dependency>

Version 1.4.0 of the runtime pulls in these modules.

ArtifactVersionWhat it does
dsl1.4.0Reads the descriptor and builds the node.
crawler1.2.1Reads sources and stores each record in the inbox.
ingester1.2.0Validates records and writes them to the data lake.
janitor1.2.1Keeps the inbox up to date, retries and cleans it.
validator1.2.0Finds and runs your validation rules.
mounter1.3.0Aggregates data lake partitions into a data mart.
infrastructure1.3.0Adapters for the file system, S3, Kafka and HTTP.
core1.2.0The model, the ports and the event bus shared by all modules.

The watchdog, which collects the health of every module, is a separate service and is not part of the runtime.

Build your first pipeline with the modules you have just added.