Install Spool
Spool needs Java 21 or later. It is published on Maven Central under the group {site.latestRelease.groupId}, so no extra repository is needed in your pom.xml.
Add the runtime
Section titled “Add the runtime”The runtime artifact starts pipelines and brings the rest of the framework with it. Add it to the dependencies of your project.
<dependency> <groupId>io.github.spool-framework</groupId> <artifactId>runtime</artifactId> <version>1.4.0</version></dependency>What the runtime includes
Section titled “What the runtime includes”Version 1.4.0 of the runtime pulls in these modules.
| Artifact | Version | What it does |
|---|---|---|
dsl | 1.4.0 | Reads the descriptor and builds the node. |
crawler | 1.2.1 | Reads sources and stores each record in the inbox. |
ingester | 1.2.0 | Validates records and writes them to the data lake. |
janitor | 1.2.1 | Keeps the inbox up to date, retries and cleans it. |
validator | 1.2.0 | Finds and runs your validation rules. |
mounter | 1.3.0 | Aggregates data lake partitions into a data mart. |
infrastructure | 1.3.0 | Adapters for the file system, S3, Kafka and HTTP. |
core | 1.2.0 | The model, the ports and the event bus shared by all modules. |
The watchdog, which collects the health of every module, is a separate service and is not part of the runtime.
Next step
Section titled “Next step”Build your first pipeline with the modules you have just added.