Concept: Dataset Pipelines
Datasets contain 1 or more pipelines. A pipeline is a container of stages that models the flow
of how data should be transformed. It consists of an ordered list of stages, each of
which define how to modify the data at that point in the pipeline. Pipelines can contain
1 or more stages.
Base datasets contain 1 pipeline, and derived datasets can contain 1 or more pipelines.
Every dataset has a Primary Pipeline.
The first stage in a pipeline brings in data from the dataset source. Stages listed after
the first stage in a pipeline take the output of the previous pipeline as the input to
the current stage. If you're familiar with ETL workflows (extract, transform, and load),
each stage is 1 step in a development pipeline.
The last stage of any pipeline is the output for that pipeline. The output of the Primary
Pipeline is the output of the entire dataset. Therefore, when you publish a dataset, the
output of the Primary Pipeline will be materialized as the data in the Prism data
source.