# `Choreo.Lab.DSL.Dataflow`
[🔗](https://github.com/code-shoily/choreo/blob/v0.14.1/lib/choreo/lab/dsl/dataflow.ex#L1)

Experimental Livebook-friendly DSL for sketching dataflow pipelines.

This Lab DSL compiles to the stable, pipe-first `Choreo.Dataflow` builders and
returns an ordinary `%Choreo.Dataflow{}`. It uses node constructors for dataflow
stages and typed edge vocabulary for normal, error, retry, and dead-letter paths.

## Examples

    iex> import Choreo.Lab.DSL.Dataflow
    ...> pipeline = dataflow do
    ...>   ingest = source("Kafka Ingest", rate: "10k/s")
    ...>   parser = transform("JSON Parser", latency_ms: 5)
    ...>   valid = conditional("Valid?")
    ...>   postgres = sink("Postgres")
    ...>   dlq = sink("Dead Letter Queue")
    ...>
    ...>   ingest ~> parser |> emits("raw events")
    ...>   parser ~> valid |> emits("parsed events")
    ...>   valid ~> postgres |> writes("valid records")
    ...>   dead_letter valid ~> dlq, "invalid records"
    ...> end
    iex> pipeline.graph.nodes[:ingest].node_type
    :source
    iex> pipeline.edge_meta[{:valid, :dlq}].path_type
    :dead_letter

Edge labels can use generic labels, typed data labels, or explicit path types:

    source ~> transform |> on("events")
    edge source ~> transform, data_type: "events"
    emits source ~> transform, "events"
    error transform ~> sink, "invalid input"
    retry transform ~> buffer, "retry later"
    dead_letter transform ~> dlq, "poison message"

# `cluster_decl`

```elixir
@type cluster_decl() :: %{id: String.t(), opts: keyword()}
```

# `edge_decl`

```elixir
@type edge_decl() :: %{from: Yog.node_id(), to: Yog.node_id(), opts: keyword()}
```

# `node_decl`

```elixir
@type node_decl() :: %{id: Yog.node_id(), builder: atom(), opts: keyword()}
```

# `buffer`

# `cluster`

# `conditional`

# `consumer`

# `consumes`

# `data`

# `dataflow`
*macro* 

Builds a `%Choreo.Dataflow{}` from a compact Lab DSL block.

# `dataflow`
*macro* 

# `dead_letter`

# `decision`

# `dlq`

# `edge`

# `edge_type`

# `emits`

# `error`

# `flow`

# `flows`

# `input`

# `join`

# `label`

# `lane`

# `merge`

# `normal`

# `on`

# `output`

# `process`

# `processor`

# `producer`

# `publishes`

# `queue`

# `rate`

# `reads`

# `retry`

# `routes`

# `sends`

# `sink`

# `source`

# `split`

# `stage`

# `taxonomy`

```elixir
@spec taxonomy() :: %{
  clusters: [atom()],
  nodes: [atom()],
  edges: [atom()],
  modifiers: [atom()],
  options: [atom()]
}
```

Returns the vocabulary supported by the dataflow DSL.

This is meant as a lightweight Livebook discovery helper when autocomplete is
not enough.

    iex> taxonomy = Choreo.Lab.DSL.Dataflow.taxonomy()
    iex> :source in taxonomy.nodes
    true
    iex> :emits in taxonomy.edges
    true
    iex> :dead_letter in taxonomy.edges
    true

# `topic`

# `transform`

# `type`

# `verbs`

```elixir
@spec verbs() :: map()
```

Compatibility alias for `taxonomy/0`.

# `weight`

# `writes`

---

*Consult [api-reference.md](api-reference.md) for complete listing*