# `Forcola.Stream`
[🔗](https://github.com/joshrotenberg/forcola/blob/v0.3.2/lib/forcola/stream.ex#L1)

Line-by-line output from a shim-supervised process as an `Enumerable`.

For CLIs that emit NDJSON or line-oriented progress (agent CLIs,
`docker events`, `git clone --progress`). The child runs in its own
process group; halting the stream, timing out, or BEAM death all kill
the whole group.

    Forcola.Stream.lines(["claude", "-p", prompt], timeout_ms: 300_000)
    |> Stream.map(&:json.decode/1)
    |> Enum.to_list()

## Termination semantics

  * A zero exit ends the stream cleanly.
  * A non-zero exit, death by signal, timeout, or spawn failure raises
    `Forcola.Stream.Error` after every line produced before death has
    been emitted. The consumers this is built for are NDJSON CLIs where
    mid-stream death matters: silently dropping the exit reproduces the
    leak this library exists to close, and a tagged final element would
    force every `Stream.map(&decode/1)` pipeline to special-case it.
    Stderr captured during the run rides in the exception.
  * Halting the stream early (`Enum.take/2`, `Stream.take_while/2`, an
    exception downstream) kills the process group and blocks until the
    shim confirms the group is dead.
  * If the consuming process dies, the port closes, the shim sees stdin
    EOF, and the group is killed.

## Idle timeout

`:timeout_ms` bounds the whole run. `:idle_timeout_ms` (optional) bounds
the gap between output frames instead, detecting a stalled producer
without capping total runtime. The two are independent and composable:
whichever bound is reached first fires, and the raised
`Forcola.Stream.Error` marks which one (`:idle_timed_out` vs
`:timed_out`).

The idle deadline resets on any child liveness signal: every STDOUT or
STDERR data frame resets it, not only newline-terminated lines. A child
that writes bytes without a newline, or writes only to stderr, still
counts as alive. On idle expiry the same early-halt kill-and-confirm
path runs (KILL the group, wait for the shim to confirm death, close the
port) and `Forcola.Stream.Error` is raised with `idle_timed_out: true`
after any lines already produced have been emitted. Omitting the option
(the default) leaves behavior identical to a run bounded only by
`:timeout_ms`.

## Backpressure

By default the shim pumps the child's stdout forward as fast as the pipes
allow; unconsumed output buffers on the BEAM side, so a consumer slower than
the producer can grow BEAM memory without bound. `:window_bytes` (or
`backpressure: true`) opts into a demand-driven mode that bounds it.

With backpressure the shim reads the child's stdout only while it holds read
credit. The Stream grants one window of credit as its consumer pulls lines;
when the window is exhausted the shim stops reading, the OS pipe fills, and
the child's next write blocks. That block is the backpressure: the producer
runs no faster than the consumer drains it. The buffering bound is roughly
`window_bytes` in flight plus one frame in transit (plus a bounded
pipe-buffer flush when the child exits). Backpressure gates the child's
stdout only; stderr is never gated, and under `merge_stderr: true` the
merged stderr bytes ride eagerly.

Interaction with `:idle_timeout_ms`: a gap between frames can be caused by
the consumer not yet granting credit rather than a stalled producer. A
consumer-driven stall does not trip the idle timeout: the idle clock is reset
each time the Stream grants credit, so time spent waiting on the consumer is
excluded. A genuinely hung producer (credit available, no output) still trips
it, and the whole-run `:timeout_ms` still bounds a consumer that never
consumes.

# `lines`

```elixir
@spec lines(
  [String.t(), ...],
  keyword()
) :: Enumerable.t()
```

Run `argv` and return its stdout as a lazy stream of lines.

Takes the same options as `Forcola.run/2`, including `:user`/`:group`
for running the child as a different user (POSIX-only, fail-closed,
privileged shim required); `:timeout_ms` is required and bounds the
whole run, not the gap between lines.

`:idle_timeout_ms` (optional, milliseconds; default `nil` = disabled)
bounds the gap between output frames: if no STDOUT or STDERR data
arrives within the interval the producer is treated as stalled, the
process group is killed, and `Forcola.Stream.Error` is raised with
`idle_timed_out: true`. It is independent of and composable with
`:timeout_ms`; whichever bound fires first wins. See the module docs for
the exact reset semantic.

`:window_bytes` (optional, positive integer) opts into demand-driven
backpressure: the shim reads the child's stdout only while it holds read
credit, and the Stream grants roughly one window as its consumer pulls
lines, so a slow consumer blocks the producer instead of buffering its
output on the BEAM. `backpressure: true` is shorthand for a default
65536-byte window. Omitting both keeps the default eager
pump. See the module docs for the buffering bound and the interaction with
`:idle_timeout_ms`.

Lines are emitted without their trailing newline. A partial line held
across frame boundaries is emitted once its newline arrives; a final
partial line with no newline is emitted before the stream terminates.

See the module docs for how termination is surfaced.

---

*Consult [api-reference.md](api-reference.md) for complete listing*