# `Blink`
[🔗](https://github.com/nerds-and-company/blink/blob/v0.10.0/lib/blink.ex#L1)

Blink provides efficient database seeding with a clean, declarative syntax.

## Example

    defmodule MyApp.Seeder do
      use Blink

      def call do
        new()
        |> with_table("users")
        |> run(MyApp.Repo)
      end

      @impl true
      def table(_seeder, "users") do
        [
          %{id: 1, name: "Alice", email: "alice@example.com"},
          %{id: 2, name: "Bob", email: "bob@example.com"}
        ]
      end
    end

## Overview

Blink simplifies database seeding by providing a structured way to build and
insert rows:

1. Create an empty `Seeder` with `new/0`.
2. Declare which tables to seed with `with_table/2`.
3. Define `table/2` clauses that return the rows to insert.
4. Run `run/2` or `run/3` to bulk-insert the rows.

## The seeder API

`use Blink` defines these functions on your module. They are the primary API,
but they do not appear in this module's function list below, because they are
defined on *your* module rather than on `Blink`:

  * `with_table(seeder, table_name)` and `with_table(seeder, table_name, opts)` —
    declare a table; the rows come from your `table/2` clause for that name.
    A list of names declares each in order.
  * `with_context(seeder, key)` — declare a context key; the value comes from
    your `context/2` clause for that key.

`use Blink` also imports `new/0` and, from this module, `put_table/2,3,4`,
`put_context/2,3`, `fetch_row!/3`, `to_row/1,2`, `to_rows/1,2`,
`copy_to_table/3,4`, `from_csv/1,2` and `from_json/1,2`, so you call all of
them unqualified.

### Choosing between with_* and put_*

Reach for `with_table/2` and `with_context/2` by default. Because the callback
runs when the table is declared, it receives the seeder built so far and can
read earlier tables and context off it — that is what makes the pipeline
declarative and lets `"posts"` derive from `"users"`.

Use `put_table/3` and `put_context/3` only when the data is already in hand at
the call site and no callback is needed:

    # Callback form: rows are computed per table, in declaration order
    new()
    |> with_table("users")
    |> with_table("posts")
    |> run(MyApp.Repo)

    # Direct form: rows already exist
    new()
    |> put_table("users", users)
    |> run(MyApp.Repo)

The two forms compose freely — a later `table/2` clause reads rows that
`put_table/3` added earlier.

### Choosing IDs

You assign primary keys yourself. Blink builds plain maps and hands them to
PostgreSQL's `COPY`, so it never asks the database to generate an ID and never
reads one back. An ID is just another value in the map you are building, no
different from a name or a timestamp — inserting a row is not what gives it
one. So a later table can reference rows that have not been inserted yet:

    def table(_seeder, "users") do
      [%{id: 1, name: "Alice"}, %{id: 2, name: "Bob"}]
    end

    def table(seeder, "posts") do
      Enum.map(seeder.tables["users"], fn user ->
        %{id: user.id, title: "Welcome, #{user.name}", user_id: user.id}
      end)
    end

Foreign keys are satisfied by insertion order, which follows the order tables
were declared, so declare parents before children.

> #### Reset the sequence for serial columns {: .warning}
>
> Inserting explicit IDs does not advance a `serial`, `bigserial`, or identity
> sequence, so the next ordinary insert your application makes can collide with
> a seeded row. Pass `reset_sequences: true` to `run/3` to advance the
> sequences after seeding; see
> [Getting Started](getting_started.html#choosing-ids).

## Seeders

Seeders are the central data unit in Blink. A `Seeder` is a struct that holds
the rows you want to seed, any contextual data you need during the seeding
process, and internal state that Blink uses to execute the bulk insert.

    %Blink.Seeder{
      tables: %{
        "table_name" => [...]
      },
      context: %{
        "key" => [...]
      },
      table_order: ...,
      table_opts: ...
    }

All keys in `tables` must match the name of a table in your database. Table
names can be either atoms or strings.

### Tables

A mapping of table names to lists of rows. These rows will be persisted to the
database when `run/2` or `run/3` is called.

### Context

Stores arbitrary data needed during the seeding process. This data is
available when building your seeds but is not inserted into the database by
`run/2` or `run/3`. Use `with_context/2` to declare context keys and define
corresponding `context/2` clauses.

## Custom Logic for Running the Seeder

By default, `run/2` and `run/3` bulk insert rows from the seeder into the
tables of a Postgres database. Internally they use Postgres' `COPY` command.

There are two ways to customize the insert behavior:

- Override the default implementation of `run/2` or `run/3`
- Pass a custom adapter to `run/3` (e.g., for non-Postgres databases)

# `context`
*optional* 

```elixir
@callback context(seeder :: Blink.Seeder.t(), key :: Blink.Seeder.key()) :: Enumerable.t()
```

Builds and returns the data to be stored under a context key in the given
`Seeder`.

Called internally by `with_context/2`. Each key passed to `with_context` must
have a corresponding `context/2` clause.

`run/2` and `run/3` ignore context data and only insert data from `:tables`.

When the callback function is missing, an `ArgumentError` is raised.

# `run`
*optional* 

```elixir
@callback run(seeder :: Blink.Seeder.t(), repo :: Ecto.Repo.t()) :: :ok
```

Specifies how to run the Seeder, performing a bulk insert of the seed data
from a `Seeder` into the given Ecto repository.

This callback function is optional, since Blink ships with a default
implementation.

# `run`
*optional* 

```elixir
@callback run(seeder :: Blink.Seeder.t(), repo :: Ecto.Repo.t(), opts :: Keyword.t()) ::
  :ok
```

# `table`
*optional* 

```elixir
@callback table(seeder :: Blink.Seeder.t(), table_name :: Blink.Seeder.key()) ::
  Enumerable.t()
```

Builds and returns the rows to be stored under a table key in the given
`Seeder`.

Called internally by `with_table/2` and `with_table/3`. Each table name passed
to `with_table` must have a corresponding `table/2` clause.

Data added to a Seeder with `table/2` is inserted into the corresponding
database table when calling `run/2` or `run/3`.

The callback can return either a list or a stream of maps. Returning a stream
enables memory-efficient seeding of large datasets.

When the callback function is missing, an `ArgumentError` is raised.

# `copy_to_table`

```elixir
@spec copy_to_table(
  rows :: Enumerable.t(),
  table_name :: String.t(),
  repo :: Ecto.Repo.t(),
  opts :: Keyword.t()
) :: :ok
```

Copies rows into a database table using database-specific bulk copy commands.

## Parameters

  * `rows` - An enumerable (list or stream) of maps where each map represents
    a row to insert. All maps must have the same keys, which correspond to the
    table columns. Using a stream allows for memory-efficient seeding of large
    datasets.
  * `table_name` - The name of the table to insert into (string or atom).
  * `repo` - An Ecto repository module.
  * `opts` - Keyword list of options:
    * `:adapter` - The adapter module to use. Defaults to
      `Blink.Adapter.Postgres`.

    All other options are adapter-specific and validated by the adapter —
    unknown keys raise `ArgumentError`. See `Blink.Adapter.Postgres` for its
    options: `:atomic` (all-or-nothing copy), `:concurrency`, `:batch_size`,
    `:timeout`, `:reset_sequences`, and `:truncate` (replace the table's
    contents instead of adding to them).

## Returns

  * `:ok` - When the copy operation succeeds

Raises an exception when the copy operation fails.

## Examples

    iex> rows = [%{id: 1, name: "Alice"}, %{id: 2, name: "Bob"}]
    iex> copy_to_table(rows, "users", MyApp.Repo)
    :ok

    # Using a stream for memory-efficient seeding
    iex> stream = Stream.map(1..1_000_000, fn i -> %{id: i, name: "User #{i}"} end)
    iex> copy_to_table(stream, "users", MyApp.Repo)
    :ok

## Notes

Column names are extracted from the first row in the enumerable, and every
row must have the same keys — a mismatch raises `Blink.RowError`.

Currently only PostgreSQL is supported via `Blink.Adapter.Postgres`.

# `fetch_row!`

```elixir
@spec fetch_row!(
  seeder :: Blink.Seeder.t(),
  table_name :: Blink.Seeder.key(),
  clauses :: Keyword.t()
) ::
  map()
```

Fetches the first row in `table_name` whose fields match all of `clauses`,
raising if none does.

A safe replacement for `Enum.find/2` over `seeder.tables[...]`: a miss
raises `ArgumentError` naming the table and clauses, instead of returning
`nil` that crashes later, far from the cause. The table must already be
declared; atom and string table names are interchangeable, as in the rest of
the seeder API.

Use this only with tables whose rows are lists — the lookup enumerates the
table, so it would consume a table built as a single-use stream.

## Examples

    def table(seeder, "posts") do
      alice = fetch_row!(seeder, "users", email: "alice@example.com")

      [%{id: 1, title: "Welcome", user_id: alice.id}]
    end

# `from_csv`

```elixir
@spec from_csv(path :: String.t(), opts :: Keyword.t()) :: Enumerable.t()
```

Reads a CSV file and returns a list or stream of maps.

Each column header becomes a string key in the resulting maps. All values are
returned as strings.

## Parameters

  * `path` - Path to the CSV file (relative or absolute)
  * `opts` - Keyword list of options. Unknown options raise `ArgumentError`:
    * `:headers` - `:infer` to read the header names from the first row
      (default), or a list of names for a file **without** a header row.
      Explicit headers do not skip the first row — on a file that has one,
      the header row comes back as a data map.
    * `:transform` - Function to transform each row map (default: identity)
    * `:stream` - When `true`, returns a stream instead of a list (default:
      `false`)

## Examples

    # Read CSV with headers in first row
    from_csv("users.csv")

    # Name the columns of a file that has no header row
    from_csv("users_no_headers.csv", headers: ["id", "name", "email"])

    # Transform values
    from_csv("users.csv", transform: fn row ->
      Map.update!(row, "id", &String.to_integer/1)
    end)

    # Stream for memory-efficient processing
    from_csv("large_users.csv", stream: true)

## Returns

A list of maps, or a stream of maps when `stream: true`.

## Notes

For JSONB columns, prefer leaving the value as the raw JSON string read from
the CSV. Since CSV values are already strings, an untransformed JSONB column is
inserted directly, skipping a JSON round trip. Only decode it into a map (via
`:transform`) when you need to inspect or modify the value before inserting —
the Postgres adapter will re-encode maps with `Jason.encode!/1` on the way in.

# `from_json`

```elixir
@spec from_json(path :: String.t(), opts :: Keyword.t()) :: [map()]
```

Reads a JSON file and returns a list of maps.

The JSON file must contain an array of objects at the root level. Each object
becomes a map with string keys.

## Parameters

  * `path` - Path to the JSON file
  * `opts` - Keyword list of options. Unknown options raise `ArgumentError`:
    * `:transform` - Function to transform each row map (default: identity)

## Examples

    # Read JSON file
    from_json("users.json")

    # Transform values
    from_json("users.json", transform: fn row ->
      Map.update!(row, "id", &String.to_integer/1)
    end)

## Returns

A list of maps.

# `put_context`

```elixir
@spec put_context(seeder :: Blink.Seeder.t(), pairs :: [{Blink.Seeder.key(), any()}]) ::
  Blink.Seeder.t()
```

Adds several `{key, value}` pairs to the seeder's context at once.

A multi-key form of `put_context/3`; pairs are applied in order and each key
must be unique (as with `put_context/3`).

## Examples

    new()
    |> put_context(user_id: user_id, project_indices: project_indices)

# `put_context`

```elixir
@spec put_context(
  seeder :: Blink.Seeder.t(),
  key :: Blink.Seeder.key(),
  value :: any()
) ::
  Blink.Seeder.t()
```

Adds `value` to the seeder's context under `key`.

A convenience wrapper over `Blink.Seeder.with_context/3` for when the context
data is already available and you do not want to define a `context/2` callback.
Raises `ArgumentError` if `key` is already present.

## Examples

    new()
    |> put_context(:generated_at, ~U[2024-01-01 00:00:00Z])

# `put_table`

```elixir
@spec put_table(
  seeder :: Blink.Seeder.t(),
  pairs :: [{Blink.Seeder.key(), Enumerable.t()}]
) ::
  Blink.Seeder.t()
```

Adds several `{table_name, rows}` pairs to the seeder at once.

A multi-table form of `put_table/3`; tables are added in order (which becomes
their insertion order) and each name must be unique. Per-table options are not
supported here — use `put_table/4` when you need them.

## Examples

    new()
    |> put_table(users: users, posts: posts)

# `put_table`

```elixir
@spec put_table(
  seeder :: Blink.Seeder.t(),
  table_name :: Blink.Seeder.key(),
  rows :: Enumerable.t(),
  opts :: Keyword.t()
) :: Blink.Seeder.t()
```

Adds `rows` to the seeder under `table_name`.

A convenience wrapper over `Blink.Seeder.with_table/4` for when the rows are
already available and you do not want to define a `table/2` callback. `rows`
may be a list or a stream. `opts` takes per-table options (for
`Blink.Adapter.Postgres`: `:batch_size`, `:concurrency`, and
`:reset_sequences`), forwarded to
`Blink.Seeder.with_table/4`. Raises `ArgumentError` if `table_name` is
already present.

## Examples

    new()
    |> put_table("users", [%{id: 1, name: "Alice"}])
    |> put_table("events", [%{id: 1, name: "Launch"}], batch_size: 1_000)

# `to_row`

```elixir
@spec to_row(
  struct(),
  opts :: Keyword.t()
) :: map()
```

Converts an Ecto schema struct into a row map for copying.

The row keeps only the schema's persisted fields — `__meta__`, associations,
and virtual fields are dropped, so a stray key cannot become a column in the
COPY statement. Values are untouched: `Ecto.Enum` atoms, calendar structs,
and maps are encoded by the adapter (see the notes on
`Blink.Adapter.Postgres.call/4`).

For a struct factory shared with the test suite this replaces a hand-rolled
`Map.from_struct/1` + `Map.take/2` pass; see the
[ExMachina guide](integrating_with_ex_machina.html). Factories that exist
for seeding alone should return plain maps instead, making conversion
unnecessary — see [Building Rows](building_rows.html).

The `:id` option is the primary-key policy:

  * `:database` (default) — the primary-key fields are dropped, so their
    columns are omitted from the COPY and the database assigns them from
    the sequence. No sequence reset is needed, but the row has no stable
    id for later tables to reference.
  * `:keep` — the struct's primary-key values are kept as they are, for
    factories that assign their own ids (`Ecto.UUID.generate/0` and the
    like).
  * any other value — set as the row's primary key, which keeps the row
    referenceable by later tables; pair this with `reset_sequences: true`
    so the application's next insert clears the seeded ids. Raises
    `ArgumentError` for a schema with a composite primary key.

Raises `ArgumentError` if `struct` is not an Ecto schema struct.

## Examples

    def table(_seeder, "products") do
      for id <- 1..200, do: to_row(build(:product), id: id)
    end

# `to_rows`

```elixir
@spec to_rows([struct()], opts :: Keyword.t()) :: [map()]
```

Converts a list of Ecto schema structs into row maps for copying.

Each struct is converted as by `to_row/2`, with the `:id` policy applying
to every row — `:database` (the default) or `:keep`; a literal id would
give every row the same primary key, so it raises `ArgumentError` here.

The `:drop_nil_columns` option (default: `false`) drops every column whose
value is `nil` in all rows, mirroring how `Repo.insert/2` omits unset
fields so database defaults apply — a struct materializes every schema
field, mostly as `nil`, and COPY would otherwise send explicit `NULL`s.
The decision is per table rather than per row because every row must have
the same keys (`Blink.RowError`).

## Examples

    def table(_seeder, "products") do
      to_rows(build_list(200, :product), drop_nil_columns: true)
    end

---

*Consult [api-reference.md](api-reference.md) for complete listing*