Track notebooks, scripts & workflows ¶

This guide walks from tracking data lineage in a notebook to tracking parameters in workflows.

To run examples, if you don’t have a lamindb instance, create one:

lamin init --storage ./test-track

Track agentic workflows¶

Sessions¶

The lamindb skill ships with the lamindb package at .agents/skills/. Ask your coding agent to copy it to wherever it reads skills from — .claude/skills/ for Claude Code, .agents/skills/ for GitHub Copilot — so that it automatically tracks agent sessions.

When the agent finishes a session with lamin finish, usage metrics are recorded in run.extra_data:

n_tokens: total tokens for the session
n_steps: number of LLM completions/turns
n_tool_calls: number of tool invocations

Plans¶

You can save an agent plan like this:

lamin save /path/to/.cursor/plans/my_task.plan.md
lamin save /path/to/.claude/plans/my_task.md

Manage notebooks and scripts¶

Call track() to save your notebook or script as a transform and start tracking inputs & outputs of a run.

import lamindb as ln

ln.track()  # initiate a tracked notebook/script run

# your code automatically tracks inputs & outputs

ln.finish()  # mark run as finished, save execution report, source code & environment

You find your notebooks and scripts in the Transform registry along with pipelines & functions:

transform = ln.Transform.get(key="my_analyses/my_notebook.ipynb")
transform.source_code             # source code
transform.runs.to_dataframe()     # all runs in a dataframe
transform.latest_run.report       # report of latest run
transform.latest_run.environment  # environment of latest run

You can use the CLI to load a transform into your current (development) directory:

lamin load --key my_analyses/my_notebook.ipynb

Here is how you’d load the notebook from the video into your local directory:

lamin load https://lamin.ai/laminlabs/lamindata/transform/F4L3oC6QsZvQ

Organize local development¶

If no development directory is set, script & notebook keys equal their filenames. Otherwise, they represent the relative path in the development directory. The exception is packaged source code, whose keys have the form pypackages/{package_name}/path/to/file.py.

To set the development directory to your current shell development directory, run:

lamin settings set dev-dir .

You can see the current status by running:

lamin info

When you cd into that directory, you will now auto-connect to the configured lamindb instance.

Sync code with git¶

To sync scripts or workflow definitions with their correponding files in a git repo, either export an environment variable:

export LAMINDB_SYNC_GIT_REPO = <YOUR-GIT-REPO-URL>

Or set the following setting:

ln.settings.sync_git_repo = <YOUR-GIT-REPO-URL>

If you work on a single project in your lamindb instance, it makes sense to set LaminDB’s dev-dir to the root of the local git repo clone.

dbs/
  project1/
    .git/
    .lamin/
    script1.py
    notebook1.ipynb
  ...

If you work on multiple projects in your lamindb instance, you can use the dev-dir as the local root and nest git repositories in it.

dbs/
  database1/
    .lamin/
    repo1/
      .git/
    repo2/
      .git/
  ...

Use projects¶

You can link the entities created during a run to a project.

import lamindb as ln

my_project = ln.Project(name="My project").save()  # create & save a project
ln.track(project="My project")  # pass project
open("sample.fasta", "w").write(">seq1\nACGT\n")  # create a dataset
ln.Artifact("sample.fasta", key="sample.fasta").save()  # auto-labeled by project

Filter entities by project, e.g., artifacts:

ln.Artifact.filter(projects=my_project).to_dataframe()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	...	is_latest	is_locked	created_at	branch_id	created_on_id	space_id	storage_id	run_id	schema_id	created_by_id
id
1	YnyJT5GYjLfcAaeZ0000	sample.fasta	None	.fasta	None	None	11	83rEPcAoBHmYiIuyBYrFKg	None	None	...	True	False	2026-08-14 13:23:15.436000+00:00	1	1	1	1	1	None	1

1 rows × 22 columns

Access entities linked to a project:

my_project.artifacts.to_dataframe()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	...	is_latest	is_locked	created_at	branch_id	created_on_id	space_id	storage_id	run_id	schema_id	created_by_id
id
1	YnyJT5GYjLfcAaeZ0000	sample.fasta	None	.fasta	None	None	11	83rEPcAoBHmYiIuyBYrFKg	None	None	...	True	False	2026-08-14 13:23:15.436000+00:00	1	1	1	1	1	None	1

1 rows × 22 columns

The same works for my_project.transforms or my_project.runs.

Use spaces¶

You can write the entities created during a run into a space that you configure on LaminHub. This is particularly useful if you want to restrict access to a space. Note that this doesn’t affect bionty entities who should typically be commonly accessible.

ln.track(space="Our team space")

Manage workflows¶

Here we’ll manage workflows with lamindb’s flow() and step() decorators, which works out-of-the-box with the majority of Python workflow managers:

tool	workflow decorator	step/task decorator	notes
`lamindb`	`@flow`	`@step`	inspired by `prefect`
`prefect`	`@flow`	`@task`	two decorators
`redun`	`@task` (on main)	`@task`	single decorator for everything
`dagster`	`@job` or `@asset`	`@op` or `@asset`	asset-centric; `@asset` is primary
`flyte`	`@workflow`	`@task`	also `@dynamic` for runtime DAGs
`airflow`	`@dag`	`@task`	TaskFlow API (modern); also supports operators
`zenml`	`@pipeline`	`@step`	inspired by `prefect`

If you’re looking for more in-depth examples or for integrating with non-decorator-based workflow managers such as Nextflow or Snakemake, see Manage computational pipelines.

tool	workflow	step/task	notes
`nextflow`	`workflow` keyword	`process` keyword	groovy-based DSL
`snakemake`	`rule` keyword	`rule` keyword	file-based DSL
`metaflow`	`FlowSpec`	`@step`	class-based
`kedro`	`Pipeline()`	`node()`	function-based

A one-step workflow¶

Decorate a function with flow() to track it as a workflow:

my_workflow.py¶

import lamindb as ln


@ln.flow()
def ingest_dataset(key: str) -> ln.Artifact:
    df = ln.examples.datasets.mini_immuno.get_dataset1()
    artifact = ln.Artifact.from_dataframe(df, key=key).save()
    return artifact


if __name__ == "__main__":
    ingest_dataset(key="my_analysis/dataset.parquet")

Let’s run the workflow:

python scripts/my_workflow.py

Query the workflow via its filename:

transform = ln.Transform.get(key="my_workflow.py")
transform.describe()

The run stored the parameter value for key:

transform.latest_run.describe()

It links output artifacts:

transform.latest_run.output_artifacts.to_dataframe()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	...	is_latest	is_locked	created_at	branch_id	created_on_id	space_id	storage_id	run_id	schema_id	created_by_id
id
3	3QtKAZNPIpanK84S0000	my_analysis/dataset.parquet	None	.parquet	dataset	DataFrame	10315	9VU_toJu0mQ3tqdMjHyUkg	None	3	...	True	False	2026-08-14 13:23:18.905000+00:00	1	1	1	1	2	None	1

1 rows × 22 columns

You can query for all runs that ran with that parameter:

ln.Run.filter(
    params__key="my_analysis/dataset.parquet",
).to_dataframe()

Show code cell output Hide code cell output

	uid	name	description	entrypoint	started_at	finished_at	params	extra_data	reference	reference_type	...	created_at	branch_id	created_on_id	space_id	transform_id	report_id	environment_id	plan_id	created_by_id	initiated_by_run_id
id
2	MemVFZn2hMT91ZMY	None	None	ingest_dataset	2026-08-14 13:23:16.969000+00:00	2026-08-14 13:23:18.909972+00:00	{'key': 'my_analysis/dataset.parquet'}	None	None	None	...	2026-08-14 13:23:16.969000+00:00	1	1	1	2	4	2	None	1	None

1 rows × 22 columns

You can also pass complex parameters and features, see: Track parameters & features.

A multi-step workflow¶

Here, the workflow calls an additional processing step:

my_workflow_with_step.py¶

import lamindb as ln


@ln.step()
def subset_dataframe(
    artifact: ln.Artifact,
    subset_rows: int = 2,
    subset_cols: int = 2,
) -> ln.Artifact:
    df = artifact.load()
    new_data = df.iloc[:subset_rows, :subset_cols]
    new_key = artifact.key.replace(".parquet", "_subsetted.parquet")
    return ln.Artifact.from_dataframe(new_data, key=new_key).save()


@ln.flow()
def ingest_dataset(key: str, subset: bool = False) -> ln.Artifact:
    df = ln.examples.datasets.mini_immuno.get_dataset1()
    artifact = ln.Artifact.from_dataframe(df, key=key).save()
    if subset:
        artifact = subset_dataframe(artifact)
    return artifact


if __name__ == "__main__":
    ingest_dataset(key="my_analysis/dataset.parquet", subset=True)

Let’s run the workflow:

python scripts/my_workflow_with_step.py

The lineage of the subsetted artifact resolves the subsetting step:

subsetted_artifact = ln.Artifact.get(key="my_analysis/dataset_subsetted.parquet")
subsetted_artifact.view_lineage()

This is the run that created the subsetted_artifact:

subsetted_artifact.run

This is the initating run that triggered the function call:

subsetted_artifact.run.initiated_by_run

These are the parameters of the run:

subsetted_artifact.run.params

These are the input artifacts:

subsetted_artifact.run.input_artifacts.to_dataframe()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	...	is_latest	is_locked	created_at	branch_id	created_on_id	space_id	storage_id	run_id	schema_id	created_by_id
id
3	3QtKAZNPIpanK84S0000	my_analysis/dataset.parquet	None	.parquet	dataset	DataFrame	10315	9VU_toJu0mQ3tqdMjHyUkg	None	3	...	True	False	2026-08-14 13:23:18.905000+00:00	1	1	1	1	2	None	1

1 rows × 22 columns

These are output artifacts:

subsetted_artifact.run.output_artifacts.to_dataframe()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	...	is_latest	is_locked	created_at	branch_id	created_on_id	space_id	storage_id	run_id	schema_id	created_by_id
id
5	InzaiZ4tVS6v66650000	my_analysis/dataset_subsetted.parquet	None	.parquet	dataset	DataFrame	3677	J0YX05CsehaJ16RhEf6DoQ	None	2	...	True	False	2026-08-14 13:23:23.035000+00:00	1	1	1	1	4	None	1

1 rows × 22 columns

A workflow with CLI arguments¶

Let’s use click to parse CLI arguments:

my_workflow_with_click.py¶

import click
import lamindb as ln


@click.command()
@click.option("--key", required=True)
@ln.flow()
def main(key: str):
    df = ln.examples.datasets.mini_immuno.get_dataset2()
    ln.Artifact.from_dataframe(df, key=key).save()


if __name__ == "__main__":
    main()

Let’s run the workflow:

python scripts/my_workflow_with_click.py --key my_analysis/dataset2.parquet

CLI arguments are tracked and accessible via run.cli_args:

run = ln.Run.filter(transform__key="my_workflow_with_click.py").first()
run.describe()

Note that it doesn’t matter whether you use click, argparse, or any other CLI argument parser.

Track parameters & features¶

We just saw that the function decorators @ln.flow() and @ln.step() track parameter values automatically. Here is how to pass parameters to ln.track():

run_track_with_params.py¶

import argparse
import lamindb as ln

if __name__ == "__main__":
    p = argparse.ArgumentParser()
    p.add_argument("--input-dir", type=str)
    p.add_argument("--downsample", action="store_true")
    p.add_argument("--learning-rate", type=float)
    args = p.parse_args()
    params = {
        "input_dir": args.input_dir,
        "learning_rate": args.learning_rate,
        "preprocess_params": {
            "downsample": args.downsample,
            "normalization": "the_good_one",
        },
    }
    ln.track(params=params)

    # your code

    ln.finish()

Run the script.

python scripts/run_track_with_params.py  --input-dir ./mydataset --learning-rate 0.01 --downsample

Query for all runs that match certain parameters:

ln.Run.filter(
    params__learning_rate=0.01,
    params__preprocess_params__downsample=True,
).to_dataframe()

Show code cell output Hide code cell output

	uid	name	description	entrypoint	started_at	finished_at	params	extra_data	reference	reference_type	...	created_at	branch_id	created_on_id	space_id	transform_id	report_id	environment_id	plan_id	created_by_id	initiated_by_run_id
id
6	zPnsxDXFrYfK6LAF	None	None	None	2026-08-14 13:23:28.334000+00:00	2026-08-14 13:23:29.322713+00:00	{'input_dir': './mydataset', 'learning_rate': ...	None	None	None	...	2026-08-14 13:23:28.334000+00:00	1	1	1	5	9	2	None	1	None

1 rows × 22 columns

Describe & get parameters:

run = ln.Run.filter(params__learning_rate=0.01).order_by("-started_at").first()
run.describe()
run.params

You can also access the CLI arguments used to start the run directly:

run.cli_args

You can also track run features in analogy to artifact features.

In contrast to params, features are validated against the Feature registry and allow to express relationships with entities in your registries.

Let’s first define labels & features.

experiment_type = ln.Record(name="Experiments", is_type=True).save()
experiment_label = ln.Record(name="Experiment1", type=experiment_type).save()
ln.Feature(name="s3_folder", dtype=str).save()
ln.Feature(name="experiment", dtype=experiment_type).save()

python scripts/run_track_with_features_and_params.py  --s3-folder s3://my-bucket/my-folder --experiment Experiment1

ln.Run.filter(s3_folder="s3://my-bucket/my-folder").to_dataframe()

Show code cell output Hide code cell output

	uid	name	description	entrypoint	started_at	finished_at	params	extra_data	reference	reference_type	...	created_at	branch_id	created_on_id	space_id	transform_id	report_id	environment_id	plan_id	created_by_id	initiated_by_run_id
id
7	GBgmVwiyXvBZzYok	None	None	None	2026-08-14 13:23:30.928000+00:00	2026-08-14 13:23:32.528296+00:00	{'example_param': 42}	None	None	None	...	2026-08-14 13:23:30.928000+00:00	1	1	1	6	10	2	None	1	None

1 rows × 22 columns

Describe & get feature values.

run2 = ln.Run.filter(
    s3_folder="s3://my-bucket/my-folder", experiment="Experiment1"
).last()
run2.describe()
run2.features.get_values()

Manage functions in scripts and notebooks¶

If you want more-fined-grained data lineage tracking in a script or notebook where you called ln.track(), you can also use the step() decorator.

In a notebook¶

@ln.step()
def subset_dataframe(
    input_artifact_key: str,
    output_artifact_key: str,
    subset_rows: int = 2,
    subset_cols: int = 2,
) -> None:
    artifact = ln.Artifact.get(key=input_artifact_key)
    dataset = artifact.load()
    new_data = dataset.iloc[:subset_rows, :subset_cols]
    ln.Artifact.from_dataframe(new_data, key=output_artifact_key).save()

Prepare a test dataset:

df = ln.examples.datasets.mini_immuno.get_dataset1(otype="DataFrame")
input_artifact_key = "my_analysis/dataset.parquet"
artifact = ln.Artifact.from_dataframe(df, key=input_artifact_key).save()

Run the function with default params:

ouput_artifact_key = input_artifact_key.replace(".parquet", "_subsetted.parquet")
subset_dataframe(input_artifact_key, ouput_artifact_key, subset_rows=1)

Query for the output:

subsetted_artifact = ln.Artifact.get(key=ouput_artifact_key)
subsetted_artifact.view_lineage()

Re-run the function with a different parameter:

subsetted_artifact = subset_dataframe(
    input_artifact_key, ouput_artifact_key, subset_cols=3
)
subsetted_artifact = ln.Artifact.get(key=ouput_artifact_key)
subsetted_artifact.view_lineage()

We created a new run:

subsetted_artifact.run

With new parameters:

subsetted_artifact.run.params

And a new version of the output artifact:

subsetted_artifact.run.output_artifacts.to_dataframe()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	...	is_latest	is_locked	created_at	branch_id	created_on_id	space_id	storage_id	run_id	schema_id	created_by_id
id
12	InzaiZ4tVS6v66650002	my_analysis/dataset_subsetted.parquet	None	.parquet	dataset	DataFrame	4299	aYJEUiAj3bJ0RkaA0Xp1qQ	None	2	...	True	False	2026-08-14 13:23:35.136000+00:00	1	1	1	1	9	None	1

1 rows × 22 columns

In a script¶

run_script_with_step.py¶

import argparse
import lamindb as ln


@ln.step()
def subset_dataframe(
    artifact: ln.Artifact,
    subset_rows: int = 2,
    subset_cols: int = 2,
    run: ln.Run | None = None,
) -> ln.Artifact:
    dataset = artifact.load(is_run_input=run)
    new_data = dataset.iloc[:subset_rows, :subset_cols]
    new_key = artifact.key.replace(".parquet", "_subsetted.parquet")
    return ln.Artifact.from_dataframe(new_data, key=new_key, run=run).save()


if __name__ == "__main__":
    p = argparse.ArgumentParser()
    p.add_argument("--subset", action="store_true")
    args = p.parse_args()

    params = {"is_subset": args.subset}

    ln.track(params=params)

    if args.subset:
        df = ln.examples.datasets.mini_immuno.get_dataset1(otype="DataFrame")
        artifact = ln.Artifact.from_dataframe(
            df, key="my_analysis/dataset.parquet"
        ).save()
        subsetted_artifact = subset_dataframe(artifact)

    ln.finish()

python scripts/run_script_with_step.py --subset

ln.view()

Show code cell output Hide code cell output

Artifact

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	...	is_latest	is_locked	created_at	branch_id	created_on_id	space_id	storage_id	run_id	schema_id	created_by_id
id
12	InzaiZ4tVS6v66650002	my_analysis/dataset_subsetted.parquet	None	.parquet	dataset	DataFrame	4299	aYJEUiAj3bJ0RkaA0Xp1qQ	None	2.0	...	True	False	2026-08-14 13:23:35.136000+00:00	1	1	1	1	9	None	1
11	InzaiZ4tVS6v66650001	my_analysis/dataset_subsetted.parquet	None	.parquet	dataset	DataFrame	3650	kqqMRFXsdeYG-XfVNfiR4A	None	1.0	...	False	False	2026-08-14 13:23:34.402000+00:00	1	1	1	1	8	None	1
7	8WZJUFdoXhZdStT40000	my_analysis/dataset2.parquet	None	.parquet	dataset	DataFrame	7019	sFlbfG5gV-yi8cG-4DGZSg	None	3.0	...	True	False	2026-08-14 13:23:26.678000+00:00	1	1	1	1	5	None	1
5	InzaiZ4tVS6v66650000	my_analysis/dataset_subsetted.parquet	None	.parquet	dataset	DataFrame	3677	J0YX05CsehaJ16RhEf6DoQ	None	2.0	...	False	False	2026-08-14 13:23:23.035000+00:00	1	1	1	1	4	None	1
3	3QtKAZNPIpanK84S0000	my_analysis/dataset.parquet	None	.parquet	dataset	DataFrame	10315	9VU_toJu0mQ3tqdMjHyUkg	None	3.0	...	True	False	2026-08-14 13:23:18.905000+00:00	1	1	1	1	2	None	1
1	YnyJT5GYjLfcAaeZ0000	sample.fasta	None	.fasta	NaN	NaN	11	83rEPcAoBHmYiIuyBYrFKg	None	NaN	...	True	False	2026-08-14 13:23:15.436000+00:00	1	1	1	1	1	None	1

6 rows × 22 columns

Feature

	uid	name	_dtype_str	unit	description	array_rank	array_size	array_shape	synonyms	default_value	...	coerce	is_locked	is_type	created_at	branch_id	created_on_id	space_id	created_by_id	run_id	type_id
id
2	EH9NUgr58tCE	experiment	cat[Record[PBrT0sVvZdqbTi2t]]	None	None	0	0	None	None	None	...	None	False	False	2026-08-14 13:23:29.686000+00:00	1	1	1	1	1	None
1	wVFBGQCvloh4	s3_folder	str	None	None	0	0	None	None	None	...	None	False	False	2026-08-14 13:23:29.679000+00:00	1	1	1	1	1	None

2 rows × 21 columns

Project

	uid	name	description	abbr	url	start_date	end_date	is_locked	is_type	created_at	branch_id	created_on_id	space_id	created_by_id	run_id	type_id
id
1	E89yUo1ms8Sb	My project	None	None	None	None	None	False	False	2026-08-14 13:23:13.317000+00:00	1	1	1	1	None	None

Record

	uid	name	description	reference	reference_type	extra_data	is_locked	is_type	created_at	branch_id	created_on_id	space_id	created_by_id	type_id	schema_id	run_id
id
2	8utzkkw7Lr99SpYe	Experiment1	None	None	None	None	False	False	2026-08-14 13:23:29.674000+00:00	1	1	1	1	1.0	None	1
1	PBrT0sVvZdqbTi2t	Experiments	None	None	None	None	False	True	2026-08-14 13:23:29.669000+00:00	1	1	1	1	NaN	None	1

Run

	uid	name	description	entrypoint	started_at	finished_at	params	extra_data	reference	reference_type	...	created_at	branch_id	created_on_id	space_id	transform_id	report_id	environment_id	plan_id	created_by_id	initiated_by_run_id
id
11	UjZEm0O9ICAWXk1A	None	None	subset_dataframe	2026-08-14 13:23:38.157000+00:00	2026-08-14 13:23:38.838632+00:00	{'artifact': 'Artifact[3QtKAZNPIpanK84S0000]',...	None	None	None	...	2026-08-14 13:23:38.157000+00:00	1	1	1	8	NaN	NaN	None	1	10.0
10	ig8nzmxpnnDgkGEc	None	None	NaN	2026-08-14 13:23:36.437000+00:00	2026-08-14 13:23:38.840514+00:00	{'is_subset': True}	None	None	None	...	2026-08-14 13:23:36.437000+00:00	1	1	1	8	13.0	2.0	None	1	NaN
9	NJNaBP5t5w7P8FK3	None	None	subset_dataframe	2026-08-14 13:23:34.459000+00:00	2026-08-14 13:23:35.142990+00:00	{'input_artifact_key': 'my_analysis/dataset.pa...	None	None	None	...	2026-08-14 13:23:34.459000+00:00	1	1	1	7	NaN	NaN	None	1	1.0
8	JW8DrZf2gAVUjqNg	None	None	subset_dataframe	2026-08-14 13:23:33.712000+00:00	2026-08-14 13:23:34.409698+00:00	{'input_artifact_key': 'my_analysis/dataset.pa...	None	None	None	...	2026-08-14 13:23:33.712000+00:00	1	1	1	7	NaN	NaN	None	1	1.0
7	GBgmVwiyXvBZzYok	None	None	NaN	2026-08-14 13:23:30.928000+00:00	2026-08-14 13:23:32.528296+00:00	{'example_param': 42}	None	None	None	...	2026-08-14 13:23:30.928000+00:00	1	1	1	6	10.0	2.0	None	1	NaN
6	zPnsxDXFrYfK6LAF	None	None	NaN	2026-08-14 13:23:28.334000+00:00	2026-08-14 13:23:29.322713+00:00	{'input_dir': './mydataset', 'learning_rate': ...	None	None	None	...	2026-08-14 13:23:28.334000+00:00	1	1	1	5	9.0	2.0	None	1	NaN
5	pYXRHLnRt9IvWomE	None	None	main	2026-08-14 13:23:24.956000+00:00	2026-08-14 13:23:26.682359+00:00	{'key': 'my_analysis/dataset2.parquet'}	None	None	None	...	2026-08-14 13:23:24.956000+00:00	1	1	1	4	8.0	2.0	None	1	NaN

7 rows × 22 columns

Storage

	uid	root	description	type	region	instance_uid	is_locked	created_at	branch_id	created_on_id	space_id	created_by_id	run_id
id
1	SSZMxGqshRec	/home/runner/work/lamindb/lamindb/docs/test-track	None	local	None	73KPGC58ahU9	False	2026-08-14 13:23:12.285000+00:00	1	1	1	1	None

Transform

	uid	key	description	kind	source_code	hash	reference	reference_type	version_tag	is_latest	is_locked	created_at	branch_id	created_on_id	space_id	environment_id	plan_id	run_id	created_by_id
id
8	L6yuTuH4SS190000	run_script_with_step.py	CLI: run_script_with_step.py	script	import argparse\nimport lamindb as ln\n\n\n@ln...	HJbjZyWWczP-VmzKQsSORg	None	None	None	True	False	2026-08-14 13:23:36.099000+00:00	1	1	1	None	None	NaN	1
7	9zHFgpyzc1pU0000	track.ipynb	NaN	function	@ln.step()\ndef subset_dataframe(\n input_a...	5kfRAQLCPwxrvAjspfdp2Q	None	None	None	True	False	2026-08-14 13:23:33.708000+00:00	1	1	1	None	None	1.0	1
6	UIyLoo0lX06E0000	run_track_with_features_and_params.py	CLI: run_track_with_features_and_params.py	script	import argparse\nimport lamindb as ln\n\n\nif ...	9MjLyvM1QzE2nPIPDRzBwg	None	None	None	True	False	2026-08-14 13:23:30.586000+00:00	1	1	1	None	None	NaN	1
5	vhayQ1F9TPXJ0000	run_track_with_params.py	CLI: run_track_with_params.py	script	import argparse\nimport lamindb as ln\n\nif __...	5RBz7zJICeKE1OSmg7gEdQ	None	None	None	True	False	2026-08-14 13:23:27.993000+00:00	1	1	1	None	None	NaN	1
4	fMYKZPoQ6RsV0000	my_workflow_with_click.py	CLI: my_workflow_with_click.py	script	import click\nimport lamindb as ln\n\n\n@click...	0eX8wmaAWkuuAvACWwL1Xg	None	None	None	True	False	2026-08-14 13:23:24.612000+00:00	1	1	1	None	None	NaN	1
3	qoGrXGhtLcRH0000	my_workflow_with_step.py	NaN	script	import lamindb as ln\n\n\n@ln.step()\ndef subs...	Ncx6UswxtCN3FZD86kgcVQ	None	None	None	True	False	2026-08-14 13:23:20.323000+00:00	1	1	1	None	None	NaN	1
2	eX2AUGaQJcTO0000	my_workflow.py	NaN	script	import lamindb as ln\n\n\n@ln.flow()\ndef inge...	uJ3fsnfaNN6EZ7Q0d8SQtw	None	None	None	True	False	2026-08-14 13:23:16.628000+00:00	1	1	1	None	None	NaN	1

The database¶

See the state of the database after we ran these different examples:

ln.view()

Show code cell output Hide code cell output

Artifact

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	...	is_latest	is_locked	created_at	branch_id	created_on_id	space_id	storage_id	run_id	schema_id	created_by_id
id
12	InzaiZ4tVS6v66650002	my_analysis/dataset_subsetted.parquet	None	.parquet	dataset	DataFrame	4299	aYJEUiAj3bJ0RkaA0Xp1qQ	None	2.0	...	True	False	2026-08-14 13:23:35.136000+00:00	1	1	1	1	9	None	1
11	InzaiZ4tVS6v66650001	my_analysis/dataset_subsetted.parquet	None	.parquet	dataset	DataFrame	3650	kqqMRFXsdeYG-XfVNfiR4A	None	1.0	...	False	False	2026-08-14 13:23:34.402000+00:00	1	1	1	1	8	None	1
7	8WZJUFdoXhZdStT40000	my_analysis/dataset2.parquet	None	.parquet	dataset	DataFrame	7019	sFlbfG5gV-yi8cG-4DGZSg	None	3.0	...	True	False	2026-08-14 13:23:26.678000+00:00	1	1	1	1	5	None	1
5	InzaiZ4tVS6v66650000	my_analysis/dataset_subsetted.parquet	None	.parquet	dataset	DataFrame	3677	J0YX05CsehaJ16RhEf6DoQ	None	2.0	...	False	False	2026-08-14 13:23:23.035000+00:00	1	1	1	1	4	None	1
3	3QtKAZNPIpanK84S0000	my_analysis/dataset.parquet	None	.parquet	dataset	DataFrame	10315	9VU_toJu0mQ3tqdMjHyUkg	None	3.0	...	True	False	2026-08-14 13:23:18.905000+00:00	1	1	1	1	2	None	1
1	YnyJT5GYjLfcAaeZ0000	sample.fasta	None	.fasta	NaN	NaN	11	83rEPcAoBHmYiIuyBYrFKg	None	NaN	...	True	False	2026-08-14 13:23:15.436000+00:00	1	1	1	1	1	None	1

6 rows × 22 columns

Feature

	uid	name	_dtype_str	unit	description	array_rank	array_size	array_shape	synonyms	default_value	...	coerce	is_locked	is_type	created_at	branch_id	created_on_id	space_id	created_by_id	run_id	type_id
id
2	EH9NUgr58tCE	experiment	cat[Record[PBrT0sVvZdqbTi2t]]	None	None	0	0	None	None	None	...	None	False	False	2026-08-14 13:23:29.686000+00:00	1	1	1	1	1	None
1	wVFBGQCvloh4	s3_folder	str	None	None	0	0	None	None	None	...	None	False	False	2026-08-14 13:23:29.679000+00:00	1	1	1	1	1	None

2 rows × 21 columns

Project

	uid	name	description	abbr	url	start_date	end_date	is_locked	is_type	created_at	branch_id	created_on_id	space_id	created_by_id	run_id	type_id
id
1	E89yUo1ms8Sb	My project	None	None	None	None	None	False	False	2026-08-14 13:23:13.317000+00:00	1	1	1	1	None	None

Record

	uid	name	description	reference	reference_type	extra_data	is_locked	is_type	created_at	branch_id	created_on_id	space_id	created_by_id	type_id	schema_id	run_id
id
2	8utzkkw7Lr99SpYe	Experiment1	None	None	None	None	False	False	2026-08-14 13:23:29.674000+00:00	1	1	1	1	1.0	None	1
1	PBrT0sVvZdqbTi2t	Experiments	None	None	None	None	False	True	2026-08-14 13:23:29.669000+00:00	1	1	1	1	NaN	None	1

Run

	uid	name	description	entrypoint	started_at	finished_at	params	extra_data	reference	reference_type	...	created_at	branch_id	created_on_id	space_id	transform_id	report_id	environment_id	plan_id	created_by_id	initiated_by_run_id
id
11	UjZEm0O9ICAWXk1A	None	None	subset_dataframe	2026-08-14 13:23:38.157000+00:00	2026-08-14 13:23:38.838632+00:00	{'artifact': 'Artifact[3QtKAZNPIpanK84S0000]',...	None	None	None	...	2026-08-14 13:23:38.157000+00:00	1	1	1	8	NaN	NaN	None	1	10.0
10	ig8nzmxpnnDgkGEc	None	None	NaN	2026-08-14 13:23:36.437000+00:00	2026-08-14 13:23:38.840514+00:00	{'is_subset': True}	None	None	None	...	2026-08-14 13:23:36.437000+00:00	1	1	1	8	13.0	2.0	None	1	NaN
9	NJNaBP5t5w7P8FK3	None	None	subset_dataframe	2026-08-14 13:23:34.459000+00:00	2026-08-14 13:23:35.142990+00:00	{'input_artifact_key': 'my_analysis/dataset.pa...	None	None	None	...	2026-08-14 13:23:34.459000+00:00	1	1	1	7	NaN	NaN	None	1	1.0
8	JW8DrZf2gAVUjqNg	None	None	subset_dataframe	2026-08-14 13:23:33.712000+00:00	2026-08-14 13:23:34.409698+00:00	{'input_artifact_key': 'my_analysis/dataset.pa...	None	None	None	...	2026-08-14 13:23:33.712000+00:00	1	1	1	7	NaN	NaN	None	1	1.0
7	GBgmVwiyXvBZzYok	None	None	NaN	2026-08-14 13:23:30.928000+00:00	2026-08-14 13:23:32.528296+00:00	{'example_param': 42}	None	None	None	...	2026-08-14 13:23:30.928000+00:00	1	1	1	6	10.0	2.0	None	1	NaN
6	zPnsxDXFrYfK6LAF	None	None	NaN	2026-08-14 13:23:28.334000+00:00	2026-08-14 13:23:29.322713+00:00	{'input_dir': './mydataset', 'learning_rate': ...	None	None	None	...	2026-08-14 13:23:28.334000+00:00	1	1	1	5	9.0	2.0	None	1	NaN
5	pYXRHLnRt9IvWomE	None	None	main	2026-08-14 13:23:24.956000+00:00	2026-08-14 13:23:26.682359+00:00	{'key': 'my_analysis/dataset2.parquet'}	None	None	None	...	2026-08-14 13:23:24.956000+00:00	1	1	1	4	8.0	2.0	None	1	NaN

7 rows × 22 columns

Storage

	uid	root	description	type	region	instance_uid	is_locked	created_at	branch_id	created_on_id	space_id	created_by_id	run_id
id
1	SSZMxGqshRec	/home/runner/work/lamindb/lamindb/docs/test-track	None	local	None	73KPGC58ahU9	False	2026-08-14 13:23:12.285000+00:00	1	1	1	1	None

Transform

	uid	key	description	kind	source_code	hash	reference	reference_type	version_tag	is_latest	is_locked	created_at	branch_id	created_on_id	space_id	environment_id	plan_id	run_id	created_by_id
id
8	L6yuTuH4SS190000	run_script_with_step.py	CLI: run_script_with_step.py	script	import argparse\nimport lamindb as ln\n\n\n@ln...	HJbjZyWWczP-VmzKQsSORg	None	None	None	True	False	2026-08-14 13:23:36.099000+00:00	1	1	1	None	None	NaN	1
7	9zHFgpyzc1pU0000	track.ipynb	NaN	function	@ln.step()\ndef subset_dataframe(\n input_a...	5kfRAQLCPwxrvAjspfdp2Q	None	None	None	True	False	2026-08-14 13:23:33.708000+00:00	1	1	1	None	None	1.0	1
6	UIyLoo0lX06E0000	run_track_with_features_and_params.py	CLI: run_track_with_features_and_params.py	script	import argparse\nimport lamindb as ln\n\n\nif ...	9MjLyvM1QzE2nPIPDRzBwg	None	None	None	True	False	2026-08-14 13:23:30.586000+00:00	1	1	1	None	None	NaN	1
5	vhayQ1F9TPXJ0000	run_track_with_params.py	CLI: run_track_with_params.py	script	import argparse\nimport lamindb as ln\n\nif __...	5RBz7zJICeKE1OSmg7gEdQ	None	None	None	True	False	2026-08-14 13:23:27.993000+00:00	1	1	1	None	None	NaN	1
4	fMYKZPoQ6RsV0000	my_workflow_with_click.py	CLI: my_workflow_with_click.py	script	import click\nimport lamindb as ln\n\n\n@click...	0eX8wmaAWkuuAvACWwL1Xg	None	None	None	True	False	2026-08-14 13:23:24.612000+00:00	1	1	1	None	None	NaN	1
3	qoGrXGhtLcRH0000	my_workflow_with_step.py	NaN	script	import lamindb as ln\n\n\n@ln.step()\ndef subs...	Ncx6UswxtCN3FZD86kgcVQ	None	None	None	True	False	2026-08-14 13:23:20.323000+00:00	1	1	1	None	None	NaN	1
2	eX2AUGaQJcTO0000	my_workflow.py	NaN	script	import lamindb as ln\n\n\n@ln.flow()\ndef inge...	uJ3fsnfaNN6EZ7Q0d8SQtw	None	None	None	True	False	2026-08-14 13:23:16.628000+00:00	1	1	1	None	None	NaN	1

Using transform versions as templates¶

A transform acts like a template upon using lamin load to load it. Consider you run:

lamin load https://lamin.ai/account/instance/transform/Akd7gx7Y9oVO0000

Upon running the returned notebook or script, you’ll automatically create a new version and be able to browse it via the version dropdown on the UI.

Additionally, you can:

label using ULabel or Record, e.g., transform.records.add(template_label)
tag with an indicative version string, e.g., transform.version = "T1"; transform.save()

A few checks at the end of this notebook:

assert run.params == {
    "input_dir": "./mydataset",
    "learning_rate": 0.01,
    "preprocess_params": {"downsample": True, "normalization": "the_good_one"},
}, run.params
assert my_project.artifacts.exists()
assert my_project.transforms.exists()
assert my_project.runs.exists()