Adapters

Adapters translate NWB Schema Language to LinkML Schema.

  • Adapter - Base Adapter Classes

  • Namespaces - Top-level container of NWB namespace indices and schema

  • Schema - Individual NWB Schema files within a namespace

  • Classes - Root methods shared between classes and groups

Adapter classes for translating from NWB schema language to LinkML

class Adapter

Abstract base class for adapters

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

property debug: bool

Whether we are in debug mode, which adds extra metadata in generated elements.

Set explicitly via _debug , or else checks for the truthiness of the environment variable NWB_LINKML_DEBUG

property logger: Logger

A logger with the name of the adapter class! See config

abstract build() → BuildResult

Generate the corresponding linkML element for this adapter

get(name: str) → Group | Dataset

Get the first item whose neurodata_type_def matches name

Convenience wrapper around walk_field_values()

get_model_with_field(field: str) → Generator[Group | Dataset, None, None]

Yield models that have a non-None value in the given field.

Useful during development to find all the ways that a given field is used.

Parameters:

field (str) – Field to search for

walk(input: BaseModel | dict | list) → Generator[BaseModel | Any | None, None, None]

Iterate through all items in the given model.

Could be a staticmethod or a function, but bound to adapters to make it available to them :)

walk_fields(input: BaseModel | dict | list, field: str | Tuple[str, ...]) → Generator[Any, None, None]

Recursively walk input for fields that match field

Parameters:

Returns:

walk_field_values(input: BaseModel | dict | list, field: Literal['neurodata_type_def'], value: Any | None = None) → Generator[Group | Dataset, None, None]

Recursively walk input for models that contain a field as a direct child with a value matching value

Parameters:

  • input (pydantic.BaseModel) – Model to walk

  • field (str) – Name of field - unlike walk_fields(), only one field can be given

  • value (Any) – Value to match for given field. If None , return models that have the field

Returns:

pydantic.BaseModel the matching model

walk_types(input: BaseModel | dict | list, get_type: Type[T] | Tuple[Type[T], Type[Unpack[Ts]]]) → Generator[T | Ts, None, None]

Walk a model, yielding items that are the same type as the given type

Parameters:
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

model_post_init(context: Any, /) → None

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

  • self – The BaseModel instance.

  • context – The context.

class ArrayAdapter(dims: List[str | None] | List[List[str | None]], shape: List[str | None] | List[List[str | None]])

Adapter that generates a ArrayExpression (or set of them) from a NWB dims/shape declaration

pivot_dims(dims: List[str | None] | List[List[str | None]] | None = None, shape: List[str | None] | List[List[str | None]] | None = None) → List[Shape]

Pivot from a list of dims and a list of shape to a list of (dim, shape) tuples

make_expression(shape: Shape) → ArrayExpression

Create the corresponding array specification from a shape

make() → List[ArrayExpression]

Create an array specification from self.dims and self.shape

make_slot() → Dict[Literal['array'], ArrayExpression] | Dict[Literal['any_of'], Dict[Literal['array'], List[ArrayExpression]]]

Make the array expressions in a dict form that can be **kwarg’d into a SlotDefinition, taking into account needing to use any_of for multiple array range specifications.

class BuildResult(schemas: ~typing.List[~linkml_runtime.linkml_model.meta.SchemaDefinition] = <factory>, classes: ~typing.List[~linkml_runtime.linkml_model.meta.ClassDefinition] = <factory>, slots: ~typing.List[~linkml_runtime.linkml_model.meta.SlotDefinition] = <factory>, types: ~typing.List[~linkml_runtime.linkml_model.meta.TypeDefinition] = <factory>, **_kwargs)

Container class for propagating nested build results back up to caller

schemas: List[SchemaDefinition]
classes: List[ClassDefinition]
slots: List[SlotDefinition]
types: List[TypeDefinition]
as_linkml() → str

Print build results as linkml-style YAML.

Note that only non-schema results will be included, as a schema usually contains all the other types.

class ClassAdapter(*, TYPE: T, cls: TI, parent: ClassAdapter | None = None)

Abstract adapter to class-like things in linkml, holds methods common to both DatasetAdapter and GroupAdapter

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

TYPE: T

The type that this adapter class handles

cls: TI
parent: ClassAdapter | None
classmethod cast_from_string(value: str | TI) → TI

Cast from YAML string to desired class

abstract build() → BuildResult

Make this abstract so it can’t be instantiated directly.

Subclasses call build_base() to get the basics true of both groups and datasets

build_base(extra_attrs: List[SlotDefinition] | None = None) → BuildResult

Build the basic class and attributes before adding any specific modifications for groups or datasets.

The main distinction in behavior for this method is whether this class has a parent class - ie this is one of the anonymous nested child datasets or groups within another group.

If the class has no parent, then…

  • Its name is inferred from its neurodata_type_def, fixed name, or neurodata_type_inc in that order

  • It is just built as normal class!

  • It will be indicated as a tree_root (which will primarily be used to invert the translation for write operations)

If the class has a parent, then…

  • If it has a neurodata_type_def or inc, that will be used as its name, otherwise concatenate parent__child, eg. TimeSeries__TimeSeriesData

  • A slot will also be made and returned with the BuildResult, which the parent will then have as one of its attributes.

build_attrs(cls: Dataset | Group) → List[SlotDefinition]

Pack the class attributes into a list of SlotDefinitions

Parameters:

cls – (Dataset | Group): Class to pack

Returns:

list[SlotDefinition]

build_name_slot() → SlotDefinition

If a class has a name, then that name should be a slot with a fixed value.

If a class does not have a name, then name should be a required attribute

References

https://github.com/NeurodataWithoutBorders/nwb-schema/issues/552#issuecomment-1700319001

Returns:

build_self_slot() → SlotDefinition

If we are a child class, we make a slot so our parent can refer to us

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'TYPE': FieldInfo(annotation=TypeVar, required=True), 'cls': FieldInfo(annotation=TypeVar, required=True), 'parent': FieldInfo(annotation=Union[ClassAdapter, NoneType], required=False, default=None)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

model_post_init(context: Any, /) → None

We need to both initialize private attributes and call the user-defined model_post_init method.

class DatasetAdapter(*, cls: Dataset, parent: ClassAdapter | None = None)

Orchestrator class for datasets - calls the set of applicable mapping classes

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

TYPE

alias of Dataset

cls: Dataset
build() → BuildResult

Build the base result, and then apply the applicable mappings.

match() → Type[DatasetMap] | None

Find the map class that applies to this class

Returns:

DatasetMap

Raises:

RuntimeError - if more than one map matches –

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'cls': FieldInfo(annotation=Dataset, required=True), 'parent': FieldInfo(annotation=Union[ClassAdapter, NoneType], required=False, default=None)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

model_post_init(context: Any, /) → None

We need to both initialize private attributes and call the user-defined model_post_init method.

parent: 'ClassAdapter' | None
class GroupAdapter(*, TYPE: ~typing.Type = <class 'nwb_schema_language.datamodel.nwb_schema_pydantic.Group'>, cls: ~nwb_schema_language.datamodel.nwb_schema_pydantic.Group, parent: ~nwb_linkml.adapters.classes.ClassAdapter | None = None)

Adapt NWB Groups to LinkML Classes

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

TYPE: Type

The type that this adapter class handles

cls: Group
build() → BuildResult

Do the translation, yielding the BuildResult

Build links specified in the links field as slots that refer to other classes, with an additional annotation specifying that they are in fact links.

Link slots can take either the object itself or the path to that object in the file hierarchy as a string.

handle_container_group(cls: Group) → BuildResult

Make a special LinkML value slot that can have any number of the objects that are of neurodata_type_inc class

Examples

- name: templates
  groups:
  - neurodata_type_inc: TimeSeries
    doc: TimeSeries objects containing template data of presented stimuli.
    quantity: '*'
  - neurodata_type_inc: Images
    doc: Images objects containing images of presented stimuli.
    quantity: '*'
handle_container_slot(cls: Group) → BuildResult

Handle subgroups that contain arbitrarily numbered classes,

eg. each of the groups in

Examples

  • name: trials neurodata_type_inc: TimeIntervals doc: Repeated experimental events that have a logical grouping. quantity: ‘?’

  • name: invalid_times neurodata_type_inc: TimeIntervals doc: Time intervals that should be removed from analysis. quantity: ‘?’

  • neurodata_type_inc: TimeIntervals doc: Optional additional table(s) for describing other experimental time intervals. quantity: ‘*’

build_datasets() → BuildResult

Build nested groups and datasets

Create ClassDefinitions for each, but then also create SlotDefinitions that will be used as attributes linking the main class to the subclasses

Datasets are simple, they are terminal classes, and all logic for creating slots vs. classes is handled by the adapter class

build_groups() → BuildResult

Build subgroups, excluding pure container subgroups

build_containers() → BuildResult

Build all container types into a single value slot

build_special_cases() → BuildResult

Special cases, at this point just for NWBFile, which has extra .specloc and specifications attrs

build_self_slot() → SlotDefinition

If we are a child class, we make a slot so our parent can refer to us

Groups are a bit more complicated because they can also behave like range declarations: eg. a group can have multiple groups with neurodata_type_inc, no name, and quantity of *, the group can then contain any number of groups of those included types as direct children

We make sure that we’re inlined as a dict so our parent class can refer to us like:

parent.{slot_name}[{name}] = self
model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'TYPE': FieldInfo(annotation=Type, required=False, default=<class 'nwb_schema_language.datamodel.nwb_schema_pydantic.Group'>), 'cls': FieldInfo(annotation=Group, required=True), 'parent': FieldInfo(annotation=Union[ClassAdapter, NoneType], required=False, default=None)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

model_post_init(context: Any, /) → None

We need to both initialize private attributes and call the user-defined model_post_init method.

parent: 'ClassAdapter' | None
class NamespacesAdapter(*, namespaces: Namespaces, schemas: List[SchemaAdapter], imported: List[NamespacesAdapter] = None)

Translate a NWB Namespace to a LinkML Schema

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

namespaces: Namespaces
schemas: List[SchemaAdapter]
imported: List[NamespacesAdapter]
classmethod from_yaml(path: Path) → NamespacesAdapter

Create a NamespacesAdapter from a nwb schema language namespaces yaml file.

Also attempts to provide imported implicitly imported schema (using the namespace key, rather than source, eg. with hdmf-common)

build(skip_imports: bool = False, progress: AdapterProgress | None = None) → BuildResult

Build the NWB namespace to the LinkML Schema

complete_namespaces() → None

After loading the namespace, and after any imports have been added afterwards, this must be called to complete the definitions of the contained schema objects.

This is not automatic because NWB doesn’t have a formal dependency resolution system, so it is often impossible to know which imports are needed until after the namespace adapter has been instantiated.

It is automatically called if it hasn’t been already by the build() method.

find_type_source(cls: str | Dataset | Group, fast: bool = False) → SchemaAdapter

Given some type (as neurodata_type_def), find the schema that it’s defined in.

Rather than returning as soon as a match is found, ensure that duplicates are not found within the primary schema, then so the same for all imported schemas.

Parameters:

  • cls (str | Dataset | Group) – The neurodata_type_def to look for the source of. If a Dataset or Group, look for the object itself (cls in schema.datasets), otherwise look for a class with a matching name.

  • fast (bool) – If True, return as soon as a match is found. If ``False`, return after checking all schemas for duplicates.

Returns:

SchemaAdapter

Raises:

KeyError – if multiple schemas or no schemas are found

to_yaml(base_dir: Path) → None

Build the schemas, saving them to yaml files according to their name

Parameters:

base_dir (Path) – Directory to save yaml files

property needed_imports: Dict[str, List[str]]

List of other, external namespaces that we need to import. Usually provided as schema with a namespace but not a source

Returns:

[‘needed_import_0’, …]}

Return type:

{‘namespace_name’

property versions: Dict[str, str]

versions for each namespace

namespace_schemas(name: str) → List[str]

Get the schemas that are defined in a given namespace

schema_namespace(name: str) → str | None

Inverse of namespace_schemas() - given a schema name, get the namespace it’s in

all_schemas() → Generator[SchemaAdapter, None, None]

Iterator over all schemas including imports

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'imported': FieldInfo(annotation=List[NamespacesAdapter], required=False, default_factory=list), 'namespaces': FieldInfo(annotation=Namespaces, required=True), 'schemas': FieldInfo(annotation=List[SchemaAdapter], required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

model_post_init(context: Any, /) → None

We need to both initialize private attributes and call the user-defined model_post_init method.

class SchemaAdapter(*, path: Path, groups: List[Group] = None, datasets: List[Dataset] = None, imports: List[SchemaAdapter | str] = None, namespace: str | None = None, version: str | None = None)

An individual schema file in nwb_schema_language

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

path: Path
groups: List[Group]
datasets: List[Dataset]
imports: List[SchemaAdapter | str]
namespace: str | None
version: str | None
property name: str

The namespace.schema name for a single schema

build() → BuildResult

Make the LinkML representation for this schema file

Things that will be populated later - id (but need to have a placeholder to instantiate) - version

property created_classes: List[Type[Group | Dataset]]

All the group and datasets created in this schema

property needed_imports: List[str]

Classes that need to be imported from other namespaces

TODO: - Need to also check classes used in links/references

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'datasets': FieldInfo(annotation=List[Dataset], required=False, default_factory=list), 'groups': FieldInfo(annotation=List[Group], required=False, default_factory=list), 'imports': FieldInfo(annotation=List[Union[SchemaAdapter, str]], required=False, default_factory=list), 'namespace': FieldInfo(annotation=Union[str, NoneType], required=False, default=None, description='String of containing namespace. Populated by NamespacesAdapter'), 'path': FieldInfo(annotation=Path, required=True), 'version': FieldInfo(annotation=Union[str, NoneType], required=False, default=None, description='Version of schema, populated by NamespacesAdapter since individual schema files dont know their version in NWB Schema Lang')}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

model_post_init(context: Any, /) → None

We need to both initialize private attributes and call the user-defined model_post_init method.