Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/source/acquisition.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,8 @@ while the StimulusEpoch represents all stimuli being presented.
|-------|------|-------------|
| `subject_id` | `str` | Subject ID (Unique identifier for the subject) |
| `specimen_id` | `Optional[str]` | Specimen ID (Specimen ID is required for in vitro imaging modalities) |
| `acquisition_start_time` | `datetime (timezone-aware)` | Acquisition start time |
| `acquisition_start_time` | `datetime (timezone-aware)` | Acquisition start time (During validation, timezone information will be moved into the acquisition_start_tz field.) |
| `acquisition_start_tz` | `Optional[pydantic_extra_types.timezone_name.TimeZoneName]` | Acquisition start timezone (Automatically populated by a validator based on acquisition_start_time.) |
| `acquisition_end_time` | `datetime (timezone-aware)` | Acquisition end time |
| `experimenters` | `List[str]` | experimenter(s) |
| `protocol_id` | `Optional[List[str]]` | Protocol ID (DOI for protocols.io) |
Expand Down
2 changes: 1 addition & 1 deletion docs/source/components/identifiers.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Code or script identifier
| `name` | `Optional[str]` | Name |
| `version` | `Optional[str]` | Code version |
| `container` | Optional[[Container](#container)] | Container |
| `run_script` | `Optional[pathlib._local.Path]` | Run script (Path to run script) |
| `run_script` | `Optional[pathlib.Path]` | Run script (Path to run script) |
| `language` | `Optional[str]` | Programming language (Programming language used) |
| `language_version` | `Optional[str]` | Programming language version |
| `input_data` | Optional[List[[DataAsset](#dataasset) or [CombinedData](#combineddata)]] | Input data (Input data used in the code or script) |
Expand Down
49 changes: 38 additions & 11 deletions docs/source/components/specimen_procedures.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,31 +29,49 @@ Description of an HCR staining round
| `probe_concentration_unit` | `str` | Probe concentration unit |


### PlanarSection

Description of a single planar section of brain tissue

| Field | Type | Title (Description) |
|-------|------|-------------|
| `coordinate_system_name` | `str` | Coordinate system name |
| `start_coordinate` | [Translation](coordinates.md#translation) | Start coordinate |
| `end_coordinate` | Optional[[Translation](coordinates.md#translation)] | End coordinate |
| `thickness` | `Optional[float]` | Slice thickness |
| `thickness_unit` | Optional[[SizeUnit](../aind_data_schema_models/units.md#sizeunit)] | Slice thickness unit |
| `partial_slice` | Optional[List[[AnatomicalRelative](../aind_data_schema_models/coordinates.md#anatomicalrelative)]] | Partial slice (If sectioning does not include the entire slice, indicate which part of the slice is retained.) |
| `output_specimen_id` | `str` | Specimen ID |
| `targeted_structure` | Optional[[BrainAtlas](../aind_data_schema_models/brain_atlas.md#ccfv3)] | Targeted structure |
| `includes_surrounding_tissue` | `Optional[bool]` | Includes surrounding tissue (Whether the section includes additional tissue surrounding the targeted structure.) |


### PlanarSectioning

Description of a sectioning procedure performed on the coronal, sagittal, or transverse/axial plane

| Field | Type | Title (Description) |
|-------|------|-------------|
| `coordinate_system` | Optional[[CoordinateSystem](coordinates.md#coordinatesystem)] | Sectioning coordinate system (Only required if different from the Procedures.coordinate_system) |
| `sections` | List[[Section](#section)] | Sections |
| `coordinate_system` | [CoordinateSystem](coordinates.md#coordinatesystem) or [Atlas](coordinates.md#atlas) or NoneType | Sectioning coordinate system (Only required if different from the Procedures.coordinate_system) |
| `sections` | List[[Section](#section) or [PlanarSection](#planarsection)] | Planar sections (Use PlanarSection for new implementations) |
| `section_orientation` | [SectionOrientation](#sectionorientation) | Sectioning orientation |


### Section

Description of a slice of brain tissue
Description of a single section of brain tissue. Slices should use PlanarSection.

| Field | Type | Title (Description) |
|-------|------|-------------|
| `output_specimen_id` | `str` | Specimen ID |
| `targeted_structure` | Optional[[BrainAtlas](../aind_data_schema_models/brain_atlas.md#ccfv3)] | Targeted structure |
| `coordinate_system_name` | `str` | Coordinate system name |
| `start_coordinate` | [Translation](coordinates.md#translation) | Start coordinate |
| `end_coordinate` | Optional[[Translation](coordinates.md#translation)] | End coordinate |
| `thickness` | `Optional[float]` | Slice thickness |
| `thickness_unit` | Optional[[SizeUnit](../aind_data_schema_models/units.md#sizeunit)] | Slice thickness unit |
| `partial_slice` | Optional[List[[AnatomicalRelative](../aind_data_schema_models/coordinates.md#anatomicalrelative)]] | Partial slice (If sectioning does not include the entire slice, indicate which part of the slice is retained.) |
| `includes_surrounding_tissue` | `Optional[bool]` | Includes surrounding tissue (Whether the section includes additional tissue surrounding the targeted structure.) |
| <del>`coordinate_system_name`</del> | `Optional[str]` | **[DEPRECATED]** Use PlanarSection instead. Coordinate system name |
| <del>`start_coordinate`</del> | Optional[[Translation](coordinates.md#translation)] | **[DEPRECATED]** Use PlanarSection instead. Start coordinate |
| <del>`end_coordinate`</del> | Optional[[Translation](coordinates.md#translation)] | **[DEPRECATED]** Use PlanarSection instead. End coordinate |
| <del>`thickness`</del> | `Optional[float]` | **[DEPRECATED]** Use PlanarSection instead. Slice thickness |
| <del>`thickness_unit`</del> | Optional[[SizeUnit](../aind_data_schema_models/units.md#sizeunit)] | **[DEPRECATED]** Use PlanarSection instead. Slice thickness unit |
| <del>`partial_slice`</del> | Optional[List[[AnatomicalRelative](../aind_data_schema_models/coordinates.md#anatomicalrelative)]] | **[DEPRECATED]** Use PlanarSection instead. Partial slice (If sectioning does not include the entire slice, indicate which part of the slice is retained.) |


### SectionOrientation
Expand All @@ -67,6 +85,15 @@ Orientation of sectioning
| `TRANSVERSE` | `Transverse` |


### Sectioning

Description of a sectioning procedure targeting a specific structure

| Field | Type | Title (Description) |
|-------|------|-------------|
| `sections` | List[[Section](#section)] | Sections |


### SpecimenProcedure

Description of surgical or other procedure performed on a specimen
Expand All @@ -75,13 +102,13 @@ Description of surgical or other procedure performed on a specimen
|-------|------|-------------|
| `procedure_type` | [SpecimenProcedureType](../aind_data_schema_models/specimen_procedure_types.md#specimenproceduretype) | Procedure type |
| `procedure_name` | `Optional[str]` | Procedure name |
| `specimen_id` | `str` | Specimen ID |
| `specimen_id` | `str or List[str]` | Specimen ID(s) |
| `start_date` | `datetime.date` | Start date |
| `end_date` | `datetime.date` | End date |
| `experimenters` | `List[str]` | experimenter(s) |
| `protocol_id` | `Optional[List[str]]` | Protocol ID (DOI for protocols.io) |
| `protocol_parameters` | `Optional[Dict[str, str]]` | Protocol parameters (Parameters defined in the protocol and their value during this procedure) |
| `procedure_details` | List[[HCRSeries](#hcrseries) or [FluorescentStain](reagent.md#fluorescentstain) or [PlanarSectioning](#planarsectioning) or [ProbeReagent](reagent.md#probereagent) or [Reagent](reagent.md#reagent) or [GeneProbeSet](reagent.md#geneprobeset)] | Procedure details (Details of the procedures, including reagents and sectioning information.) |
| `procedure_details` | List[[HCRSeries](#hcrseries) or [FluorescentStain](reagent.md#fluorescentstain) or [Sectioning](#sectioning) or [PlanarSectioning](#planarsectioning) or [ProbeReagent](reagent.md#probereagent) or [Reagent](reagent.md#reagent) or [GeneProbeSet](reagent.md#geneprobeset)] | Procedure details (Details of the procedures, including reagents and sectioning information.) |
| `notes` | `Optional[str]` | Notes |


83 changes: 75 additions & 8 deletions src/aind_data_schema/components/specimen_procedures.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
"""Specimen procedures module for AIND data schema."""

import warnings
from datetime import date
from enum import Enum
from typing import Dict, List, Optional
from typing import Dict, List, Optional, Union

from aind_data_schema_models.brain_atlas import BrainStructureModel
from aind_data_schema_models.coordinates import AnatomicalRelative
Expand Down Expand Up @@ -30,10 +31,67 @@ class SectionOrientation(str, Enum):


class Section(DataModel):
"""Description of a slice of brain tissue"""
"""Description of a single section of brain tissue. Slices should use PlanarSection."""

output_specimen_id: str = Field(..., title="Specimen ID")
targeted_structure: Optional[BrainStructureModel] = Field(default=None, title="Targeted structure")
includes_surrounding_tissue: Optional[bool] = Field(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible to indicate which of these fields will become required in v3.0 (similar to how we mark the deprecated fields)? It might just help to get that information so future upgrades will be easier. (and so we remember to make them required)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather track this with tickets #1723

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Users won't see the tickets, so they won't know to include fields that will be required in the future. But, users also don't read the schema ...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to how we do deprecation messages we can also do "v3" messaging so it shows up in the documentation that this will be a required field in the future. If we do that we should do a pass over the whole schema to add those flags everywhere we can find them though. Do you want to open a ticket for that work? We probably have to do it together

default=None,
title="Includes surrounding tissue",
description="Whether the section includes additional tissue surrounding the targeted structure.",
)

coordinate_system_name: Optional[str] = Field(
default=None, title="Coordinate system name", deprecated="Use PlanarSection instead"
)
start_coordinate: Optional[Translation] = Field(
default=None, title="Start coordinate", deprecated="Use PlanarSection instead"
)
end_coordinate: Optional[Translation] = Field(
default=None, title="End coordinate", deprecated="Use PlanarSection instead"
)
thickness: Optional[float] = Field(default=None, title="Slice thickness", deprecated="Use PlanarSection instead")
thickness_unit: Optional[SizeUnit] = Field(
default=None, title="Slice thickness unit", deprecated="Use PlanarSection instead"
)
partial_slice: Optional[List[AnatomicalRelative]] = Field(
default=None,
title="Partial slice",
description="If sectioning does not include the entire slice, indicate which part of the slice is retained.",
deprecated="Use PlanarSection instead",
)

@model_validator(mode="after")
def deprecated_coordinate_fields(self):
"""Warn if deprecated coordinate fields are used"""
deprecated_fields = []
if self.coordinate_system_name is not None:
deprecated_fields.append("coordinate_system_name")
if self.start_coordinate is not None:
deprecated_fields.append("start_coordinate")
if self.end_coordinate is not None:
deprecated_fields.append("end_coordinate")
if self.thickness is not None:
deprecated_fields.append("thickness")
if self.thickness_unit is not None:
deprecated_fields.append("thickness_unit")
if self.partial_slice is not None:
deprecated_fields.append("partial_slice")

if deprecated_fields:
warnings.warn(
(
f"Section fields {deprecated_fields} are deprecated. "
"Use PlanarSection for sections with coordinate data."
),
DeprecationWarning,
)

return self


class PlanarSection(Section):
"""Description of a single planar section of brain tissue"""

# Coordinates
coordinate_system_name: str = Field(..., title="Coordinate system name")
Expand All @@ -54,22 +112,31 @@ def check_one_of_end_thickness(self):

if not self.end_coordinate and not self.thickness:
raise OneOfError(
"Section",
"PlanarError",
["end_coordinate", "thickness"],
)
return self


class PlanarSectioning(DataModel):
class Sectioning(DataModel):
"""Description of a sectioning procedure targeting a specific structure"""

sections: List[Section] = Field(..., title="Sections")


class PlanarSectioning(Sectioning):
"""Description of a sectioning procedure performed on the coronal, sagittal, or transverse/axial plane"""

coordinate_system: Optional[CoordinateSystem | Atlas] = Field(
default=None,
title="Sectioning coordinate system",
description="Only required if different from the Procedures.coordinate_system",
) # note: exact field name is used by a validator
)

sections: List[Union[Section, PlanarSection]] = Field(
..., title="Planar sections", description="Use PlanarSection for new implementations"
)

sections: List[Section] = Field(..., title="Sections")
section_orientation: SectionOrientation = Field(..., title="Sectioning orientation")


Expand Down Expand Up @@ -99,7 +166,7 @@ class SpecimenProcedure(DataModel):

procedure_type: SpecimenProcedureType = Field(..., title="Procedure type")
procedure_name: Optional[str] = Field(default=None, title="Procedure name")
specimen_id: str = Field(..., title="Specimen ID")
specimen_id: Union[str, List[str]] = Field(..., title="Specimen ID(s)")
start_date: date = Field(..., title="Start date")
end_date: date = Field(..., title="End date")
experimenters: List[str] = Field(
Expand All @@ -114,7 +181,7 @@ class SpecimenProcedure(DataModel):
)

procedure_details: DiscriminatedList[
HCRSeries | FluorescentStain | PlanarSectioning | ProbeReagent | Reagent | GeneProbeSet
HCRSeries | FluorescentStain | Sectioning | PlanarSectioning | ProbeReagent | Reagent | GeneProbeSet
] = Field(
default=[],
title="Procedure details",
Expand Down
51 changes: 39 additions & 12 deletions tests/test_procedures.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
""" test Procedures """

import unittest
import warnings
from datetime import date
from unittest.mock import patch

Expand All @@ -24,6 +25,7 @@
)
from aind_data_schema.components.specimen_procedures import (
HCRSeries,
PlanarSection,
PlanarSectioning,
Section,
SectionOrientation,
Expand Down Expand Up @@ -326,6 +328,26 @@ def test_validate_procedure_type_multiple(self):
)
self.assertIn("SpecimenProcedure.procedure_details should only contain one type of model", repr(e.exception))

def test_section_deprecated_coordinate_fields(self):
"""Test that using deprecated coordinate fields in Section raises deprecation warnings"""
with warnings.catch_warnings(record=True) as w:
warnings.simplefilter("always")
section = Section(
output_specimen_id="section1",
coordinate_system_name="CCFv3",
start_coordinate=Translation(translation=[0.5, 1.0, 0.0, 1.0]),
thickness=100.0,
thickness_unit=SizeUnit.UM,
partial_slice=[AnatomicalRelative.LEFT],
)
deprecation_warnings = [warning for warning in w if issubclass(warning.category, DeprecationWarning)]
self.assertGreaterEqual(len(deprecation_warnings), 1)
section_warnings = [warning for warning in deprecation_warnings if "Section fields" in str(warning.message)]
self.assertEqual(len(section_warnings), 1)
self.assertIn("partial_slice", str(section_warnings[0].message))
self.assertIn("PlanarSection", str(section_warnings[0].message))
self.assertEqual(section.output_specimen_id, "section1")

def test_coordinate_volume_validator(self):
"""Test validator for list lengths on BrainInjection"""

Expand Down Expand Up @@ -452,20 +474,25 @@ def test_sectioning(self):
)
self.assertIsNotNone(sectioning_procedure)

valid_section = PlanarSection(
output_specimen_id="123456_001",
coordinate_system_name="BREGMA_ARI",
start_coordinate=Translation(
translation=[0.3, 0, 0, 0],
),
thickness=100.0,
thickness_unit=SizeUnit.UM,
)
self.assertIsNotNone(valid_section)

# Raise error if neither end_coordinate nor thickness is provided
with self.assertRaises(OneOfError):
PlanarSectioning(
coordinate_system=CoordinateSystemLibrary.BREGMA_ARI,
sections=[
Section(
output_specimen_id="123456_001",
coordinate_system_name="BREGMA_ARI",
start_coordinate=Translation(
translation=[0.3, 0, 0],
),
),
],
section_orientation=SectionOrientation.CORONAL,
PlanarSection(
output_specimen_id="123456_001",
coordinate_system_name="BREGMA_ARI",
start_coordinate=Translation(
translation=[0.3, 0, 0],
),
)

def test_validate_subject_specimen_ids(self):
Expand Down