Annotating Time Intervals ⏱️

Overview

AqNWB provides the TimeIntervals type for recording time intervals (such as experimental epochs, trials, or invalid times), where each interval is defined by a start_time and a stop_time. Additional user-defined columns can be appended to store any per-interval metadata.

NWB provides several standard TimeIntervals tables that can be created via convenience methods on NWBFile:

• epochs: NWBFile::createEpochs for storing time intervals marking coarse-grained experimental phases or subdivisions of a recording session, such as baseline, task, rest, or sleep stages
• trials: NWBFile::createTrials for storing time intervals corresponding to repeated experimental units with consistent structure, such as individual stimulus-response-reward cycles.
• invalid_times: NWBFile::createInvalidTimes for indicating time intervals that should be removed from analysis.

AqNWB supports two complementary acquisition patterns:

Pattern	When to use
Row-based (addRow / addRows)	Intervals arrive one at a time during acquisition; the total count is not known in advance.
Column-based (write full column vectors)	All interval data are available in memory at once and can be written in a single bulk call per column. Add additional complete columns after acquisition is completed.

Both patterns share the same setup steps (I/O, NWBFile, table creation, and startRecording) and differ only in how data are written to the table. The two patterns are not exclusive and hybrid approaches are possible, but require caution. In particular when using column-based append, we must ensure that all columns have been appropriately updated such that all columns have indeed the same number of rows before we use addRow or addRows to avoid inconsistent data.

Note

Chunked storage is required for when the total number of rows is not known in advance and we need to expand the columns at runtime. The rowChunkSize parameter of the createDefaultDataSpecs controls how many rows are stored per HDF5 chunk. In practice, the number of rows per chunk should typically be the same for all columns of a table. The chunking behavior can also be customized via the individual DataSpec of the columns. Avoid creating too small chunks (e.g,. single rows) as well as too large chunks (e.g., millions of rows per chunk).

Note

TimeIntervals is intended for storing general annotations of time ranges. Depending on the application (e.g., when intervals are generated by data acquisition or automatic data processing), it can be useful to describe intervals (or instantaneous events) in time as an EventsTable.

Setup: I/O, NWBFile, and table creation

1. Create the I/O object

    std::shared_ptr<BaseIO> io = createIO("HDF5", path);
    io->open();

2. Create the NWBFile

    auto nwbfile = NWB::NWBFile::create(io);
    auto subjectSpec =
        AQNWB::NWB::Subject::SubjectSpec()
            .withSubjectId("mouse001")
            .withSpecies("Mus musculus")
            .withSex("M")
            .withAge("P90D")
            .withDescription(
                "Wild type mouse used for electrophysiology study");
    std::string currentTime = getCurrentTime();
    Status initStatus = nwbfile->initialize(generateUuid(),
                                            "a recording session",
                                            "data collection info",
                                            currentTime,
                                            currentTime,
                                            subjectSpec);
    AQNWB::checkStatus(initStatus, "NWBFile initialization");

3. Configure and create the TimeIntervals table

Before creating the table, build a list of column specifications using TimeIntervals::createDefaultDataSpecs. This factory returns the default set of columns (always including the id, start_time, and stop_time columns) and allows you to decide to include the optional tags column and control the HDF5 chunk size used for each column.

To add a custom column to the table, create a VectorData::DataSpec (or a spec for any other VectorData subtype) and push it onto the spec vector before passing it to NWBFile::createTimeIntervals. The spec bundles the column name, dataset configuration (data type, initial size, chunk size), and description in one object. Once the spec list is ready, pass it to NWBFile::createTimeIntervals together with the table name and description:

    // Create a TimeIntervals table for trials. The table will live at
    // /intervals/trials inside the NWB file. We request the optional tags
    // column.
    auto columnSpecs = NWB::TimeIntervals::createDefaultDataSpecs(
        NWB::NWBFile::INTERVALS_PATH + "/trials",
        100,  // number of rows in a chunk (chunked storage is required to
              // support append when the total number of rows is not known in
              // advance)
        true);  // create tags column
 
    // Optionally add a custom column to the spec list before creating the
    // table. Here we add a string "condition" column with a chunk size of 100
    // rows.
    IO::ArrayDataSetConfig conditionConfig(
        BaseDataType::V_STR,  // data type
        SizeArray {0},  // initial size (0 = extensible)
        SizeArray {100});  // chunk size
    columnSpecs.push_back(NWB::VectorData::createDataSpec(
        "condition", conditionConfig, "Trial condition label."));
 
    auto trialsTable = nwbfile->createTimeIntervals(
        "trials",  // name of the table
        "Experimental trials.",  // description
        columnSpecs);  // pre-built column spec list

Note

If you do not require custom columns (or want to create them later via addColumn), then you can bypass the creation of the table configuration and instead configure the table directly via NWBFile::createTimeIntervals or the convenience methods NWBFile::createEpochs, NWBFile::createTrials, and NWBFile::createInvalidTimes.

4. Start the recording

    Status startStatus = io->startRecording();

Note

When using HDF5IO, calling startRecording enables SWMR mode by default. No new datasets or groups can be added after this point unless the file is closed and reopened.

Pattern 1: Row-based acquisition

Use this pattern when intervals are detected one at a time during an ongoing recording session and the total interval count is not known in advance.

Add rows

Call addRow to append a single interval, or addRows to append several intervals at once. Each row is expressed as a DynamicTable::RowData, which is an std::unordered_map<std::string, CellValue> that maps column names to typed scalar or vector values. Row IDs are auto-generated (0, 1, 2, …) when not explicitly supplied.

    // During acquisition, append individual rows, each
    // representing a detected interval using addRow().
    // Each row is a map from column name to value.  The row ID is
    // auto-generated (0, 1, 2, …) when not supplied.
    // All columns — including custom ones — must be provided for every row.
    AQNWB::Types::RowData row0 = {
        {"start_time", 0.123f},
        {"stop_time", 1.456f},
        {"tags", std::vector<std::string> {"correct", "fast"}},
        {"condition", std::string("visual_stimulus")}};
    Status s0 = trialsTable->addRow(row0);
 
    // We can also append multiple rows at once using addRows().
    std::vector<AQNWB::Types::RowData> moreRows = {
        {{"start_time", 2.123f},
         {"stop_time", 3.456f},
         {"tags", std::vector<std::string> {"incorrect"}},
         {"condition", std::string("auditory_stimulus")}},
        {{"start_time", 4.123f},
         {"stop_time", 5.456f},
         {"tags", std::vector<std::string> {"correct", "slow"}},
         {"condition", std::string("visual_stimulus")}}};
    Status s1 = trialsTable->addRows(moreRows);
 
    io->flush();  // optional, flush data to disk

Note

addRow and addRows write data immediately to the underlying HDF5 datasets by extending the chunked arrays one row (or batch of rows) at a time. Call io->flush() at any point to ensure data are moved to disk.

Stop the recording

    io->stopRecording();
    io->close();

Pattern 2: Column-based (bulk) acquisition

Use this pattern when all interval data are collected in memory first and can be written to the file in a single pass. This is common for post-acquistion or offline processing pipelines.

Add data to columns

After startRecording, retrieve each column object via the corresponding record*() accessor on the table, then call writeDataBlock to write the full column vector in one call. Finally, set the row IDs with setRowIDs.

    // Create a TimeIntervals table for epochs.
    auto columnSpecs = NWB::TimeIntervals::createDefaultDataSpecs(
        NWB::NWBFile::INTERVALS_PATH + "/epochs",
        100,  // row chunk size
        false);  // no tags column
 
    auto epochsTable = nwbfile->createTimeIntervals(
        "epochs", "Experimental epochs.", columnSpecs);

Custom columns that were not part of the original spec list can be added after table creation but before startRecording() using addColumn. Create a VectorData object, initialize it with an ArrayDataSetConfig, and pass it to addColumn:

    // Custom columns can also be added after table creation but before
    // startRecording() using addColumn(DataSpecPtr).  This is the recommended
    // approach and is consistent with the DataSpec-based approach used by
    // initialize(). The alternative approach would be to create VectorData
    // columns directly and then adding them via addColumn().
    auto labelSpec = NWB::VectorData::createDataSpec(
        "label",
        IO::ArrayDataSetConfig(BaseDataType::V_STR,  // variable-length string
                               SizeArray {0},  // initial size (0 = extensible)
                               SizeArray {100}),  // chunk size
        "Epoch label.");
    Status addColStatus = epochsTable->addColumn(labelSpec);
    REQUIRE(addColStatus == Status::Success);

    Status startStatus = io->startRecording();

Write all columns — including any custom ones — after startRecording:

    // After the recording session, write all interval data as full columns in a
    // single call per column.  This is efficient when the complete dataset is
    // available in memory at write time.
    std::vector<float> startTimes = {0.100f, 10.600f, 20.100f};
    std::vector<float> stopTimes = {10.000f, 20.000f, 30.000f};
    std::vector<std::string> labels = {"baseline", "stimulation", "recovery"};
    std::vector<int> rowIds = {0, 1, 2};
 
    SizeArray dataShape = {startTimes.size()};
    SizeArray positionOffset = {0};
 
    // Write the start_time column
    auto startTimeColumn = epochsTable->readStartTime();
    Status startStatusWrite = startTimeColumn->recordData()->writeDataBlock(
        dataShape, positionOffset, BaseDataType::F32, startTimes.data());
    REQUIRE(startStatusWrite == Status::Success);
 
    // Write the stop_time column
    auto stopTimeColumn = epochsTable->readStopTime();
    Status stopStatusWrite = stopTimeColumn->recordData()->writeDataBlock(
        dataShape, positionOffset, BaseDataType::F32, stopTimes.data());
    REQUIRE(stopStatusWrite == Status::Success);
 
    // Write the custom "label" column
    auto labelCol = epochsTable->readColumn<NWB::VectorData>("label");
    Status labelStatusWrite = labelCol->recordData()->writeDataBlock(
        dataShape, positionOffset, BaseDataType::V_STR, labels);
    REQUIRE(labelStatusWrite == Status::Success);
 
    // Write the row IDs
    Status idStatus = epochsTable->setRowIDs(rowIds);
    REQUIRE(idStatus == Status::Success);
 
    io->flush();  // optional, flush data to disk

Stop the recording

    io->stopRecording();
    io->close();

Further reading

• Reading DynamicTable. See the Reading DynamicTable section in the reading tutorial for details on how to read data from a TimeIntervals table.
• Tips and best practices. See the Tips and best practices section in the Events tutorial for general advice on chunking, writing, and customizing tables, which apply equally to TimeIntervals.