[Bugfix] Fix VectorData height calculation when data is BoundPipe

[ehennestad]

Motivation

Following HDMF PR 1180, datasets written by PyNWB is stored by default as expandable datasets with maxSize = Inf on all dimensions. When MatNWB loads such a dataset, it wraps it in a BoundPipe (DataPipe). If more than one dimension is expandable, BoundPipe cannot reliably infer a single append axis and falls back to the first dimension.

This caused an issue in types.util.dynamictable.checkConfig, which validates DynamicTable objects on construction by checking that all VectorData columns have the same height. For VectorData backed by a BoundPipe, the height was computed from the inferred axis. For multidimensional DynamicTable columns, however, the table row dimension in MatNWB corresponds to the last MATLAB dimension because NWB/HDF5 and MATLAB use reversed dimension order.

As a result, a loaded types.core.PlaneSegmentation.image_mask with MATLAB size 100 x 100 x 30 (HDF5 size 30 x 100 x 100) was assigned a height of 100 instead of the correct height 30, causing checkConfig to fail.

This PR fixes that by computing the height of BoundPipe-backed VectorData from the last MATLAB dimension instead of the inferred axis.

Added unit test:
tests.unit.dynamicTableTest/testCheckConfigUsesLastDimForBoundDataPipe

How to test the behavior?

Opening ophys_tutorial file written by pynwb v3.14

Before:

nwbFile = nwbRead('ophys_tutorial.nwb', 'savedir', '.');

Error using assert
Special column `id` of DynamicTable needs to match the detected height of 100. Found 30 IDs.

Error in types.util.dynamictable.checkConfig (line 60)
    assert(tableHeight == numIds, ...
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in types.core.PlaneSegmentation (line 87)
            types.util.dynamictable.checkConfig(obj);
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in io.createParsedType (line 28)
    typeInstance = feval(typeName, varargin{:}); % Create the type.
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in io.parseGroup (line 86)
    parsed = io.createParsedType(info.Name, Type.typename, kwargs{:});
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in io.parseGroup (line 34)
    subg = io.parseGroup(filename, group, blacklist);
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in io.parseGroup (line 34)
    subg = io.parseGroup(filename, group, blacklist);
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in io.parseGroup (line 34)
    subg = io.parseGroup(filename, group, blacklist);
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in io.parseGroup (line 34)
    subg = io.parseGroup(filename, group, blacklist);
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error in nwbRead (line 99)
        nwb = io.parseGroup(filename, h5info(filename), blackList);
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

After:

nwbFile = nwbRead('ophys_tutorial.nwb', 'savedir', '.');

<no error>

Checklist

• Have you ensured the PR description clearly describes the problem and solutions?
• Have you checked to ensure that there aren't other open or previously closed Pull Requests for the same change?
• If this PR fixes an issue, is the first line of the PR description fix #XX where XX is the issue number?

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.48%. Comparing base (270e871) to head (12b3b25).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #802      +/-   ##
==========================================
- Coverage   95.53%   95.48%   -0.06%     
==========================================
  Files         192      192              
  Lines        7011     7014       +3     
==========================================
- Hits         6698     6697       -1     
- Misses        313      317       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ehennestad]

Add explicit case for empty DataPipe data.

in MATLAB, size([], n) = 1 if n>= 3.

So for nd DataPipe with empty data, the VectorData height/length would be 1 instead of 0.

[Copilot]

Pull request overview

Fixes DynamicTable construction failures when VectorData is backed by a bound DataPipe (i.e., BoundPipe) with multiple extendable dimensions by computing column “height” from the last MATLAB dimension rather than the bound pipe’s inferred append axis.

Changes:

• Update types.util.dynamictable.checkConfig to derive bound DataPipe height from size(data) (last MATLAB dimension).
• Add a unit test reproducing the multi-extendable-dimension bound-pipe scenario.
• Adjust a system test DataPipe configuration for multi-dimensional appending.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
+types/+util/+dynamictable/checkConfig.m	Changes height detection for bound DataPipe-backed VectorData to use MATLAB dimension ordering.
+tests/+unit/dynamicTableTest.m	Adds a unit test to ensure DynamicTable validation succeeds for a multi-dim bound DataPipe.
+tests/+system/DynamicTableTest.m	Updates DataPipe axis/maxSize and sample row shape for multi-dimensional appends.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

This new unit test covers the multi-dimensional bound-pipe case, but the updated checkConfig logic can also affect bound pipes that are effectively vectors stored as a 2-D [N 1] dataset (e.g., created from DataPipe('data', rand(N,1))). Adding a companion assertion/test that a bound [N 1] DataPipe column yields a table height of N would help prevent regressions once the bound-pipe height logic is adjusted.

Suggested change

	function testToTableUsesRowCountForBoundColumnVectorDataPipe(testCase)
	fileName = strrep(testCase.getRandomFilename(), '.nwb', '.h5');
	datasetName = "/column_vector";
	numRows = 10;
	columnVectorData = rand(numRows, 1);
	dataPipe = types.untyped.DataPipe( ...
	'data', columnVectorData, ...
	'maxSize', [Inf, 1], ...
	'chunkSize', [numRows, 1]);
	fileId = H5F.create(fileName);
	dataPipe.export(fileId, datasetName, {});
	H5F.close(fileId);
	boundPipe = testCase.verifyWarning( ...
	@() types.untyped.DataPipe('filename', fileName, 'path', datasetName), ...
	'NWB:BoundPipe:InvalidPipeShape');
	columnVector = types.hdmf_common.VectorData( ...
	'description', 'Column vector data stored as an [N 1] dataset', ...
	'data', boundPipe);
	idColumn = types.hdmf_common.ElementIdentifiers('data', int64((0:numRows-1)'));
	dynamicTable = types.hdmf_common.DynamicTable( ...
	'description', 'test table with bound column vector data pipe', ...
	'colnames', {'column_vector'}, ...
	'column_vector', columnVector, ...
	'id', idColumn);
	convertedTable = dynamicTable.toTable();
	testCase.verifyEqual(size(boundPipe), [numRows, 1]);
	testCase.verifyEqual(height(convertedTable), numRows);
	end