Poster presentation at the Society for Neuroscience Annual Meeting 2018 (527.11/MMM28):

Authors:

*O. RUEBEL¹, A. TRITT², N. H. CAIN³, B. DICHTER⁴, J.-C. FILLION-ROBIN⁵, D. OZTURK⁵, L. M. FRANK⁶, E. F. CHANG⁷, F. T. SOMMER⁸, K. SVOBODA⁹, M. GRAUER⁵, W. SCHROEDER⁵, L. NG³, K. BOUCHARD¹⁰;
¹Computat. Res. Div., ²Lawrence Berkeley Natl. Lab., Berkeley, CA; ³Allen Inst. for Brain Sci., Seattle, WA; ⁴Sanford Univ., Stanford, CA; ⁵Kitware, Inc., Clifton Park, NY; ⁶Dept. of Physiol., UC San Francisco, San Francisco, CA; ⁷Neurosurg., UCSF, San Francisco, CA; ⁸Univ. California, Helen Wills Neurosci Inst., Helen Wills Neurosci. Inst., Berkeley, CA; ⁹HHMI / Janelia Farm Res. Campus, Ashburn, VA; ¹⁰Biol. Systems and Engin., LBNL/UCB, Berkeley, CA

Abstract:

Neurodata Without Borders: Neurophysiology (NWB:N) is an emerging unified data standard for neurophysiology data, focused primarily on the dynamics of groups of neurons measured under a large range of experimental conditions. Here we describe recent advances in the NWB:N data standardization ecosystem with a particular focus on data organization, advanced data input/output (I/O), schema extension, and community building. We have enhanced the NWB:N specification language and extension APIs via support for compound data types, enabling storage of complex data types, e.g., tables. In addition, object/region references provide advanced mechanisms for explicit cross-referencing of data. Together, these methods have enabled significant improvements in the organization of complex metadata in NWB:N, e.g., for describing electrodes and ROIs. To support advanced data I/O needs, we have extended the PyNWB Python API for NWB:N to support optimization of the layout of data arrays on disk via chunking and compression, enabling significantly reduced I/O and storage cost. PyNWB also enables the use and creation of external links, facilitating modular storage and enhanced organization of large data collections across files. Finally, we have extended PyNWB via novel, customizable interfaces for iterative data write with broad applications to data streaming, on-the-fly data generation, and progressive, out-of-core conversion of large data arrays. As part of our community engagement efforts, more than 65 users from more than 20 different laboratories have participated this year in NWB:N development and user training hackathon events organized by Lawrence Berkeley National Laboratory (LBNL), the Allen Institute for Brain Science, and Kitware. Here we demonstrate the application of our system to diverse neurophysiology use cases by the Frank Lab (USCF), Chang Lab (UCSF), Bouchard Lab (LBNL) and the Allen Institute for Brain Science among others. The NWB:N software ecosystem empowers users to easily access, use, and analyze NWB:N data, integrate NWB:N with user code bases, and develop new extensions for NWB:N. This work builds the critical foundation towards advanced methods for data management, provenance, query/discovery, and advanced, high-performance visualization and analysis.