부산대학교 도서관

The BRAIN platform that we developed over the last 2.5 years is a collaborative platform for the neuroscience research community. Such platforms are crucial for expediting research in data-heavy research communities such as the neuroscience optical imaging community. However, it is important to also build-in quality assurance within the platform to ensure that the results can be trusted. Testing the accuracy of scientific algorithms by comparing the actual results to the expected requires tedious manual efforts, which typically has to be repeated for each new context. Metamorphic testing has been shown to be a powerful technique for scientific data analysis applications where asserting correctness is usually a manual, laborious, and challenge task. It uses relations between a set of inputs and a set of outputs to detect issues instead of comparing actual and expected results and thus addresses the above problem. In this paper we report on a feasibility study where we applied metamorphic testing to one step in the calcium imaging pipeline analysis, namely the step to extract the locations of the neuron cells. We tested the cell registration module of CaImAn, one popular software library for calcium imaging analysis, for robustness against slight changes in the input images, i.e. we applied linear transformations such as small degrees of rotation and blurring to the input images, and then evaluated the neurons identified by the cell registration module for both the original and the transformed inputs. Our results show that for transformations such as rotation and translation, CaImAn’s cell registration unexpectedly identified more neurons for the transformed input than for the original image. Meanwhile, for transformations such as padding, CaImAn identified fewer neurons when run on the the transformed inputs. Our results demonstrate the need for evaluating third party libraries for confidence. Furthermore, it also demonstrates that metamorphic testing is a viable quality assurance technique for the BRAIN platform as well as for similar scientific applications where it is difficult to detect analysis inaccuracies.