Automation of Sample Analysis for Scientific Research Biolaboratory

The client is a biotechnological laboratory based at a major university, with a focus on tissue engineering and biotechnologies. The client has a vast amount of raw data obtained from spectrophotometers, confocal microscopes, EEG/ECG/spirometry of laboratory animals, and other unprocessed results. As part of scientific grants, the laboratory collaborates with several universities from the USA and Europe.

Client's Main Requests: Development and implementation of a fully automated process for analyzing stained tissue samples. A typical task involves a researcher in one country uploading 4,000 microphotographs into the system, and a researcher in another country receiving a finished document with detailed statistical analysis and data on the studied sample. Implementation of JupiterHub for performing routine tasks. Ensure primary migration from Matlab and GNU Octave to a stack consisting of NumPy, SciPy, matplotlib/seaborn.

To address the issue, we involved an expert from our team who had previously worked in a scientific laboratory with similar tasks. The main challenge faced by the client was the correct segmentation of areas that were simultaneously stained with multiple dyes, followed by morphometric analysis and the labor-intensive manual data processing. We suggested the use of a color deconvolution method based on the paper "Ruifrok AC, Johnston DA. Quantification of histochemical staining by color deconvolution" implemented in Python code.

The proposed MVP produced results that fully satisfied the client in terms of segmentation quality, specificity, and sensitivity. We optimized the final solution for GPU use, significantly enhancing performance. Based on this solution, we established a workflow with automatic processing of samples from the uploaded pool and exporting the obtained pandas dataframes as processed tabular data. Additionally, we made several improvements for detecting objects with specific parameters in the images to enrich statistical data.
The second major challenge was the initial adaptation of the client's toolset from GNU Octave to the Python stack consisting of NumPy, SciPy for collaborative work in JupiterHub. The client decided to change the stack since a limited number of employees were proficient in Matlab, complicating the joint use of the developed toolkit. We deployed a JupiterHub installation on a Kubernetes cluster with quota allocation through PersistentVolumeClaims. To facilitate migration, we compiled and analyzed a series of typical data processing tasks previously solved with Octave, for example, typical tasks for analyzing raw data with cyclo-periodic tests of materials on the Instron 8801 device, spectral analysis of electromyogram recordings. We developed documentation, standard Docker images with the proper environment, and helped rewrite typical tasks using NumPy, SciPy libraries. For example, for Fast Fourier Transform of unprocessed data requiring spectral analysis.
As a result of our work:

1. Reduced labor costs of laboratory personnel on routine project tasks by 27%.
2. Fully automated pipeline from microphotograph upload to making analyzed sample data available for the teams, including quasi-proteomic analysis.
3. Provided migration of the main toolkit to more common standard Python libraries, enabling more scientists to participate in their development.
4. Prepared a platform for launching JupiterNotebook with a ready-to-use environment, understandable to researchers needing data analysis but not developers.