Group Normalization Research Articles

Normalization in Training U-Net for 2-D Biomedical Semantic Segmentation

2D biomedical semantic segmentation is important for robotic vision in surgery. Segmentation methods based on Deep Convolutional Neural Network (DCNN) can out-perform conventional methods in terms of both accuracy and levels of automation. One common issue in training a DCNN for biomedical semantic segmentation is the internal covariate shift where the training of convolutional kernels is encumbered by the distribution change of input features, hence both the training speed and performance are decreased. Batch Normalization (BN) is the first proposed method for addressing internal covariate shift and is widely used. Instance Normalization (IN) and Layer Normalization (LN) have also been proposed. Group Normalization (GN) is proposed more recently and has not yet been applied to 2D biomedical semantic segmentation, however, no specific validations on GN were given. Most DCNNs for biomedical semantic segmentation adopt BN as the normalization method by default, without reviewing its performance. In this paper, four normalization methods - BN, IN, LN and GN are compared in details, specifically for 2D biomedical semantic segmentation. U-Net is adopted as the basic DCNN structure. Three datasets regarding the Right Ventricle (RV), aorta, and Left Ventricle (LV) are used for the validation. The results show that detailed subdivision of the feature map, i.e. GN with a large group number or IN, achieves higher accuracy. This accuracy improvement mainly comes from better model generalization. Codes are uploaded and maintained at Xiao-Yun Zhou's Github.

Group Normalization for Genomic Data

Data normalization is a crucial preliminary step in analyzing genomic datasets. The goal of normalization is to remove global variation to make readings across different experiments comparable. In addition, most genomic loci have non-uniform sensitivity to any given assay because of variation in local sequence properties. In microarray experiments, this non-uniform sensitivity is due to different DNA hybridization and cross-hybridization efficiencies, known as the probe effect. In this paper we introduce a new scheme, called Group Normalization (GN), to remove both global and local biases in one integrated step, whereby we determine the normalized probe signal by finding a set of reference probes with similar responses. Compared to conventional normalization methods such as Quantile normalization and physically motivated probe effect models, our proposed method is general in the sense that it does not require the assumption that the underlying signal distribution be identical for the treatment and control, and is flexible enough to correct for nonlinear and higher order probe effects. The Group Normalization algorithm is computationally efficient and easy to implement. We also describe a variant of the Group Normalization algorithm, called Cross Normalization, which efficiently amplifies biologically relevant differences between any two genomic datasets.

Synthesis and Evaluation of Insulin−Human Serum Albumin Conjugates

A series of human insulin maleimido derivatives with short and long linkers was synthesized by exploiting the variations in the pK(a) values and environment of the three amino groups present in the protein. The syntheses were accomplished in organic solvent because of maleimide's instability in basic aqueous media. The derivatives thus obtained were conjugated to the free thiol on Cys34 of human serum albumin (HSA) and purified. A structure-activity relationship based on in vitro receptor binding and activation results for this series of insulin-HSA conjugates showed that the best compounds were attached at the B1 position of insulin with either short or long linkers. Two conjugates were administered subcutaneously to streptozotocin-induced diabetic rats and found to possess blood glucose normalizing activity up to 8 h post-administration. The return to diabetic plasma glucose levels was not observed within the time frame of the experiment (48 h). In comparison, the insulin-treated group's normalization activity lasted 2 h and returned to a diabetic level at 8 h. The onset of the conjugate activities were delayed by 1 h when compared to the activity of human insulin. The study results led to the identification of CJC-1575 as a potent and long lasting human insulin analogue.