Reducing Objective Function Mismatch in Deep Clustering with the Unsupervised Companion Objective

Daniel J Trosten,Michael C Kampffmeyer,Robert Jenssen

doi:10.7557/18.5709

Abstract

Preservation of local similarity structure is a key challenge in deep clustering. Many recent deep clustering methods therefore use autoencoders to help guide the model's neural network towards an embedding which is more reflective of the input space geometry. However, recent work has shown that autoencoder-based deep clustering models can suffer from objective function mismatch (OFM). In order to improve the preservation of local similarity structure, while simultaneously having a low OFM, we develop a new auxiliary objective function for deep clustering. Our Unsupervised Companion Objective (UCO) encourages a consistent clustering structure at intermediate layers in the network -- helping the network learn an embedding which is more reflective of the similarity structure in the input space. Since a clustering-based auxiliary objective has the same goal as the main clustering objective, it is less prone to introduce objective function mismatch between itself and the main objective. Our experiments show that attaching the UCO to a deep clustering model improves the performance of the model, and exhibits a lower OFM, compared to an analogous autoencoder-based model.

Highlights

Preservation of local similarity structure is a key time series [18]
In order the unsupervised clustering loss is attached to the to improve the preservation of local similarity struc- code space of the autoencoder, and the model is ture, while simultaneously having a low objective function mismatch (OFM), we fine tuned using either the clustering loss alone, or develop a new auxiliary objective function for deep both the clustering loss, and the reconstruction loss clustering
Our experiments show that attaching the UCO to a deep clustering model improves the performance of the model, and exhibits a lower OFM, compared to an analogous autoencoder-based model

Summary

Introduction

Preservation of local similarity structure is a key time series [18]. The ever growing amount of unlachallenge in deep clustering. Recent work has shown that deep neural networks that have been pre-trained autoencoder-based deep clustering models can suffer as autoencoders [20, 1, 21, 10]. In these models, from objective function mismatch (OFM). In order the unsupervised clustering loss is attached to the to improve the preservation of local similarity struc- code space of the autoencoder, and the model is ture, while simultaneously having a low OFM, we fine tuned using either the clustering loss alone, or develop a new auxiliary objective function for deep both the clustering loss, and the reconstruction loss clustering.

Methods

Results

Conclusion