Skip to main content

2024 | OriginalPaper | Buchkapitel

Clustering-Friendly Representation Learning for Enhancing Salient Features

verfasst von : Toshiyuki Oshima, Kentaro Takagi, Kouta Nakata

Erschienen in: Advances in Knowledge Discovery and Data Mining

Verlag: Springer Nature Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Recently, representation learning with contrastive learning algorithms has been successfully applied to challenging unlabeled datasets. However, these methods are unable to distinguish important features from unimportant ones under simply unsupervised settings, and definitions of importance vary according to the type of downstream task or analysis goal, such as the identification of objects or backgrounds. In this paper, we focus on unsupervised image clustering as the downstream task and propose a representation learning method that enhances features critical to the clustering task. We extend a clustering-friendly contrastive learning method and incorporate a contrastive analysis approach, which utilizes a reference dataset to separate important features from unimportant ones, into the design of loss functions. Conducting an experimental evaluation of image clustering for three datasets with characteristic backgrounds, we show that for all datasets, our method achieves higher clustering scores compared with conventional contrastive analysis and deep clustering methods.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Abid, A., Zhang, M.J., Bagaria, V.K., Zou, J.: Exploring patterns enriched in a dataset with contrastive principal component analysis. Nat. Commun. 9, 2134 (2018)CrossRef Abid, A., Zhang, M.J., Bagaria, V.K., Zou, J.: Exploring patterns enriched in a dataset with contrastive principal component analysis. Nat. Commun. 9, 2134 (2018)CrossRef
3.
Zurück zum Zitat Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. In: NeurIPS, pp. 9912–9924. Curran Associates, Inc. (2020) Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. In: NeurIPS, pp. 9912–9924. Curran Associates, Inc. (2020)
4.
Zurück zum Zitat Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: ICML, pp. 1597–1607. PMLR (2020) Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: ICML, pp. 1597–1607. PMLR (2020)
5.
Zurück zum Zitat Chen, X., He, K.: Exploring simple siamese representation learning. In: CVPR, pp. 15750–15758. IEEE (2021) Chen, X., He, K.: Exploring simple siamese representation learning. In: CVPR, pp. 15750–15758. IEEE (2021)
6.
Zurück zum Zitat Dang, Z., Deng, C., Yang, X., Wei, K., Huang, H.: Nearest neighbor matching for deep clustering. In: CVPR, pp. 13693–13702 (2021) Dang, Z., Deng, C., Yang, X., Wei, K., Huang, H.: Nearest neighbor matching for deep clustering. In: CVPR, pp. 13693–13702 (2021)
7.
Zurück zum Zitat Deng, L.: The mnist database of handwritten digit images for machine learning research. IEEE Signal Process. Mag. 29(6), 141–142 (2012)CrossRef Deng, L.: The mnist database of handwritten digit images for machine learning research. IEEE Signal Process. Mag. 29(6), 141–142 (2012)CrossRef
8.
Zurück zum Zitat Dirie, A.H., Abid, A., Zou, J.: Contrastive multivariate singular spectrum analysis. In: 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1122–1127 (2019) Dirie, A.H., Abid, A., Zou, J.: Contrastive multivariate singular spectrum analysis. In: 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1122–1127 (2019)
9.
Zurück zum Zitat Ge, R., Zou, J.: Rich component analysis. In: ICML, pp. 1502–1510. PMLR (2016) Ge, R., Zou, J.: Rich component analysis. In: ICML, pp. 1502–1510. PMLR (2016)
10.
Zurück zum Zitat Li, Y., Hu, P., Liu, Z., Peng, D., Zhou, J.T., Peng, X.: Contrastive clustering. In: AAAI, pp. 8547–8555. AAAI Press (2021) Li, Y., Hu, P., Liu, Z., Peng, D., Zhou, J.T., Peng, X.: Contrastive clustering. In: AAAI, pp. 8547–8555. AAAI Press (2021)
11.
Zurück zum Zitat Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV) (2015) Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV) (2015)
12.
Zurück zum Zitat McInnes, L., Healy, J., Melville, J.: UMAP: uniform manifold approximation and projection for dimension reduction. arXiv:1802.03426 (2018) McInnes, L., Healy, J., Melville, J.: UMAP: uniform manifold approximation and projection for dimension reduction. arXiv:​1802.​03426 (2018)
14.
Zurück zum Zitat Park, S., et al.: Improving unsupervised image clustering with robust learning. In: CVPR (2021) Park, S., et al.: Improving unsupervised image clustering with robust learning. In: CVPR (2021)
15.
Zurück zum Zitat Piosenka, G.: Birds 400 - species image classification (2022) Piosenka, G.: Birds 400 - species image classification (2022)
16.
Zurück zum Zitat Rougetet, A.: Landscape pictures (2020) Rougetet, A.: Landscape pictures (2020)
17.
Zurück zum Zitat Shota, M., Yukako, T.: Application of contrastive representation learning to unsupervised defect classification in semiconductor manufacturing. In: AEC/APC Symposium Asia 2021 (2021) Shota, M., Yukako, T.: Application of contrastive representation learning to unsupervised defect classification in semiconductor manufacturing. In: AEC/APC Symposium Asia 2021 (2021)
18.
Zurück zum Zitat Tao, Y., Takagi, K., Nakata, K.: Clustering-friendly representation learning via instance discrimination and feature decorrelation. In: ICLR (2021) Tao, Y., Takagi, K., Nakata, K.: Clustering-friendly representation learning via instance discrimination and feature decorrelation. In: ICLR (2021)
19.
Zurück zum Zitat Tsai, T.W., Li, C., Zhu, J.: Mice: mixture of contrastive experts for unsupervised image clustering. In: ICLR (2021) Tsai, T.W., Li, C., Zhu, J.: Mice: mixture of contrastive experts for unsupervised image clustering. In: ICLR (2021)
21.
Zurück zum Zitat Wu, Z., Xiong, Y., Yu, S.X., Lin, D.: Unsupervised feature learning via non-parametric instance discrimination. In: CVPR, pp. 3733–3742. IEEE (2018) Wu, Z., Xiong, Y., Yu, S.X., Lin, D.: Unsupervised feature learning via non-parametric instance discrimination. In: CVPR, pp. 3733–3742. IEEE (2018)
22.
Zurück zum Zitat Li, Y., Yang, M., Peng, D., Li, T., Huang, J., Peng, X.: Twin contrastive learning for online clustering. Int. J. Comput. Vision 130(9), 2205–2221 (2022)CrossRef Li, Y., Yang, M., Peng, D., Li, T., Huang, J., Peng, X.: Twin contrastive learning for online clustering. Int. J. Comput. Vision 130(9), 2205–2221 (2022)CrossRef
23.
Zurück zum Zitat Zhou, S., et al.: A comprehensive survey on deep clustering: taxonomy, challenges, and future directions (2022) Zhou, S., et al.: A comprehensive survey on deep clustering: taxonomy, challenges, and future directions (2022)
Metadaten
Titel
Clustering-Friendly Representation Learning for Enhancing Salient Features
verfasst von
Toshiyuki Oshima
Kentaro Takagi
Kouta Nakata
Copyright-Jahr
2024
Verlag
Springer Nature Singapore
DOI
https://doi.org/10.1007/978-981-97-2242-6_17

Premium Partner