Skip to main content
Top

2024 | OriginalPaper | Chapter

Instance-Ambiguity Weighting for Multi-label Recognition with Limited Annotations

Authors : Daniel Shrewsbury, Suneung Kim, Young-Eun Kim, Heejo Kong, Seong-Whan Lee

Published in: Advances in Knowledge Discovery and Data Mining

Publisher: Springer Nature Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Multi-label recognition with limited annotations has been gaining attention recently due to the costs of thorough dataset annotation. Despite significant progress, current methods for simulating partial labels utilize a strategy that uniformly omits labels, which inadequately prepares models for real-world inconsistencies and undermines their generalization performance. In this paper, we consider a more realistic partial label setting that correlates label absence with an instance’s ambiguity, and propose the novel Ambiguity-Aware Instance Weighting (AAIW) to specifically address the performance decline caused by such ambiguous instances. This strategy dynamically modulates instance weights to prioritize learning from less ambiguous instances initially, then gradually increasing the weight of complex examples without the need for predetermined sequencing of data. This adaptive weighting not only facilitates a more natural learning progression but also enhances the model’s ability to generalize from increasingly complex patterns. Experiments on standard multi-label recognition benchmarks demonstrate the advantages of our approach over state-of-the-art methods.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Ahmad, M., Lee, S.W.: Human action recognition using multi-view image sequences. In: ICAFGR, pp. 523 – 528 (2006) Ahmad, M., Lee, S.W.: Human action recognition using multi-view image sequences. In: ICAFGR, pp. 523 – 528 (2006)
2.
go back to reference Baruch, E.B., et al.: Asymmetric loss for multi-label classification. In: ICCV, pp. 82–91 (2020) Baruch, E.B., et al.: Asymmetric loss for multi-label classification. In: ICCV, pp. 82–91 (2020)
3.
go back to reference Ben-Baruch, E., et al.: Multi-label classification with partial annotations using class-aware selective loss. In: CVPR, pp. 4754–4762 (2021) Ben-Baruch, E., et al.: Multi-label classification with partial annotations using class-aware selective loss. In: CVPR, pp. 4754–4762 (2021)
4.
go back to reference Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: ICML, pp. 41–48 (2009) Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: ICML, pp. 41–48 (2009)
5.
go back to reference Chen, M., Zheng, A.X., Weinberger, K.Q.: Fast image tagging. In: ICML (2013) Chen, M., Zheng, A.X., Weinberger, K.Q.: Fast image tagging. In: ICML (2013)
6.
go back to reference Chen, T., Pu, T., Wu, H., Xie, Y., Lin, L.: Structured semantic transfer for multi-label recognition with partial labels. In: AAAI, pp. 339–346 (2022) Chen, T., Pu, T., Wu, H., Xie, Y., Lin, L.: Structured semantic transfer for multi-label recognition with partial labels. In: AAAI, pp. 339–346 (2022)
7.
go back to reference Chen, T., Xu, M., Hui, X., Wu, H., Lin, L.: Learning semantic-specific graph representation for multi-label image recognition. In: ICCV, pp. 522–531 (2019) Chen, T., Xu, M., Hui, X., Wu, H., Lin, L.: Learning semantic-specific graph representation for multi-label image recognition. In: ICCV, pp. 522–531 (2019)
8.
go back to reference Chen, Z.M., Wei, X.S., Wang, P., Guo, Y.: Multi-label image recognition with graph convolutional networks. In: CVPR (2019) Chen, Z.M., Wei, X.S., Wang, P., Guo, Y.: Multi-label image recognition with graph convolutional networks. In: CVPR (2019)
9.
go back to reference Cole, E., Mac Aodha, O., Lorieul, T., Perona, P., Morris, D., Jojic, N.: Multi-label learning from single positive labels. In: CVPR (2021) Cole, E., Mac Aodha, O., Lorieul, T., Perona, P., Morris, D., Jojic, N.: Multi-label learning from single positive labels. In: CVPR (2021)
10.
go back to reference Ding, Z., et al.: Exploring structured semantic prior for multi label recognition with incomplete labels. In: CVPR, pp. 3398–3407 (2023) Ding, Z., et al.: Exploring structured semantic prior for multi label recognition with incomplete labels. In: CVPR, pp. 3398–3407 (2023)
12.
go back to reference He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: CVPR (2020) He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: CVPR (2020)
13.
go back to reference He, S., Guo, T., Dai, T., Qiao, R., Shu, X., Ren, B., Xia, S.T.: Open-vocabulary multi-label classification via multi-modal knowledge transfer. In: AAAI (2023) He, S., Guo, T., Dai, T., Qiao, R., Shu, X., Ren, B., Xia, S.T.: Open-vocabulary multi-label classification via multi-modal knowledge transfer. In: AAAI (2023)
14.
go back to reference Huynh, D., Elhamifar, E.: Interactive multi-label CNN learning with partial labels. In: CVPR (2020) Huynh, D., Elhamifar, E.: Interactive multi-label CNN learning with partial labels. In: CVPR (2020)
15.
go back to reference Kapoor, A., Jain, P., Viswanathan, R.: Multilabel classification using bayesian compressed sensing. In: NeurIPS, pp. 2645–2653 (2012) Kapoor, A., Jain, P., Viswanathan, R.: Multilabel classification using bayesian compressed sensing. In: NeurIPS, pp. 2645–2653 (2012)
16.
go back to reference Kim, Y., Kim, J., Akata, Z., Lee, J.: Large loss matters in weakly supervised multi-label classification. In: CVPR, pp. 14136–14145 (2022) Kim, Y., Kim, J., Akata, Z., Lee, J.: Large loss matters in weakly supervised multi-label classification. In: CVPR, pp. 14136–14145 (2022)
17.
go back to reference Kim, Y., Kim, J.M., Jeong, J., Schmid, C., Akata, Z., Lee, J.: Bridging the gap between model explanations in partially annotated multi-label classification. In: CVPR, pp. 3408–3417 (2023) Kim, Y., Kim, J.M., Jeong, J., Schmid, C., Akata, Z., Lee, J.: Bridging the gap between model explanations in partially annotated multi-label classification. In: CVPR, pp. 3408–3417 (2023)
18.
go back to reference Krishna, R., et al.: Visual genome: connecting language and vision using crowdsourced dense image annotations. Int. J. Comput. Vis. 123(1), 32–73 (2017)MathSciNetCrossRef Krishna, R., et al.: Visual genome: connecting language and vision using crowdsourced dense image annotations. Int. J. Comput. Vis. 123(1), 32–73 (2017)MathSciNetCrossRef
19.
go back to reference Lee, M.S., Yang, Y.M., Lee, S.W.: Automatic video parsing using shot boundary detection and camera operation analysis. Pattern Recogn. 34, 711–719 (2001)CrossRef Lee, M.S., Yang, Y.M., Lee, S.W.: Automatic video parsing using shot boundary detection and camera operation analysis. Pattern Recogn. 34, 711–719 (2001)CrossRef
20.
go back to reference Lin, T.Y., et al.: Microsoft coco: common objects in context. In: ECCV, pp. 740–755 (2014) Lin, T.Y., et al.: Microsoft coco: common objects in context. In: ECCV, pp. 740–755 (2014)
21.
go back to reference Litrico, M., Del Bue, A., Morerio, P.: Guiding pseudo-labels with uncertainty estimation for source-free unsupervised domain adaptation. In: CVPR (2023) Litrico, M., Del Bue, A., Morerio, P.: Guiding pseudo-labels with uncertainty estimation for source-free unsupervised domain adaptation. In: CVPR (2023)
22.
go back to reference Liu, F., Xiang, T., Hospedales, T.M., Yang, W., Sun, C.: Semantic regularisation for recurrent image annotation. In: CVPR, pp. 4160–4168 (2016) Liu, F., Xiang, T., Hospedales, T.M., Yang, W., Sun, C.: Semantic regularisation for recurrent image annotation. In: CVPR, pp. 4160–4168 (2016)
23.
go back to reference Liu, W., Wang, H., Shen, X., Tsang, I.W.H.: The emerging trends of multi-label learning. IEEE Trans. Pattern Anal. Mach. Intell. 44, 7955–7974 (2020)CrossRef Liu, W., Wang, H., Shen, X., Tsang, I.W.H.: The emerging trends of multi-label learning. IEEE Trans. Pattern Anal. Mach. Intell. 44, 7955–7974 (2020)CrossRef
24.
go back to reference Mitchell, H.B., Schaefer, P.A.: A “soft” k-nearest neighbor voting scheme. Int. J. Intell. Syst. 16(4), 459–468 (2001) Mitchell, H.B., Schaefer, P.A.: A “soft” k-nearest neighbor voting scheme. Int. J. Intell. Syst. 16(4), 459–468 (2001)
25.
go back to reference Nam, W.J., Gur, S., Choi, J., Wolf, L., Lee, S.W.: Relative attributing propagation: interpreting the comparative contributions of individual units in deep neural networks. In: AAAI, pp. 2501–2508 (2020) Nam, W.J., Gur, S., Choi, J., Wolf, L., Lee, S.W.: Relative attributing propagation: interpreting the comparative contributions of individual units in deep neural networks. In: AAAI, pp. 2501–2508 (2020)
26.
go back to reference Park, L.A.F., Simoff, S.: Using entropy as a measure of acceptance for multi-label classification. In: Advances in Intelligent Data Analysis XIV, pp. 217–228 (2015) Park, L.A.F., Simoff, S.: Using entropy as a measure of acceptance for multi-label classification. In: Advances in Intelligent Data Analysis XIV, pp. 217–228 (2015)
27.
go back to reference Pu, T., Chen, T., Wu, H., Lin, L.: Semantic-aware representation blending for multi-label image recognition with partial labels. In: AAAI, pp. 2091–2098 (2022) Pu, T., Chen, T., Wu, H., Lin, L.: Semantic-aware representation blending for multi-label image recognition with partial labels. In: AAAI, pp. 2091–2098 (2022)
28.
go back to reference Radford, A., et al.: Learning transferable visual models from natural language supervision. In: ICML (2021) Radford, A., et al.: Learning transferable visual models from natural language supervision. In: ICML (2021)
29.
go back to reference Rajeswar, S., López, P.R., Singhal, S., Vázquez, D., Courville, A.C.: Multi-label iterated learning for image classification with label ambiguity. In: CVPR, pp. 4773–4783 (2021) Rajeswar, S., López, P.R., Singhal, S., Vázquez, D., Courville, A.C.: Multi-label iterated learning for image classification with label ambiguity. In: CVPR, pp. 4773–4783 (2021)
30.
go back to reference Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015) Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)
31.
go back to reference Sun, X., Hu, P., Saenko, K.: Dualcoop: fast adaptation to multi-label recognition with limited annotations. In: NeurIPS (2022) Sun, X., Hu, P., Saenko, K.: Dualcoop: fast adaptation to multi-label recognition with limited annotations. In: NeurIPS (2022)
32.
go back to reference Vasisht, D., Damianou, A., Varma, M., Kapoor, A.: Active learning for sparse bayesian multilabel classification. In: SIGKDD, pp. 472–481 (2014) Vasisht, D., Damianou, A., Varma, M., Kapoor, A.: Active learning for sparse bayesian multilabel classification. In: SIGKDD, pp. 472–481 (2014)
33.
go back to reference Wang, Y., et al.: Multi-label classification with label graph superimposing. In: AAAI, vol. 34, pp. 12265–12272 (2020) Wang, Y., et al.: Multi-label classification with label graph superimposing. In: AAAI, vol. 34, pp. 12265–12272 (2020)
34.
go back to reference Wu, B., Liu, Z., Wang, S., Hu, B.G., Ji, Q.: Multi-label learning with missing labels. In: ICPR, pp. 1964–1968 (2014) Wu, B., Liu, Z., Wang, S., Hu, B.G., Ji, Q.: Multi-label learning with missing labels. In: ICPR, pp. 1964–1968 (2014)
35.
go back to reference Zhang, B., et al.: Flexmatch: boosting semi-supervised learning with curriculum pseudo labeling. In: NeurIPS, vol. 34, pp. 18408–18419 (2021) Zhang, B., et al.: Flexmatch: boosting semi-supervised learning with curriculum pseudo labeling. In: NeurIPS, vol. 34, pp. 18408–18419 (2021)
36.
go back to reference Zhang, X., Song, Y., Zuo, F., Wang, X.: Towards imbalanced large scale multi-label classification with partially annotated labels. In: SERA, pp. 195–200 (2023) Zhang, X., Song, Y., Zuo, F., Wang, X.: Towards imbalanced large scale multi-label classification with partially annotated labels. In: SERA, pp. 195–200 (2023)
37.
go back to reference Zhang, Y., et al.: Simple and robust loss design for multi-label learning with missing labels. arXiv abs/2112.07368 (2021) Zhang, Y., et al.: Simple and robust loss design for multi-label learning with missing labels. arXiv abs/2112.07368 (2021)
38.
go back to reference Zhou, D., Chen, P., Wang, Q., Chen, G., Heng, P.A.: Acknowledging the unknown for multi-label learning with single positive labels. arXiv abs/2203.16219 (2022) Zhou, D., Chen, P., Wang, Q., Chen, G., Heng, P.A.: Acknowledging the unknown for multi-label learning with single positive labels. arXiv abs/2203.16219 (2022)
Metadata
Title
Instance-Ambiguity Weighting for Multi-label Recognition with Limited Annotations
Authors
Daniel Shrewsbury
Suneung Kim
Young-Eun Kim
Heejo Kong
Seong-Whan Lee
Copyright Year
2024
Publisher
Springer Nature Singapore
DOI
https://doi.org/10.1007/978-981-97-2242-6_13

Premium Partner