Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Progress in Artificial Intelligence

22-01-2024 | Regular Paper

Collaborative and dynamic kernel discriminant analysis for large-scale problems: applications in multi-class learning and novelty detection

Authors: F. Dufrenois, A. Khatib, M. Hamlich, D. Hamad

Published in: Progress in Artificial Intelligence

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

We present CKDA a new multi-class collaborative learning strategy based on multiple kernel discriminant analysis learners. The principle of CKDA is to share the computational and memory footprint of the kernel Gram matrix between several weak KDA learners, i.e. with a limited storage capacity. Based on bagging techniques, CKDA has the particularity to generate as many classifiers as necessary to minimize the overall empirical risk. The number of learners generated is fully data dependent and not specified beforehand, allowing CKDA to reach high compression rates. Moreover, since each KDA learner is formulated as a multi-response kernel regression problem, we derive a regularized kernel Mahalanobis distance as classification measure. Lastly, based on the definition of Mahalanobis distance, the problem of novelty detection is also addressed by computing novelty thresholds during the learning stage. Extensive experiments on several large-scale data sets show the effectiveness of the proposed algorithm both in multi-class and novelty detection problems.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
Alexandre-Cortizo, E., Rosa-Zurera, M., Lopez-Ferreras, F.: Application of Fisher linear discriminant analysisto speech/music classification. In: EUROCON 2005—The International Conference on Computer as a Tool, vol. 2, pp. 1666–1669 (2005)
2.
Pang, S., Ozawa, S., Kasabov, N.: Incremental linear discriminant analysis for classification of data streams. IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 35(5), 905–914 (2005)CrossRef
3.
4.
Bodesheim, P., Freytag, A., Rodner, E., Kemmler, M., Denzler, J.: Kernel null space methods for novelty detection. In: Computer Vision and Pattern Recognition, pp. 3374–3381 (2013)
5.
Sebastian, M., Alexander, J.S., Bernhard, S.: An improved training algorithm for kernel fisher discriminants. In: International Workshop on Artificial Intelligence and Statistics, pp. 209–215 (2001)
6.
Cai, D., He, X., Han, J.: Speed up kernel discriminant analysis. VLDB J. 20, 21–33 (2011)CrossRef
7.
Cameron, M., Christopher, M.: Recursive sampling for the Nyström method. In: NIPS, pp. 3834–3846 (2017)
8.
Dufrenois, F.: Incremental and compressible kernel null discriminant analysis. Pattern Recogn. 127, 108642 (2022)CrossRef
9.
Li, W.-H., Zhong, Z., Zheng, W.-S.: One-pass person re-identification by sketch online discriminant analysis. Pattern Recogn. 93, 237–250 (2017)CrossRef
10.
Liberty, E.: Simple and deterministic matrix sketching. In: The 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Chicago, USA, pp. 581–588 (2013)
11.
Rahimi, A., Recht, B.: Random features for large-scale kernel machines. In: Proceedings of the 20th International Conference on Neural Information Processing Systems, pp. 1177–1184. Curran Associates Inc., Red Hook (2007)
12.
Ye, H., Li, Y., Chen, C., Zhang, Z.: Fast Fisher discriminant analysis with randomized algorithms. Pattern Recogn. 72, 82–92 (2017)CrossRef
13.
Dai, G., Yeung, D.-Y.: Boosting kernel discriminant analysis and its application on tissue classification of gene expression data. In: IJCAI, pp. 744–749 (2007)
14.
Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting. Ann. Stat. 28, 337–407 (2000)MathSciNetCrossRef
15.
Allwein, E., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: a unifying approach for margin classifiers. J. Mach. Learn. Res. 1, 113–141 (2000)MathSciNet
16.
Hastie, T., Buja, A., Tibshirani, R.: Penalized discriminant analysis. Ann. Stat. 23, 73–102 (2000)MathSciNet
17.
Hastie, T., Tibshirani, R., Buja, A.: Flexible discriminant analysis by optimal scoring. J. Am. Stat. Assoc. 89, 1255–1270 (2000)MathSciNetCrossRef
18.
Roth, V., Steinhage, V.: Nonlinear discriminant analysis using kernel functions. In: Proceedings of the 12th International Conference on Neural Information Processing Systems. MIT Press, Cambridge, pp. 568–574 (1999)
19.
Ye, J.: Least squares linear discriminant analysis. In: Proceedings of the 24th International Conference on Machine Learning. Association for Computing Machinery, New York, pp. 1087–1093 (2007)
20.
Sun, L., Ji, S., Ye, J.: A least squares formulation for a class of generalized eigenvalue problems in machine learning. In: Proceedings of the 26th Annual International Conference on Machine Learning, New York, pp. 977–984 (2009)
21.
De La Torre, F.: A least-squares framework for component analysis. IEEE Trans. Pattern Anal. Mach. Intell. 34(6), 1041–1055 (2012)CrossRef
22.
Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. Johns Hopkins University Press (1996)
23.
Stewart, G.W.: Matrix algorithms. Basic decompositions, vol. 1. Society for Industrial Mathematics (1998)CrossRef
24.
Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998)CrossRef
25.
LeCun, Y., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)CrossRef
26.
27.
Bousmalis, K., Trigeorgis, G., Silberman, N., Krishnan, D., Erhan, D.: Domain separation networks. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain, pp. 343–351 (2016)
28.
29.
Ferrag, M.A., et al.: Deep learning for cyber security intrusion detection: approaches, datasets, and comparative study. J. Inf. Secur. Appl. 50, 102419 (2020)
30.
Krizhevsky, A.: Learning multiple layers of features from tiny images (2009)
31.
Liu, J., Lian, Z., Wang, Y., Xiao, J.: Incremental kernel null space discriminant analysis for novelty detection. In: CVPR, pp. 4123–4131 (2017)
32.
Wang, Y., Ding, Y., He, X., Fan, X., Lin, C., Li, F., Wang, T., Luo, Z., Luo, J.: Novelty detection and online learning for chunk data streams. IEEE Trans. Pattern Anal. Mach. Intell. 43(7), 2400–2412 (2021)CrossRef
33.
Mina, G., Perry, D.J., Phillips, J.M.: Streaming kernel principal component analysis. In: Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, Cadiz, Spain, pp. 1365–1374 (2016)
34.
Wang, Z., Xu, W., Hu, J., Guo, J.: A multiclass SVM method via probabilistic error-correcting output codes. In: 2010 International Conference on Internet Technology and Applications, pp. 1–4 (2010)
35.
36.
Jiang, H., Haoyu W., Wenhao H., Deovrat K., Arin C.: Fast incremental SVDD learning algorithm with the Gaussian kernel. In: AAAI Conference on Artificial Intelligence (2017)
37.
Tax, D.M., Duin, R.P.: Support vector domain description. Pattern Recognit. Lett. 20, 1191–1199 (1999)CrossRef
38.
Perera, P., Patel, V.M.: Deep transfer learning for multiple class novelty detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11536–11544 (2019)
39.
Tan, C., et al.: A survey on deep transfer learning. In: International Conference on Artificial Neural Networks (2018)
40.
Meidan, Y., et al.: N-BaIoT-network-based detection of IoT botnet attacks using deep autoencoders. IEEE Pervasive Comput. 17(3), 12–22 (2018)
Metadata
Title
Collaborative and dynamic kernel discriminant analysis for large-scale problems: applications in multi-class learning and novelty detection
Authors
F. Dufrenois
A. Khatib
M. Hamlich
D. Hamad
Publication date
22-01-2024
Publisher
Springer Berlin Heidelberg
Published in
Progress in Artificial Intelligence
Print ISSN: 2192-6352
Electronic ISSN: 2192-6360
DOI
https://doi.org/10.1007/s13748-023-00309-6

Premium Partner

    Image Credits