nach oben

Erschienen in:

2023 | OriginalPaper | Buchkapitel

H\(^2\)-Nets: Hyper-hodge Convolutional Neural Networks for Time-Series Forecasting

verfasst von : Yuzhou Chen, Tian Jiang, Yulia R. Gel

Erschienen in: Machine Learning and Knowledge Discovery in Databases: Research Track

Verlag: Springer Nature Switzerland

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Hypergraphs recently have emerged as a new promising alternative to describe complex dependencies in spatio-temporal processes, resulting in the newest trend in multivariate time series forecasting, based semi-supervised learning of spatio-temporal data with Hypergraph Convolutional Networks. Nevertheless, such recent approaches are often limited in their capability to accurately describe higher-order interactions among spatio-temporal entities and to learn hidden interrelations among network substructures. Motivated by the emerging results on simplicial convolution, we introduce the concepts of Hodge theory and Hodge Laplacians, that is, a higher-order generalization of the graph Laplacian, to hypergraph learning. Furthermore, we develop a novel framework for hyper-simplex-graph representation learning which describes complex relationships among both graph and hyper-simplex-graph simplices and, as a result, simultaneously extracts latent higher-order spatio-temporal dependencies. We provide theoretical foundations behind the proposed hyper-simplex-graph representation learning and validate our new Hodge-style Hyper-simplex-graph Neural Networks (H\(^2\)-Nets) on 7 real world spatio-temporal benchmark datasets. Our experimental results indicate that H\(^2\)-Nets outperforms the state-of-the-art methods by a significant margin, while demonstrating lower computational costs.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Weighted Multivariate Mean Reversion for Online Portfolio Selection

Nächstes Kapitel Overcoming Catastrophic Forgetting for Fine-Tuning Pre-trained GANs

Agarwal, S., Branson, K., Belongie, S.: Higher order learning with graphs. In: Proceedings of the International Conference on Machine Learning, pp. 17–24 (2006)

Arya, D., Gupta, D.K., Rudinac, S., Worring, M.: Hypersage: generalizing inductive representation learning on hypergraphs. arXiv preprint arXiv:2010.04558 (2020)

Atwood, J., Towsley, D.: Diffusion-convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 29. Curran Associates, Inc. (2016)

Bai, L., Yao, L., Li, C., Wang, X., Wang, C.: Adaptive graph convolutional recurrent network for traffic forecasting. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 17804–17815. Curran Associates, Inc. (2020)

Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018)

Barbarossa, S., Sardellitti, S., Ceci, E.: Learning from signals defined over simplicial complexes. In: IEEE Data Science Workshop. IEEE (2018)

Benson, A.R., Abebe, R., Schaub, M.T., Jadbabaie, A., Kleinberg, J.: Simplicial closure and higher-order link prediction. Proc. Natl. Acad. Sci. USA 115(48), E11221–E11230 (2018)

Bilal, U., Jemmott, J.B., Schnake-Mahl, A., Murphy, K., Momplaisir, F.: Racial/ethnic and neighbourhood social vulnerability disparities in COVID-19 testing positivity, hospitalization, and in-hospital mortality in a large hospital system in Pennsylvania: a prospective study of electronic health records. Lancet Reg. Health-Am. 10, 100220 (2022)

Bodnar, C., et al.: Weisfeiler and lehman go topological: message passing simplicial networks. In: ICLR 2021 Workshop on Geometrical and Topological Representation Learning (2021)

10.

Bolla, M.: Spectra, euclidean representations and clusterings of hypergraphs. Discret. Math. 117(1–3), 19–39 (1993)

11.

Bronstein, M.M., Bruna, J., Cohen, T., Veličković, P.: Geometric deep learning: grids, groups, graphs, geodesics, and gauges (2021)

12.

Bunch, E., You, Q., Fung, G., Singh, V.: Simplicial 2-complex convolutional neural networks. In: NeurIPS 2020 Workshop on Topological Data Analysis and Beyond (2020)

13.

Cao, D., et al.: Spectral temporal graph neural network for multivariate time-series forecasting. In: Advances in Neural Information Processing Systems (2020)

14.

Chen, Y., Jiang, T., Heleno, M., Moreira, A., Gel, Y.R.: Evaluating distribution system reliability with hyperstructures graph convolutional nets. In: IEEE International Conference on Big Data, pp. 1793–1800 (2022)

15.

Chen, Y., Batsakis, S., Poor, H.V.: Higher-order spatio-temporal neural networks for COVID-19 forecasting. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1–5. IEEE (2023)

16.

Chen, Y., Gel, Y., Poor, H.V.: Time-conditioned dances with simplicial complexes: zigzag filtration curve based supra-hodge convolution networks for time-series forecasting. Adv. Neural. Inf. Process. Syst. 35, 8940–8953 (2022)

17.

Chen, Y., Gel, Y.R., Poor, H.V.: BScNets: block simplicial complex neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence (2022)

18.

Chen, Y., Heleno, M., Moreira, A., Gel, Y.R.: Topological graph convolutional networks solutions for power distribution grid planning. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 123–134 (2023)

19.

Chen, Y., Jacob, R.A., Gel, Y.R., Zhang, J., Poor, H.V.: Learning power grid outages with higher-order topological neural networks. IEEE Trans. Power Syst. (2023)

20.

Chen, Y., Segovia, I., Gel, Y.R.: Z-GCNETs: time zigzags at graph convolutional networks for time series forecasting. In: Proceedings of the International Conference on Machine Learning, pp. 1684–1694 (2021)

21.

Chen, Y., Segovia-Dominguez, I., Coskunuzer, B., Gel, Y.: Tamp-s2gcnets: coupling time-aware multipersistence knowledge representation with spatio-supra graph convolutional networks for time-series forecasting. In: International Conference on Learning Representations (2022)

22.

Ebli, S., Defferrard, M., Spreemann, G.: Simplicial neural networks. In: Topological Data Analysis and Beyond workshop at Advances in Neural Information Processing Systems (2020)

23.

Feng, Y., You, H., Zhang, Z., Ji, R., Gao, Y.: Hypergraph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3558–3565 (2019)

24.

Geng, X., et al.: Spatiotemporal multi-graph convolution network for ride-hailing demand forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3656–3663 (2019)

25.

Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 1263–1272. PMLR (2017)

26.

Guo, S., Lin, Y., Feng, N., Song, C., Wan, H.: Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 922–929 (2019)

27.

Hajij, M., Istvan, K., Zamzmi, G.: Cell complex neural networks. In: NeurIPS 2020 Workshop on Topological Data Analysis and Beyond (2020)

28.

Hamilton, W.L., Ying, R., Leskovec, J.: Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems (2017)

29.

Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRef

30.

Hsiao, W.Y., Liu, J.Y., Yeh, Y.C., Yang, Y.H.: Compound word transformer: learning to compose full-song music over dynamic directed hypergraphs. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 4503–4511 (2021)

31.

Huang, J., Yang, J.: UniGNN: a unified framework for graph and hypergraph neural networks. In: Proceedings of the International Joint Conference on Artificial Intelligence (2021)

32.

Jiang, J., Wei, Y., Feng, Y., Cao, J., Gao, Y.: Dynamic hypergraph neural networks. In: Proceedings of the International Joint Conference on Artificial Intelligence (2019)

33.

Karimi, A.M., Wu, Y., Koyuturk, M., French, R.H.: Spatiotemporal graph neural network for performance prediction of photovoltaic power systems. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 15323–15330 (2021)

34.

Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: Proceedings of the International Conference on Learning Representations (2017)

35.

Lai, G., Chang, W.C., Yang, Y., Liu, H.: Modeling long-and short-term temporal patterns with deep neural networks. In: ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 95–104 (2018)

36.

Li, Y., Yu, R., Shahabi, C., Liu, Y.: Diffusion convolutional recurrent neural network: data-driven traffic forecasting. In: Proceedings of the International Conference on Learning Representations (2018)

37.

Liang, Y., et al.: UrbanFM: inferring fine-grained urban flows. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM (2019)

38.

Lim, L.H.: Hodge laplacians on graphs. SIAM Rev. 62(3), 685–715 (2020)MathSciNetCrossRefMATH

39.

Oreshkin, B.N., Carpov, D., Chapados, N., Bengio, Y.: N-beats: neural basis expansion analysis for interpretable time series forecasting. In: Proceedings of the International Conference on Learning Representations (2019)

40.

Pan, Z., Liang, Y., Wang, W., Yu, Y., Zheng, Y., Zhang, J.: Urban traffic prediction from spatio-temporal data using deep meta learning. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM (2019)

41.

Rangapuram, S.S., Seeger, M.W., Gasthaus, J., Stella, L., Wang, Y., Januschowski, T.: Deep state space models for time series forecasting. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

42.

Reitsma, M.B., et al.: Racial/ethnic disparities in COVID-19 exposure risk, testing, and cases at the subcounty level in California: study examines racial/ethnic disparities in COVID-19 risk, testing, and cases. Health Aff. 40(6), 870–878 (2021)MathSciNetCrossRef

43.

Roddenberry, T.M., Glaze, N., Segarra, S.: Principled simplicial neural networks for trajectory prediction. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 139, pp. 9020–9029. PMLR (2021)

44.

Roddenberry, T.M., Segarra, S.: HodgeNet: graph neural networks for edge data. In: The 53rd Asilomar Conference on Signals, Systems, and Computers. IEEE (2019)

45.

Sawhney, R., Agarwal, S., Wadhwa, A., Derr, T., Shah, R.R.: Stock selection via spatiotemporal hypergraph attention network: a learning to rank approach. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 497–504 (2021)

46.

Schaub, M.T., Benson, A.R., Horn, P., Lippner, G., Jadbabaie, A.: Random walks on simplicial complexes and the normalized hodge 1-laplacian. SIAM Rev. 62(2), 353–391 (2020)MathSciNetCrossRefMATH

47.

Schaub, M.T., Zhu, Y., Seby, J.B., Roddenberry, T.M., Segarra, S.: Signal processing on higher-order networks: livin’on the edge... and beyond. Signal Process. 187, 108149 (2021)

48.

Sen, R., Yu, H.F., Dhillon, I.S.: Think globally, act locally: a deep neural network approach to high-dimensional time series forecasting. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

49.

Song, C., Lin, Y., Guo, S., Wan, H.: Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 01, pp. 914–921 (2020)

50.

Srinivasan, D., Chan, C.W., Balaji, P.: Computational intelligence-based congestion prediction for a dynamic urban street network. Neurocomputing 72(10–12), 2710–2716 (2009)CrossRef

51.

Sun, X., et al.: Heterogeneous hypergraph embedding for graph classification. In: Proceedings of the ACM International Conference on Web Search and Data Mining, pp. 725–733 (2021)

52.

Tran, A., Mathews, A., Ong, C.S., Xie, L.: Radflow: a recurrent, aggregated, and decomposable model for networks of time series. In: Proceedings of the International World Wide Web Conference, pp. 730–742 (2021)

53.

Wang, J., Zhang, Y., Wei, Y., Hu, Y., Piao, X., Yin, B.: Metro passenger flow prediction via dynamic hypergraph convolution networks. IEEE Trans. Intell. Transp. Syst. 22(12), 7891–7903 (2021)CrossRef

54.

Wu, S., Rizoiu, M.A., Xie, L.: Estimating attention flow in online video networks. Proc. ACM Hum.-Comput. Interact. 3, 1–25 (2019)

55.

Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Yu, P.S.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32(1), 4–24 (2021)MathSciNetCrossRef

56.

Wu, Z., Pan, S., Long, G., Jiang, J., Zhang, C.: Graph WaveNet for deep spatial-temporal graph modeling. In: Proceedings of the International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization (2019)

57.

Xia, L., Huang, C., Xu, Y., Dai, P., Bo, L., Zhang, X., Chen, T.: Spatial-temporal sequential hypergraph network for crime prediction with dynamic multiplex relation learning. In: Proceedings of the International Joint Conference on Artificial Intelligence (2021)

58.

Yadati, N., Nimishakavi, M., Yadav, P., Nitin, V., Louis, A., Talukdar, P.: Hypergcn: a new method of training graph convolutional networks on hypergraphs. In: Advances in Neural Information Processing Systems, pp. 1511–1522 (2019)

59.

Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton-based action recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence (2018)

60.

Yao, H., Tang, X., Wei, H., Zheng, G., Li, Z.: Revisiting spatial-temporal similarity: a deep learning framework for traffic prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence (2019)

61.

Yao, H., et al.: Deep multi-view spatial-temporal network for taxi demand prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence (2018)

62.

Yi, J., Park, J.: Hypergraph convolutional recurrent neural network. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining August 2020. ACM (2020)

63.

Yin, H., Benson, A.R., Leskovec, J.: Higher-order clustering in networks. Phys. Rev. E 97(5), 052306 (2018)CrossRef

64.

Yin, N., et al.: Dynamic hypergraph convolutional network. In: IEEE International Conference on Data Engineering (2022)

65.

Yu, B., Yin, H., Zhu, Z.: Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In: Proceedings of the International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization (2018)

66.

Zhang, J., Zheng, Y., Qi, D.: Deep spatio-temporal residual networks for citywide crowd flows prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 1655–1661 (2017)

67.

Zhang, J., Zheng, Y., Qi, D., Li, R., Yi, X.: DNN-based prediction model for spatio-temporal data. In: Proceedings of the ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM (2016)

68.

Zheng, W., Yan, L., Gou, C., Wang, F.Y.: Two heads are better than one: hypergraph-enhanced graph reasoning for visual event ratiocination. In: Proceedings of the International Conference on Machine Learning, vol. 139, pp. 12747–12760 (2021)

Titel: H-Nets: Hyper-hodge Convolutional Neural Networks for Time-Series Forecasting
verfasst von: Yuzhou Chen
Tian Jiang
Yulia R. Gel
Verlag: Springer Nature Switzerland
Buch: Machine Learning and Knowledge Discovery in Databases: Research Track
Print ISBN: 978-3-031-43423-5

Electronic ISBN: 978-3-031-43424-2

Copyright-Jahr: 2023
DOI: https://doi.org/10.1007/978-3-031-43424-2_17

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner