nach oben

Erschienen in:

2024 | OriginalPaper | Buchkapitel

Evaluation of a Red Team Automation Tool in Live Cyber Defence Exercises

verfasst von : Hannes Holm, Jenni Reuben

Erschienen in: ICT Systems Security and Privacy Protection

Verlag: Springer Nature Switzerland

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

This paper presents an evaluation of the red team automation tool Lore in two live-fire cyber defense exercises (CDX). During the CDXs, Lore and manual “red” teams subjected 72 network security analysts (i.e., defenders; the “blue” side) to various threats such as software exploits and shell commands. Ten hypotheses related to how the actions by manual red teams and Lore are perceived and managed by the security analysts are examined. Evaluations were made by studying the subjective judgements of the analysts and by comparing the objective ground truth to their submitted incident reports. The results show that none of the null hypotheses could be rejected. In other words, the security analysts could not tell the difference between the actions made by the manual red team and those made by Lore, and their performance was similar regardless of the source of the threats.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Quantum-Secure Communication for Trusted Edge Computing with IoT Devices

Nächstes Kapitel Automated and Improved Detection of Cyber Attacks via an Industrial IDS Probe

CCDCOE, “Locked Shields” (2022), https://ccdcoe.org/exercises/locked-shields/.

See the National Guard Bureau home page for further information, https://www.nationalguard.mil/.

NIST, “NICE Framework Resource Center”, https://www.nist.gov/itl/applied-cybersecurity/nice/nice-framework-resource-center.

Abbott, R.G., McClain, J., Anderson, B., Nauer, K., Silva, A., Forsythe, C.: Automated performance assessment in cyber training exercises, p. 7

Andrew, A., Spillard, S., Collyer, J., Dhir, N.: Developing optimal causal cyber-defence agents via cyber security simulation. arXiv preprint arXiv:2207.12355 (2022)

Bashir, M., Lambert, A., Guo, B., Memon, N., Halevi, T.: Cybersecurity competitions: the human angle 13(5), 74–79. https://doi.org/10.1109/MSP.2015.100

Delacre, M., Lakens, D., Leys, C.: Why psychologists should by default use Welch’s t-test instead of student’s t-test. 30(1), 92–101. https://doi.org/10.5334/irsp.82, http://www.rips-irsp.com/articles/10.5334/irsp.82/

Dutta, A., Chatterjee, S., Bhattacharya, A., Halappanavar, M.: Deep reinforcement learning for cyber system defense under dynamic adversarial uncertainties. arXiv preprint arXiv:2302.01595 (2023)

Gustafsson, T., Almroth, J.: Cyber range automation overview with a case study of CRATE. In: Asplund, M., Nadjm-Tehrani, S. (eds.) NordSec 2020. LNCS, vol. 12556, pp. 192–209. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-70852-8_12CrossRef

Henshel, D.S., et al.: Predicting proficiency in cyber defense team exercises. In: MILCOM 2016 - 2016 IEEE Military Communications Conference, pp. 776–781 (2016). https://doi.org/10.1109/MILCOM.2016.7795423

Holm, H.: Lore A Red Team Emulation Tool, p. 1.https://doi.org/10.1109/TDSC.2022.3160792

Holm, H., Sommestad, T.: SVED: scanning, vulnerabilities, exploits and detection. In: 2016 IEEE Military Communications Conference, pp. 976–981. IEEE (2016)

10.

Li, L., Fayad, R., Taylor, A.: Cygil: a cyber gym for training autonomous agents over emulated network systems. arXiv preprint arXiv:2109.03331 (2021)

11.

Lif, P., Varga, S., Wedlin, M., Lindahl, D., Persson, M.: Evaluation of information elements in a cyber incident report. In: 2020 IEEE European Symposium on Security and Privacy Workshops (EuroS &PW), pp. 17–26. IEEE (2020)

12.

Maennel, K., Ottis, R., Maennel, O.: Improving and measuring learning effectiveness at cyber defense exercises. In: Lipmaa, H., Mitrokotsa, A., Matulevičius, R. (eds.) NordSec 2017. LNCS, vol. 10674, pp. 123–138. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70290-2_8CrossRef

13.

McIntosh, S.E.: The wingman-philosopher of mig alley: John boyd and the ooda loop. Air Power History 58(4), 24–33 (2011). http://www.jstor.org/stable/26276108

14.

Miehling, E., Rasouli, M., Teneketzis, D.: Optimal defense policies for partially observable spreading processes on Bayesian attack graphs. In: Proceedings of the Second ACM Workshop on Moving Target Defense, pp. 67–76 (2015)

15.

Miller, D., Alford, R., Applebaum, A., Foster, H., Little, C., Strom, B.: Automated adversary emulation: a case for planning and acting with unknowns. Technical report, MITRE CORP MCLEAN VA MCLEAN (2018)

16.

Mäses, S., Hallaq, B., Maennel, O.: Obtaining Better Metrics for Complex Serious Games Within Virtualised Simulation Environments, p. 9

17.

Nhu, N.X., Nghia, T.T., Quyen, N.H., Pham, V.H., Duy, P.T., et al.: Leveraging deep reinforcement learning for automating penetration testing in reconnaissance and exploitation phase. In: 2022 RIVF International Conference on Computing and Communication Technologies, pp. 41–46. IEEE (2022)

18.

Rajivan, P., Moriano, P., Kelley, T., Camp, L.J.: What can johnny do?–Factors in an end-user expertise instrument. In: HAISA, pp. 199–208 (2016)

19.

Sarraute, C., Buffet, O., Hoffmann, J.: Penetration testing== pomdp solving? arXiv preprint arXiv:1306.4714 (2013)

20.

Stupp, C.: Sweden tests cyber defenses as war and nato bid raise security risks. https://www.wsj.com/articles/sweden-tests-cyber-defenses-as-war-and-nato-bid-raise-security-risks-11663925402

21.

Sultana, M., Taylor, A., Li, L.: Autonomous network cyber offence strategy through deep reinforcement learning. In: Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications III, vol. 11746, pp. 490–502. SPIE (2021)

22.

Zilberman, P., Puzis, R., Bruskin, S., Shwarz, S., Elovici, Y.: Sok: a survey of open-source threat emulators. arXiv preprint arXiv:2003.01518 (2020)

Titel: Evaluation of a Red Team Automation Tool in Live Cyber Defence Exercises
verfasst von: Hannes Holm
Jenni Reuben
Verlag: Springer Nature Switzerland
Buch: ICT Systems Security and Privacy Protection
Print ISBN: 978-3-031-56325-6

Electronic ISBN: 978-3-031-56326-3

Copyright-Jahr: 2024
DOI: https://doi.org/10.1007/978-3-031-56326-3_13

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner