Skip to main content

2024 | OriginalPaper | Buchkapitel

SHIELD: A Specialized Dataset for Hybrid Blind Forensics of World Leaders

verfasst von : Qingran Lin, Xiang Li, Beilin Chu, Renying Wang, Xianhao Chen, Yuzhe Mao, Zhen Yang, Linna Zhou, Weike You

Erschienen in: Digital Forensics and Cyber Crime

Verlag: Springer Nature Switzerland

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The speech videos of public figures, such as movie celebrities and world leaders, have an extensive influence on the Internet. However, the authenticity of these videos is often difficult to ascertain. These videos may have been carefully imitated by comedians or manipulated using Deepfake methods, which creates significant obstacles for the video forensics of specific characters. Moreover, the vast amount of data on social networking platforms renders manual screening impractical. To specifically address this issue, we present SHIELD, which stands for Specialized dataset for Hybrid blInd forEnsics of worLd leaDers. Unlike most previous public Deepfake datasets that only contain Deepfake samples, this dataset exquisitely includes a collection that can quickly test this issue, encompassing both impersonator and Deepfake videos. We provide a detailed dataset production process and conduct an elaborate experiment under the hybrid blind detection scenario. Our findings reveal the limitations of existing methods, demonstrate the potential of identity-based models, and illustrate the increased challenges posed by SHIELD.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
6.
Zurück zum Zitat Afchar, D., Nozick, V., Yamagishi, J., Echizen, I.: Mesonet: a compact facial video forgery detection network. In: WIFS, pp. 1–7. IEEE (2018) Afchar, D., Nozick, V., Yamagishi, J., Echizen, I.: Mesonet: a compact facial video forgery detection network. In: WIFS, pp. 1–7. IEEE (2018)
7.
Zurück zum Zitat Agarwal, S., Farid, H., El-Gaaly, T., Lim, S.: Detecting deep-fake videos from appearance and behavior. In: WIFS, pp. 1–6. IEEE (2020) Agarwal, S., Farid, H., El-Gaaly, T., Lim, S.: Detecting deep-fake videos from appearance and behavior. In: WIFS, pp. 1–6. IEEE (2020)
8.
Zurück zum Zitat Agarwal, S., Farid, H., Gu, Y., He, M., Nagano, K., Li, H.: Protecting world leaders against deep fakes. In: CVPR Workshops, pp. 38–45. Computer Vision Foundation/IEEE (2019) Agarwal, S., Farid, H., Gu, Y., He, M., Nagano, K., Li, H.: Protecting world leaders against deep fakes. In: CVPR Workshops, pp. 38–45. Computer Vision Foundation/IEEE (2019)
9.
Zurück zum Zitat Bohácek, M., Farid, H.: Protecting president zelenskyy against deep fakes. CoRR abs/2206.12043 (2022) Bohácek, M., Farid, H.: Protecting president zelenskyy against deep fakes. CoRR abs/2206.12043 (2022)
11.
Zurück zum Zitat Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: CVPR, pp. 1800–1807. IEEE Computer Society (2017) Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: CVPR, pp. 1800–1807. IEEE Computer Society (2017)
12.
Zurück zum Zitat Cozzolino, D., Rössler, A., Thies, J., Nießner, M., Verdoliva, L.: Id-reveal: identity-aware deepfake video detection. In: ICCV, pp. 15088–15097. IEEE (2021) Cozzolino, D., Rössler, A., Thies, J., Nießner, M., Verdoliva, L.: Id-reveal: identity-aware deepfake video detection. In: ICCV, pp. 15088–15097. IEEE (2021)
13.
Zurück zum Zitat Das, S., Seferbekov, S.S., Datta, A., Islam, M.S., Amin, M.R.: Towards solving the deepfake problem : an analysis on improving deepfake detection using dynamic face augmentation. In: ICCVW, pp. 3769–3778. IEEE (2021) Das, S., Seferbekov, S.S., Datta, A., Islam, M.S., Amin, M.R.: Towards solving the deepfake problem : an analysis on improving deepfake detection using dynamic face augmentation. In: ICCVW, pp. 3769–3778. IEEE (2021)
14.
Zurück zum Zitat Deng, J., Guo, J., Xue, N., Zafeiriou, S.: Arcface: additive angular margin loss for deep face recognition. In: CVPR, pp. 4690–4699. Computer Vision Foundation/IEEE (2019) Deng, J., Guo, J., Xue, N., Zafeiriou, S.: Arcface: additive angular margin loss for deep face recognition. In: CVPR, pp. 4690–4699. Computer Vision Foundation/IEEE (2019)
15.
Zurück zum Zitat Deng, J., Guo, J., Zhou, Y., Yu, J., Kotsia, I., Zafeiriou, S.: Retinaface: Single-stage dense face localisation in the wild. CoRR abs/1905.00641 (2019) Deng, J., Guo, J., Zhou, Y., Yu, J., Kotsia, I., Zafeiriou, S.: Retinaface: Single-stage dense face localisation in the wild. CoRR abs/1905.00641 (2019)
16.
Zurück zum Zitat Dolhansky, B., Howes, R., Pflaum, B., Baram, N., Canton-Ferrer, C.: The deepfake detection challenge (DFDC) preview dataset. CoRR abs/1910.08854 (2019) Dolhansky, B., Howes, R., Pflaum, B., Baram, N., Canton-Ferrer, C.: The deepfake detection challenge (DFDC) preview dataset. CoRR abs/1910.08854 (2019)
17.
Zurück zum Zitat Dong, X., et al.: Protecting celebrities from deepfake with identity consistency transformer. In: CVPR, pp. 9458–9468. IEEE (2022) Dong, X., et al.: Protecting celebrities from deepfake with identity consistency transformer. In: CVPR, pp. 9458–9468. IEEE (2022)
18.
Zurück zum Zitat He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778. IEEE Computer Society (2016) He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778. IEEE Computer Society (2016)
19.
Zurück zum Zitat Hernandez, F., Nguyen, V., Ghannay, S., Tomashenko, N., Estève, Y.: TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation. In: Karpov, A., Jokisch, O., Potapova, R. (eds.) SPECOM 2018. LNCS (LNAI), vol. 11096, pp. 198–208. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99579-3_21CrossRef Hernandez, F., Nguyen, V., Ghannay, S., Tomashenko, N., Estève, Y.: TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation. In: Karpov, A., Jokisch, O., Potapova, R. (eds.) SPECOM 2018. LNCS (LNAI), vol. 11096, pp. 198–208. Springer, Cham (2018). https://​doi.​org/​10.​1007/​978-3-319-99579-3_​21CrossRef
20.
Zurück zum Zitat Huang, J., Wang, X., Du, B., Du, P., Xu, C.: Deepfake MNIST+: a deepfake facial animation dataset. In: ICCVW, pp. 1973–1982. IEEE (2021) Huang, J., Wang, X., Du, B., Du, P., Xu, C.: Deepfake MNIST+: a deepfake facial animation dataset. In: ICCVW, pp. 1973–1982. IEEE (2021)
21.
Zurück zum Zitat Khalid, H., Tariq, S., Kim, M., Woo, S.S.: Fakeavceleb: a novel audio-video multimodal deepfake dataset. In: NeurIPS Datasets and Benchmarks (2021) Khalid, H., Tariq, S., Kim, M., Woo, S.S.: Fakeavceleb: a novel audio-video multimodal deepfake dataset. In: NeurIPS Datasets and Benchmarks (2021)
22.
Zurück zum Zitat Korshunov, P., Marcel, S.: Deepfakes: a new threat to face recognition? assessment and detection. CoRR abs/1812.08685 (2018) Korshunov, P., Marcel, S.: Deepfakes: a new threat to face recognition? assessment and detection. CoRR abs/1812.08685 (2018)
23.
Zurück zum Zitat Kwon, P., You, J., Nam, G., Park, S., Chae, G.: Kodf: a large-scale korean deepfake detection dataset. In: ICCV, pp. 10724–10733. IEEE (2021) Kwon, P., You, J., Nam, G., Park, S., Chae, G.: Kodf: a large-scale korean deepfake detection dataset. In: ICCV, pp. 10724–10733. IEEE (2021)
24.
Zurück zum Zitat Le, T., Nguyen, H.H., Yamagishi, J., Echizen, I.: Openforensics: large-scale challenging dataset for multi-face forgery detection and segmentation in-the-wild. In: ICCV, pp. 10097–10107. IEEE (2021) Le, T., Nguyen, H.H., Yamagishi, J., Echizen, I.: Openforensics: large-scale challenging dataset for multi-face forgery detection and segmentation in-the-wild. In: ICCV, pp. 10097–10107. IEEE (2021)
25.
Zurück zum Zitat Li, J., Xie, H., Li, J., Wang, Z., Zhang, Y.: Frequency-aware discriminative feature learning supervised by single-center loss for face forgery detection. In: CVPR, pp. 6458–6467, Computer Vision Foundation/IEEE (2021) Li, J., Xie, H., Li, J., Wang, Z., Zhang, Y.: Frequency-aware discriminative feature learning supervised by single-center loss for face forgery detection. In: CVPR, pp. 6458–6467, Computer Vision Foundation/IEEE (2021)
26.
Zurück zum Zitat Li, Y., Chang, M., Lyu, S.: In ictu oculi: exposing AI generated fake face videos by detecting eye blinking. CoRR abs/1806.02877 (2018) Li, Y., Chang, M., Lyu, S.: In ictu oculi: exposing AI generated fake face videos by detecting eye blinking. CoRR abs/1806.02877 (2018)
27.
Zurück zum Zitat Li, Y., Yang, X., Sun, P., Qi, H., Lyu, S.: Celeb-df: a large-scale challenging dataset for deepfake forensics. In: CVPR, pp. 3204–3213. Computer Vision Foundation/IEEE (2020) Li, Y., Yang, X., Sun, P., Qi, H., Lyu, S.: Celeb-df: a large-scale challenging dataset for deepfake forensics. In: CVPR, pp. 3204–3213. Computer Vision Foundation/IEEE (2020)
28.
Zurück zum Zitat Matern, F., Riess, C., Stamminger, M.: Exploiting visual artifacts to expose deepfakes and face manipulations. In: WACV Workshops, pp. 83–92. IEEE (2019) Matern, F., Riess, C., Stamminger, M.: Exploiting visual artifacts to expose deepfakes and face manipulations. In: WACV Workshops, pp. 83–92. IEEE (2019)
29.
Zurück zum Zitat Prajwal, K.R., Mukhopadhyay, R., Namboodiri, V.P., Jawahar, C.V.: A lip sync expert is all you need for speech to lip generation in the wild. In: ACM Multimedia, pp. 484–492. ACM (2020) Prajwal, K.R., Mukhopadhyay, R., Namboodiri, V.P., Jawahar, C.V.: A lip sync expert is all you need for speech to lip generation in the wild. In: ACM Multimedia, pp. 484–492. ACM (2020)
31.
Zurück zum Zitat Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nießner, M.: Faceforensics++: learning to detect manipulated facial images. In: ICCV, pp. 1–11. IEEE (2019) Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nießner, M.: Faceforensics++: learning to detect manipulated facial images. In: ICCV, pp. 1–11. IEEE (2019)
32.
Zurück zum Zitat Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., Sebe, N.: First order motion model for image animation. In: NeurIPS, pp. 7135–7145 (2019) Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., Sebe, N.: First order motion model for image animation. In: NeurIPS, pp. 7135–7145 (2019)
33.
Zurück zum Zitat Tan, M., Le, Q.V.: Efficientnet: rethinking model scaling for convolutional neural networks. In: ICML. Proceedings of Machine Learning Research, vol. 97, pp. 6105–6114. PMLR (2019) Tan, M., Le, Q.V.: Efficientnet: rethinking model scaling for convolutional neural networks. In: ICML. Proceedings of Machine Learning Research, vol. 97, pp. 6105–6114. PMLR (2019)
34.
Zurück zum Zitat Thies, J., Zollhöfer, M., Nießner, M.: Deferred neural rendering: image synthesis using neural textures. ACM Trans. Graph. 38(4), 66:1–66:12 (2019) Thies, J., Zollhöfer, M., Nießner, M.: Deferred neural rendering: image synthesis using neural textures. ACM Trans. Graph. 38(4), 66:1–66:12 (2019)
35.
Zurück zum Zitat Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., Nießner, M.: Face2face: real-time face capture and reenactment of RGB videos. Commun. ACM 62(1), 96–104 (2019)CrossRef Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., Nießner, M.: Face2face: real-time face capture and reenactment of RGB videos. Commun. ACM 62(1), 96–104 (2019)CrossRef
37.
Zurück zum Zitat Yang, X., Li, Y., Lyu, S.: Exposing deep fakes using inconsistent head poses. In: ICASSP, pp. 8261–8265. IEEE (2019) Yang, X., Li, Y., Lyu, S.: Exposing deep fakes using inconsistent head poses. In: ICASSP, pp. 8261–8265. IEEE (2019)
38.
Zurück zum Zitat Zi, B., Chang, M., Chen, J., Ma, X., Jiang, Y.: Wilddeepfake: a challenging real-world dataset for deepfake detection. In: ACM Multimedia, pp. 2382–2390. ACM (2020) Zi, B., Chang, M., Chen, J., Ma, X., Jiang, Y.: Wilddeepfake: a challenging real-world dataset for deepfake detection. In: ACM Multimedia, pp. 2382–2390. ACM (2020)
Metadaten
Titel
SHIELD: A Specialized Dataset for Hybrid Blind Forensics of World Leaders
verfasst von
Qingran Lin
Xiang Li
Beilin Chu
Renying Wang
Xianhao Chen
Yuzhe Mao
Zhen Yang
Linna Zhou
Weike You
Copyright-Jahr
2024
DOI
https://doi.org/10.1007/978-3-031-56583-0_6

Premium Partner