Skip to main content

19.02.2024

MaGNIFIES: Manageable GAN Image Augmentation Framework for Inspection of Electronic Systems

verfasst von: Pallabi Ghosh, Gijung Lee, Mengdi Zhu, Olivia P. Dizon-Paradis, Ulbert J. Botero, Damon L. Woodard, Domenic Forte

Erschienen in: Journal of Hardware and Systems Security

Einloggen, um Zugang zu erhalten

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Electronic counterfeiting is a long-lasting problem that continues to cost original manufacturers billions, fund organized crime, and jeopardize national security and mission-critical infrastructures. Manual inspection is a popular and standardized way to detect counterfeit electronic components, but it is time-consuming and requires subject matter experts for classification. State-of-the-art machine learning, deep learning, and computer vision-based physical inspection methods are promising to alleviate these issues. However, the main bottleneck for doing so is a lack of high-quality, publicly available counterfeit image data for training. Producing such datasets is also time-consuming and often requires expensive equipment. In addition, most test labs are not allowed to freely publish images taken from their customer’s chips. One solution to this data shortage bottleneck can be addressed by augmenting synthetic data. In this paper, (i) data multiplication using Progressive GAN, StyleGAN, and classical methods in counterfeit data domain is explored; (ii) a novel framework, named MaGNIFIES, is proposed; and (iii) an efficient Convolutional Neural Network architecture is proposed, which can detect defective parts by training only on the synthetic dataset generated using (i) or (ii). For proof of concept, we have used low-quality images of resistors and capacitors with and without scratch defects as counterfeit and golden components respectively. We have also illustrated how our approach using MaGNIFIES addresses the shortcomings of the existing augmentation methods. Separate data augmentation detection models are trained with each type of augmented data generated using MaGNIFIES, as well as existing techniques, and tested on a test set of real data.
Literatur
1.
Zurück zum Zitat Mehta D, Lu H, Paradis OP et al (2020) The big hack explained: detection and prevention of pcb supply chain implants. ACM J Emerg Technol Comput Syst (JETC) 16(4):1–25CrossRef Mehta D, Lu H, Paradis OP et al (2020) The big hack explained: detection and prevention of pcb supply chain implants. ACM J Emerg Technol Comput Syst (JETC) 16(4):1–25CrossRef
3.
Zurück zum Zitat Frontier economics (2016) The economic impacts of counterfeiting and piracy. Report prepared for BASCAP and INTA. International Chamber of Commerce Frontier economics (2016) The economic impacts of counterfeiting and piracy. Report prepared for BASCAP and INTA. International Chamber of Commerce
5.
Zurück zum Zitat Cardoso B (2021) The dark side of the chip shortage: Counterfeits. X-ray News Cardoso B (2021) The dark side of the chip shortage: Counterfeits. X-ray News
6.
Zurück zum Zitat Guin U, DiMase D, Tehranipoor M (2014) Counterfeit integrated circuits: Detection, avoidance, and the challenges ahead. J Electron Test 30(1):9–23CrossRef Guin U, DiMase D, Tehranipoor M (2014) Counterfeit integrated circuits: Detection, avoidance, and the challenges ahead. J Electron Test 30(1):9–23CrossRef
7.
Zurück zum Zitat Guin U, Forte D, Tehranipoor M (2013) Anti-counterfeit techniques: From design to resign. 2013 14th International workshop on microprocessor test and verification. IEEE, pp 89–94 Guin U, Forte D, Tehranipoor M (2013) Anti-counterfeit techniques: From design to resign. 2013 14th International workshop on microprocessor test and verification. IEEE, pp 89–94
10.
Zurück zum Zitat Ghosh P, Bhattacharya A, Forte D et al (2019) Automated defective pin detection for recycled microelectronics identification. Journal of Hardware and Systems Security 3(3):250–260CrossRef Ghosh P, Bhattacharya A, Forte D et al (2019) Automated defective pin detection for recycled microelectronics identification. Journal of Hardware and Systems Security 3(3):250–260CrossRef
11.
Zurück zum Zitat Ghosh P, Botero UJ, Ganji F, Woodard D, Chakraborty RS, Forte D (2020) Automated detection and localization of counterfeit chip defects by texture analysis in infrared (ir) domain. 2020 IEEE Physical Assurance and Inspection of Electronics (PAINE). IEEE, pp 1–6 Ghosh P, Botero UJ, Ganji F, Woodard D, Chakraborty RS, Forte D (2020) Automated detection and localization of counterfeit chip defects by texture analysis in infrared (ir) domain. 2020 IEEE Physical Assurance and Inspection of Electronics (PAINE). IEEE, pp 1–6
12.
Zurück zum Zitat Ghosh P, Forte D, Woodard DL, Chakraborty RS (2018) Automated detection of pin defects on counterfeit microelectronics. ISTFA 2018: Proceedings from the 44th International Symposium for Testing and Failure Analysis. ASM International, p 57 Ghosh P, Forte D, Woodard DL, Chakraborty RS (2018) Automated detection of pin defects on counterfeit microelectronics. ISTFA 2018: Proceedings from the 44th International Symposium for Testing and Failure Analysis. ASM International, p 57
13.
Zurück zum Zitat Ghosh P, Ganji F, Forte D et al (2019) Automated framework for unsupervised counterfeit integrated circuit detection by physical inspection Ghosh P, Ganji F, Forte D et al (2019) Automated framework for unsupervised counterfeit integrated circuit detection by physical inspection
14.
Zurück zum Zitat Asadizanjani N, Tehranipoor M, Forte D (2017) Counterfeit electronics detection using image processing and machine learning. J Phys Conf Ser 787:012023. IOP PublishingCrossRef Asadizanjani N, Tehranipoor M, Forte D (2017) Counterfeit electronics detection using image processing and machine learning. J Phys Conf Ser 787:012023. IOP PublishingCrossRef
15.
Zurück zum Zitat Mahmood K, Carmona PL, Shahbazmohamadi S et al (2015) Real-time automated counterfeit integrated circuit detection using x-ray microscopy. Appl Opt 54(13):D25–D32CrossRef Mahmood K, Carmona PL, Shahbazmohamadi S et al (2015) Real-time automated counterfeit integrated circuit detection using x-ray microscopy. Appl Opt 54(13):D25–D32CrossRef
16.
Zurück zum Zitat Shahbazmohamadi S, Forte D, Tehranipoor M (2014) Advanced physical inspection methods for counterfeit IC detection. ISTFA 2014: Conference Proceedings from the 40th International Symposium for Testing and Failure Analysis. ASM International, p 55 Shahbazmohamadi S, Forte D, Tehranipoor M (2014) Advanced physical inspection methods for counterfeit IC detection. ISTFA 2014: Conference Proceedings from the 40th International Symposium for Testing and Failure Analysis. ASM International, p 55
17.
Zurück zum Zitat Kuo C-W, Ashmore JD, Huggins D, Kira Z (2019) Data-efficient graph embedding learning for pcb component detection. 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, pp 551–560 Kuo C-W, Ashmore JD, Huggins D, Kira Z (2019) Data-efficient graph embedding learning for pcb component detection. 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, pp 551–560
18.
Zurück zum Zitat Lu H, Mehta D, Paradis OP et al (2020) Fics-pcb: A multi-modal image dataset for automated printed circuit board visual inspection. IACR Cryptol ePrint Arch 2020:366 Lu H, Mehta D, Paradis OP et al (2020) Fics-pcb: A multi-modal image dataset for automated printed circuit board visual inspection. IACR Cryptol ePrint Arch 2020:366
19.
Zurück zum Zitat Jessurun N, Dizon-Paradis OP, Harrison J, Ghosh S, Tehranipoor MM, Woodard DL, Asadizanjani N (2022) FPIC: a novel semantic dataset for optical PCB assurance. arXiv preprint arXiv:2202.08414 Jessurun N, Dizon-Paradis OP, Harrison J, Ghosh S, Tehranipoor MM, Woodard DL, Asadizanjani N (2022) FPIC: a novel semantic dataset for optical PCB assurance. arXiv preprint arXiv:​2202.​08414
20.
Zurück zum Zitat Nathan Jessurun, Daniel E. Capecci, Olivia P. Dizon Paradis et al (2022) Semi-supervised semantic annotator (S3A): toward efficient semantic labeling. In: Agarwal M, Calloway C, Niederhut D, Shupe D (eds) Proceedings of the 21st Python in Science Conference, pp 7–12. https://doi.org/10.25080/majora-212e5952-001 Nathan Jessurun, Daniel E. Capecci, Olivia P. Dizon Paradis et al (2022) Semi-supervised semantic annotator (S3A): toward efficient semantic labeling. In: Agarwal M, Calloway C, Niederhut D, Shupe D (eds) Proceedings of the 21st Python in Science Conference, pp 7–12. https://​doi.​org/​10.​25080/​majora-212e5952-001
21.
Zurück zum Zitat Fridman Y, Rusanovsky M, Oren G (2021) Changechip: A reference-based unsupervised change detection for pcb defect detection. 2021 IEEE physical assurance and inspection of electronics (PAINE). IEEE, pp 1–8 Fridman Y, Rusanovsky M, Oren G (2021) Changechip: A reference-based unsupervised change detection for pcb defect detection. 2021 IEEE physical assurance and inspection of electronics (PAINE). IEEE, pp 1–8
25.
Zurück zum Zitat Anzai Y (2012) Pattern recognition and machine learning. Elsevier Anzai Y (2012) Pattern recognition and machine learning. Elsevier
26.
Zurück zum Zitat Goodfellow I, Pouget-Abadie J, Mirza M et al (2020) Generative adversarial networks. Commun ACM 63(11):139–144MathSciNetCrossRef Goodfellow I, Pouget-Abadie J, Mirza M et al (2020) Generative adversarial networks. Commun ACM 63(11):139–144MathSciNetCrossRef
28.
Zurück zum Zitat Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196 Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:​1710.​10196
29.
Zurück zum Zitat Karras T, Laine S, Aittala M, Hellsten J, Lehtinen J, Aila T (2020) Analyzing and improving the image quality of stylegan. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8110–8119 Karras T, Laine S, Aittala M, Hellsten J, Lehtinen J, Aila T (2020) Analyzing and improving the image quality of stylegan. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8110–8119
30.
Zurück zum Zitat Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134 Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134
31.
32.
Zurück zum Zitat Wright RE (1995) Logistic regression. American Psychological Association, pp 217–244 Wright RE (1995) Logistic regression. American Psychological Association, pp 217–244
33.
Zurück zum Zitat Ramchoun H, Ghanou Y, Ettaouil M, Janati Idrissi MA (2016) Multilayer perceptron: Architecture optimization and training. Int J Interact Multimed Artif Intell Ramchoun H, Ghanou Y, Ettaouil M, Janati Idrissi MA (2016) Multilayer perceptron: Architecture optimization and training. Int J Interact Multimed Artif Intell
36.
Zurück zum Zitat Mittal A, Moorthy AK, Bovik AC (2011) Blind/referenceless image spatial quality evaluator. 2011 conference record of the forty fifth asilomar conference on signals, systems and computers (ASILOMAR). IEEE, pp 723–727 Mittal A, Moorthy AK, Bovik AC (2011) Blind/referenceless image spatial quality evaluator. 2011 conference record of the forty fifth asilomar conference on signals, systems and computers (ASILOMAR). IEEE, pp 723–727
37.
Zurück zum Zitat Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708. IEEEADSMathSciNetCrossRefPubMed Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708. IEEEADSMathSciNetCrossRefPubMed
38.
Zurück zum Zitat Mittal A, Soundararajan R, Bovik AC (2012) Making a “completely blind” image quality analyzer. IEEE Signal Process Lett 20(3):209–212. IEEEADSCrossRef Mittal A, Soundararajan R, Bovik AC (2012) Making a “completely blind” image quality analyzer. IEEE Signal Process Lett 20(3):209–212. IEEEADSCrossRef
39.
Zurück zum Zitat Ferzli R, Karam LJ (2009) A no-reference objective image sharpness metric based on the notion of just noticeable blur (jnb). IEEE Trans Image Process 18(4):717–728ADSMathSciNetCrossRefPubMed Ferzli R, Karam LJ (2009) A no-reference objective image sharpness metric based on the notion of just noticeable blur (jnb). IEEE Trans Image Process 18(4):717–728ADSMathSciNetCrossRefPubMed
Metadaten
Titel
MaGNIFIES: Manageable GAN Image Augmentation Framework for Inspection of Electronic Systems
verfasst von
Pallabi Ghosh
Gijung Lee
Mengdi Zhu
Olivia P. Dizon-Paradis
Ulbert J. Botero
Damon L. Woodard
Domenic Forte
Publikationsdatum
19.02.2024
Verlag
Springer International Publishing
Erschienen in
Journal of Hardware and Systems Security
Print ISSN: 2509-3428
Elektronische ISSN: 2509-3436
DOI
https://doi.org/10.1007/s41635-024-00145-7