Building a Manufacturing Product Defect Classification Model and Application using H2O Hydrogen Torch, H2O MLOps, and H2O Wave

Primary Authors: Nishaanthini Gnanavel and Genevieve Richards

Effective product quality control is of utmost importance in the manufacturing industry. The presence of defective components can have adverse effects on various aspects, including escalating production costs, compromising product quality, diminishing product longevity, and leading to substantial resource wastage. However, by promptly identifying defects and implementing suitable measures, significant reductions in operational expenses and quality-related expenditures can be achieved.

To mitigate these expenses, one possible approach is to conduct manual inspections during the production process. However, this method is time-consuming, expensive, and can lead to production delays, ultimately impacting market responsiveness. To address this challenge, the implementation of a machine vision defect detection system becomes imperative. Such a system has the potential to decrease production costs by minimizing the quantity of rejected and reprocessed faulty materials. It also aids in minimizing wastage of labor, materials, and machine time, reducing energy consumption, and facilitating the identification of underlying causes for defects.

Manufacturing Product Defect Classification Solution using H2O AI Cloud

Our customer data science team, recently developed a solution to detect different types of defects on manufacturing products (steel sheets) using deep learning techniques and deployed the model in MLOps. The solution is then packaged in an AI application which can be accessed by different stakeholders.

H2O’s AI cloud offers a diverse range of components that facilitate the complete lifecycle of developing, deploying, and utilizing machine learning or deep learning models. In this particular solution, we have leveraged three crucial components:

• AI Engine – H2O Hydrogen Torch: The H2O Hydrogen Torch application empowers data scientists of all levels, whether novice or experienced, to develop intricate deep learning models to address diverse problem domains like computer vision, natural language processing, and audio processing. Notably, this can be accomplished without requiring extensive coding knowledge. Utilizing the capabilities of H2O Hydrogen Torch, we successfully constructed a sophisticated multi-class deep learning model. The primary objective of this model was to accurately predict the presence and specific type of defects on micro thin steel sheets.

• Deployment: H2O MLOps: Upon conducting model tuning and identifying the optimal model using Hydrogen Torch, we proceeded to deploy it utilizing H2O MLOps. This advanced product facilitates the seamless export of Hydrogen Torch models to a collaborative MLOps project. Within this environment, users gain access to comprehensive features for efficient management, deployment, and monitoring of the deployed model.

• Application: H2O Wave: The development of the application was accomplished utilizing H2O Wave, an open-source Python development framework designed to streamline the creation of real-time interactive AI applications. With its user-friendly interface, H2O Wave empowers data scientists, machine learning engineers, and software developers to efficiently develop cutting-edge applications.

Model Details

Dataset: For model training purposes, a dataset comprising surface images of hot-rolled steel sheets was utilized. To prepare the dataset, the images were converted to grayscale and subsequently labeled into six distinct categories.

Model: By employing the H2O Hydrogen Torch model, numerous image classification models were trained, incorporating diverse backbones like efficientnets, resnets, mobilenets, and VGG architectures. These models were integrated together to enhance overall performance. The selected loss function for training was CrossEntropy, accompanied by the Adam W optimization algorithm.

Deployment: Through the utilization of H2O MLOps, the scoring pipeline generated from the Hydrogen Torch AI engine was deployed as a standalone model. This deployment offered a REST endpoint and allowed inference calls to be made using CURL request parameters. The REST endpoint was seamlessly integrated into the AI application, providing configurability. This implies that in the future, if there is a need to modify the deployment options from single to batch, or single to champion/challenger model, or even enable AB testing mode, all such options can be supported effortlessly.

Application: The entire solution, model, and the deployment were wrapped in a Wave Application to view the results, set alerts, monitor the predictions etc. The wave app has a range of functionalities including:

• Bring your own data –  Upload two types of files: a zip file of images or a video file as an input.
• Bring your own Model – Use the existing model or can create and deploy your own model in MLOps and use its endpoint URL in the app.
• Email Alerts – Receive notifications and alert emails about the detected defects during the scoring process. These can be sent based on your configurations in the configuration tab. There are two types of alert emails. The first one is instance defect alert emails where alert emails are sent during the scoring process when a defect is detected with prediction score above the alert threshold set by the user. The second one is the batch defect summary alert email which is sent after the scoring process to give a summary about the defect classification process.
• Global Monitoring View – This view showcases the results of the defect classification process after the scoring process and helps to analyze the defect classification process image by image.

The dashboard features various useful visualizations relevant to the scoring results. The Defect Classification Analysis by image names section of the dashboard displays both the raw input image of the hot rolled steel strips and a spectral image that highlights the area of the defect more clearly to the user. The spectral image is generated using OpenCV Saliency, which helps the user analyze the defect in that strip more effectively. 

The Product Defect Classification app provides a reliable and efficient solution to the problem of product quality control in the manufacturing industry. With the ability to detect defects, classify them and provide timely and accurate results, this application helps in reducing operational and quality related costs by minimizing rejected and reprocessed defective materials. Ultimately, our Product Defect Classification app is a valuable tool for the manufacturing industry to improve product quality, reduce costs, and increase overall efficiency.

For any queries, please reach out at cloud-feedback@h2o.ai Contact us if you are interested in discussing this use-case or any other use case further.