Smart manufacturing

The utilization of AI in manufacturing processes has ushered in a new era of data-driven optimization and problem-solving. AI is a pivotal cornerstone, especially in Industry 4.0 and 5.0 applications, yielding advanced technologies to guide a new era of heightened efficiency, productivity, and refined decision-making. In this capacity, AI emerges as an indispensable force driving transformative advancements within the manufacturing domain, where data-driven insights and smart automation catalyze innovation and shape the future of industrial processes. The assessment of SoA approaches in the manufacturing realm helped us identify three concrete use cases for further investigation: (1) Rare event prediction, (2) Anomaly detection using process knowledge workflow, and (3) Explaining the root cause of an event through causal knowledge.

PROJECTS

Project-1:Rare Event Prediction

There are different types of events that can be observed within autonomous systems. For example, our consideration on this topic led us to explore “rare events” as they are particularly important for smart manufacturing applications. Rare events are occurrences that take place with a significantly lower frequency than more common regular events. In manufacturing, predicting such events is particularly important, as they lead to unplanned downtime, shortening equipment lifespan, and high energy consumption. The occurrence of events is usually considered rare if observed in 5-10% of all instances, while the occurrence of less than 1% is considered extremely rare. The rarity of events is inversely correlated with the maturity of a manufacturing industry. Typically, the rarity of events causes the multivariate data generated within a manufacturing process to be highly imbalanced, which leads to bias in predictive models. To this end, we conducted a thorough survey to investigate prior art along four dimensions: (i) data acquisition methods, (ii) data preprocessing techniques, (iii) algorithmic approaches, and (iv) evaluation/assessment approaches. Figure X below summarizes the methods we looked at within each dimension

Regarding modeling rare event prediction, we first evaluated the role of data enrichment approaches combined with supervised machine-learning techniques. Predominantly, we explored using three data enrichment approaches; data augmentation, sampling and imputation. Our study is based on open real-world datasets we obtained from public data sources. Figure 1 includes the approaches to learning from rare event data that we exemplified from the comprehensive survey.

Project-2:Process Knowledge Workflow using PDDL

We proposed Process Knowledge Workflow (PKW) using Planning Domain Definition Language (PDDL) to detect anomalies in the assembly line (Figure 2&3). Specifically, the PKW uses multimodal data from robot sensors, conveyor belt sensors and camera to detect anomalies. The images from the camera will be processed using state-of-the-art object detection techniques to identify the objects of interest and the interactions among them. The data from the sensors will be utilized to detect the movement of the desired objects. Using this information, PKW will be constructed for the expected process. Then the PKW can serve as an orchestrator to identify anomalies in the sensor values. Based on the sensor values, the PKW will also have the ability to define and name the anomalies. We conducted an extensive survey of the list of sensors in the assembly line from neXt Future Factories lab and studied the type and range of data being generated. Using this, the first version of the PKW ontology will be constructed. A demonstration of PKW can be found in Figure 2.

Project-3:Explaining the root cause of an event through causal knowledge

Our goal is to utilize the process knowledge workflow to construct a sequential causal Bayesian network (Figure 4) that captures both the input-output sequence of the process and the agent information. Specifically, we aim to create a model that can track the cause-and-effect relationships between the input commands, robot actions, and pipeline outputs in the assembly process. By doing so, we will be able to predict the impact of a given input/output and action combination on the assembly pipeline. If an anomaly occurs, we will use the causal Bayesian network to identify the possible root cause and explain the event.

Project-4:AI Empowered Personalized Education

We have surveyed the latest AI technologies for personalized education and have identified synergies with some active projects on using AI for education at AIISC. There is growing use of KGs along with statistical AI (i.e., neural network based) techniques for EduTech solutions using AI for Education. One use of KG is for capturing domain knowledge. For example, the EduTech company Embibe.com has developed comprehensive KG for all subjects and courses it supports. Another use of KG is for creating personalized KG to capture the history of student’s interaction with an AI based education application (e.g., what are all the topics students has learned and how well so that the AI based application can guide the next learning objective and help teachers and the students with personalized assessment). At AIISC we are using AI for education through gaming (e.g., teach strategies for solving Rubrics cube) and support analogy-based pedagogy to improve learning about complex topics with the aid of analogy with more familiar concepts. We identify Dynamic KG to be another interesting application for personalized education. Specifically, the use of a Dynamic KG would allow us to keep the course materials up to date with the latest developments while catering to the emerging demands of students’ learning goals. We believe our active project on AI supported Analogy based personalized learning has an interesting use-case for effectively explaining difficult manufacturing concepts to students. For example, consider the case where students are learning about a new concept: Data Interoperability. Using the analogy about the role of a translator in multilingual communication, we can explain the need for data interoperability to ensure seamless communication between different computer systems and software applications. This approach was tested in the workforce development context as part of the Summer AI Camp for High School students (https://aiisc.ai/safari/).

TEAM MEMBERS

Advised By - Amit Sheth

1. Ruwan Wickramarachchi
2. Chathurangi Shyalika
3. Utkarshani Jamini
4. Revathy Venkatramanan
5. Vishal Pallagani

PUBLICATIONS

1. Shyalika, C., Wickramarachchi, R., & Sheth, A., A Comprehensive Survey on Rare Event Prediction. arXiv preprint "https://arxiv.org/pdf/2309.11356.pdf", 2023
2. Wickramarachchi, R., Henson, C., & Sheth, A., CLUE-AD: A Context-Based Method for Labeling Unobserved Entities in Autonomous Driving Data. 37th AAAI Conference on Artificial Intelligence (AAAI), February, 2023.
3. Wickramarachchi, R., C. Henson & A. Sheth, "Knowledge-Based Entity Prediction for Improved Machine Perception in Autonomous Systems," in IEEE Intelligent Systems, vol. 37, no. 5, pp. 42-49, 1 Sept.-Oct. 2022, doi: 10.1109/MIS.2022.3181015.
4. Kalach, F. E., R. Wickramarachchi, R. Harik & A. Sheth, "A Semantic Web Approach to Fault Tolerant Autonomous Manufacturing," in IEEE Intelligent Systems, vol. 38, no. 1, pp. 69-75, 1 Jan.-Feb. 2023, doi: 10.1109/MIS.2023.3235677.
5. Jaimini, Utkarshani, Tongtao Zhang, Georgia Olympia Brikis, & Amit Sheth. "iMetaverseKG: Industrial Metaverse Knowledge Graph to Promote Interoperability in Design and Engineering Applications." IEEE Internet Computing 26, no. 6 (2022): 59-67.
6. Jaimini, Utkarshani, & Amit Sheth. "CausalKG: Causal Knowledge Graph Explainability using interventional and counterfactual reasoning." IEEE Internet Computing 26, no. 1 (2022): 43-50.
7. R. Venkataramanan, A. Tripathy, M. Foltin, H. Y. Yip, A. Justine & A. Sheth, "Knowledge Graph Empowered Machine Learning Pipelines for Improved Efficiency, Reusability, and Explainability," in IEEE Internet Computing, vol. 27, no. 1, pp. 81-88, 1 Jan.-Feb. 2023, doi: 10.1109/MIC.2022.3228087.
8. Shyalika, C., Wickramarachchi, R., & Sheth, A., Towards Rare Event Prediction in Manufacturing Domain. CSE Research Symposium, University of South Carolina, April 14, 2023. [Poster presentation]
9. Shyalika, C., Wickramarachchi, R., & Sheth, A., Multivariate Data Augmentation for Rare Event Prediction in Manufacturing. CRA-WP Grad Cohort for Women, April 21, 2023. [Poster presentation]
10. Shyalika, C., Wickramarachchi, R., Harik, R., & Sheth, A., Evaluating the Role of Multivariate Data Augmentation Towards Rare Event Prediction in Manufacturing. Discover USC Symposium, University of South Carolina, April 21, 2023. [Poster presentation]

Blog Posts

1. Shyalika, C., Wickramarachchi, R., & Sheth, A., A Comprehensive Survey on Rare Event Prediction. AIHub, Available at "https://aihub.org/2023/11/22/a-comprehensive-survey-on-rare-event-prediction/", 2023

FUNDING

• NSF Award#: 2119654
• RII Track 2 FEC: Enabling Factory to Factory (F2F) Networking for Future Manufacturing
• Timeline: Oct 2021 - Sept 2025
• Award Amount: $739,239

---

• NSF Award#: 2133842
• EAGER: Advancing Neuro-symbolic AI with Deep Knowledge-infused Learning
• Timeline: 01 July 2021 - 30 June 2022
• Award Amount: $139,999