Smart manufacturing

From Knoesis wiki
Jump to: navigation, search

The utilization of AI in manufacturing processes has ushered in a new era of data-driven optimization and problem-solving. AI is a pivotal cornerstone, especially in Industry 4.0 and 5.0 applications, yielding advanced technologies to guide a new era of heightened efficiency, productivity, and refined decision-making. In this capacity, AI emerges as an indispensable force driving transformative advancements within the manufacturing domain, where data-driven insights and smart automation catalyze innovation and shape the future of industrial processes. The assessment of SoA approaches in the manufacturing realm helped us identify three concrete use cases for further investigation: (1) Rare event prediction, (2) Anomaly detection using process knowledge workflow, and (3) Explaining the root cause of an event through causal knowledge.


Project-1:Rare Event Prediction

There are different types of events that can be observed within autonomous systems. For example, our consideration on this topic led us to explore “rare events” as they are particularly important for smart manufacturing applications. Rare events are occurrences that take place with a significantly lower frequency than more common regular events. In manufacturing, predicting such events is particularly important, as they lead to unplanned downtime, shortening equipment lifespan, and high energy consumption. The occurrence of events is usually considered rare if observed in 5-10% of all instances, while the occurrence of less than 1% is considered extremely rare. The rarity of events is inversely correlated with the maturity of a manufacturing industry. Typically, the rarity of events causes the multivariate data generated within a manufacturing process to be highly imbalanced, which leads to bias in predictive models. To this end, we conducted a thorough survey to investigate prior art along four dimensions: (i) data acquisition methods, (ii) data preprocessing techniques, (iii) algorithmic approaches, and (iv) evaluation/assessment approaches. Figure X below summarizes the methods we looked at within each dimension

Regarding modeling rare event prediction, we first evaluated the role of data enrichment approaches combined with supervised machine-learning techniques. Predominantly, we explored using three data enrichment approaches; data augmentation, sampling and imputation. Our study is based on open real-world datasets we obtained from public data sources. Figure 1 includes the approaches to learning from rare event data that we exemplified from the comprehensive survey.

Figure 1: Approaches to learning from rare event data

Project-2:RI2AP: Robust and Interpretable 2D Anomaly Prediction in Assembly Pipelines

Predicting anomalies in manufacturing assembly lines is crucial for reducing time and labor costs and improving processes. For instance, in rocket assembly, premature part failures can lead to significant financial losses and labor inefficiencies. With the abundance of sensor data in the Industry 4.0 era, machine learning (ML) offers potential for early anomaly detection. However, current ML methods for anomaly prediction have limitations, with F1 measure scores of only 50% and 66% for prediction and detection, respectively. This is due to challenges like the rarity of anomalous events, scarcity of high-fidelity simulation data (actual data are expensive), and the complex relationships between anomalies not easily captured using traditional ML approaches. Specifically, these challenges relate to two dimensions of anomaly prediction: predicting when anomalies will occur and understanding the dependencies between them. This paper introduces a new method called Robust and Interpretable 2D Anomaly Prediction (RI2AP) designed to address both dimensions effectively. RI2AP is demonstrated on a rocket assembly simulation, showing up to a 30-point improvement in F1 measure compared to current ML methods. This highlights its potential to enhance automated anomaly prediction in manufacturing. Additionally, RI2AP includes a novel interpretation mechanism inspired by a causal-influence framework, providing domain experts with valuable insights into sensor readings and their impact on predictions. Finally, the RI2AP model was deployed in a real manufacturing setting for assembling rocket parts. Results and insights from this deployment demonstrate the promise of RI2AP for anomaly prediction in manufacturing assembly pipelines.

Figure 2: R12AP

Project-3:Multimodal Hybrid Fusion for Robust and Explainable Anomaly Prediction in Assembly Pipelines

In modern assembly pipelines, identifying anomalies is crucial to ensuring product quality and operational efficiency. In complex manufacturing environments, contemporary methods frequently prove inadequacy in managing the intricacies of multi-modal data. This project proposes a novel hybrid fusion approach for multi-modal anomaly prediction in assembly pipelines. Our research builds upon two primary approaches in multi-modal learning: time series-image fusion learning models and joint models for time series and image representation learning. We review existing baselines, including feature-level fusion, where vision and time-series data are combined at the beginning using a unified model, and decision-level fusion, where separate models for each modality are fused mid-process. We introduce a hybrid fusion mechanism that leverages both feature-level and decision-level fusion techniques. The proposed approach involves a two-stage process: data preprocessing and hybrid fusion modeling. We demonstrate and evaluate the new method using the multi-modal dataset that we have made public and conduct comprehensive ablation studies to evaluate the impact of our preprocessing and hybrid fusion stages compared to traditional baselines. This study demonstrates that a hybrid fusion approach can effectively harness the complementary strengths of time-series and image data, offering a robust solution for anomaly prediction in assembly pipelines and provide higher accuracy in predicting anomalies.

Project-4:Process Knowledge Workflow using PDDL

We proposed Process Knowledge Workflow (PKW) using Planning Domain Definition Language (PDDL) to detect anomalies in the assembly line (Figure 2&3). Specifically, the PKW uses multimodal data from robot sensors, conveyor belt sensors and camera to detect anomalies. The images from the camera will be processed using state-of-the-art object detection techniques to identify the objects of interest and the interactions among them. The data from the sensors will be utilized to detect the movement of the desired objects. Using this information, PKW will be constructed for the expected process. Then the PKW can serve as an orchestrator to identify anomalies in the sensor values. Based on the sensor values, the PKW will also have the ability to define and name the anomalies. We conducted an extensive survey of the list of sensors in the assembly line from neXt Future Factories lab and studied the type and range of data being generated. Using this, the first version of the PKW ontology will be constructed. A demonstration of PKW can be found in Figure 2.

Figure 3: Process Knowledge Workflow (PKW)

Figure 4: Monitoring Events at Each Step

Project-5:Explaining the root cause of an event through causal knowledge

Our goal is to utilize the process knowledge workflow to construct a sequential causal Bayesian network (Figure 4) that captures both the input-output sequence of the process and the agent information. Specifically, we aim to create a model that can track the cause-and-effect relationships between the input commands, robot actions, and pipeline outputs in the assembly process. By doing so, we will be able to predict the impact of a given input/output and action combination on the assembly pipeline. If an anomaly occurs, we will use the causal Bayesian network to identify the possible root cause and explain the event.

Figure 5: Sequential Causal Bayesian Network

Project-6:AI Empowered Personalized Education

We have surveyed the latest AI technologies for personalized education and have identified synergies with some active projects on using AI for education at AIISC. There is growing use of KGs along with statistical AI (i.e., neural network based) techniques for EduTech solutions using AI for Education. One use of KG is for capturing domain knowledge. For example, the EduTech company has developed comprehensive KG for all subjects and courses it supports. Another use of KG is for creating personalized KG to capture the history of student’s interaction with an AI based education application (e.g., what are all the topics students has learned and how well so that the AI based application can guide the next learning objective and help teachers and the students with personalized assessment). At AIISC we are using AI for education through gaming (e.g., teach strategies for solving Rubrics cube) and support analogy-based pedagogy to improve learning about complex topics with the aid of analogy with more familiar concepts. We identify Dynamic KG to be another interesting application for personalized education. Specifically, the use of a Dynamic KG would allow us to keep the course materials up to date with the latest developments while catering to the emerging demands of students’ learning goals. We believe our active project on AI supported Analogy based personalized learning has an interesting use-case for effectively explaining difficult manufacturing concepts to students. For example, consider the case where students are learning about a new concept: Data Interoperability. Using the analogy about the role of a translator in multilingual communication, we can explain the need for data interoperability to ensure seamless communication between different computer systems and software applications. This approach was tested in the workforce development context as part of the Summer AI Camp for High School students. See (

AI in manufacturing presentation for high school students

Industrial visit to McNair Aerospace Center


Advised By - Dr.Amit Sheth

AIISC Collaborators:

  1. Ruwan Wickramarachchi
  2. Chathurangi Shyalika
  3. Utkarshani Jamini
  4. Revathy Venkatramanan
  5. Vishal Pallagani
  6. Yuxin Zi
  7. Kaushik Roy
  8. Renjith Prasad

External Collaborators:

  1. Fadi El Kalach


  1. Dr.Ramy Harik
  2. Dr.Vignesh Narayanan


  1. El Kalach, F., Yousif, I., Wuest, T., R. Harik & A. Sheth, Cognitive manufacturing: definition and current trends. Journal of Intelligent Manufacturing (2024),
  2. Jaimini, U., Henson C., & Sheth, A., "Causal Neurosymbolic AI: A Synergy Between Causality and Neurosymbolic Methods" IEEE Intelligent Systems vol. 39, no. 3 (2024).
  3. Shyalika, C., Roy, K., Prasad, R., Kalach, F. E., Zi, Y., Mittal, P., Narayanan, V., R. Harik & A. Sheth, (2024). RI2AP: Robust and Interpretable 2D Anomaly Prediction in Assembly Pipelines. Sensors, 24(10), 3244.
  4. Shyalika, C.; Roy, K.; Prasad, R.; Kalach, F.E.; Zi, Y.; Mittal, P.; Narayanan, V.; Harik, R.; Sheth, A. RI2AP: Robust and Interpretable 2D Anomaly Prediction in Assembly Pipelines. Preprints 2024, 2024041959.
  5. Shyalika, C., Wickramarachchi, R., & Sheth, A., A Comprehensive Survey on Rare Event Prediction. arXiv preprint "", 2023
  6. Wickramarachchi, R., Henson, C., & Sheth, A., CLUE-AD: A Context-Based Method for Labeling Unobserved Entities in Autonomous Driving Data. 37th AAAI Conference on Artificial Intelligence (AAAI), February, 2023.
  7. Wickramarachchi, R., C. Henson & A. Sheth, "Knowledge-Based Entity Prediction for Improved Machine Perception in Autonomous Systems," in IEEE Intelligent Systems, vol. 37, no. 5, pp. 42-49, 1 Sept.-Oct. 2022, doi: 10.1109/MIS.2022.3181015.
  8. Kalach, F. E., R. Wickramarachchi, R. Harik & A. Sheth, "A Semantic Web Approach to Fault Tolerant Autonomous Manufacturing," in IEEE Intelligent Systems, vol. 38, no. 1, pp. 69-75, 1 Jan.-Feb. 2023, doi: 10.1109/MIS.2023.3235677.
  9. Jaimini, U., Henson C., & Sheth, A., "An Ontology Design Pattern for Representing Causality" ISWC 2023 Workshop on Ontology Design Pattern.
  10. Jaimini, U., Tongtao, Z., Georgia, B., & Sheth, A., "iMetaverseKG: Industrial Metaverse Knowledge Graph to Promote Interoperability in Design and Engineering Applications." IEEE Internet Computing 26, no. 6 (2022): 59-67.
  11. Jaimini, U., & Sheth, U., "CausalKG: Causal Knowledge Graph Explainability using interventional and counterfactual reasoning." IEEE Internet Computing 26, no. 1 (2022): 43-50.
  12. R. Venkataramanan, A. Tripathy, M. Foltin, H. Y. Yip, A. Justine & A. Sheth, "Knowledge Graph Empowered Machine Learning Pipelines for Improved Efficiency, Reusability, and Explainability," in IEEE Internet Computing, vol. 27, no. 1, pp. 81-88, 1 Jan.-Feb. 2023, doi: 10.1109/MIC.2022.3228087.


  1. Shyalika, C., Roy, K., Prasad, R., Zi, Y., Mittal, P., Kalach, F. E., Narayanan, V., R. Harik & A. Sheth, “RI2AP: Robust and Interpretable 2D Anomaly Prediction in Assembly Pipelines”. DiscoverUSC-2024, University of South Carolina, April 19, 2024. [Honoured with a "Best Poster" award]
  2. Shyalika, C., Roy, K., Prasad, R., Zi, Y., Mittal, P., Kalach, F. E., Narayanan, V., R. Harik & A. Sheth, “RI2AP: Robust and Interpretable 2D Anomaly Prediction in Assembly Pipelines”. CSE Research Symposium, University of South Carolina, March 15, 2024.
  3. Shyalika, C., Wickramarachchi, R., & Sheth, A., Towards Rare Event Prediction in Manufacturing Domain. CSE Research Symposium, University of South Carolina, April 14, 2023.
  4. Shyalika, C., Wickramarachchi, R., & Sheth, A., Multivariate Data Augmentation for Rare Event Prediction in Manufacturing. CRA-WP Grad Cohort for Women, April 21, 2023.
  5. Shyalika, C., Wickramarachchi, R., Harik, R., & Sheth, A., Evaluating the Role of Multivariate Data Augmentation Towards Rare Event Prediction in Manufacturing. Discover USC Symposium, University of South Carolina, April 21, 2023.


  1. Shyalika, C., Wickramarachchi, R., & Sheth, A., A Comprehensive Survey on Rare Event Prediction. AIHub, Available at "", 2023


  • NSF Award#: 2119654
  • RII Track 2 FEC: Enabling Factory to Factory (F2F) Networking for Future Manufacturing
  • Timeline: Oct 2021 - Sept 2025
  • Award Amount: $739,239

  • NSF Award#: 2133842
  • EAGER: Advancing Neuro-symbolic AI with Deep Knowledge-infused Learning
  • Timeline: 01 July 2021 - 30 June 2022
  • Award Amount: $139,999