Neurosymbolic Artificial Intelligence Research at AIISC

From Knoesis wiki
Revision as of 15:34, 1 March 2024 by Utkarshanijaimini (Talk | contribs) (→‎Journal Publications)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Neurosymbolic AI Overview

Humans interact with the environment using a combination of perception transforming sensory inputs from their environment into symbols, and cognition - mapping symbols to knowledge about the environment for supporting abstraction, reasoning by analogy, and long-term planning. Human perception-inspired machine perception, in the context of artificial intelligence (AI), refers to large-scale pattern recognition from raw data using neural networks trained using self-supervised learning objectives such as next-word prediction or object recognition. On the other hand, machine cognition encompasses more complex computations, such as using knowledge of the environment to guide reasoning, analogy, and long-term planning. Humans can also control and explain their cognitive functions. This seems to require the retention of symbolic mappings from perception outputs to knowledge about their environment. For example, humans can follow and explain the guidelines and safety constraints driving their decision making in safety-critical applications such as health care, criminal justice, and autonomous driving.

Embodying intelligent behavior in an artificial intelligence system must involve both perception, processing raw data and cognition, using background knowledge to support abstraction, analogy, reasoning, and planning. Symbolic structures represent this background knowledge explicitly. Although neural networks are a powerful tool for processing and extracting patterns from data, they lack explicit representations of background knowledge, hindering the reliable evaluation of their cognition capabilities. Furthermore, applying appropriate safety standards while providing explainable outcomes guided by concepts from background knowledge is crucial for establishing trustworthy models of cognition for decision support.

Outcomes Achieved So far

  1. The rapid improvement in language models suggests that they will achieve almost optimal performance levels for large-scale perception. Knowledge graphs are suitable for symbolic structures that bridge the cognition and perception aspects because they support real-world dynamism. Unlike static and brittle symbolic logics, such as first-order logic, they are easy to update. In addition to their suitability for enterprise use cases and established standards for portability, knowledge graphs are part of a mature ecosystem of algorithms that enable highly efficient graph management and querying. This scalability allows for modeling large and complex datasets with millions or billions of nodes.
  2. We find that combining language models and knowledge graphs are most effective in current implementations. However, it also suggests that future knowledge graphs have the potential to model heterogeneous types of application- and domain-level knowledge beyond schemas. This includes workflows, constraint specifications, and process structures
  3. Combining such enhanced knowledge graphs with high-capacity neural networks would provide the end user with an extremely high degree of algorithmic- and application-level utility. The concern for safety is behind the recent push to withhold further rollout of generative AI systems such as GPT* as current systems could significantly harm individuals and society without additional guardrails. We believe that guidelines, policy, and regulations can be encoded via extended forms of knowledge graphs, which in turn can provide explainability accountability, rigorous auditing capabilities, and safety. Encouragingly, progress is being made on all these fronts swiftly, and the future looks promising

Journal Publications

  1. Purohit, H., Shalin, V. L., Sheth, A. P., & Sheth, A. (2020). Knowledge graphs to empower humanity-inspired AI systems. IEEE Internet Computing, 24(4), 48–54. https://doi.org/10.1109/MIC.2020.3013683
  2. Gaur, M., Faldu, K., & Sheth, A. (2021). Semantics of the black-box: Can knowledge graphs help make deep learning systems more interpretable and explainable? IEEE Internet Computing, 25 (1), Jan-Feb 2021, pp. 51–59. https://doi.org/10.1109/MIC.2020.3031769
  3. Sheth, A., & Thirunarayan, K. (2021). The duality of data and knowledge across the three waves of AI. IT Professional, 23(3), 35–45. https://doi.org/10.1109/MITP.2021.3070985
  4. Sheth, A., Gaur, M., Roy, K., & Faldu, K. (2021). Knowledge-intensive language understanding for explainable ai. IEEE Internet Computing, 25(5), 19-24. (Link: https://ieeexplore.ieee.org/abstract/document/9514440)
  5. Sheth, A., Gaur, M., Roy, K., Venkataraman, R., & Khandelwal, V. (2022). Process knowledge-infused ai: Toward user-level explainability, interpretability, and safety. IEEE Internet Computing, 26(5), 76-84. (Link: https://ieeexplore.ieee.org/abstract/document/9889132)
  6. Jaimini, U. and Sheth, A., 2022. CausalKG: Causal Knowledge Graph Explainability using interventional and counterfactual reasoning. IEEE Internet Computing, 26(1), pp.43-50. (Link: https://ieeexplore.ieee.org/abstract/document/9706608)
  7. Sheth, A., Roy, K., & Gaur, M. (2023). Neurosymbolic Artificial Intelligence (Why, What, and How). IEEE Intelligent Systems, 38(3), 56-62. (Lin: https://ieeexplore.ieee.org/abstract/document/10148662/)
  8. Sheth, A., & Roy, K. (2024). Neurosymbolic Value-Inspired AI (Why, What, and How). IEEE Intelligent Systems. (Link: https://arxiv.org/pdf/2312.09928)

Conference, Symposium, and Workshop Publications

  1. Pallagani, V., Roy, K., Muppasani, B., Fabiano, F., Loreggia, A., Murugesan, K., ... & Sheth, A. (2024). On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS). International Conference on Automated Planning and Scheduling (ICAPS). (Link: https://arxiv.org/pdf/2401.02500)
  2. Zi, Y., Veeramani, H., Roy, K., & Sheth, A. (2024). RDR: the Recap, Deliberate, and Respond Method for Enhanced Language Understanding. AAAI Workshop on Neuro-Symbolic Learning and Reasoning in the era of Large Language Models. (Link: https://openreview.net/forum?id=hNQJI0KS3T)
  3. Zi, Y., Roy, K., Narayanan, V., & Sheth, A. (2024). Exploring Alternative Approaches to Language Modeling for Learning from Data and Knowledge. AAAI Spring Symposium on Empowering Machine Learning and Large Language Models with Domain and Commonsense Knowledge. (Link: https://scholarcommons.sc.edu/cgi/viewcontent.cgi?article=1619&context=aii_fac_pub)
  4. Roy, K., Oltramari, A., Zi, Y., Shyalika, C., Narayanan, V., & Sheth, A. (2024). Causal Event Graph-Guided Language-based Spatiotemporal Question Answering. AAAI Spring Symposium on Empowering Machine Learning and Large Language Models with Domain and Commonsense Knowledge. (Link: https://scholarcommons.sc.edu/cgi/viewcontent.cgi?article=1618&context=aii_fac_pub)
  5. Roy, Kaushik, et al. "Knowledge-Infused Self Attention Transformers." KDD Workshop on Knowledge Augmented Methods for NLP (2023). (Link: https://arxiv.org/pdf/2306.13501)
  6. Jaimini, Utkarshani, Cory Henson, and Amit Sheth. "An Ontology Design Pattern for Representing Causality." 14th Workshop on Ontology Design Pattern at 22nd International Semantic Web Conference (2023). (Link: https://scholarcommons.sc.edu/cgi/viewcontent.cgi?article=1615&context=aii_fac_pub)
  7. Roy, Kaushik, et al. "Process Knowledge-infused Learning for Clinician-friendly Explanations." AAAI Second Symposium on Human Partnership with Medical Artificial Intelligence (2023). (Link: https://arxiv.org/pdf/2306.09824)
  8. Roy, Kaushik, and Vipula Rawte. "TDLR: Top Semantic-Down Syntactic Language Representation." NeurIPS'22 Workshop on All Things Attention: Bridging Different Perspectives on Attention. 2022. (Link: https://attention-learning-workshop.github.io/2022/papers/rawte-tdlr_top_semanticdown_syntactic_language_representation.pdf)
  9. Keyur Faldu, Amit Sheth, Prashant Kikani, Hemang Akbari. KI-BERT: Infusing Knowledge Context for Better Language and Domain Understanding. arXiv:2104.08145 [cs.CL] https://doi.org/10.48550/arXiv.2104.08145
  10. Kursuncu, U., Gaur, M., & Sheth, A. (2020). Knowledge infused learning (K-IL): Towards deep incorporation of knowledge in deep learning. Proceedings of the AAAI 2020 Spring Symposium on Combining Machine Learning and Knowledge Engineering in Practice (AAAI-MAKE).
  11. Gaur, M., Desai, A., Faldu, Keyur, & Sheth, A. (2020). Explainable AI using knowledge graphs. ACM CoDS-COMAD Conference. https://aiisc.ai/xaikg/
  12. Roy, K., Zhang, Q., Gaur, M., & Sheth, A. (2021). Knowledge infused policy gradients for adaptive pandemic control. Proceedings of the AAAI 2021 Spring Symposium on Combining Machine Learning and Knowledge Engineering (AAAI-MAKE 2021).
  13. Roy, K., Zhang, Q., Gaur, M., & Sheth, A. (2021). Knowledge infused policy gradients with upper confidence bound for relational bandits, In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 35-50. Springer, Cham, 2021.

Application Paper Publications

  1. Wickramarachchi, R., Henson, C., & Sheth, A. (2021). Knowledge-infused learning for entity prediction in driving scenes. Frontiers in big Data, 4, 759110. (Link: https://www.frontiersin.org/articles/10.3389/fdata.2021.759110/full)
  2. Venkataramanan, R., Padhee, S., Rao, S. R., Kaoshik, R., Rajan, A. S., & Sheth, A. (2023). Ki-Cook: clustering multimodal cooking representations through knowledge-infused learning. Frontiers in Big Data, 6. (Link: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10406211/)
  3. Wickramarachchi, R., Henson, C., & Sheth, A. (2023, June). CLUE-AD: a context-based method for labeling unobserved entities in autonomous driving data. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 37, No. 13, pp. 16491-16493). (Link: https://ojs.aaai.org/index.php/AAAI/article/view/27089/26861)
  4. Roy, K., Khandelwal, V., Goswami, R., Dolbir, N., Malekar, J., & Sheth, A. (2023). Demo Alleviate: Demonstrating artificial intelligence-enabled virtual assistance for telehealth: The mental health case. Thirty-Seventh AAAI Conference on Artificial Intelligence. (Link: https://scholarcommons.sc.edu/aii_fac_pub/566/)
  5. Roy, K., Gaur, M., Rawte, V., Kalyan, A., & Sheth, A. (2023). ProKnow: Process Knowledge for Safety Constrained and Explainable Question Generation for Mental Health Diagnostic Assistance., Frontiers Big Data, 09 January 2023 (Link : https://www.frontiersin.org/articles/10.3389/fdata.2022.1056728/full)
  6. Roy, K., Khandelwal, V., Surana, H., Vera, V., Sheth, A., & Heckman, H. (2024). GEAR-Up: Generative AI and External Knowledge-based Retrieval Upgrading Scholarly Article Searches for Systematic Reviews. AAAI Conference on Artificial Intelligence. (Link: https://arxiv.org/pdf/2312.09948)

Keynotes, Tutorials and Talks

  1. Sheth, A., Knowledge-infused NLU for Addiction and Mental Health Research, Keynotes at the MAISoN21, Aug 2021, and the ASONAM21, Sept 2021.Presentation-GDriveVideo-MAISoN@IJCAI
  2. Sheth, A., Don’t Handicap AI without Explicit Knowledge, Keynotes at the IEEE Services, Sept 2021, and the DEXA2021, Oct 2021. Presenation-GDriveVideo IEEE SErvicesVideo-DEXA
  3. Amit Sheth, Semantics of the Black-Box: Towards Knowledge-infused Learning --- Keynote talk at the Semantic Machine Learning Workshop in conjunction with the 15th IEEE International Conference on Semantic Computing, February 2021, https://youtu.be/cx-l0XDk9Tw
  4. Sheth, A., Augmented Personalized Health: an explicit knowledge enhanced neural symbolic Health approach to patient empowerment for managing chronic disease burden. Keynote at the SWAT4HCLS (Semantic Web Applications and Tools for Healthcare and Life Sciences), ​​January 2022. SlidesharePresentation-GDriveVideo
  5. Amit Sheth, From NLP to NLU: Why we need varied, comprehensive, and stratified knowledge (Neuro-symbolic AI), Keynote at KnowledgeNLP at AAAI2023, February 2023. Slides Abstract Video
  6. Roy, K. (2024). Knowledge-infused Neurosymbolic Artificial Intelligence for Mental Healthcare. Intelligent Clinical Care Center, University of Florida. (Slides: https://lnkd.in/eVCzinAk)

Links to previous years' publications: http://wiki.aiisc.ai/index.php/Advancing_Neuro-symbolic_AI_with_Deep_Knowledge-infused_Learning, and https://wiki.aiisc.ai/index.php?title=EAGER:_Knowledge-guided_neurosymbolic_AI_with_guardrails_for_safe_virtual_health_assistants

Funding

EAGER: Advancing Neuro-symbolic AI with Deep Knowledge-infused Learning

  • NSF Award#: 2133842
  • Award Period of Performance:   2022-2024
  • Award Amount: $139,999

EAGER: Knowledge-guided neurosymbolic AI with guardrails for safe virtual health assistants

  • NSF Award #: 2335967
  • Award Period of Performance:   2023-2025
  • Award Amount: $200,000

Personnel

Retrieved from "https://wiki.aiisc.ai/index.php?title=Neurosymbolic_Artificial_Intelligence_Research_at_AIISC&oldid=13667"