METACOG-26: Symposium on Artificial Metacognition

We are proposing METACOG-2026 to be part of the AAAI Fall Symposium. This page is designed to provide some basic information.

Symposium Description

Artificial Metacognition is the ability for an AI-enabled system to reason about itself.  The idea is rooted in cognitive psychology [1, 2]. In the early 2000s, AAAI held several symposia on this topic [3, 4]. Recently, due to the advent of the large language model (LLM), neurosymbolic AI, and the need for more robust, resilient and secure AI-enabled systems, this topic has re-emerged in a series of small venues [5, 6],  an “emerging trends” talk at AAAI-2026 [7], and a forthcoming special issue of IEEE Intelligent Systems.  The study of metacognition goes beyond related topics such as out-of-distribution detection and uncertainty quantification by not only detecting when a model could potentially be in an error mode, but determining aspects about security, computational efficiency, power usage, explainability, and corrective action in a unified, often cognitively inspired, framework.  Recent events have brought together researchers from computer science, cognitive psychology, electrical engineering, mechanical engineering, systems engineering, and mathematics.  The proposed Symposium on Artificial Metacognition will continue this exploration by inviting papers featuring a variety of methodologies that have been explored in the recent literature, including stress testing of robotic systems, model introspection, model certification, performance prediction, critique models for LLMs, metacognitive rule learning, and metacognitive extensions to cognitive architectures (e.g.,  ACT-R and the Common Model of Cognition).

Objectives

The objectives of the symposium are as follows:

  • Survey and synthesize current approaches to metacognition in AI-enabled systems, including monitoring, control, and metareasoning.
  • Understand the requirements for and trade-offs among various metacognitive approaches.
  • Identify novel methods for metacognition that improve AI performance in operational, out-of-distribution, and cross-domain settings.
  • Identify application areas suitable for the deployment of metacognitive methods, including autonomy, cyber, vision, health, intelligence, robotics, and decision support.
  • Foster cross-disciplinary collaboration between AI, cognitive psychology, cognitive modeling, control theory, and systems engineering.
  • Examine the relationship between artificial metacognition and human operators, including trust, calibration, and human-AI teaming.

Topics of Interest

Specific topics to be covered include, but are not limited to:

  • AI Agents with Metacognition (LLM-based agents, autonomous agents, and embodied agents that self-monitor and self-regulate).
  • Cognitive model architectures with metacognitive extensions (e.g., ACT-R, Soar, the Common Model of Cognition).
  • Metacognitive rule learning (data-driven and neuro-symbolic learning of error-detection and constraint rules).
  • Critique models (training, evaluation, and deployment of models that produce natural-language feedback on the outputs of other AI-enabled systems).
  • Explainable performance prediction of black-box AI-enabled systems.
  • Stress testing of reinforcement learning and perception systems.
  • Metacognitive monitoring vs. metacognitive control, including metareasoning and resource regulation.
  • Neuro-symbolic AI architectures for metacognition.
  • Self-adaptive, self-healing, and self-repairing AI-enabled systems for new domains.
  • Out-of-distribution detection and correction, abductive inference, and consistency-based verification as metacognitive cues.
  • Trust calibration, human-in-the-loop metacognition, and human-AI teaming.
  • Datasets, benchmarks, and evaluation methodology for artificial metacognition.
  • Applications of artificial metacognition to autonomy, robotics, cyber operations, and decision support.

Symposium Format

METACOG-26 will be a 2.5-day fully in-person symposium following the standard AAAI Fall Symposium structure. Programming will combine traditional paper sessions with extended discussion, a poster session, three invited keynotes, and a closing panel. This structure is consistent with the symposium series’ emphasis on intimate forums and substantive discussion. We anticipate accepting up to 14 full papers (oral) and up to 12 posters. Submission will be peer-reviewed by the organizing committee and a small program committee drawn from the prior workshops. Accepted papers will be published in the AAAI Technical Report series.

Organizing Committee

Paulo Shakarian (Syracuse)

Nathaniel D. Bastian (DARPA)

Francesco Restuccia (Northeastern)

Christian Lebiere (Carnegie Mellon)

Arslan Basharat (KitWare)

Andrea Stocco (University of Washington)

Keynotes

John Laird. Laird is the John L. Tishman Emeritus Professor of Engineering at the University of Michigan and Co-Director of the Center for Integrated Cognition. He is one of the original developers of the Soar cognitive architecture, begun with Allen Newell and Paul Rosenbloom in the early 1980s, and is a foundational figure in research on general intelligent agents and unified theories of cognition. His work spans cognitive architecture, reinforcement learning, problem solving, knowledge representation, and interactive task learning, and he has led long-running efforts to extend Soar with episodic and semantic memory, mental imagery, and metacognitive control. Laird is a Fellow of AAAI, ACM, and the Cognitive Science Society, and a past recipient of the AAAI Classic Paper Award. He brings deep expertise in cognitive architectures as a substrate for artificial metacognition.

Mark Steyvers. Steyvers is Professor and Chair of the Department of Cognitive Sciences at the University of California, Irvine, with joint appointments in the Donald Bren School of Information and Computer Sciences and in Psychological Science in the School of Social Ecology, and is affiliated with the Center for Machine Learning and Intelligent Systems. He received his Ph.D. from Indiana University in 2000 and completed postdoctoral training at Stanford University. His research focuses on high-level cognition — including metacognition, wisdom of crowds, episodic and semantic memory, decision making, and human-AI collaboration — combining computational modeling, machine learning, and behavioral experiments. He has been funded by NSF, NIH, IARPA, the U.S. Navy, and AFOSR, and is a past President of the Society of Mathematical Psychology. He brings expertise on metacognitive judgment, calibration, and human-AI teaming — areas central to the symposium’s cognitive-science track.

Susmit Jha. Jha is a Program Manager in the Information Innovation Office (I2O) at the Defense Advanced Research Projects Agency (DARPA), which he joined in August 2025. His research focuses on trustworthy and steerable generative AI, controllable multi-agent systems, and adversarial robustness for AI deployed in dynamic, real-world settings. Before joining DARPA, Jha was a Technical Director in the Computer Science Laboratory at SRI International, where he led the Neuro-Symbolic Computing and Intelligence research group, with funding from DARPA, IARPA, ARPA-H, NSA, ARL, DLA, and NSF. He served on the DARPA ISAT Study Group (2023–2025), has published over 100 peer-reviewed papers in venues such as NeurIPS, ICML, ICLR, AAAI, and CVPR, and received the 10-Year Most Influential Paper Award at IEEE/ACM ICSE 2020 for his work on program synthesis. He holds a Ph.D. in Computer Science from UC Berkeley.

Sponsorship

KitWare

Please contact Paulo Shakarian (pashakar <at> syr.edu) for inquiries about sponsorship.

References

[1] Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906–911.

[2] Nelson, T. O., & Narens, L. (1990). Metamemory: A theoretical framework and new findings. In G. H. Bower (Ed.), The Psychology of Learning and Motivation, Vol. 26 (pp. 125–173). Academic Press.

[3] Anderson, M. L., & Oates, T. (Eds.). (2005). Metacognition in Computation: Papers from the 2005 AAAI Spring Symposium (March 21–23, 2005, Stanford, CA). AAAI Technical Report SS-05-04. AAAI Press.

[4] Cox, M. T., & Raja, A. (Eds.). (2011). Metareasoning: Thinking about Thinking. MIT Press. (Based on the AAAI-08 Workshop on Metareasoning, Chicago, July 2008.)

[5] Shakarian, P., & Wei, H. (Eds.). (2025). Metacognitive Artificial Intelligence. Cambridge University Press. (Proceedings of METACOG-23, the First Workshop on Metacognitive Prediction of AI Behavior, Nov. 13–15, 2023, Scottsdale, AZ.)

[6] METACOG-25: Second Workshop on Metacognitive Prediction of AI Behavior, held in conjunction with the SIAM International Conference on Data Mining (SDM-25), May 1, 2025, Alexandria, VA.

[7] Shakarian, P. (2026). Toward Artificial Metacognition. In Proceedings of the AAAI Conference on Artificial Intelligence (Emerging Trends in AI track), Singapore, January 2026.