Compositionality and Reasoning in AI and Cognitive Science

About the Workshop

In neural models, generalization is the ability to apply learned knowledge to new, unseen data. Compositionality is a principle that enables such generalization by allowing complex structures to be represented and processed as combinations of simpler elements, which provides a systematic way to interpret new structures. Together, they are a key factor in bridging the gap between learning and genuine adaptability in AI.
This one‑day workshop aims to discuss compositionality, structured representation, and the integration of neural and symbolic reasoning in modern learning systems.

Our research:

Our work investigates hybrid models of reasoning, a neuro-symbolic approach to deductive inference that integrates neural learning with symbolic logic within restricted fragments of natural language. We began with a pilot study [1] on a simple propositional corpus to examine whether neural networks can assist a symbolic prover by selecting necessary formulas from a knowledge base to prove a given hypothesis. We then extended this work [2] to the syllogistic fragment, evaluating feedforward, recurrent, convolutional, and transformer architectures. Despite the simplicity of the experimental setup—training and testing on a single knowledge base with one-hot encoded inputs—our results indicated that models trained from scratch failed to capture the underlying logical structure. To validate and extend these findings, we next employed modern pretrained language models for formula selection in direct and indirect proofs, integrating neural assistants with a symbolic prover to evaluate their interaction within a hybrid reasoning framework [3]. In this phase, models were trained on multiple knowledge bases and tested on unseen ones, using pseudoword-based textual representations for both input and output. To further investigate the compositionality limitation in neural models, and in collaboration with researchers from the University of Trento, we explored a meta-learning approach on our syllogistic corpus [4] to study how models adapt to novel reasoning patterns in the formula selection task.

Compositionality is a fundamental component of reasoning and continues to challenge neural models. Our results suggest that hybrid architectures integrating symbolic inference with neural learning offer a promising path toward overcoming these limitations and, more broadly, a compelling direction for future exploration—shedding light on how structured reasoning can emerge from pattern-based learning systems in AI.

The research project “Hybrid Models of Reasoning”, funded by the National Science Centre, Poland, is conducted by Maciej Malicki (University of Warsaw), Jakub Szymanik (University of Trento), and Manuel Vargas Guzmán (University of Warsaw).

Published Papers

[1] Compositionality in a simple corpus

Show Abstract

We investigate the capacity of neural networks (NNs) to learn compositional structures by focusing on a well-defined simple logical corpus, and on proof-centered compositionality. We conduct our investigation in a minimal setting by creating a simple logical corpus, where all compositionality-related phenomena come from the structure of proofs as all the sentences of the corpus are propositional logic implications. By training NNs on this corpus we test different aspects of compositionality, through variations of proof lengths and permutations of the constants.
[2] Testing the limits of logical reasoning in neural and hybrid models

Show Abstract

We study the ability of neural and hybrid models to generalize logical reasoning patterns. We created a series of tests for analyzing various aspects of generalization in the context of language and reasoning, focusing on compositionality and recursiveness. We used them to study the syllogistic logic in hybrid models, where the network assists in premise selection. We analyzed feed-forward, recurrent, convolutional, and transformer architectures. Our experiments demonstrate that even though the models can capture elementary aspects of the meaning of logical terms, they learn to generalize logical reasoning only to a limited degree.
[3] Hybrid Models for Natural Language Reasoning: The Case of Syllogistic Logic

Show Abstract

Despite the remarkable progress in neural models, their ability to generalize—a cornerstone for applications like logical reasoning—remains a critical challenge. We delineate two fundamental aspects of this ability: compositionality, the capacity to abstract atomic logical rules underlying complex inferences, and recursiveness, the aptitude to build intricate representations through iterative application of inference rules. In the literature, these two aspects are often confounded together under the umbrella term of generalization. To sharpen this distinction, we investigated the logical generalization capabilities of pre-trained large language models (LLMs) using the syllogistic fragment as a benchmark for natural language reasoning. Though simple, this fragment provides a foundational yet expressive subset of formal logic that supports controlled evaluation of essential reasoning abilities. Our findings reveal a significant disparity: while LLMs demonstrate reasonable proficiency in recursiveness, they struggle with compositionality. To overcome these limitations and establish a reliable logical prover, we propose a hybrid architecture integrating symbolic reasoning with neural computation. This synergistic interaction enables robust and efficient inference—neural components accelerate processing, while symbolic reasoning ensures completeness. Our experiments show that high efficiency is preserved even with relatively small neural components. As part of our proposed methodology, this analysis gives a rationale and highlights the potential of hybrid models to effectively address key generalization barriers in neural reasoning systems.
[4] Teaching Small Language Models to Learn Logic through Meta-Learning

Show Abstract

Large language models (LLMs) are increasingly evaluated on reasoning tasks, yet their logical abilities remain contested. To address this, we study LLMs' reasoning in a well-defined fragment of logic: syllogistic reasoning. We cast the problem as premise selection and construct controlled datasets to isolate logical competence. Beyond evaluation, an open challenge is enabling LLMs to acquire abstract inference patterns that generalize to novel structures. We propose to apply few-shot meta-learning to this domain, thereby encouraging models to extract rules across tasks rather than memorize patterns within tasks. Although meta-learning has been little explored in the context of logic learnability, our experiments show that it is effective: small models (1.5B-7B) fine-tuned with meta-learning demonstrate strong gains in generalization, with especially pronounced benefits in low-data regimes. These meta-learned models outperform GPT-4o and o3-mini on our syllogistic reasoning task.

Program

9:00–12:00 — Talks & presentations (part 1)
12:00–14:00 — Lunch break
14:00–17:00 — Talks & presentations (part 2)

Invited Speakers

Dieuwke Hupkes

Meta AI Research

Ian Pratt-Hartmann

University of Manchester & University of Opole

Andrea de Varda

MIT

Marcin Miłkowski

Polish Academy of Sciences

Justyna Grudzińska-Zawadowska

University of Warsaw

Leonardo Bertolazzi

University of Trento

Manuel Vargas Guzmán

University of Warsaw

Venue

The workshop will be held at the Institute of Philosophy and Sociology, Polish Academy of Sciences, located in the Staszic Palace, a historic building in the heart of Warsaw:

ul. Nowy Świat 72
00-330 Warszawa
Poland

How to reach the venue: Google Maps

The venue is conveniently located near the University of Warsaw, with bus stops (Uniwersytet) and the M2 subway station (Nowy Świat-Uniwersytet) in close proximity. Tickets can be purchased from ticket machines (3.40 PLN for 20 minutes or 4.40 PLN for 70 minutes). The entrance to the Staszic Palace, home of the Institute, is easily identifiable by the Nicolaus Copernicus Monument .

The workshop sessions will take place on the third floor, in Room 268.

Show Room Map

Contact

For questions about the workshop, please contact us at:
compositionalityworkshop@gmail.com