In the ever-evolving quest to endow machines with human-like reasoning, a central challenge is understanding how concepts relate—not just by sharing examples, but by sharing the underlying properties and meanings that define them. Ben Goertzel’s recent work, “Intensional Inheritance Between Concepts: An Information-Theoretic Interpretation”, introduces a groundbreaking framework to address this challenge. His approach quantifies intensional inheritance, providing AI and AGI systems with a principled way to assess how knowledge of one concept informs our understanding of another.
Imagine asking a seemingly simple question: “If something is an animal, what does that tell us about whether it is a cat?” While the answer seems obvious to humans, representing this relationship computationally is far from trivial. Traditional AI approaches often rely on extensional inheritance—examining whether instances of one concept belong to another—but this ignores the richness of shared properties and probabilistic nuances that define meaning. Intensional inheritance, in contrast, captures how concepts inherit meaning and properties from one another, allowing machines to reason about concepts in a way that mirrors human intuition.
The Problem: Measuring Meaning Between Concepts
The research asks a fundamental question:
"How can we quantify the information that one concept conveys about another?"
To answer this, Goertzel models concepts as sets of properties with associated degrees, representing how strongly an element exhibits each property:
F = {F1, F2, ..., Fn}, with degrees {d1, d2, ..., dn}
W = {W1, W2, ..., Wm}, with degrees {e1, e2, ..., em}
Here, degrees di and ej indicate the probability or extent to which an element possesses a property. Overlaps and dependencies among properties are handled using interaction information, enabling the framework to account for complex interrelationships that go beyond simple overlaps.
This formulation allows AI systems to move past rigid hierarchies and model semantic richness, where one concept can inform another in probabilistic and nuanced ways.
Two Complementary Approaches: Shannon and Algorithmic Information Theory
Goertzel develops the framework using two information-theoretic lenses: Shannon information theory and algorithmic information theory. Both approaches yield formulas that quantify intensional inheritance, yet they provide complementary insights into the nature of knowledge and meaning.
Shannon Information Approach
Using Shannon’s framework, mutual information measures how much knowing F reduces uncertainty about W:
I(F; W) = H(W) - H(W|F) = H(F) + H(W) - H(F, W)
Where:
- H(F) and H(W) are the entropies of F and W
- H(F, W) is the joint entropy, incorporating dependencies among properties
- H(W|F) is the conditional entropy of W given F
Interaction information among multiple properties adjusts for overlapping influences:
Interaction Information = Σ (−1)^(|S|+1) * I(S) for all subsets S of properties
Conditional probability, which gives the likelihood of W given F, is expressed as:
P(W|F) = P(W) * 2^(I(F;W))
This elegant formula captures the essence of intensional inheritance, reflecting both the individual contributions of properties and their interactions.
Algorithmic Information Approach
Parallel to Shannon’s statistical perspective, algorithmic information theory frames the problem in terms of Kolmogorov complexity, measuring the minimal description length of a concept:
I(F:W) = K(W) - K(W|F) = K(W) + K(F) - K(F, W)
Where K(·) denotes the complexity of a concept or property. Conditional probabilities follow similarly:
P(W|F) ≈ P(W) * 2^(I(F:W))
Remarkably, despite the conceptual differences, Shannon and algorithmic frameworks converge mathematically, offering AI researchers multiple yet coherent tools to model meaning transfer between concepts.
Special Cases and Simplifications
For mutually exclusive properties, calculations simplify considerably. If:
- n = number of properties in F
- m = number of properties in W
- k = number of overlapping properties
- s = total unique properties = n + m - k
Then, using Shannon’s framework:
P(W|F) = k / n
And in the algorithmic framework, assuming equal property complexities:
I(F:W) = log(k / n)
P(W|F) = P(W) * 2^(I(F:W)) = (m / s) * (k / n)
Such simplifications highlight how intensional inheritance naturally generalizes extensional inheritance, which considers only set membership without accounting for overlapping or probabilistic properties.
Implications for AI and AGI
The ability to quantify intensional inheritance has profound implications:
- Knowledge Representation: Systems like OpenCog Hyperon and Probabilistic Logic Networks can model hierarchical relationships with greater semantic depth.
- Reasoning and Inference: Probabilistic and nuanced relationships allow AI to make informed predictions about concepts beyond mere instances.
- Generalization: By incorporating property degrees and interactions, machines can generalize knowledge to novel situations more effectively.
For example, an AI might understand that “cats are mammals” (extensional) and also recognize that “cats inherit characteristics of mammals, like being warm-blooded, but also add unique traits like purring and retractable claws” (intensional). This level of nuance is crucial for human-level reasoning and AGI.
Reflections and Future Directions
Goertzel’s framework bridges philosophical insight, formal logic, and information theory, offering a quantitative foundation for reasoning about meaning itself. By demonstrating that extensional inheritance is a special case of intensional inheritance, he provides a unifying perspective on conceptual relationships.
Looking forward, this work opens pathways to:
- Unified Information-Theoretic Models of Knowledge combining statistical and algorithmic perspectives
- Advanced AGI Reasoning Systems capable of interpreting, predicting, and manipulating abstract concepts
- Integration with Learning Systems, where observed patterns in data can inform property-based reasoning hierarchies
In short, this research demonstrates that meaning is more than just examples—it is embedded in the informational structure of concepts themselves, offering AI systems a path toward understanding, not just observing, the world.
References
- Goertzel, B., Intensional Inheritance Between Concepts: An Information-Theoretic Interpretation. SingularityNET Foundation, TrueAGI Inc., 2025.
- Van de Cruys, T., Two multivariate generalizations of pointwise mutual information, Proceedings of the Workshop on Distributional Semantics and Compositionality, 2011.
- Fitting, M., Intensional Logic, 2006.
- Goertzel, B. et al., OpenCog Hyperon: A Framework for AGI at the Human Level and Beyond, 2023.
- Li, M., Vitanyi, P., An Introduction to Kolmogorov Complexity and Its Applications, Springer, 3rd ed., 2008.