The HMH framework represents a breakthrough in connecting symbolic AI with vector-based neural networks. It allows complex data structures to be stored in fixed-size vectors without losing the hierarchical relationship of the data.
Core Algebraic Formulas
These formulas allow the system to "pack" and "unpack" information:
- Circular Convolution (Binding):
- Fourier Transform Recovery (Unbinding):
- Modular Bundling (Combining Slots):
Logical Integration Formulas
This bridges the gap between hypervectors (h) and Probabilistic Logic (p):
- Morphism ():
- Crosstalk and Separation:
Comparison of Efficiencies
| Approach | Storage | Bind/Unbind | Query |
| Concatenation-based | O(D * L) | - | O(N*D*L) |
| HMH + FFT | O(D) | O(D log D) | O(N*D) |
| HMH + XOR + HNSW | O(D + index) | O(Dw) | O(log N) |
1. The Core Concept The document introduces HMH as a way to combine two powerful AI methods: "hierarchical embeddings" (which organize information like a tree or an ontology) and "compositional hypervectors" (which are high-dimensional vectors that can be mathematically combined). The goal is to create a system that is as smart as human logic but as fast and flexible as neural networks.
2. How the Vectors are Structured Instead of just being a list of numbers, HMH vectors are "modular." Each type of data (like a person, an action, or a place) is given its own section or "module" in the vector. These modules are then divided into "slots" that represent specific roles, such as:
- Subject: Who is doing the action.
- Object: What the action is being done to.
- Effect: What happens as a result. This allows the AI to store a complex sentence or idea in a single, fixed-size vector.
3. Key Mathematical Operations The system uses "Binding" and "Bundling" to manage data:
- Binding: This links a "Role" to a "Filler" (e.g., binding the role of "Driver" to the person "John"). It uses a math operation called "Circular Convolution" or "XOR" to mix them together into one vector.
- Bundling: This is the process of adding multiple bound pairs together to form a complete thought or memory.
- Unbinding: This is the reverse process, where the AI "unpacks" the vector to find out who the subject was or what the object was.
4. Logical Reasoning (PLN Integration) A major part of the paper explains how these vectors help "Probabilistic Logic Networks" (PLN). By using hypervectors, the AI can check if two ideas are "orthogonal" (totally different) or "similar." This prevents the AI from getting confused during complex logic chains. It uses a "Separation Score" to make sure that different pieces of information don't interfere with each other, leading to more reliable and "clean" reasoning.
5. Cognitive Memory Systems The framework is designed to work within a larger AI architecture called PRIMUS. It acts as a bridge between different types of memory:
- Declarative Memory: Facts and rules.
- Episodic Memory: Sequences of events (like a timeline).
- Procedural Memory: How to perform tasks. By using HMH, all these different types of memory can be searched and retrieved using the same vector-based "language."
6. Efficiency and Performance The paper concludes that this method is much more efficient than traditional AI. It can handle deeper hierarchies and more complex data without requiring more computer memory. By using a specialized indexing method (like HNSW), the AI can search through millions of these complex ideas in a fraction of a second.