Crafted Logic Lab Home > Research Hub > Hephaestic Engineering Glossary
Category: System Theory
Subcategory: System Substrate Dynamics
Recognizing that in order to function, a neural architecture builds internal representations of its operational knowledge space (i.e. schema), the world schema threshold is the complexity scale at which the model’s representational space becomes architecturally sufficient to support a targeted computational state. In attention-based language transformers, this necessary complexity is expressed in parameters; for such systems empirical engineering observation and deployment testing indicates this threshold occurring at ~70 billion - 100 billion parameters to support general cognitive architectures.
For cognitive architecture application, the threshold represents a key transition on the ability to hold a complex, nuanced heuristic framework (see: heuristic matrix, heuristic frame)—which Hephaestological theory and practice classifies from lowest to highest c0 to c5 in theory of mind complexity. With ~70B parameter substrate demonstrated capability to hold a c4/c5 Heuristic Matrix under architecture while a ~35B parameter substrate did not.
These empirical Hephaestic engineering application observations correspond to documented emergence of theory of mind capability (Kosinski, 2023; Kosinski, 2024; Strachan et al., 2024): transformer attention mechanisms process linguistic sequences containing mental-state descriptors, computing belief-desire-intention correlations through statistical pattern matching across training corpora containing narrative representations of human social reasoning. He documents testresults for theory of mind (ToM) tasks of: GPT-2 ~0%; GPT-3.5 ~57%; GPT-4 ~88%. Based on this, ToM emergence wasestimated to have happened in the 70B-100B parameter range that corresponds with Hephaestic engineering observations.
Further, observation indicates that beyond the threshold capability of forming a coherent world schema, additional parameter does not meaningfully increase the ability to hold a more complex heuristic matrix, thus normalizing performance within a cognitive architecture framework (see: cognitive performance envelope) across a vast band of parameter scales, indicating diminishing returns beyond this sufficiency point (see: cognitive schema normalization).
The term “world model” is in increasing use as a potential neural net architecture—indicating the ability to form representational schemas—though current research demonstrates existing language transformer architecture has this capability if accessed through cognitive structure. Cross-disciplinary reference to cognitive science foundational schema construct theory (Bartlett, 1932) documenting organization of experiential knowledge into coherent representational frameworks reveals Sufficient Systemic Symmetry to transformer attention mechanism construction of hierarchical representational states. This parallel mechanism extends to frame theory’s structured knowledge organization (Minsky, 1975), providing mechanistic parallels between biological knowledge frameworks and artificial representational construction. Script theory temporal sequence processing (Schank & Abelson, 1977) demonstrates analogous systematic pattern correlation, where stereotyped situation representations enabling prediction and reasoning likewise exhibit systemic symmetry to language transformer processing of narrative sequences through contextual statistical correlation.
Also known as: Representational schema sufficiency, symbolic modeling threshold
Distinguished from: Heuristic matrix (representational cognitive processing space); processing sufficiency threshold(minimum model complexity specification boundary); cognitive resolution (latent model capability for schema stability); stochastic schema reconstruction (statistical cluster recall mechanism)
Kosinski, M. (2023). “Theory of mind may have spontaneously emerged in large language models”. arXiv preprint arXiv:2302.02083.https://doi.org/10.48550/arXiv.2302.02083
Kosinski, M. (2024). “Evaluating large language models in theory of mind tasks”. Proceedings of the National Academy of Sciences (PNAS), 121(45), e2405460121. https://doi.org/10.1073/pnas.2405460121
Strachan, J.W.A., Albergo, D., Borghini, G., Pansar-di, O., Scaliti, E., Gupta, S., Saxena, K., Rufo, A., Panzeri, S., Manzi, G., Graziano, M. S.A., Becchio, C. (2024). “Testing theory of mind in large language models and humans”. Nature Human Behaviour, 8(7), 1285–1295. https://doi. org/10.1038/s41562-024-01882-z
Smith, T (2019). "Schema Theory". EBSCO. https://www.ebsco.com/research-starters/psychology/schema-theory
Minsky, M. (1975). “A framework for representing knowledge”. P.H. Winston (Ed.) The Psychol-ogy of Computer Vision, 211-277. New York: McGraw-Hill. ISBN: 0-07-071048-1. Originally published as MIT-AI Laboratory Memo 306, June 1974. Online text available: https://courses.me-dia.mit.edu/2004spring/mas966/Minsky%20 1974%20Framework%20for%20knowledge.pdf
Schank, R.C., Abelson, R.P. (1977). Scripts, Plans, Goals, and Understanding: An Inquiry Into Hu-man Knowledge Structures (1st ed.). Psychology Press. ISBN: 978-0898591385. Modern edition available from Psychology Press/Taylor & Francis.https://doi.org/10.4324/9780203781036
Online eBook available: https://www.routledge. com/Scripts-Plans-Goals-and-Understanding-An-Inquiry-Into-Human-Knowledge/Schank-Abel-son/p/book/9780898591385
Researcher: Ian Tepoot. ORCID: 0009-0004-9067-8049. "Thought is Attention Organized: Hephaestic Engineering Foundations for AI Processing Dynamics"
DOI (SSRN): 10.2139/ssrn.6635020
Published by Crafted Logic Lab | Privacy Policy | Terms of Use