SCLM
Stateful Coherent Language Model
SCLM adds persistent latent memory to any transformer language model, enabling true coherence across long conversations and multi-turn generation. Unlike RAG or extended context windows, SCLM maintains a compressed, evolving latent state that persists across turns without growing memory requirements.
The Problem
Transformers forget everything between turns
Standard transformer models have no persistent state. Every conversation turn is processed independently unless the entire history is re-fed into the context window. This approach is computationally expensive, has hard length limits, and degrades as context grows.
RAG systems retrieve external documents but cannot maintain the evolving internal state of a conversation: who said what, what changed, what was established earlier.
The Solution
A persistent latent state that evolves with the conversation
SCLM adds only 2–5% additional parameters to any existing transformer. These parameters maintain a compressed latent state vector that is updated at every turn and injected back into the model at configurable layers.
Key advantage: Memory does not grow with conversation length. A 10-turn and a 1000-turn conversation use identical memory footprint. Entity coherence is maintained regardless of conversation length.
Architecture
Five components, one coherent memory
The SCLM memory system is built on five interlocking components. Each one addresses a specific failure mode of memory in language models.
Encapsulation
GRU-style state update from hidden states that compresses conversation history into a fixed-size latent vector.
Alignment
Cross-attention between the latent state and transformer hidden layers, ensuring memory influences generation at every step.
Revision
Drift detection and correction that prevents the latent state from accumulating errors over long conversations.
Coherence
Mixture-of-Experts for consistency, where multiple expert heads maintain coherent entity representations.
Propagation
State injection into transformer layers, with memory injected at configurable depth levels for fine-grained control.
Benchmarks
Minimal overhead, measurable coherence improvement
| Base Model | EARCP Parameters | Overhead | Entity Retention |
|---|---|---|---|
| Mistral-7B | 91.7M | 2.4% | 85% |
| LLaMA-7B | 91.7M | 2.4% | 83% |
| Phi-2 | 52.3M | 1.9% | 81% |
Entity Retention measures the model's ability to correctly reference entities introduced earlier in a conversation. Benchmarks conducted on internal test sets. Independent replication encouraged.
Installation & Quick Start
Works with any HuggingFace transformer
1. Install
# Basic installation
pip install saclm
# With quantization support
pip install saclm[quantization]
# Full installation
pip install saclm[full]2. Load and use
from sclm import SCLMModel
# Load pre-trained SCLM model
model = SCLMModel.from_pretrained(
"amewebstudio/ananke-sclm",
load_in_4bit=True # Optional: 4-bit quantization
)
# Start new conversation
model.reset_state()
# Build context: entities are remembered
model.add_context("The wizard Elara lives in Silverwood.")
model.add_context("Her familiar is a silver cat named Nimbus.")
# Generate: Elara and Nimbus persist in memory
output = model.generate(
"One day, Elara decided to",
max_new_tokens=50
)
# "One day, Elara decided to take Nimbus on a
# journey through Silverwood..."3. Advanced configuration
from sclm import SCLMConfig
config = SCLMConfig(
latent_state_dim=256, # State vector dimension
n_experts=2, # Number of MoE experts
state_injection_layers=[8, 16], # Injection depth
alpha_inject=0.02, # Injection strength
)
model = SCLMModel.from_pretrained(
"model-name",
config=config
)
# Use optimized presets
from sclm.config import get_preset
config = get_preset("mistral-7b")
# Also available: "llama-7b", "phi-2", "tiny"# Freeze memory for local edits
model.freeze_state()
output = model.generate("The sword was RED")
model.unfreeze_state()
# Memory unchanged: global context preservedUse Cases
Any application that requires memory
Interactive Fiction & Games
Characters, places, and story elements persist across chapters and sessions. No need to re-establish context at every scene.
model.add_context("Queen Lyra rules Eldoria.")
model.add_context("Advisor Marcus serves the crown.")
output = model.generate("Marcus approached the throne")
# Queen Lyra and Marcus are remembered correctlyLong Conversations
Memory persists without growing the context window. Suitable for customer support, therapy assistants, and long-form interviews.
# Memory persists without growing context
for turn in conversation_turns:
model.add_context(turn)
response = model.generate("Based on our discussion,")
# All prior context is in memory, not in windowCreative Writing
Maintain narrative consistency across chapters, characters, and timelines in long-form creative writing workflows.
model.add_context("Chapter 1: Sarah found a map.")
# Chapter 2: Sarah is remembered
output = model.generate("Chapter 2: The next morning, Sarah")
# Correct reference to Sarah and the mapLicensing
Dual licensing model
Free License (BSL-1.1)
Commercial License
Required for organizations with revenue over $100,000 USD/year, SaaS products, or redistribution in proprietary products.
@article{amega2025sclm,
title={SCLM: Stateful Coherent Language Models with EARCP Architecture},
author={Amega, Mike},
year={2025},
note={Ameforge Research Lab, Proprietary}
}