AmeforgeAmeforge
Proprietary · Dual LicenseEARCP ArchitecturePatent PendingPyPI: pip install saclm

SCLM

Stateful Coherent Language Model

SCLM adds persistent latent memory to any transformer language model, enabling true coherence across long conversations and multi-turn generation. Unlike RAG or extended context windows, SCLM maintains a compressed, evolving latent state that persists across turns without growing memory requirements.

2–5%
Parameter Overhead
vs. base model size
85%
Entity Retention
Mistral-7B benchmark
∞
Context Length
No window limit
BSL-1.1
License
Dual: free + commercial

Transformers forget everything between turns

Standard transformer models have no persistent state. Every conversation turn is processed independently unless the entire history is re-fed into the context window. This approach is computationally expensive, has hard length limits, and degrades as context grows.

RAG systems retrieve external documents but cannot maintain the evolving internal state of a conversation: who said what, what changed, what was established earlier.

A persistent latent state that evolves with the conversation

SCLM adds only 2–5% additional parameters to any existing transformer. These parameters maintain a compressed latent state vector that is updated at every turn and injected back into the model at configurable layers.

Key advantage: Memory does not grow with conversation length. A 10-turn and a 1000-turn conversation use identical memory footprint. Entity coherence is maintained regardless of conversation length.

Five components, one coherent memory

The SCLM memory system is built on five interlocking components. Each one addresses a specific failure mode of memory in language models.

E

Encapsulation

GRU-style state update from hidden states that compresses conversation history into a fixed-size latent vector.

A

Alignment

Cross-attention between the latent state and transformer hidden layers, ensuring memory influences generation at every step.

R

Revision

Drift detection and correction that prevents the latent state from accumulating errors over long conversations.

C

Coherence

Mixture-of-Experts for consistency, where multiple expert heads maintain coherent entity representations.

P

Propagation

State injection into transformer layers, with memory injected at configurable depth levels for fine-grained control.

Input → Transformer Hidden States
↓
[Encapsulation] → Latent State (GRU update)
↓
[Alignment] → Cross-attention: State ↔ Hidden
↓
[Revision] → Drift detection + correction
↓
[Coherence] → MoE consistency check
↓
[Propagation] → Inject into transformer layers
↓
Output: Coherent generation with persistent memory

Minimal overhead, measurable coherence improvement

Base ModelEARCP ParametersOverheadEntity Retention
Mistral-7B91.7M2.4%
85%
LLaMA-7B91.7M2.4%
83%
Phi-252.3M1.9%
81%

Entity Retention measures the model's ability to correctly reference entities introduced earlier in a conversation. Benchmarks conducted on internal test sets. Independent replication encouraged.

Works with any HuggingFace transformer

1. Install

bash
# Basic installation
pip install saclm

# With quantization support
pip install saclm[quantization]

# Full installation
pip install saclm[full]

2. Load and use

python
from sclm import SCLMModel

# Load pre-trained SCLM model
model = SCLMModel.from_pretrained(
    "amewebstudio/ananke-sclm",
    load_in_4bit=True  # Optional: 4-bit quantization
)

# Start new conversation
model.reset_state()

# Build context: entities are remembered
model.add_context("The wizard Elara lives in Silverwood.")
model.add_context("Her familiar is a silver cat named Nimbus.")

# Generate: Elara and Nimbus persist in memory
output = model.generate(
    "One day, Elara decided to",
    max_new_tokens=50
)
# "One day, Elara decided to take Nimbus on a
#  journey through Silverwood..."

3. Advanced configuration

python
from sclm import SCLMConfig

config = SCLMConfig(
    latent_state_dim=256,        # State vector dimension
    n_experts=2,                  # Number of MoE experts
    state_injection_layers=[8, 16],  # Injection depth
    alpha_inject=0.02,            # Injection strength
)

model = SCLMModel.from_pretrained(
    "model-name",
    config=config
)

# Use optimized presets
from sclm.config import get_preset
config = get_preset("mistral-7b")
# Also available: "llama-7b", "phi-2", "tiny"
Edit Mode: local changes without affecting memory
python
# Freeze memory for local edits
model.freeze_state()
output = model.generate("The sword was RED")
model.unfreeze_state()
# Memory unchanged: global context preserved

Any application that requires memory

Interactive Fiction & Games

Characters, places, and story elements persist across chapters and sessions. No need to re-establish context at every scene.

model.add_context("Queen Lyra rules Eldoria.")
model.add_context("Advisor Marcus serves the crown.")
output = model.generate("Marcus approached the throne")
# Queen Lyra and Marcus are remembered correctly

Long Conversations

Memory persists without growing the context window. Suitable for customer support, therapy assistants, and long-form interviews.

# Memory persists without growing context
for turn in conversation_turns:
    model.add_context(turn)

response = model.generate("Based on our discussion,")
# All prior context is in memory, not in window

Creative Writing

Maintain narrative consistency across chapters, characters, and timelines in long-form creative writing workflows.

model.add_context("Chapter 1: Sarah found a map.")

# Chapter 2: Sarah is remembered
output = model.generate("Chapter 2: The next morning, Sarah")
# Correct reference to Sarah and the map

Dual licensing model

🆓

Free License (BSL-1.1)

✓Personal and hobbyist projects
✓Academic research (non-profit)
✓Small businesses (revenue < $100,000 USD/year)
Download on HuggingFace

Commercial License

Required for organizations with revenue over $100,000 USD/year, SaaS products, or redistribution in proprietary products.

✓Legal compliance for enterprise use
✓Priority technical support
✓Right to redistribute
✓Warranty and indemnification
✓Custom feature development
Contact for Commercial License
bibtex
@article{amega2025sclm,
  title={SCLM: Stateful Coherent Language Models with EARCP Architecture},
  author={Amega, Mike},
  year={2025},
  note={Ameforge Research Lab, Proprietary}
}