Make the hardware do more.
Geocentric treats training speed, memory use, context, evaluation, vision, and inference as one connected systems problem.
Training efficiency across four resource axes
EPICYCLE combines progressive depth, context folding, selective token loss, and memory-efficient optimization.
| Gear | Constraint | Mechanism |
|---|---|---|
| DEFERENT | Speed | Start with fewer active layers and bring the rest online during training. |
| HORIZON | Speed, token use | Train shorter independent sequences before expanding to full context. |
| EQUANT | Quality per token | Experiment with selective difficult-token loss after warmup. |
| ARMILLARY | Optimizer memory | Reduce persistent optimizer state to free capacity for larger models. |
The model remains an ordinary decoder-only transformer at inference time.
Current systems results
Inference: 74.26 → 88.27 tokens/sec in a controlled Apple M4 generation comparison, an 18.87% increase.
Training: context folding reached about 1.10× throughput in an M4 systems ablation.
Optimizer state: experimental balanced momentum measured 65.38% lower resident optimizer-state memory in one embedding-heavy 11.2M-parameter configuration.
These are separate controlled experiments, not one combined speedup number.
Evaluation that tells you what changed
PARALLAX tests the model across multiple failure modes instead of collapsing everything into one score.
| Text modeling | Language-model quality across multiple genres. |
|---|---|
| Context use | Whether additional context actually improves predictions. |
| Completion ranking | Whether true continuations outrank false alternatives. |
| Instruction following | Rule-based checks for requested behavior. |
| Degeneration | Looping and repetition behavior. |
| Performance | Prefill, decode throughput, and peak memory. |
| Vision grounding | Whether image information improves the right caption prediction. |
Optional text watermarking can travel with a checkpoint
Geocentric's watermarking work stores identity and strength with the model so attribution can be applied consistently during generation.
Add images without rebuilding the language decoder
A vision tower encodes image patches and a projector maps them into the language model's embedding space. The decoder keeps the same core architecture.
From raw data to a finished model
| Tokenizer | Train and validate a model-specific vocabulary. |
|---|---|
| Data | Prepare large corpora into reusable tokenized shards. |
| Pretraining | Train the base causal language model from scratch. |
| Post-training | Instruction tuning and capability shaping. |
| Evaluation | Measure quality, context use, stability, and performance. |
| Release | Package weights, configuration, tokenizer, and model metadata. |