SEMQ’s semantic abstraction hits FP32 accuracy (92.27%) while beating 4-bit quantization

A new approach to AI “math” is aiming at the real, expensive part of building systems: not training, but what your infrastructure has to lug around after. The SEMQ Group, founded by CEO and Andrés Mac Allister, says SEMQ can preserve semantic usefulness without clinging to the full numerical precision of embedding vectors. In at least one benchmark, it lands at 92.27% accuracy, effectively matching an FP32 baseline at 92.26%, while 4-bit quantization registers 56.05% accuracy.

That specific gap is the whole point. In SEMQ’s framing, conventional quantization shrinks embeddings by reducing precision, but it also erodes the parts semantic systems actually rely on. SEMQ tries a different move: instead of compressing embedding weights (especially the embeddings themselves), it separates the semantics from the representation. The claim is that semantic relationships depend primarily on the relative orientation of embedding vectors, so preserving absolute magnitude matters less than preserving relational geometry. If you are running retrieval, classification, agent memory, or any pipeline where embeddings become “state” you must reuse, reproduce, audit, or transfer, this could change the storage and operational math.

To appreciate why the stakes are high, remember what embeddings are in the usual setup. Embedding model weights include embeddings, which map tokens to vectors, and those vectors are commonly stored in Full-Precision (FP32). FP32 uses 4 bytes per parameter. For a 7B parameter model, that implies about 28 GB of disk space and memory. Quantization reduces that footprint: FP16/BF16 uses 2 bytes per parameter, cutting the estimate to about 14 GB. Smaller options include FP8, INT8/Q8, Q6, Q5, Q4, Q3, and Q2. But the trade-off is precision, and the answers get worse as precision drops.

SEMQ, short for Symbolic Embedding Multi-Quantization, is built around an alternate idea described in a paper published earlier this year. SEMQ “replaces raw vectors with fixed-dimensional symbolic structures” designed to preserve relational properties like relative similarity ordering and neighborhood structure. It also aims to decouple representation from metrics, indexing, and execution semantics. In plainer terms: SEMQ wants a semantic abstraction layer that retains how things relate in embedding space, without needing to store every coordinate as high-precision numerical magnitudes.

Mac Allister’s email to The Register makes the motivation explicit. In conventional embedding systems, semantic state is stored as high-precision coordinates that encode both magnitude and direction. The SEMQ question, according to him, is whether a substantial part of useful semantic information could be represented through structural relationships: how components move relative to one another, which regions they occupy, and what directional configuration they form. SEMQ’s representation approach aims to capture relative geometry rather than enumerating independent floating-point magnitudes. He argues this matters because semantic systems care about relationships, similarity, neighborhood structure, continuity, retrieval behavior, change over time, not just each numeric value in isolation.

What does that look like in practice? SEMQ is described as portable and reproducible, with the goal of being able to reproduce, audit, compare, and transfer semantic state across processes. Mac Allister points to validation tests that convert embedding-based semantic state into a deterministic.semq representation, restore it, and evaluate stability for retrieval and classification operations. In one benchmark using the Banking77 dataset from MTEB and the all-MiniLM-L6-v2 embedding model, the FP32 baseline achieved 92.26 percent accuracy, while SEMQ achieved 92.27 percent accuracy, effectively matching within 0.03 percentage points. In the same semantic classification setting, SEMQ substantially outperformed 4-bit quantization, which registered 56.05 percent accuracy, 36.22 percentage points below FP32.

Mac Allister is careful about generalizing. He frames the result as evidence that preserving semantic structure can be materially different from simply reducing numerical precision. That distinction could matter for enterprises that have already invested in quantization strategies, because the conversation is shifting from “how small can we make embeddings” to “what properties must survive the shrink.” If semantic state is treated like operational data, not just model internals, the ability to inspect semantic behavior, reproduce it deterministically, and move it between systems becomes a governance and reliability feature, not only a performance tweak.

Adoption is positioned as non-disruptive. Mac Allister says SEMQ can be applied either at the point of data ingestion, using an SDK to encode vectors generated by an embedding model on documents into.semq artifacts, or at query time to load, query, compare, restore, and verify the encoding. The claim is that a team can adopt SEMQ without replacing its LLM, embedding model, vector database, or agent framework. He describes it as starting alongside the existing stack as a sidecar layer, then becoming the representation used for selected retrieval or memory workloads. Potential use cases include making embeddings or memory state portable across systems, reproducing semantic state across runs or machines, auditing model changes, reducing dependence on opaque or hard-to-reproduce stateful pipelines, and diffing semantic state. He even mentions extending SEMQ beyond semantic embeddings: in research,.semq files snapshot and restore transformer KV-cache state across process boundaries, described as a runtime-state workflow for pausing, transferring, and resuming an active model session.

For decision-makers, the underlying question is what cost and risk you are truly paying for when you run AI workloads. Quantization reduces bytes, but SEMQ reframes what should be preserved and why. Mac Allister says SEMQ targets infrastructure overhead tied to semantic state, and he also says interest is coming from teams where reproducibility, state portability, lower overhead, and inspectability are operationally important. His company is working through a Founding Design Partnership Program with organizations exploring applications in enterprise AI, retrieval, agent memory, and auditable AI workflows, including some AI infrastructure hyperscalers and some companies at the AI application layer. He says NDAs prevent naming organizations publicly, but he characterizes interest as coming from big companies facing a “big problem” for operational execution. If that is your world, SEMQ is worth watching because it attacks both the storage bill and the audit trail.