TY - GEN
T1 - Data Compression for AI Model Training
AU - Kruger, Dov
AU - Kumar, Yulia
AU - Li, Juan
N1 - Publisher Copyright: © 2026 IEEE.
PY - 2026
Y1 - 2026
N2 - Training AI models requires ingesting massive amounts of training data. We define Parametric Matching (PM), a grammar-based compression technique that parses structured text into abstract syntax trees, replaces format patterns with compact tokens, and moves numerical parameters into homogeneous streams of tokens that can additionally be bit compressed using LZMA. The results are input streams that are equivalent to the original interleaved text and data in formats such as SVG, 3D-models such as OBJ, PDF, but highly compressed and much more amenable to processing. PM has many advantages. Loading data is much faster (a factor of 20 for 3D models, and 100 to 200 times faster to load). Just as important, by preprocessing the semantics, ambiguous cases can be resolved once, and can be turned into embedding vectors more efficiently, and potentially more accurately. Our experiments with SVG, G-Code, OBJ and PDF files show large compression. The results in Table 1 show that the PM algorithm can create data objects that are not only more compressible but far faster to load because the binary format can be instantly used, in this case to load OpenGL and render without parsing ASCII data. When compressing a binary object that is half the size of the original ASCII representation, by definition LZMA takes half the time to compress because there are fewer bytes to process.
AB - Training AI models requires ingesting massive amounts of training data. We define Parametric Matching (PM), a grammar-based compression technique that parses structured text into abstract syntax trees, replaces format patterns with compact tokens, and moves numerical parameters into homogeneous streams of tokens that can additionally be bit compressed using LZMA. The results are input streams that are equivalent to the original interleaved text and data in formats such as SVG, 3D-models such as OBJ, PDF, but highly compressed and much more amenable to processing. PM has many advantages. Loading data is much faster (a factor of 20 for 3D models, and 100 to 200 times faster to load). Just as important, by preprocessing the semantics, ambiguous cases can be resolved once, and can be turned into embedding vectors more efficiently, and potentially more accurately. Our experiments with SVG, G-Code, OBJ and PDF files show large compression. The results in Table 1 show that the PM algorithm can create data objects that are not only more compressible but far faster to load because the binary format can be instantly used, in this case to load OpenGL and render without parsing ASCII data. When compressing a binary object that is half the size of the original ASCII representation, by definition LZMA takes half the time to compress because there are fewer bytes to process.
KW - data compression
KW - embedding
KW - komogorov complexity
KW - large lan- guage model
KW - llm
KW - parametric matching
UR - https://www.scopus.com/pages/publications/105041005864
U2 - 10.1109/DCC66757.2026.00071
DO - 10.1109/DCC66757.2026.00071
M3 - Conference contribution
T3 - Data Compression Conference Proceedings
SP - 444
BT - Proceedings - DCC 2026
A2 - Bilgin, Ali
A2 - Fowler, James E.
A2 - Serra-Sagrista, Joan
A2 - Ye, Yan
A2 - Storer, James A.
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2026 Data Compression Conference, DCC 2026
Y2 - 24 March 2026 through 27 March 2026
ER -