[cond-mat/0108530] Language Trees and Zipping
In this letter we present a very general method to extract information from a generic string of characters, e.g. a text, a DNA sequence or a time series. Based on data-compression techniques, its key point is the computation of a suitable measure of the remoteness of two bodies of knowledge. We present the implementation of the method to linguistic motivated problems, featuring highly accurate results for language recognition, authorship attribution and language classification.
[cond-mat/0108530] Language Trees and Zipping Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Condensed Matter > Statistical Mechanics arXiv:cond-mat/0108530 (cond-mat) [Submitted on 31 Aug 2001 ( v1 ), last revised 19 Dec 2001 (this version, v2)] Title: Language Trees and Zipping Authors: Dario Benedetto , Emanuele Caglioti , Vittorio Loreto View a PDF of the paper titled Language Trees and Zipping, by Dario Benedetto and 1 other authors View PDF Abstract: In this letter we present a very general me
Explore this link on the map →related reading
- Can gzip be a language model?nathan.rs
- annaabrandenberger.github.io
- [2309.10668] Language Modeling Is Compressionarxiv.org
- Data Compression Explainedmattmahoney.net
- GitHub - socketteer/loom: Multiversal tree writing interface for human-AI collaboration · GitHubgithub.com
- Tokenized and Continuous Embedding Compressions of Protein Sequence and Structurebiorxiv.org
- How Lossless Data Compression Works | Quanta Magazinequantamagazine.org
- GitHub - adlnlp/doc_gcn · GitHubgithub.com
- arxiv.org/pdf/2505.24832arxiv.org
- Language, trees, and geometry in neural networkspair-code.github.io
- GitHub - ccusage/ccusage: npx ccusage · GitHubgithub.com
- GitHub - oughtinc/ice: Interactive Composition Explorer: a debugger for compositional language model programs · GitHubgithub.com