✳flâneur — a map of the web's best reading

The Intricate Link Between Compression and Prediction

mindfulmodeler.substack.com · 828 words · saved by 1 readers

You can beat deep learning for text classification with gzip + a short Python script, according to this paper.1 The idea: Use k-nearest neighbors based on compressed documents. Thanks for reading Mindful Modeler! Subscribe for free to receive new posts and support my work. Subscribe Here’s how it works: Compress (zip) the new, unlabeled document and measure file size (= C(x)). Compress 1 training document and measure file size (= C(y)). Concatenate the new document and the training document, compress the resulting doc, and measure file size (= C(xy)). Compute the normalized compression distance: Repeat this procedure for all documents in training data. This gives you the distances between your new document and all training docs. Pick the k documents with the smallest NCD and classify the new document according to the majority class of the k documents. But why does this work at all? If you haven't heard about the amazing connection between compression and prediction — you're in for a tr

The Intricate Link Between Compression and Prediction How Gzip and K-Nearest Neighbors Can Outperform Deep Learning Models Christoph Molnar Oct 24, 2023 22 3 1 Share You can beat deep learning for text classification with gzip + a short Python script, according to this paper . 1 The idea: Use k-nearest neighbors based on compressed documents. Thanks for reading Mindful Modeler! Subscribe for free to receive new posts and support my work. Subscribe Here’s how it works: Compress (zip) the new, unlabeled document and measure file size (= C(x)). Compress 1 training document and measure file size (

Explore this link on the map →

related reading