why lowercase letters save data | endtimes.dev
For example: I took the front page of Hacker News and rewrote the title of each article in sentence case instead of title case — reducing the size by 31 bytes. Sentence case: The cat sat on the mat Title case: The Cat Sat on the Mat How can it be true that changing a few uppercase letters to lowercase letters saves data? The answer is compression. It isn't intuitive, but once you understand how text compression works, it will begin to make sense. In this article I hope to… Text compression is more effective when… Replacing uppercase characters with their more common lowercase counterparts helps with all three of these. To understand why this works you need to understand how compression works. To explain how text compression works, we'll specifically look at the deflate algorithm commonly used in zip files. The principles are the same for other algorithms too. Deflate uses two compression methods — Huffman encoding and LZSS — both of these compression methods are effected by replacing u
Lowercase letters and uppercase letters use the same amout of data — 1 byte each. So, it's surprising to learn that swapping uppercase letters for lowercase letters saves data. For example: I took the front page of Hacker News and rewrote the title of each article in sentence case instead of title case — reducing the size by 31 bytes . Sentence case: The cat sat on the mat Title case: The Cat Sat on the Mat How can it be true that changing a few uppercase letters to lowercase letters saves data? The answer is compression . It isn't intuitive, but once you understand how text compression works,
Explore this link on the map →related reading
- How Lossless Data Compression Works | Quanta Magazinequantamagazine.org
- Data Compression Explainedmattmahoney.net
- ChatGPT Is a Blurry JPEG of the Web | The New Yorkernewyorker.com
- [2309.10668] Language Modeling Is Compressionarxiv.org
- 500'000€ Prize for Compressing Human Knowledgeprize.hutter1.net
- Run-length encoding - Wikipediaen.wikipedia.org
- How does image compression work? | Lee Robinsonleerob.com
- Smaller and faster data compression with Zstandard - Engineering at Metaengineering.fb.com
- Hutter Prize - Wikipediaen.wikipedia.org
- Variable-length encoding - Wikipediaen.wikipedia.org
- Fabrice Bellard's Home Pagebellard.org
- Can gzip be a language model?nathan.rs