diff --git a/doc/decompression.sgml b/doc/decompression.sgml new file mode 100644 index 000000000..e4a8b4ca7 --- /dev/null +++ b/doc/decompression.sgml @@ -0,0 +1,164 @@ + + +
+Decompressing data with cc65 +<author><url url="mailto:colin@colino.net" name="Colin Leroy-Mira"> + +<abstract> +How to decompress data using one of cc65's runtime decompressors. +</abstract> + +<!-- Table of contents --> +<toc> + +<!-- Begin the document --> + +<sect>Overview<p> + +cc65 ships with multiple decompressors, each having pros and cons. This page +will detail each of them, and how to use them. + + + +<sect>LZ4<p> + +The LZ4 format is widely known. It has a number of drawbacks, though: + +There are many LZ4 subformats available, and generating LZ4 data compatible +with the cc65 decompressor requires some work. The cc65 decompressor works +on raw LZ4 data with no header, which makes it difficult to generate compressed +data simply using the <tt/lz4/ command-line utility. + +This also means that the function needs to be passed the expected decompressed +size as argument. This makes generating compressed LZ4 data even harder. + +The simplest way to generate "correct" LZ4 data for the cc65 decompressor is +to write a small C utility, like this one (example stripped of any error checking): +<tscreen><verb> + FILE *fp; + size_t read_bytes, wrote_bytes; + char header[2]; + char in_buf[MAX_COMPRESSED_DATA_SIZE*16]; + char out_buf[MAX_COMPRESSED_DATA_SIZE]; + + fp = fopen(argv[1], "rb"); + read_bytes = fread(in_buf, 1, sizeof(in_buf), fp); + fclose(fp); + + wrote_bytes = LZ4_compress_HC(in_buf, out_buf, read_bytes, sizeof(out_buf), 16); + header[0] = (read_bytes & 0xFF); + header[1] = (read_bytes & 0xFFFF) >> 8; + + fp = fopen("DATA.LZ4", "wb"); + fwrite(header, 1, sizeof(header), fp); + fwrite(out_buf, 1, wrote_bytes, fp); + fclose(fp); +</verb></tscreen> + +Decompressing in a cc65 program then looks like: +<tscreen><verb> + int fd; + int decompressed_size; + char compressed[MAX_COMPRESSED_DATA_SIZE]; + char destination[MAX_COMPRESSED_DATA_SIZE*16]; + + fd = open("DATA.LZ4", O_RDONLY); + read(fd, &decompressed_size, sizeof(int)); + read(fd, compressed, MAX_COMPRESSED_DATA_SIZE); + close(fd); + + decompress_lz4(compressed, destination, decompressed_size); +</verb></tscreen> + +<sect>LZSA<p> + +The LZSA formats come from Emmanuel Marty and has its code hosted in the +<url url="https://github.com/emmanuel-marty/lzsa" name="Github LZSA repository">. + +Compressing data is simple, from a command-line or shell: +<tscreen><verb> + lzsa -r -f 1 input.bin DATA.LZSA #For lzsa1 format + lzsa -r -f 2 input.bin DATA.LZSA #For lzsa2 format +</verb></tscreen> + +Decompressing is then as simple as possible: +<tscreen><verb> + int fd; + char compressed[MAX_COMPRESSED_DATA_SIZE]; + char destination[MAX_COMPRESSED_DATA_SIZE*16]; + + fd = open("DATA.LZSA", O_RDONLY); + read(fd, compressed, MAX_COMPRESSED_DATA_SIZE); + close(fd); + + decompress_lzsa1(compressed, destination); + // or + // decompress_lzsa2(compressed, destination); +</verb></tscreen> + +<sect>ZX02<p> + +The ZX02 formats come from <tt/dmsc/ and has its code hosted in the +<url url="https://github.com/dmsc/zx02" name="Github ZX02 repository">. + +Compressing data is simple, from a command-line or shell: +<tscreen><verb> + zx02 -f input.bin DATA.ZX02 +</verb></tscreen> + +Decompressing is then as simple as possible: +<tscreen><verb> + int fd; + char compressed[MAX_COMPRESSED_DATA_SIZE]; + char destination[MAX_COMPRESSED_DATA_SIZE*16]; + + fd = open("DATA.ZX02", O_RDONLY); + read(fd, compressed, MAX_COMPRESSED_DATA_SIZE); + close(fd); + + decompress_zx02(compressed, destination); +</verb></tscreen> + +<sect>In-place decompression<p> + +As memory is often sparse in the cc65 targets, it is often possible to decompress +data "in-place", requiring only the destination buffer and no compressed data +buffer. But as the cc65 decompressors do not support backwards compressed data, +it is necessary to have the compressed data at the <tt/end/ of the destination +buffer, <tt/and/ that the destination buffer has a few extra bytes (8 are enough) +so that decompressing does not overwrite the end of the compressed data too soon: + +<tscreen><verb> + #define BUFFER_SIZE = (MAX_UNCOMPRESSED_DATA_SIZE) + 8; + + int fd; + int compressed_size; + char dest_buf[BUFFER_SIZE]; + char *end_of_buf = dest_buf + BUFFER_SIZE; + + fd = open("DATA.ZX02", O_RDONLY); + compressed_size = read(fd, dest_buf, MAX_UNCOMPRESSED_DATA_SIZE); + close(fd); + memmove(end_of_buf - compressed_size, dest_buf, compressed_size); + decompress_zx02(end_of_buf - compressed_size, dest_buf); +</verb></tscreen> + +<sect>Which decompressor to choose<p> + +The best decompressor depends on your use-case and whether you favor size or +speed. This table allows for a simple comparison. +Decompression speed is the number of uncompressed bytes per second at 1MHz. + +<table><tabular ca="rrrr"> +<bf/Decompressor/|<bf/Approximate compression ratio/|<bf/Decompressor size/|<bf/Decompression speed/@<hline> +<bf/LZ4/|40.7%|272 bytes|18.6kB/s@<hline> +<bf/LZSA1/|46.3%|202 bytes|26.8kB/s@<hline> +<bf/LZSA2/|52.1%|267 bytes|22.5kB/s@<hline> +<bf/ZX02/|52.8%|138 bytes|16.3kB/s@<hline> +<bf/ZX02 (fast)/|52.8%|172 bytes|18.2kB/s +</tabular> +</table> + +In short, if you want to have the smallest amount of data, go with ZX02. If you +want the fastest decompression speed, go with LZSA1. +</article> diff --git a/doc/index.sgml b/doc/index.sgml index e39fd11b4..bd75ec5e4 100644 --- a/doc/index.sgml +++ b/doc/index.sgml @@ -71,6 +71,9 @@ <tag><htmlurl url="debugging.html" name="debugging.html"></tag> Debug programs, using the VICE emulator. + <tag><htmlurl url="decompression.html" name="decompression.html"></tag> + Information about 6502-friendly decompressors shipped in cc65's runtime. + </descrip>