the10thwiz/cc65
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #2847 from colinleroy/zx02-faster-decompressor

Browse Source 
Add a 9% faster ZX02 decompressor
This commit is contained in:
Bob Andrews
2025-11-03 13:34:29 +01:00
committed by GitHub
parent 9390e5c2ce d6dd0fa865
commit 083f89a9ca
6 changed files with 791 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

164
doc/decompression.sgml Normal file
View File

@@ -0,0 +1,164 @@
<!doctype linuxdoc system>
<article>
<title>Decompressing data with cc65
<author><url url="mailto:colin@colino.net" name="Colin Leroy-Mira">
<abstract>
How to decompress data using one of cc65's runtime decompressors.
</abstract>
<!-- Table of contents -->
<toc>
<!-- Begin the document -->
<sect>Overview<p>
cc65 ships with multiple decompressors, each having pros and cons. This page
will detail each of them, and how to use them.
<sect>LZ4<p>
The LZ4 format is widely known. It has a number of drawbacks, though:
There are many LZ4 subformats available, and generating LZ4 data compatible
with the cc65 decompressor requires some work. The cc65 decompressor works
on raw LZ4 data with no header, which makes it difficult to generate compressed
data simply using the <tt/lz4/ command-line utility.
This also means that the function needs to be passed the expected decompressed
size as argument. This makes generating compressed LZ4 data even harder.
The simplest way to generate "correct" LZ4 data for the cc65 decompressor is
to write a small C utility, like this one (example stripped of any error checking):
<tscreen><verb>
FILE *fp;
size_t read_bytes, wrote_bytes;
char header[2];
char in_buf[MAX_COMPRESSED_DATA_SIZE*16];
char out_buf[MAX_COMPRESSED_DATA_SIZE];
fp = fopen(argv[1], "rb");
read_bytes = fread(in_buf, 1, sizeof(in_buf), fp);
fclose(fp);
wrote_bytes = LZ4_compress_HC(in_buf, out_buf, read_bytes, sizeof(out_buf), 16);
header[0] = (read_bytes & 0xFF);
header[1] = (read_bytes & 0xFFFF) >> 8;
fp = fopen("DATA.LZ4", "wb");
fwrite(header, 1, sizeof(header), fp);
fwrite(out_buf, 1, wrote_bytes, fp);
fclose(fp);
</verb></tscreen>
Decompressing in a cc65 program then looks like:
<tscreen><verb>
int fd;
int decompressed_size;
char compressed[MAX_COMPRESSED_DATA_SIZE];
char destination[MAX_COMPRESSED_DATA_SIZE*16];
fd = open("DATA.LZ4", O_RDONLY);
read(fd, &amp;decompressed_size, sizeof(int));
read(fd, compressed, MAX_COMPRESSED_DATA_SIZE);
close(fd);
decompress_lz4(compressed, destination, decompressed_size);
</verb></tscreen>
<sect>LZSA<p>
The LZSA formats come from Emmanuel Marty and has its code hosted in the
<url url="https://github.com/emmanuel-marty/lzsa" name="Github LZSA repository">.
Compressing data is simple, from a command-line or shell:
<tscreen><verb>
lzsa -r -f 1 input.bin DATA.LZSA #For lzsa1 format
lzsa -r -f 2 input.bin DATA.LZSA #For lzsa2 format
</verb></tscreen>
Decompressing is then as simple as possible:
<tscreen><verb>
int fd;
char compressed[MAX_COMPRESSED_DATA_SIZE];
char destination[MAX_COMPRESSED_DATA_SIZE*16];
fd = open("DATA.LZSA", O_RDONLY);
read(fd, compressed, MAX_COMPRESSED_DATA_SIZE);
close(fd);
decompress_lzsa1(compressed, destination);
// or
// decompress_lzsa2(compressed, destination);
</verb></tscreen>
<sect>ZX02<p>
The ZX02 formats come from <tt/dmsc/ and has its code hosted in the
<url url="https://github.com/dmsc/zx02" name="Github ZX02 repository">.
Compressing data is simple, from a command-line or shell:
<tscreen><verb>
zx02 -f input.bin DATA.ZX02
</verb></tscreen>
Decompressing is then as simple as possible:
<tscreen><verb>
int fd;
char compressed[MAX_COMPRESSED_DATA_SIZE];
char destination[MAX_COMPRESSED_DATA_SIZE*16];
fd = open("DATA.ZX02", O_RDONLY);
read(fd, compressed, MAX_COMPRESSED_DATA_SIZE);
close(fd);
decompress_zx02(compressed, destination);
</verb></tscreen>
<sect>In-place decompression<p>
As memory is often sparse in the cc65 targets, it is often possible to decompress
data "in-place", requiring only the destination buffer and no compressed data
buffer. But as the cc65 decompressors do not support backwards compressed data,
it is necessary to have the compressed data at the <tt/end/ of the destination
buffer, <tt/and/ that the destination buffer has a few extra bytes (8 are enough)
so that decompressing does not overwrite the end of the compressed data too soon:
<tscreen><verb>
#define BUFFER_SIZE = (MAX_UNCOMPRESSED_DATA_SIZE) + 8;
int fd;
int compressed_size;
char dest_buf[BUFFER_SIZE];
char *end_of_buf = dest_buf + BUFFER_SIZE;
fd = open("DATA.ZX02", O_RDONLY);
compressed_size = read(fd, dest_buf, MAX_UNCOMPRESSED_DATA_SIZE);
close(fd);
memmove(end_of_buf - compressed_size, dest_buf, compressed_size);
decompress_zx02(end_of_buf - compressed_size, dest_buf);
</verb></tscreen>
<sect>Which decompressor to choose<p>
The best decompressor depends on your use-case and whether you favor size or
speed. This table allows for a simple comparison.
Decompression speed is the number of uncompressed bytes per second at 1MHz.
<table><tabular ca="rrrr">
<bf/Decompressor/|<bf/Approximate compression ratio/|<bf/Decompressor size/|<bf/Decompression speed/@<hline>
<bf/LZ4/|40.7%|272 bytes|18.6kB/s@<hline>
<bf/LZSA1/|46.3%|202 bytes|26.8kB/s@<hline>
<bf/LZSA2/|52.1%|267 bytes|22.5kB/s@<hline>
<bf/ZX02/|52.8%|138 bytes|16.3kB/s@<hline>
<bf/ZX02 (fast)/|52.8%|172 bytes|18.2kB/s
</tabular>
</table>
In short, if you want to have the smallest amount of data, go with ZX02. If you
want the fastest decompression speed, go with LZSA1.
</article>

23
doc/funcref.sgml
View File

@@ -920,6 +920,7 @@ communication, see also <tt>testcode/lib/ser-test.c</tt>.
<itemize>
<item><ref id="decompress_zx02" name="decompress_zx02">
<item><ref id="decompress_zx02_fast" name="decompress_zx02_fast">
</itemize>
@@ -3580,6 +3581,28 @@ smallest and has the best compression ratio.
</quote>
<sect1>decompress_zx02_fast<label id="decompress_zx02_fast"><p>
<quote>
<descrip>
<tag/Function/Uncompress a ZX02 buffer.
<tag/Header/<tt/<ref id="zx02.h" name="zx02.h">/
<tag/Declaration/<tt/void decompress_zx02_fast (const unsigned char* src, unsigned char* const dst);/
<tag/Description/<tt/decompress_zx02_fast/ uncompresses a ZX02 buffer with format 2.
<tag/Notes/<itemize>
<item>Use <tt/zx02 input.bin output.zx02/ to compress your input.
<item>The project and compressor can be found at <url url="https://github.com/dmsc/zx02">
<item>ZX02 is the slowest decompressor shipped with cc65 runtime, but is also the
smallest and has the best compression ratio.
<item><tt/decompress_zx02_fast/ is 9% faster than <tt/decompress_zx02/, but 34
bytes bigger.
</itemize>
<tag/Availability/cc65
<tag/Example/None.
</descrip>
</quote>
<sect1>detect_c128<label id="detect_c128"><p>
<quote>

3
doc/index.sgml
View File

@@ -71,6 +71,9 @@
<tag><htmlurl url="debugging.html" name="debugging.html"></tag>
Debug programs, using the VICE emulator.
<tag><htmlurl url="decompression.html" name="decompression.html"></tag>
Information about 6502-friendly decompressors shipped in cc65's runtime.
</descrip>