![]() ![]() Compression can be a bit or a lot slower than gzip, depending on compression level. zstd ★ - Probably the best algorithm for good compression. For uncompressible data, xz expands the data less than lzip (0.01 vs 1.4), but at the cost of making it more vulnerable to undetected corruption because xz stores uncompressible data uncompressed, and corruption in the uncompressed packets of a LZMA2 stream can't be detected by the decoder, leaving the (optional) check sequence as the only.Can be accessed in userspace using squashfuse. ![]() Supposed to be faster than gzip at the expense of lower compression ratios. snappy - Default in Apache Parquet files.QuickLZ - QuickLZ is the world's fastest compression library, reaching 308 Mbyte/s per core.pixz - Pixz (pronounced pixie) is a parallel, indexing version of xz.This allows files to be extracted on at a time. Use -ms=off to disable solid compression (compressing multiple files together). lzo - "Offers pretty fast compression and extremely fast decompression.".lz4 ★ - Probably the best algorithm for fast compression.Not that much development and lingering bugs. Best if you want to compress very large text files. Even with integrity checking enabled, my. Your choice of compression ( lzma/ bzip2/ gzip). Using zstd -no-check is significantly unsafer than using xz -checknone. Includes the bsdcat, bsdcpio, and bsdtar binaries. libarchive ★ - Provides a common interface to different compression algorithms.gzip - Single-file/stream lossless data compression utility.FastLZ - Small & portable byte-aligned LZ77 compression.bzip2 - Freely available, patent free, high-quality data compressor.Note: The first column with numbers 1.9 indicates the compression setting passed to gzip, bzip2 and lzmash (e.g. Heres an attempt to give you an idea how the various choices compare. Knowing which one to use can be so confusing. ★ - Generally better compression than gzip with comparable runtime. Theres gzip, bzip2, xz, lzip, lzma, lzop and less free tools like rar, zip, arc to choose from. Z is a simple, safe and convenient front-end for the compress(1), uncompress(1), gzip(1), bzip2(1), lzip(1), xz(1), tar(1), zip(1) and unzip(1) utilities.This either calls for a wrapper, or I personally CBB to use that pipeline. OTOH, encrypting with another tool makes for four++ commands (including key management and key storage), and four is larger than two. So sue both Solomon and his friend Reed for not coming up with next algorithm!Ĭan't recall from the top of my head if lzip segments+encrypts, but if it does, and does that multithreaded - well then, add some parity and you'll be fine. It's old, probably older than your smoothie-age githubs. There was even a version with Thread Building Blocks support which is still cached, but I am not aware of its commit history. Heck, xz can be viewed as a fork of it, and in fact it uses one of it's algorithms!įor parity, I'd go with PAR 2.0 format because, well, is there any better?. ![]() Speaking of long term data storage, I'd go with segmented, encrypted 7-Zip with a couple of parity volumes added.ħz format is stable enough for my tastes (15+ years no problem, and it does warn you fairly when you do need that parity applied), opensource (so you can easily find old versions if need arises), encrypts (as long as you trust AES), compresses well (using quite some RAM depending on options), decompresses fast enough (but zstd is faster). ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |