Commit Graph

26 Commits

Author SHA1 Message Date
Manu Linares
eae7d0c4a1 stdlib: std::compression::zip and std::compression::deflate (#2930)
* stdlib: implement `std::compression::zip` and `std::compression::deflate`

- C3 implementation of DEFLATE (RFC 1951) and ZIP archive handling.
- Support for reading and writing archives using STORE and DEFLATE
methods.
- Decompression supports both fixed and dynamic Huffman blocks.
- Compression using greedy LZ77 matching.
- Zero dependencies on libc.
- Stream-based entry reading and writing.
- Full unit test coverage.

NOTE: This is an initial implementation. Future improvements could be:

- Optimization of the LZ77 matching (lazy matching).
- Support for dynamic Huffman blocks in compression.
- ZIP64 support for large files/archives.
- Support for encryption and additional compression methods.

* optimizations+refactoring

deflate:
- replace linear search with hash-based match finding.
- implement support for dynamic Huffman blocks using the Package-Merge
algorithm.
- add streaming decompression.
- add buffered StreamBitReader.

zip:
- add ZIP64 support.
- add CP437 and UTF-8 filename encoding detection.
- add DOS date/time conversion and timestamp preservation.
- add ZipEntryReader for streaming entry reads.
- implement ZipArchive.extract and ZipArchive.recover helpers.

other:
- Add `set_modified_time` to std::io;
- Add benchmarks and a few more unit tests.

* zip: add archive comment support

add tests

* forgot to rename the benchmark :(

* detect utf8 names on weird zips

fix method not passed to open_writer

* another edge case where directory doesn't end with /

* testing utilities

- detect encrypted zip
- `ZipArchive.open_writer` default to DEFLATE

* fix zip64 creation, add tests

* fix ZIP header endianness for big-endian compatibility

Update ZipLFH, ZipCDH, ZipEOCD, Zip64EOCD, and Zip64Locator structs to
use little-endian bitstruct types from std::core::bitorder

* fix ZipEntryReader position tracking and seek logic ZIP_METHOD_STORE

added a test to track this

* add package-merge algorithm attribution

Thanks @konimarti

* standalone deflate_benchmark.c3 against `miniz`

* fix integer overflows, leaks and improve safety

* a few safety for 32-bit systems and tests

* deflate compress optimization

* improve match finding, hash updates, and buffer usage

* use ulong for zip offsets

* style changes (#18)

* style changes

* update tests

* style changes in `deflate.c3`

* fix typo

* Allocator first. Some changes to deflate to use `copy_to`

* Fix missing conversion on 32 bits.

* Fix deflate stream. Formatting. Prefer switch over if-elseif

* - Stream functions now use long/ulong rather than isz/usz for seek/available.
- `instream.seek` is replaced by `set_cursor` and `cursor`.
- `instream.available`, `cursor` etc are long/ulong rather than isz/usz to be correct on 32-bit.

* Update to constdef

* Fix test

---------

Co-authored-by: Book-reader <thevoid@outlook.co.nz>
Co-authored-by: Christoffer Lerno <christoffer@aegik.com>
2026-02-20 20:41:34 +01:00
Christoffer Lerno
cdabe8fd9e - Create optional with ~ instead of ?. return io::EOF?; becomes return io::EOF~.
- Deprecated use of `?` to create optional.
2026-01-20 16:10:28 +01:00
Walther Chen
f023db8638 fix ByteBuffer.grow (#2622)
* fix ByteBuffer.grow

---------

Co-authored-by: Christoffer Lerno <christoffer@aegik.com>
2025-12-06 17:57:11 +01:00
BWindey
12eea4a98d [STDLIB] Add macro return types (#2487)
* add return types to macros where applicable
* std::time::clock::now() -> clock::now()
2025-09-18 14:06:58 +02:00
Christoffer Lerno
e605a21fd3 Revert "Revert 0.7.6 code for 0.7.5 re-release"
This reverts commit d1349c9cfb.
2025-09-05 23:30:35 +02:00
Christoffer Lerno
d1349c9cfb Revert 0.7.6 code for 0.7.5 re-release 2025-09-05 18:42:54 +02:00
niedlich
cf14787552 Typo fixes (#2457)
* fix typos in comments and strings
* fix typos in symbols (and some comments/strings)
* fix typos in releasenotes.md

---------

Co-authored-by: Christoffer Lerno <christoffer@aegik.com>
2025-09-04 01:07:07 +02:00
Christoffer Lerno
a03d821602 - Use @pool_init() to set up a temp pool on a thread. Only the main thread has implicit temp pool setup.
- `tmem` is now a variable.
2025-03-21 17:08:58 +01:00
Christoffer Lerno
25bccf4883 New faults and syntax (#2034)
- Remove `[?]` syntax.
- Change `int!` to `int?` syntax.
- New `fault` declarations.
- Enum associated values can reference the calling enum.
2025-03-10 00:11:35 +01:00
Christoffer Lerno
c0b80eccad Change @return! syntax to require ":" after faults. Update all contracts to consistently use ":" before the description. 2025-03-05 17:11:45 +01:00
Christoffer Lerno
2a895ec7be First 0.7 update, removing all deprecated features. 2025-03-03 00:32:20 +01:00
Christoffer Lerno
33b05bcfeb More deprecations in lib6, and updates to lib7 2025-02-27 11:10:41 +01:00
Christoffer Lerno
4961d0433f - Circumvent Aarch64 miscompilations of atomics.
- Fixes to ByteBuffer allocation/free.
- Fix issue where compiling both for asm and object file would corrupt the obj file output.
2025-02-12 12:50:30 +01:00
Christoffer Lerno
31cd839063 Switch to <* *> docs. Fix issue with dynamically loaded C3 libs with other C3 code. 2024-10-14 02:14:34 +02:00
Christoffer Lerno
ed5d338a39 Added new style named arguments. 2024-09-05 22:13:22 +02:00
Christoffer Lerno
e293c435af 0.6.0: init_new/init_temp removed. LinkedList API rewritten. List "pop" and "remove" function now return Optionals. RingBuffer API rewritten. Allocator interface changed. Deprecated Allocator, DString and mem functions removed. "identity" functions are now constants for Matrix and Complex numbers. @default implementations for interfaces removed. any* => any, same for interfaces. Emit local/private globals as "private" in LLVM, following C "static". Updated enum syntax. Add support [rgba] properties in vectors. Improved checks of aliased "void". Subarray -> slice. Fix of llvm codegen enum check. Improved alignment handling. Add --output-dir #1155. Removed List/Object append. GenericList renamed AnyList. Remove unused "unwrap". Fixes to cond. Optimize output in dead branches. Better checking of operator methods. Disallow any from implementing dynamic methods. Check for operator mismatch. Remove unnecessary bitfield. Remove numbering in --list* commands Old style enum declaration for params/type, but now the type is optional. Add note on #1086. Allow making distinct types out of "void", "typeid", "anyfault" and faults. Remove system linker build options. "Try" expressions must be simple expressions. Add optimized build to Mac tests. Register int. assert(false) only allowed in unused branches or in tests. Compile time failed asserts is a compile time error. Remove current_block_is_target. Bug when assigning an optional from an optional. Remove unused emit_zstring. Simplify phi code. Remove unnecessary unreachable blocks and remove unnecessary current_block NULL assignments. Proper handling of '.' and Win32 '//server' paths. Add "no discard" to expression blocks with a return value. Detect "unsigned >= 0" as errors. Fix issue with distinct void as a member #1147. Improve callstack debug information #1184. Fix issue with absolute output-dir paths. Lambdas were not type checked thoroughly #1185. Fix compilation warning #1187. Request jump table using @jump for switches. Path normalization - fix possible null terminator out of bounds. Improved error messages on inlined macros.
Upgrade of mingw in CI. Fix problems using reflection on interface types #1203. Improved debug information on defer. $foreach doesn't create an implicit syntactic scope.
Error if `@if` depends on `@if`. Updated Linux stacktrace. Fix of default argument stacktrace. Allow linking libraries directly by file path. Improve inlining warning messages. Added `index_of_char_from`. Compiler crash using enum nameof from different module #1205. Removed unused fields in find_msvc. Use vswhere to find msvc. Update tests for LLVM 19
2024-06-12 10:14:26 +02:00
Christoffer Lerno
748c737e8f 0.5.4: Hash variables accept designated initializers. @safemacro overrides the need for @ in macro names. Fixes to macro context evaluation. Updated allocator api. Removed install_win_reqs.bat. Deterministic @init for MacOS. Fixed temp memory issue with formatter. Support LLVM 19. Add support to compare bitstructs using == and !=. Support Windows .def files. Removed invalid grammar from grammar.y. Support compile time folding of &|^~ for bitstructs. output project setting now respected. Fix issue where constants were not properly constant folded. Add temp_push/pop. Aliased declarations caused errors when used in initializers. Fix export output. Fix of const ternary #1118. Fix of $$MODULE in nested macros #1117. Fix debug info on globals. out now correctly detects subscript[] use #1116. Lateral implicit imports removed. Default to '.' if no libdir is specified. Improved error messages for --lib. Fix raylib snake example. Overzealous local escape check corrected #1127. Improved yacc grammar #1128. --linker argument #1067. Fixes to the matrix operations #1130. Added GenericList. 2024-02-15 21:39:33 +01:00
Christoffer Lerno
deb4cc7c4b 0.5.3: Single-module not respected. Fix issue with compiler defined types. Fix optimization levels for projects. Use GEP i8 on offsets. Optimize foreach on len 1 arrays. Move panic blocks last. Fix generic module wildcard imports. Deprecate init_temp / init_new. Fix issue with macro vaarg and untyped lists. Fix extern const globals. 2024-01-14 15:34:54 +01:00
Christoffer Lerno
ffb0021d04 Use backtrace on windows. Updated backtrace API 2023-11-17 00:36:42 +01:00
Christoffer Lerno
f39aa1a41e Add location tracking for memory allocations. 2023-11-09 22:03:25 +01:00
Christoffer Lerno
1e38ccdd2b Fix missing free on GrowableBitSet. init_new/init_temp for GrowableBitSet, LinkedList, List, HashMap, DString, ByteBuffer. Interface to_string renamed to_new_string. Change in allocator usage, malloc is now heap. Added new_array, new_zero_array, new, new_clear, clone. Concat => concat_new. string::printf => string::new_format, string::tprintf => string::tformat. "to_*" are now "to_new_*" and "to_temp_*". "from_*" is "new_from*" 2023-11-09 01:59:49 +01:00
Christoffer Lerno
1aa038c92f Interface based streams. Fix for initializing with a force unwrap inside. Allow $define to take a list. Allow $define to return error on argument type mismatch in call. Fixed broken bit operations on boolean vectors. 2023-10-31 01:06:59 +01:00
Christoffer Lerno
89d4c2cab7 Allocator uses protocols. Fix bug where it was not possible to pass a ref variable as a ref variable. Correct codegen for !anyptr. 2023-10-14 12:39:46 +02:00
Pierre Curto
d61482dffc fix Object.free (#982)
* lib/std/collections: add HashMap.@each_entry()
* lib/std/json: fix Object.free() when object is a map
* lib/std/collections: fix allocator use in Object.{set,set_at,append}
* lib/std: add char.from_hex
* lib/std/collections: print arrays and objects compactly
* lib/std/io: fix Formatter.vprintf result
* lib/std/io/stream: rename module for ByteBuffer
* lib/std/io/stream: make Scanner a Stream reader
* lib/std/io: make std{in,err,out} return File* if no libc
2023-09-12 13:49:52 +02:00
Christoffer Lerno
9a6d83f526 Updated stream API. 2023-09-03 01:14:15 +02:00
Pierre Curto
6808a38c9f add std::io::stream::ByteBuffer; fix std::io::Path::walk (#895)
* lib/std/io/stream: add some inlines

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>

* lib/std/io/stream add ByteBuffer

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>

* lib/std/io/path: fix free of paths in walk

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>

* lib/std/bits: remove unnecessary receiver type

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>

---------

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>
2023-07-29 21:21:45 +02:00