Commit Graph

52 Commits

Author SHA1 Message Date
Manu Linares
eae7d0c4a1 stdlib: std::compression::zip and std::compression::deflate (#2930)
* stdlib: implement `std::compression::zip` and `std::compression::deflate`

- C3 implementation of DEFLATE (RFC 1951) and ZIP archive handling.
- Support for reading and writing archives using STORE and DEFLATE
methods.
- Decompression supports both fixed and dynamic Huffman blocks.
- Compression using greedy LZ77 matching.
- Zero dependencies on libc.
- Stream-based entry reading and writing.
- Full unit test coverage.

NOTE: This is an initial implementation. Future improvements could be:

- Optimization of the LZ77 matching (lazy matching).
- Support for dynamic Huffman blocks in compression.
- ZIP64 support for large files/archives.
- Support for encryption and additional compression methods.

* optimizations+refactoring

deflate:
- replace linear search with hash-based match finding.
- implement support for dynamic Huffman blocks using the Package-Merge
algorithm.
- add streaming decompression.
- add buffered StreamBitReader.

zip:
- add ZIP64 support.
- add CP437 and UTF-8 filename encoding detection.
- add DOS date/time conversion and timestamp preservation.
- add ZipEntryReader for streaming entry reads.
- implement ZipArchive.extract and ZipArchive.recover helpers.

other:
- Add `set_modified_time` to std::io;
- Add benchmarks and a few more unit tests.

* zip: add archive comment support

add tests

* forgot to rename the benchmark :(

* detect utf8 names on weird zips

fix method not passed to open_writer

* another edge case where directory doesn't end with /

* testing utilities

- detect encrypted zip
- `ZipArchive.open_writer` default to DEFLATE

* fix zip64 creation, add tests

* fix ZIP header endianness for big-endian compatibility

Update ZipLFH, ZipCDH, ZipEOCD, Zip64EOCD, and Zip64Locator structs to
use little-endian bitstruct types from std::core::bitorder

* fix ZipEntryReader position tracking and seek logic ZIP_METHOD_STORE

added a test to track this

* add package-merge algorithm attribution

Thanks @konimarti

* standalone deflate_benchmark.c3 against `miniz`

* fix integer overflows, leaks and improve safety

* a few safety for 32-bit systems and tests

* deflate compress optimization

* improve match finding, hash updates, and buffer usage

* use ulong for zip offsets

* style changes (#18)

* style changes

* update tests

* style changes in `deflate.c3`

* fix typo

* Allocator first. Some changes to deflate to use `copy_to`

* Fix missing conversion on 32 bits.

* Fix deflate stream. Formatting. Prefer switch over if-elseif

* - Stream functions now use long/ulong rather than isz/usz for seek/available.
- `instream.seek` is replaced by `set_cursor` and `cursor`.
- `instream.available`, `cursor` etc are long/ulong rather than isz/usz to be correct on 32-bit.

* Update to constdef

* Fix test

---------

Co-authored-by: Book-reader <thevoid@outlook.co.nz>
Co-authored-by: Christoffer Lerno <christoffer@aegik.com>
2026-02-20 20:41:34 +01:00
Christoffer Lerno
cdabe8fd9e - Create optional with ~ instead of ?. return io::EOF?; becomes return io::EOF~.
- Deprecated use of `?` to create optional.
2026-01-20 16:10:28 +01:00
Christoffer Lerno
eccc6700dc - Fix appending to c:\ or \ #2569. 2025-11-09 21:45:19 +01:00
Christoffer Lerno
52ececba37 - Fix to Path handling c:\foo and \home parent. #2569 2025-11-08 23:42:47 +01:00
Christoffer Lerno
69b3263a00 - Added path::home_directory, path::documents_directory, path::videos_directory, path::pictures_directory, path::desktop_directory, path::screenshots_directory,
`path::public_share_directory`, `path::templates_directory`, `path::saved_games_directory`, `path::music_directory`, `path::downloads_directory`.
  Fix codegen bug in expressions like `foo(x()) ?? io::EOF?` causing irregular crashes.
2025-09-06 02:27:10 +02:00
Christoffer Lerno
239d249f01 - Added $kindof compile time function.
- Deprecated `@typekind` macro in favour of `$kindof`.
- Deprecated `@typeis` macro in favour of `$typeof(#foo) == int`.
2025-08-27 20:38:12 +02:00
Kiana
506e63284b Tidy up path.c3 (#2309)
* Tidy up path.c3
2025-07-20 12:26:44 +02:00
Christoffer Lerno
fad87b294b mkdir/rmdir would not work properly with substring paths on non-windows platforms. 2025-06-30 21:41:52 +02:00
Christoffer Lerno
a03d821602 - Use @pool_init() to set up a temp pool on a thread. Only the main thread has implicit temp pool setup.
- `tmem` is now a variable.
2025-03-21 17:08:58 +01:00
Christoffer Lerno
72608ce01d - Temp allocator now supports more than 2 in-flight stacks.
- Printing stacktrace uses its own temp allocator.
- `@pool` no longer takes an argument.
- `Allocator` interface removes `mark` and `reset`.
- DynamicArenaAllocator has changed init function.
- Added `BackedArenaAllocator` which is allocated to a fixed size, then allocates on the backing allocator and supports mark/reset.
2025-03-18 15:16:22 +01:00
Christoffer Lerno
5c77c9a754 - Change distinct -> typedef.
- Order of attribute declaration is changed for `alias`.
- Added `LANGUAGE_DEV_VERSION` env constant.
- Rename `anyfault` -> `fault`.
- Changed `fault` -> `faultdef`.
- Added `attrdef` instead of `alias` for attribute aliases.
2025-03-15 20:10:47 +01:00
Christoffer Lerno
8b49e6c14d Rename def to alias. 2025-03-13 11:22:27 +01:00
Christoffer Lerno
b8ae2b06d6 Rename @return! to @return? 2025-03-12 21:40:30 +01:00
Christoffer Lerno
25bccf4883 New faults and syntax (#2034)
- Remove `[?]` syntax.
- Change `int!` to `int?` syntax.
- New `fault` declarations.
- Enum associated values can reference the calling enum.
2025-03-10 00:11:35 +01:00
Christoffer Lerno
c0b80eccad Change @return! syntax to require ":" after faults. Update all contracts to consistently use ":" before the description. 2025-03-05 17:11:45 +01:00
Christoffer Lerno
c40198b016 - new_* functions in general moved to version without new_ prefix.
- `string::new_from_*` changed to `string::from_*`.
- `String.to_utf16_copy` and related changed to `String.to_utf16`.
- `String.to_utf16_tcopy` and related changed to `String.to_temp_utf16`
- `mem::temp_new` changed to `mem::tnew`.
- `mem::temp_alloc` and related changed to `mem::talloc`.
- `mem::temp_new_array` changed to `mem::temp_array`.
2025-03-03 00:32:20 +01:00
Christoffer Lerno
2a895ec7be First 0.7 update, removing all deprecated features. 2025-03-03 00:32:20 +01:00
Christoffer Lerno
e77d1fb646 - Increase precedence of (Foo) { 1, 2 }
- Add `--enable-new-generics` to enable `Foo{int}` generic syntax.
2025-02-18 00:26:22 +01:00
Christoffer Lerno
8c58b31bbd Remove <[]> experimental generic syntax. 2025-02-13 12:53:46 +01:00
Christoffer Lerno
c4212c4649 - Test runner will also check for leaks.
- `write` of qoi would leak memory.
- Issue when having an empty `Path` or just "."
- `set_env` would leak memory.
2025-02-10 00:39:02 +01:00
Christoffer Lerno
e1d546225f Update stdlib with new syntax for short function decl. 2025-02-08 23:04:59 +01:00
Christoffer Lerno
9530fe8fcd Fix regression for parsing types and switch to the "new" generic syntax that's being tested. 2025-01-26 22:51:27 +01:00
Christoffer Lerno
83f8d24892 Update sponsors. More comments in path.c3 2024-10-26 13:00:54 +02:00
Christoffer Lerno
31cd839063 Switch to <* *> docs. Fix issue with dynamically loaded C3 libs with other C3 code. 2024-10-14 02:14:34 +02:00
Christoffer Lerno
ed5d338a39 Added new style named arguments. 2024-09-05 22:13:22 +02:00
Christoffer Lerno
3ccb4b9ec3 $exec may now provide a stdin parameter. Deprecated path.append, path.tappend, getcwd, tgetcwd, path.absolute, ls. Deprecated env::get_config_dir, replaced by env::new_get_config_dir. Added path.has_extension, path.new_append, path.temp_append, new_cwd, temp_cwd, path.new_absolute, new_ls, temp_ls. Added dstring.replace Updated win escapes for exec. 2024-08-15 00:31:47 +02:00
Christoffer Lerno
e293c435af 0.6.0: init_new/init_temp removed. LinkedList API rewritten. List "pop" and "remove" function now return Optionals. RingBuffer API rewritten. Allocator interface changed. Deprecated Allocator, DString and mem functions removed. "identity" functions are now constants for Matrix and Complex numbers. @default implementations for interfaces removed. any* => any, same for interfaces. Emit local/private globals as "private" in LLVM, following C "static". Updated enum syntax. Add support [rgba] properties in vectors. Improved checks of aliased "void". Subarray -> slice. Fix of llvm codegen enum check. Improved alignment handling. Add --output-dir #1155. Removed List/Object append. GenericList renamed AnyList. Remove unused "unwrap". Fixes to cond. Optimize output in dead branches. Better checking of operator methods. Disallow any from implementing dynamic methods. Check for operator mismatch. Remove unnecessary bitfield. Remove numbering in --list* commands Old style enum declaration for params/type, but now the type is optional. Add note on #1086. Allow making distinct types out of "void", "typeid", "anyfault" and faults. Remove system linker build options. "Try" expressions must be simple expressions. Add optimized build to Mac tests. Register int. assert(false) only allowed in unused branches or in tests. Compile time failed asserts is a compile time error. Remove current_block_is_target. Bug when assigning an optional from an optional. Remove unused emit_zstring. Simplify phi code. Remove unnecessary unreachable blocks and remove unnecessary current_block NULL assignments. Proper handling of '.' and Win32 '//server' paths. Add "no discard" to expression blocks with a return value. Detect "unsigned >= 0" as errors. Fix issue with distinct void as a member #1147. Improve callstack debug information #1184. Fix issue with absolute output-dir paths. Lambdas were not type checked thoroughly #1185. Fix compilation warning #1187. Request jump table using @jump for switches. Path normalization - fix possible null terminator out of bounds. Improved error messages on inlined macros.
Upgrade of mingw in CI. Fix problems using reflection on interface types #1203. Improved debug information on defer. $foreach doesn't create an implicit syntactic scope.
Error if `@if` depends on `@if`. Updated Linux stacktrace. Fix of default argument stacktrace. Allow linking libraries directly by file path. Improve inlining warning messages. Added `index_of_char_from`. Compiler crash using enum nameof from different module #1205. Removed unused fields in find_msvc. Use vswhere to find msvc. Update tests for LLVM 19
2024-06-12 10:14:26 +02:00
Christoffer Lerno
748c737e8f 0.5.4: Hash variables accept designated initializers. @safemacro overrides the need for @ in macro names. Fixes to macro context evaluation. Updated allocator api. Removed install_win_reqs.bat. Deterministic @init for MacOS. Fixed temp memory issue with formatter. Support LLVM 19. Add support to compare bitstructs using == and !=. Support Windows .def files. Removed invalid grammar from grammar.y. Support compile time folding of &|^~ for bitstructs. output project setting now respected. Fix issue where constants were not properly constant folded. Add temp_push/pop. Aliased declarations caused errors when used in initializers. Fix export output. Fix of const ternary #1118. Fix of $$MODULE in nested macros #1117. Fix debug info on globals. out now correctly detects subscript[] use #1116. Lateral implicit imports removed. Default to '.' if no libdir is specified. Improved error messages for --lib. Fix raylib snake example. Overzealous local escape check corrected #1127. Improved yacc grammar #1128. --linker argument #1067. Fixes to the matrix operations #1130. Added GenericList. 2024-02-15 21:39:33 +01:00
Christoffer Lerno
1e38ccdd2b Fix missing free on GrowableBitSet. init_new/init_temp for GrowableBitSet, LinkedList, List, HashMap, DString, ByteBuffer. Interface to_string renamed to_new_string. Change in allocator usage, malloc is now heap. Added new_array, new_zero_array, new, new_clear, clone. Concat => concat_new. string::printf => string::new_format, string::tprintf => string::tformat. "to_*" are now "to_new_*" and "to_temp_*". "from_*" is "new_from*" 2023-11-09 01:59:49 +01:00
Christoffer Lerno
99cfaa1583 Refactor protocols. 2023-10-06 22:31:41 +02:00
Christoffer Lerno
49c4595457 Dynamic protocols. 2023-10-05 15:20:41 +02:00
Christoffer Lerno
c5404c6573 get_env for Win32, @pierrec's get_config_dir and get_home_dir 2023-09-25 16:29:49 +02:00
Christoffer Lerno
a1ecf2211f as_str() replaced by str_view() 2023-09-24 23:50:16 +02:00
Pierre Curto
e4febe62ef lib/std/io: make PathWalker return an optional
Signed-off-by: Pierre Curto <pierre.curto@gmail.com>
2023-08-02 23:05:28 +02:00
Pierre Curto
6808a38c9f add std::io::stream::ByteBuffer; fix std::io::Path::walk (#895)
* lib/std/io/stream: add some inlines

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>

* lib/std/io/stream add ByteBuffer

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>

* lib/std/io/path: fix free of paths in walk

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>

* lib/std/bits: remove unnecessary receiver type

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>

---------

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>
2023-07-29 21:21:45 +02:00
Christoffer Lerno
499c82b089 Updated indentation to C3 standard. 2023-07-26 14:01:24 +02:00
Pierre Curto
59b077223b use IoError.UNSUPPORTED_OPERATION instead of asserts; improve Path.walk
Signed-off-by: Pierre Curto <pierre.curto@gmail.com>
2023-07-26 13:05:12 +02:00
Pierre Curto
242006d05d add is_absolute and absolute methods to path::Path (#882)
* lib/std/io/os: remove unnecessary dup in native_ls

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>

* lib/std/core: add String.index_of_char and String.rindex_of_char

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>

* lib/std/io: add Path.is_absolute

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>

* lib/std/io: add Path.absolute

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>

* lib/std: fix Path.normalize on files starting with `.`; add Path.walk

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>

---------

Signed-off-by: Pierre Curto <pierre.curto@gmail.com>
2023-07-25 23:23:56 +02:00
Christoffer Lerno
b74b62e242 Fixes to broken docs parsing. Fixes #880. 2023-07-25 17:55:46 +02:00
Christoffer Lerno
5c2e82fc8b More use of temp allocator. 2023-07-19 02:46:43 +02:00
Christoffer Lerno
4dcfb7a675 Fix of #862 where enums declarations where not regenerated. Updated @pool implementation. 2023-07-18 23:23:53 +02:00
Christoffer Lerno
e2676a5c7f Change zero terminated wide strings to use WString over Char16 2023-07-09 20:10:17 +02:00
Christoffer Lerno
4f7b42cdc4 New generic syntax and ad hoc invocation. 2023-07-06 23:43:36 +02:00
Christoffer Lerno
848a5212ef Fixes #811. Update some stdlib methods. Header printout correctly picks a name. #804 2023-07-03 13:17:43 +02:00
Christoffer Lerno
68f6cb1286 Updated organization file functions, env naming. 2023-06-24 10:29:17 +02:00
Christoffer Lerno
d5b01d3a8f Native ls 2023-06-21 16:27:53 +02:00
Christoffer Lerno
4c1edfb941 Dev (#777)
* The new @if directive.
2023-06-10 23:16:28 +02:00
Christoffer Lerno
ddd0497922 Better lowering of distinct types. Noreturn function call expr recognized as a "jump" for escape analysis. Preferring "def" in libs. To upper / to lower for ascii. Initial dynlib support. 2023-05-21 21:41:01 +02:00
Christoffer Lerno
172d561f07 Change syntax of $if, $assert, $include, $echo. Introduces $error 2023-05-06 12:18:00 +02:00
Christoffer Lerno
809321e20c Updated grammar. Removal of elif. Removal of ':' ';' in some ct statements. Empty faults is now an error. Remove "define" for types. Remove "private". Better errors on incorrect bitstruct syntax. Introduction of wildcard type rather than optional wildcard. Removal of scaled vector type. mkdir and rmdir. Disallow define @Foo() = { @inline }. Add handling for @optreturn and change it to @return!. Restrict interface style functions. Updated x64 ABI. stdlib updates to string. Removed deprecated functions. Update how variadics are implemented. Extended error messages. x86 ABI fixes. Shift check fixes. '!' and '?' are flipped. No trailing ',' allowed in functions. Fix to string parsing. Allow l suffix. Simplifying flatpath. any replaces variant, anyfault replaces anyerr. Allow getting the underlying type of anyfault. De-duplicate string constants. Fix of readme. Extended list. Fix of "(MyEnum)x + 1". Clock and DateTime types. Fixes to array concat. 2023-04-21 10:56:39 +02:00