CI: Major refactor, expanded coverage, and ~80% speedup (#2736)

* netbsd fail on error

- Group each BSD test in its own subshell. Without this, if an error
occurs, it exits, but we don't know "where" it failed
- added *usesh: true* to BSD for faster copying into the virtualbox VM

- Some test_suite_runners wheren't correctly printing. added
--no-terminal
- For testing I've changed *build-msvc* from Debug to RelWithDebInfo

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* separate runs

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* fix typo

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* Split msvc-debug into separate workflow

Run msvc-debug only on maintainer pushes, keep CI fast and still catches
errors.

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* Fix build and test errors on NetBSD

* Set proper sizes for various types used for threads
* Add num_cpu function
* Add sysctl constants
* Allow testproject to build on NetBSD
* Tweaks to the linker code so that it adds correct flags and finds dynamic linker on NetBSD

* bsd: force system linker test

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* system linker on openbsd

remove lld from netbsd

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* use cc linker in openbsd, and re-add lld for netbsd

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* openbsd: fixes

- implement num_cpu() for OPENBSD
- testproject: add openbsd
- linker.c: fix for openbsd
- bsd refactor main.yml and more fixes

* openbsd: fix for unit tests and test_suite_runner

openbsd now passes all tests except the 'dynlib-test' which yields
'relocation error' when running `cc test.c -L. -ladd -Wl,-rpath=.`

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* openbsd: guard/disable unit test test_ct_intlog2()

* build-msys2-clang: add all tests that build-msvc has

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* Fix Windows import library generation by adding /IMPLIB support and properly declaring static_lib_name()

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* build-msys2-mingw: add all tests that build-msvc has and enable it

configure CMake to link LLVM and LLD statically

* Revert "Revert windows"

This reverts commit 197f82d829.

* win-llvm 21.1.8

uses the RelWithDebInfo+Assertions

* refactor(CI): Centralize test logic and fix Docker execution

This commit refactors the GitHub Actions workflow to improve
maintainability and correct a flaw in the Docker build.

- Ensures Docker based tests run *inside* the generated container, not
on the host. This also fixes environment isolation and user permission
errors.
- Centralizes all common test steps (examples, libs, unit tests, etc)
into a single, auto-detecting Bash script: `scripts/tools/ci_tests.sh`.
- Centralize all release artifact packaging into a single
`scripts/tools/package_build.sh`
- fixes using correct paths for test_suite_runner when running on
windows under bash.

- This significantly reduces YAML duplication, making `main.yml` more
readable and easier to maintain.

This one supersedes pull/2677, and includes the fixes for:

- fix openbsd and netbsd
- added win-llvm 21.1.8 and fixed CMakeLists.txt
- added full unit-tests and suite for build-msys2-mingw and
build-msys2-clang

* macos: add/fix build for llvm (19,20,21)

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* ci: run tests in temp dir to avoid workspace pollution

Executes ci_tests.sh in a disposable temporary directory by copying
resources/ and test/ folders, preventing build artifacts from cluttering
the source tree.

This makes it easy to run `scripts/tools/ci_tests.sh` locally on any
platform.

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* add ilammy/msvc-dev-cmd@ and Ninja

we can simplify the windows runner and run everything in bash

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* windows: drain subprocess stderr before join to avoid test_suite_runner hang

* move last test to `ci_tests.sh`

thus completing the centralization of tests in `ci_tests.sh`

* Fix Windows CI cache

Updates the cache key to include hashes of CMakeLists.txt and the
workflow file.

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* `package_build.sh` runs in temp dir to avoid workspace pollution

- add cleanup

* nix: use `ci_tests.sh`

* remove annoying build-mac warnings

This now checks if package already installed trying to prevent
annoying warnings in the github ci:

Fixes:
```
build-mac (Release, 19)curl 8.17.0 is already installed and up-to-date.
To reinstall 8.17.0, run: brew reinstall curl
```

* refactor test_suite_runner

- less allocations, cleaner code, more maintainable.
- now can be killed and the temp dirs are correctly cleaned up, and the
output is correct.

* main.yml: bsd: change to using rsync, is twice as fast.

* default to `--wincrt=dynamic` when `--sanitize=address` on windows

Default to dynamic, as static ASan is removed in LLVM 21+ for Windows
https://github.com/llvm/llvm-project/pull/107899

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>

* Formatting (indent with tab, align with space). Fix K&R braces. Style: keep all cases multiline if one is.

---------

Signed-off-by: Manuel Barrio Linares <mbarriolinares@gmail.com>
Co-authored-by: Christoffer Lerno <christoffer@aegik.com>
Co-authored-by: Alex Garrett <limit.ordinal17@gmail.com>
This commit is contained in:
Manu Linares
2026-01-13 21:16:06 -03:00
committed by GitHub
parent 15fc435d92
commit 04eb6fc451
20 changed files with 1516 additions and 1649 deletions

View File

@@ -2,7 +2,7 @@ cmake_minimum_required(VERSION 3.20)
set(C3_LLVM_MIN_VERSION 17)
set(C3_LLVM_MAX_VERSION 22)
set(C3_LLVM_DEFAULT_VERSION 19)
set(C3_LLVM_DEFAULT_VERSION 21)
if (CMAKE_CURRENT_SOURCE_DIR STREQUAL CMAKE_CURRENT_BINARY_DIR)
message(FATAL_ERROR "In-tree build detected, please build in a separate directory")
@@ -56,12 +56,30 @@ set(CMAKE_FIND_PACKAGE_SORT_DIRECTION DEC)
set(CMAKE_C_STANDARD 11)
set(CMAKE_CXX_STANDARD 17)
# Use /MT or /MTd
set(CMAKE_MSVC_RUNTIME_LIBRARY "MultiThreaded$<$<CONFIG:Debug>:Debug>")
if(MSVC)
message(STATUS "MSVC version ${MSVC_VERSION}")
add_compile_options(/utf-8)
if(C3_WITH_LLVM)
FetchContent_GetProperties(LLVM_Windows)
if(NOT LLVM_Windows_URL MATCHES "msvcrt")
set(MSVC_CRT_SUFFIX "")
message(STATUS "Detected STATIC LLVM (libcmt)")
else()
set(MSVC_CRT_SUFFIX "DLL")
message(STATUS "Detected DYNAMIC LLVM (msvcrt)")
endif()
endif()
# Force the Runtime to Release (/MT or /MD) even in Debug
# This is required to match our RelWithDebInfo LLVM builds
set(CMAKE_MSVC_RUNTIME_LIBRARY "MultiThreaded${MSVC_CRT_SUFFIX}")
set_property(GLOBAL PROPERTY MSVC_RUNTIME_LIBRARY "MultiThreaded${MSVC_CRT_SUFFIX}")
add_compile_definitions(_ITERATOR_DEBUG_LEVEL=0)
# Suppresses the LNK4098 mismatch warning in Debug builds
set(CMAKE_EXE_LINKER_FLAGS_DEBUG "${CMAKE_EXE_LINKER_FLAGS_DEBUG} /NODEFAULTLIB:libcmtd /NODEFAULTLIB:msvcrtd")
else()
add_compile_options(-gdwarf-3 -fno-exceptions)
@@ -77,6 +95,7 @@ set(C3_USE_MIMALLOC OFF CACHE BOOL "Use built-
set(C3_MIMALLOC_TAG "v1.7.3" CACHE STRING "Used version of mimalloc")
set(C3_USE_TB OFF CACHE BOOL "Use TB")
set(C3_LLD_DIR "" CACHE STRING "Use custom LLD directory")
set(C3_LLD_INCLUDE_DIR "" CACHE STRING "Use custom LLD include directory")
set(C3_ENABLE_CLANGD_LSP OFF CACHE BOOL "Enable/Disable output of compile commands during generation")
set(LLVM_CRT_LIBRARY_DIR "" CACHE STRING "Use custom llvm's compiler-rt directory")
set(TCC_LIB_PATH "/usr/lib/tcc/libtcc1.a" CACHE STRING "Use custom libtcc1.a path")
@@ -142,11 +161,11 @@ if(C3_WITH_LLVM)
endif()
FetchContent_Declare(
LLVM_Windows
URL https://github.com/c3lang/win-llvm/releases/download/llvm_19_1_5/llvm-19.1.5-windows-amd64-msvc17-libcmt.7z
URL https://github.com/c3lang/win-llvm/releases/download/llvm_21_1_8/llvm-21.1.8-windows-amd64-msvc17-libcmt.7z
)
FetchContent_Declare(
LLVM_Windows_debug
URL https://github.com/c3lang/win-llvm/releases/download/llvm_19_1_5/llvm-19.1.5-windows-amd64-msvc17-libcmt-dbg.7z
URL https://github.com/c3lang/win-llvm/releases/download/llvm_21_1_8/llvm-21.1.8-windows-amd64-msvc17-libcmt-dbg.7z
)
if(CMAKE_BUILD_TYPE STREQUAL "Debug")
message("Loading Windows LLVM debug libraries, this may take a while...")
@@ -273,7 +292,12 @@ if(C3_WITH_LLVM)
else()
message(STATUS "Looking for shared lld libraries in ${LLVM_LIBRARY_DIRS}")
find_library(LLVM NAMES libLLVM.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH REQUIRED)
#find_library(LLVM NAMES libLLVM.so PATHS ${LLVM_LIBRARY_DIRS} NO_DEFAULT_PATH REQUIRED)
if(UNIX AND NOT WIN32)
find_library(LLVM NAMES libLLVM.so PATHS ${LLVM_LIBRARY_DIRS} REQUIRED)
else()
find_library(LLVM NAMES libLLVM.a LLVM.lib PATHS ${LLVM_LIBRARY_DIRS} REQUIRED)
endif()
set(llvm_libs ${LLVM})
# These don't seem to be reliable on windows.
@@ -435,10 +459,18 @@ if(C3_WITH_LLVM)
target_compile_definitions(c3c PUBLIC LLVM_AVAILABLE=1)
add_library(c3c_wrappers STATIC wrapper/src/wrapper.cpp)
if (MSVC)
# Use the same detected CRT for the wrapper
set_target_properties(c3c_wrappers PROPERTIES MSVC_RUNTIME_LIBRARY "MultiThreaded${MSVC_CRT_SUFFIX}")
set_target_properties(miniz PROPERTIES MSVC_RUNTIME_LIBRARY "MultiThreaded${MSVC_CRT_SUFFIX}")
target_compile_options(c3c PRIVATE
"$<$<CONFIG:Debug>:/EHa>"
"$<$<CONFIG:Release>:/EHsc>")
endif()
if(C3_LLD_INCLUDE_DIR)
target_include_directories(c3c_wrappers PRIVATE ${C3_LLD_INCLUDE_DIR})
endif()
else()
target_sources(c3c PRIVATE src/utils/hostinfo.c)
target_compile_definitions(c3c PUBLIC LLVM_AVAILABLE=0)
@@ -552,13 +584,21 @@ if(MSVC)
endif()
if(C3_WITH_LLVM)
set(clang_lib_dir ${llvm_dir}/lib/clang/${C3_LLVM_VERSION}/lib/windows)
set(sanitizer_runtime_libraries
${clang_lib_dir}/clang_rt.asan-x86_64.lib
${clang_lib_dir}/clang_rt.asan_dynamic-x86_64.lib
${clang_lib_dir}/clang_rt.asan_dynamic-x86_64.dll
${clang_lib_dir}/clang_rt.asan_dynamic_runtime_thunk-x86_64.lib)
# The the sanitizer libs are in the folder "lib/clang/21/lib/windows/" but use the find anyway
file(GLOB_RECURSE FOUND_ASAN_LIB "${llvm_dir}/*clang_rt.asan_dynamic-x86_64.lib")
if(FOUND_ASAN_LIB)
list(GET FOUND_ASAN_LIB 0 _asan_path)
get_filename_component(_asan_dir "${_asan_path}" DIRECTORY)
set(sanitizer_runtime_libraries
${_asan_dir}/clang_rt.asan_dynamic-x86_64.lib
${_asan_dir}/clang_rt.asan_dynamic-x86_64.dll
${_asan_dir}/clang_rt.asan_dynamic_runtime_thunk-x86_64.lib)
message(STATUS "Found Sanitizer binaries at: ${_asan_dir}")
else()
message(WARNING "Could not find sanitizer runtime libraries in ${llvm_dir}")
endif()
endif()
else()
if (C3_WITH_LLVM AND NOT LLVM_ENABLE_RTTI)
target_compile_options(c3c_wrappers PRIVATE -fno-rtti)