Global Test Infrastructure Metrics: High-level summary of total asserted tests, modular suites, spec files, and verification phases.

Key Topics & Information

• Total Asserted Test Blocks: 1,260+ tests
• Engine & Core Bindings Specs: 70 files (498 it-blocks)
• Toolchain & CLI Specs: 19 files (217 it-blocks)
• Modular In-Engine Test Suites: 84 suites across 51 categories (536 test-blocks)
• External Ecosystem Specs: 3 files (9 it-blocks)
• In-Editor Tool Testers: ToolTester2D and ToolTester3D
• Standalone Host Runners: Regular (tests.exe) and Portable Embedded GDPC (tests_portable.exe)
• Verification Execution Phases: 6 distinct tiers (0 through 6)

The Six-Phase Verification Pipeline: Detailed overview of the 6 execution phases from pre-flight checks to multi-addon smoke testing.

Lapis organizes all testing into a strict 6-phase pipeline ensuring that syntax, binding generation, memory safety, editor tooling, standalone binary embedding, and real-time physics simulation are verified:

Crystal Specifications Breakdown: Overview of the 70 engine unit specs, 19 CLI toolchain specs, and radare2 crash forensics specs.

Phase 1 executes headless Crystal specs validating binding mechanics, type marshaling, and CLI tools:

Phase 1a: Engine & Core Bindings Specs (spec/ - 70 files, 498 tests)

• core_spec.cr: Core Variant type conversions, StringName allocations, ObjectDB class lookups.
• math_spec.cr: Vector2, Vector3, Vector4, Transform2D, Transform3D, Basis, Quaternion, Color math.
• features_spec.cr: DSL syntax: node, @[Export], custom getters/setters, default values, signals.
• safety_spec.cr: Dead pointer protection, #check_alive!, DisposedObjectError enforcement.
• safety_and_bindings_spec.cr: Boehm GC lifetime retention, dynamic scaling (220+ properties), 10+ signal args.
• scene_pipeline_spec.cr: Typed instantiation via > operator, fluent .configure chaining.
• signal_operators_spec.cr: First-class signal operators (+=, -=, >>, >, <<).
• concurrency_spec.cr: Cooperative fibers, Fiber.yield, non-blocking channels, mutex contention.
• thread_safety_spec.cr: ThreadAffinityError when mutating SceneTree from background OS threads.
• match_spec.cr: Polymorphic pattern matching over Variant instances and downcasting.

Phase 1b: Lapis Toolchain & CLI Specs (tools/lapis/spec/ - 19 files, 217 tests)

• cli_spec.cr & commands_spec.cr: Argument parsing, subcommand dispatches, flag normalization.
• standalone_spec.cr: Standalone game compilation, CRT linking, runtime DLL synchronization.
• tui_spec.cr & tui_views_spec.cr: Double-buffered ANSI terminal user interface and split views.
• analyze_spec.cr: Binary auditing with radare2, ASLR/DEP hardening checks, symbol budgeting.
• export_templates_spec.cr: Automated Godot export template downloading and checksum verification.

Phase 1d: Debugger & Crash Forensics Specs (17 files)

• crash_handler_spec.cr: Windows SEH / Unix SIGSEGV interception and crash dump writing.
• plugin_forensics_spec.cr: Automated register extraction (RCX/RDI) and ObjectDB dead-pointer detection.
• decompiler_spec.cr: Disassembly into assembly (pdf) and pseudo-C (pdc) via radare2.

it "detects freed nodes and raises DisposedObjectError defensively" do
  node = Godot.create(Godot::Node2D)
  node_id = node.instance_id
  node.alive?.should be_true

  node.destroy
  node.alive?.should be_false
  node.destroyed?.should be_true

  expect_raises(Godot::DisposedObjectError) do
    node.set_position(Godot::Vector2.new(10.0_f32, 20.0_f32))
  end
end

Headless In-Editor Tool Tests: Executing @tool scripts, live reload, and clean shutdown verification in headless Godot Editor.

Phase 2 boots the genuine Godot Editor in headless mode with CRYSTAL_TOOL_TEST=1 and GODOT_RUN_TOOL_TESTS=1.

Key Components:

1. ToolTester2D (src/main.cr): Active @tool class derived from Godot::Node2D. Auto-executes Registry.run_category("2D", self), verifying 2D transform calculations, CanvasItem rendering, and in-editor tool buttons (ExportToolButton). Outputs .tool_tests_passed and junit_tool_2d.xml.
2. ToolTester3D (src/main.cr): Active @tool class derived from Godot::Node3D. Auto-executes Registry.run_category("3D", self), verifying 3D transforms, MeshInstance3D generation, DirectionalLight3D, and Camera3D projections. Outputs junit_tool_3d.xml.
3. Editor Live Reload & Shadow DLL Loading (spec/editor_reload_spec.cr): Tests the Windows shadow DLL loading cycle where modified game libraries are recompiled and hot-reloaded via crystal_bridge.cpp while the editor remains open.
4. Clean Shutdown Protocol (scripts/verify_editor.ps1): Verifies that ResourceFormatLoaderCrystal handles .cr files as Script resources with zero loader errors, and ensures upon exit that exactly ZERO ERROR: BUG: Unreferenced static string to 0 errors occur.

@[Tool]
node ToolTester2D < Godot::Node2D do
  @[ExportToolButton("▶ Run 2D Tool Tests")]
  property run_tests_button = ->run_tool_tests

  def _ready
    if Godot.editor_hint?
      Godot.print("[ToolTester2D] Editor detected. Auto-executing in-editor tests...")
      run_tool_tests
    end
  end

  def run_tool_tests
    results = Registry.run_category("2D", self)
    # ... asserts results and writes .tool_tests_passed ...
  end
end

Standalone Binary Host Runners: Validating the pure LibGodot host paradigm with regular and portable embedded GDPC executables.

Phase 3 validates the Standalone Host Execution Paradigm (Mode B), where Crystal boots first and embeds the Godot engine as a dynamic shared library (libgodot.dll / libgodot.so).

Phase 3a: Regular Standalone Runner (bin/tests.exe)

• Executed via: tests.exe --headless --rendering-driver opengl3 --audio-driver Dummy --quit-after 600 -- --autorun
• Accompanied by external tests.pck on disk.
• Proves that Crystal's CRT and Boehm GC cleanly bootstrap the Godot C-API and main loop.

Phase 3b: Portable Standalone Runner in Isolated Sandbox (bin/tests_portable.exe)

• The complete virtual filesystem res:// is appended directly to the executable using Godot's GDPC format.
• Tested in an isolated sandbox directory (scratch/test_portable_sandbox/) containing NO .pck file on disk.
• Proves that portable standalone distributions unpack and execute completely self-contained without external assets.

Runtime Test Suites Comprehensive Inventory: Exhaustive directory of all 84 modular runtime suites in spec/suites/ across 14 engine subsystems.

Phase 4 executes scenes/main_test_runner.tscn, running all 84 modular suites (536+ test cases):

Quantitative Zero Memory Leak Verification: Mathematical leak verification using Godot Performance monitors and Boehm GC.

Memory leaks are fatal in native game development. Lapis enforces quantitative zero-leak verification via Lapis::Test.assert_no_leak, leveraging Godot's internal engine performance monitors:

def assert_no_leak(max_delta_objects : Int32 = 0, name : String = "Operation", &block)
  5.times { Fiber.yield }
  GC.collect

  perf = Godot.performance
  initial_objects = perf.get_monitor(Godot::Performance::Monitor::ObjectCount).to_i64
  initial_nodes   = perf.get_monitor(Godot::Performance::Monitor::ObjectNodeCount).to_i64

  yield

  5.times { Fiber.yield }
  GC.collect

  final_objects = perf.get_monitor(Godot::Performance::Monitor::ObjectCount).to_i64
  final_nodes   = perf.get_monitor(Godot::Performance::Monitor::ObjectNodeCount).to_i64

  delta_objects = (final_objects - initial_objects).abs
  delta_nodes   = (final_nodes - initial_nodes).abs

  assert_lte delta_objects, max_delta_objects, "#{name}: Object leak detected!"
  assert_eq delta_nodes, 0, "#{name}: Node leak detected!"
end

Developer Test Commands Reference: Reference guide for invoking tests via Make and the Lapis CLI.