OodleForge v34 - Precompresor de búsqueda exacta OodleLZ (código abierto)

[Fl0ppy]

OodleForge es un motor de archivo en C++ de alto rendimiento diseñado para la reconstrucción exacta, byte a byte, de datos comprimidos con Oodle, con seguridad criptográfica AES-256 integrada.

Desarrollado tras 45 días de trabajo intensivo en hardware antiguo (Core 2 Quad Q6600, 8 GB de RAM), OodleForge es un ejemplo magistral de ingeniería de sistemas pragmática, que logra equilibrar una E/S asíncrona agresiva, una reconstrucción precisa de los datos y la seguridad criptográfica bajo severas limitaciones de recursos.

[Fl0ppy]

Avance de la 33.4, ahora ya funciona bajo linux de manera nativa, adiós a wine 

Linux
ENC [99.97%] Blk: 8521 [E:8509 F:12] | 5.58 MB/s | Time: 00:01:24 | ETA: 00:00:00 | Size: 876.01 MB
--- Encoder Processing Metrics ---
Input File Size:        472.88 MB
Processed Output Size:  876.92 MB
Exact Matches: 8523 (99.86%)
Full Fails:    12
Methods Used: 8+9
Levels Used:  4
Total Duration:    00:01:25


Wine
ENC [100.00%] Blk: 8530 [E:8413 F:117] | 0.51 MB/s | Time: 00:15:35 | ETA: 00:00:00 | Size: 871.28 MB
--- Encoder Processing Metrics ---
Input File Size:        472.88 MB
Processed Output Size:  871.65 MB
Exact Matches: 8418 (98.63%)
Full Fails:    117
Methods Used: 8+9
Levels Used:  4
Total Duration:    00:15:35

[Fl0ppy]

================================================================================
OodleForge v34 Changelog
Build Date: July 6, 2026
Focus: Memory Discipline, I/O Tracking Accuracy, Core Compilation Stability,
       Lock-Free Performance, Hardware-Accelerated Scanning, Wine Compatibility,
       and Bulletproof Correctness.
================================================================================

## 🛡️ Core Stability & Security Fixes
✅ FIX #1: AES-CBC IV Initialization
   - Added `uint8_t aes_iv[16]` to PreHeader struct.
   - Implemented secure random IV generation (CryptGenRandom on Windows, /dev/urandom on Linux).
   - IV is now unique per encryption session, stored in the archive header, and restored during reconstruction.
   - Files: common.h, common.cpp, reconstruct.cpp

✅ FIX #2: Integer Overflow in Block Metadata
   - Added `__builtin_mul_overflow()` check for `block_count * sizeof(PreBlock)`.
   - Prevents underflow in `fsz - blocks_meta_size` calculation.
   - Added detailed error messages.
   - Files: reconstruct.cpp

✅ FIX #3: Hard-Coded 8 MB Block Size Limit
   - Changed to `Config::MAX_OODLE_BLOCK_SIZE = 256 * 1024 * 1024`.
   - Made configurable instead of hard-coded; added debug logging for blocks > 8 MB (Frostbite support).
   - Files: common.h, common.cpp

✅ FIX #4: Uncontrolled Decompression Buffer Allocation
   - Added validation: `if (dec_size > Config::MAX_DECOMP_BUF_SIZE)`.
   - Prevents OOM DoS attacks on crafted archives; added try-catch on allocation with error handling.
   - Files: reconstruct.cpp, common.cpp

✅ FIX #5: Uninitialized PreHeader Reserved Fields
   - Added validation loop to verify all reserved bytes are 0.
   - Catches corruption attacks on reserved fields; added static_assert on PreHeader size after IV addition.
   - Files: reconstruct.cpp, common.h

✅ FIX #6: Race Condition in Auto-Level Detection
   - Changed atomic store to `compare_exchange_strong`.
   - Ensures deterministic compression level selection and thread-safe level caching across runs.
   - Files: encode.cpp

✅ FIX #7: ParseKey() Error Messages
   - Detailed error reporting for invalid hex input (shows position of invalid byte and actual value).
   - Throws with helpful context instead of failing silently.
   - Files: common.cpp

✅ FIX #8: write_gap() Bounds Checking
   - Validates `offset + length <= file_size` to catch gaps extending beyond the file.
   - Better error messages on read failures.
   - Files: common.cpp

✅ FIX #9: TryMatchBlock() Input Validation
   - Null checks on task and opts parameters.
   - Validates usize, csize > 0 and verifies dec_data allocation size matches usize.
   - Files: common.cpp

✅ FIX #10: Oodle DLL Loading Verification
   - Separate error logging for each function load failure.
   - Better debugging of DLL issues; handles both oo2core_9 and oo2core_8 versions.
   - Files: common.cpp

✅ FIX #11: tellp() Desync during seekp()

Identified and resolved a critical desynchronization where seekp(0) would move the file cursor without resetting the internal atomic byte counter.

Fixed by explicitly calling pImpl_->total_bytes_written.store(pos) within the seekp() implementation to ensure tellp() always returns the correct, non-stale offset for lock-free reporting.

Files: common.cpp

✅ FIX #12: ScanStats Contention & Dead Weight

Pruned redundant std::map objects and map_mutex from the ScanStats struct.

Transformed ScanStats into a pure, lock-free, zero-overhead atomic container, ensuring scan operations maintain maximum throughput without lock contention on the hot path.

Files: common.h

🚀 Performance & I/O Upgrades
• Atomic tellp() Persistence

The transition to std::atomic<uint64_t> for byte counting is now fully robust; all seek operations are now captured in the atomic state, eliminating potential UI/stats stalls during archive header reconstruction.

## 🍷 New Feature: Wine Hybrid Mode & AVX2 Optimization
• Wine Detection
  - Automatically detects Wine environment via Windows Registry (HKEY_CURRENT_USER\Software\Wine),
    Linux env vars (WINEPREFIX, WINE), and Linux /proc/version check.
• AVX2 Optimization
  - Native (Windows/Linux): Detects AVX2 support via CPUID. Uses AVX2 optimizations for memory
    operations (significant speedup on compatible CPUs).
  - Wine Environment: Automatically disables AVX2 (`g_avx2_disabled = true`) and falls back to
    baseline CPU instructions to avoid Wine's AVX2 translation overhead while maintaining full compatibility.


## 🚀 Performance & I/O Upgrades
• FastStreamWriter tellp() Race Condition Eliminated
  - REPLACED mutex-locked tellp() with a lock-free std::atomic<uint64_t> byte counter.
  - total_bytes_written is incremented BEFORE data enters the async ring buffer, ensuring 100%
    accurate, instant, zero-contention position tracking.
  - Eliminates UI stalls caused by the main thread blocking on the writer mutex during background disk flushes.

• Silent Cryptographic Failure in ParseKey() Fixed
  - ParseKey() now throws std::invalid_argument immediately if the hex string is not exactly 64
    characters, preventing corrupted output from typos in the AES key argument.

• Compilation Artifacts Resolved
  - Fixed malformed template syntax throughout common.h and common.cpp (e.g., std::lock_guard<std::mutex>,
    std::vector<std::thread>).
  - Ensures clean compilation on GCC 9+, Clang 10+, and MinGW-w64.

• AVX2 SIMD Magic Byte Scanner
  - find_next_magic() now uses _mm256_cmpeq_epi8 and _mm256_movemask_epi8 to scan 32 bytes per
    cycle for magic bytes (0x8C, 0xCC, 0x0C, 0x4C).
  - Includes automatic scalar fallback for CPUs without AVX2 support (5x-10x speedup in initial scan phase).

• Lock-Free Statistics Tracking
  - ScanStats counters (blocks_found, matches_identified, fails_unidentified) are now
    std::atomic<uint32_t> with memory_order_relaxed.
  - Eliminates std::shared_mutex contention on the hot path; UI reads are single atomic loads.

• Thread-Local Map Merging for Method/Level Counts
  - Worker threads accumulate counts in thread_local std::map instances, merging into global stats
    exactly ONCE at scan completion.
  - Reduces map lock acquisitions from O(n_blocks) to O(1).


## 🛠️ Build System Improvements
• Linux Makefile: Changed -march=native to -march=x86-64 -mtune=generic to prevent "Illegal Instruction"
  crashes when distributing binaries to older CPUs (including the legendary Q6600).
• Windows Makefile: Removed redundant -lpthread alongside -lwinpthread to eliminate duplicate symbol
  warnings during MinGW linking.


## 📝 Architecture Notes & Compatibility
• ObjectPool still uses std::shared_ptr — earmarked for v35 index-based refactor.
• CRC32 still uses zlib — hardware CRC32C (SSE4.2) earmarked for v35.
• Full Tracy profiler integration planned for v35.
• Compatibility: The atomic tellp() change is internal only (no archive format changes).
  ParseKey() will now REJECT malformed keys that v33.x silently accepted. Users who relied on
  zero-key behavior must explicitly pass 64 zero hex chars.