chore: update Windows build to use PCRE2 as default

Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>

Author: fzipi

.github/workflows/test-ci-windows.yml

@@ -100,6 +100,7 @@ jobs:
         -A $cmakeArch `
         -DWITH_LUA=ON `
         -DWITH_YAJL=ON `
+        -DWITH_PCRE_JIT=ON `
         -S IIS -B "iis\build"
 
     - name: Build IIS Module

README_WINDOWS.md

@@ -8,13 +8,14 @@ by Tom Donovam, 4/2/2011
 
 Dependency | Tested with | Note
 ----|------|----
-Microsoft Visual Studio C++ | Visual Studio 2013 (aka VC12) |
-[CMake build system](http://www.cmake.org/) | CMake v3.8.2 |
-[Apache 2.4.x](http://httpd.apache.org/) | Apache 2.4.27 | Apache must be built from source using the same Visual Studio compiler as mod_security.
-[PCRE, Perl Compatible Regular Expression library](http://www.pcre.org/) | PCRE v8.40
-[LibXML2](http://xmlsoft.org/) | LibXML2 v2.9.4 |
-[Lua Scripting Language](http://www.lua.org/) | Lua v5.3.4
-[cURL multiprotocol file transfer library](http://curl.haxx.se/) | cURL v7.54.0
+Microsoft Visual Studio C++ | Visual Studio 2019 (aka VS16) |
+[CMake build system](http://www.cmake.org/) | CMake v4.2.3 |
+[Apache 2.4.x](http://httpd.apache.org/) | Apache 2.4.66 | Apache must be built from source using the same Visual Studio compiler as mod_security.
+[PCRE2, Perl Compatible Regular Expression library](https://www.pcre.org/) | PCRE2 v10.47 | ModSecurity v2 now uses PCRE2 by default (not legacy PCRE)
+[LibXML2](http://xmlsoft.org/) | LibXML2 v2.15.1 |
+[Lua Scripting Language](http://www.lua.org/) | Lua v5.4.8
+[cURL multiprotocol file transfer library](http://curl.haxx.se/) | cURL v8.18.0
+[zlib compression library](https://zlib.net/) | zlib v1.3.1
 
 
 ## Before building
@@ -23,22 +24,25 @@ The directory where you build software from source ( ``C:\work`` in this exmaple
 must contain the Apache source you used to build the Apache web serverand the mod_security source
 
     Apache source is in             C:\work\httpd-2.4.27    in this example.
-    Apache has been installed to    C:\Apache2427           in this example.
+    Apache has been installed to    C:\Apache2466           in this example.
     Mod_security source is in       C:\work\mod_security    in this example.
 
 ## Download and untar the prerequisite library sources:
 
-    Download pcre-8.40.tar.gz     from ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/
-    untar it into C:\work\  creating C:\work\pcre-8.40
+    Download pcre2-10.47.zip from https://github.com/PCRE2Project/pcre2/releases/download/pcre2-10.47/
+    unzip it into C:\work\  creating C:\work\pcre2-10.47
 
-    Download libxml2-2.9.4.tar.gz    from ftp://xmlsoft.org/libxml2/
-    untar it into C:\work\ creating C:\work\libxml2-2.9.4
+    Download libxml2-2.15.1.tar.gz from https://download.gnome.org/sources/libxml2/2.15/
+    untar it into C:\work\ creating C:\work\libxml2-2.15.1
 
-    Download lua-5.3.4.tar.gz from http://www.lua.org/ftp/
-    untar it into C:\work\ creating C:\work\lua-5.3.4
+    Download lua-5.4.8.tar.gz from http://www.lua.org/ftp/
+    untar it into C:\work\ creating C:\work\lua-5.4.8
 
-    Download curl-7.54.0.tar.gz from http://curl.haxx.se/download.html
-    untar it into C:\work\ creating C:\work\curl-7.54.0
+    Download curl-8.18.0.zip from https://curl.se/download/
+    unzip it into C:\work\ creating C:\work\curl-8.18.0
+
+    Download zlib-1.3.1.tar.gz from https://github.com/madler/zlib/releases/download/v1.3.1/
+    untar it into C:\work\ creating C:\work\zlib-1.3.1
 
 ## Setup your build environment:
 
@@ -71,54 +75,58 @@ If OpenSSL and zlib support were included when you built Apache 2.4, and you wan
 
 ## Build
 
-### PCRE-8.40
+### PCRE2-10.47
 
-    CD C:\work\pcre-8.40
-    CMAKE   -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_SHARED_LIBS=True
+    CD C:\work\pcre2-10.47
+    CMAKE -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_SHARED_LIBS=True -DPCRE2_BUILD_PCRE2_8=ON -DPCRE2_SUPPORT_JIT=ON
     NMAKE
 
-### LibXML2-2.9.4
+Note: PCRE2 with JIT support provides better performance and fixes stack overflow issues present in older PCRE versions.
+
+### LibXML2-2.15.1
 
-    CD C:\work\libxml2-2.9.4\win32
+    CD C:\work\libxml2-2.15.1\win32
     CSCRIPT configure.js iconv=no vcmanifest=yes zlib=yes
     NMAKE -f Makefile.msvc
 
-### Lua-5.3.4
+### Lua-5.4.8
 
-    CD C:\work\lua-5.3.4\src
+    CD C:\work\lua-5.4.8\src
     CL /Ox /arch:SSE2 /GF /GL /Gy /FD /EHsc /MD  /Zi /TC /wd4005 /D "_MBCS" /D "LUA_CORE" /D "LUA_BUILD_AS_DLL" /D "_CRT_SECURE_NO_WARNINGS" /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D "_WIN32" /D "_WINDLL" /c *.c
     DEL lua.obj luac.obj
     LINK /DLL /LTCG /DEBUG /OUT:lua5.1.dll *.obj
     IF EXIST lua5.1.dll.manifest MT  -manifest lua5.1.dll.manifest -outputresource:lua5.1.dll;2
 
-### cURL-7.54.0
+### cURL-8.18.0
 
-    CD C:\work\curl-7.54.0
-    CMAKE   -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_SHARED_LIBS=True -DCURL_ZLIB=True
-    NMAKE
+    CD C:\work\curl-8.18.0\winbuild
+    SET ARCH=x64
+    NMAKE /f Makefile.vc mode=dll ENABLE_WINSSL=yes MACHINE=%ARCH% WITH_ZLIB=dll
 
 ### ModSecurity-2.9.x
 
     CD C:\work\mod_security\apache2
-    NMAKE -f Makefile.win APACHE=C:\Apache2427 PCRE=C:\work\pcre-8.40 LIBXML2=C:\work\libxml2-2.9.4 LUA=C:\work\lua-5.3.4\src
+    NMAKE -f Makefile.win APACHE=C:\Apache2466 PCRE=C:\work\pcre2-10.47 LIBXML2=C:\work\libxml2-2.15.1 LUA=C:\work\lua-5.4.8\src CURL=C:\work\curl-8.18.0
 
 ## Install ModSecurity and run Apache
 
-Copy these five files to ``C:\Apache2427\bin``:
+Copy these files to ``C:\Apache2466\bin``:
 
-    C:\work\pcre-8.40\pcre.dll C:\Apache2427\bin\
-    C:\work\lua-5.3.4\src\lua5.1.dll C:\Apache2427\bin\
-    C:\work\libxml2-2.9.4\win32\bin.msvc\libxml2.dll  C:\Apache2427\bin\
-    C:\work\curl-7.54.0\libcurl.dll  C:\Apache2427\bin\
-    C:\work\mod_security\apache2\mlogc-src\mlogc.exe
+    C:\work\pcre2-10.47\pcre2-8.dll C:\Apache2466\bin\
+    C:\work\lua-5.4.8\src\lua5.1.dll C:\Apache2466\bin\
+    C:\work\libxml2-2.15.1\win32\bin.msvc\libxml2.dll C:\Apache2466\bin\
+    C:\work\curl-8.18.0\builds\libcurl-vc-x64-release-dll-zlib-dll-ipv6-sspi-schannel-obj-lib\libcurl.dll C:\Apache2466\bin\
+    C:\work\mod_security\apache2\mlogc-src\mlogc.exe C:\Apache2466\bin\
 
-Copy this one file to ``C:\Apache2427\modules``:
+Copy this one file to ``C:\Apache2466\modules``:
 
     C:\work\mod_security\apache2\mod_security2.so
 
-You may also copy ``C:\work\curl-7.54.0\curl.exe`` to ``C:\Apache2427\bin``, if you want to use the cURL command-line program.
+You may also copy ``C:\work\curl-8.18.0\curl.exe`` to ``C:\Apache2466\bin``, if you want to use the cURL command-line program.
+
+Download OWASP CRS from https://github.com/coreruleset/coreruleset/releases/latest and unzip it into ``C:\Apache2466\conf\owasp_crs``
 
-Download the core rules from http://sourceforge.net/projects/mod-security/files/modsecurity-crs/0-CURRENT/ and unzip them into ``C:\Apache2427\conf\modsecurity_crs``
+For example, download ``coreruleset-4.x.x.zip``, extract it, and rename the extracted directory to ``owasp_crs``.
 
 Add configuration directives to your Apache conf\httpd.conf:
 
@@ -130,8 +138,9 @@ Add configuration directives to your Apache conf\httpd.conf:
     <IfModule security2_module>
         SecRuleEngine On
         SecDataDir   logs
-        Include conf/modsecurity_crs/*.conf
-        Include conf/modsecurity_crs/base_rules/*.conf
+        # Include OWASP CRS configuration
+        Include conf/owasp_crs/crs-setup.conf
+        Include conf/owasp_crs/rules/*.conf
         SecAuditEngine RelevantOnly
         SecAuditLogRelevantStatus "^(?:5|4\d[^4])"
         SecAuditLogType Serial
@@ -144,31 +153,31 @@ Add configuration directives to your Apache conf\httpd.conf:
 Edit the top of ``C:\work\mod_security\apache2\mlogc-src\Makefile.win`` and set your local paths
 
         # Path to Apache httpd installation
-        BASE = C:\Apache2427
+        BASE = C:\Apache2466
 
         # Paths to required libraries
-        PCRE = C:\work\pcre-8.40
-        CURL = C:\work\curl-7.54.0
+        PCRE = C:\work\pcre2-10.47
+        CURL = C:\work\curl-8.18.0
 
         # Linking libraries
         LIBS = $(BASE)\lib\libapr-1.lib \
                $(BASE)\lib\libaprutil-1.lib \
-               $(PCRE)\pcre.lib \
-               $(CURL)\libcurl_imp.lib \
+               $(PCRE)\pcre2-8.lib \
+               $(CURL)\libcurl.lib \
                wsock32.lib
 
 Build the ``mlogc.exe`` program:
 
         CD  C:\work\mod_security_trunk\mlogc
         NMAKE -f Makefile.win
 
-Copy ``mlocg.exe`` to ``C:\Apache2427\bin\``
+Copy ``mlocg.exe`` to ``C:\Apache2466\bin\``
 
-Create a new command file ``C:\Apache2427\bin\mlogc.bat`` with one line:
+Create a new command file ``C:\Apache2466\bin\mlogc.bat`` with one line:
 
-        C:\Apache2427\bin\mlogc.exe C:\Apache2427\conf\mlogc.conf
+        C:\Apache2466\bin\mlogc.exe C:\Apache2466\conf\mlogc.conf
 
-Create a new configuration file ``C:\Apache2427\conf\mlogc.conf`` to control the piped-logging program ``mlogc.exe``.
+Create a new configuration file ``C:\Apache2466\conf\mlogc.conf`` to control the piped-logging program ``mlogc.exe``.
 Here is an example ``conf\mlogc.conf``:
 
     CollectorRoot       "C:/Apache2427/logs"

iis/Makefile.win

@@ -1,6 +1,7 @@
 ###########################################################################
 #
-# Usage:   NMAKE -f Makefile.win APACHE={httpd installion dir}  PCRE={pcre dir} LIBXML2={LibXML2 dir} [   LUA={Lua dir} ]
+# Usage:   NMAKE -f Makefile.win APACHE={httpd installion dir}  PCRE={pcre2 dir} LIBXML2={LibXML2 dir} [   LUA={Lua dir} ]
+# Note: ModSecurity v2 uses PCRE2 by default (not legacy PCRE). Set PCRE to your pcre2 build directory.
 #
 !IF "$(APACHE)" == "" || "$(PCRE)" == "" || "$(LIBXML2)" == "" || "$(CURL)" == ""
 !ERROR NMAKE arguments: APACHE=dir PCRE=dir LIBXML2=dir CURL=dir are required to build mod_security2 for Windows
@@ -9,7 +10,7 @@
 # Linking libraries
 LIBS = $(APACHE)\lib\libapr-1.lib \
        $(APACHE)\lib\libaprutil-1.lib \
-       $(PCRE)\pcre.lib \
+       $(PCRE)\pcre2-8.lib \
        $(CURL)\libcurl.lib \
        $(LIBXML2)\win32\bin.msvc\libxml2.lib \
        "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "comdlg32.lib" "advapi32.lib" "shell32.lib" "ole32.lib" \
@@ -24,7 +25,7 @@ LINK = link.exe
 
 MT = mt
 
-DEFS = /nologo /O2 /LD /W3 /wd4244 /wd4018 -DWITH_YAJL -DWIN32 -DWINNT -Dinline=APR_INLINE -DAP_DECLARE_STATIC -D_MBCS -D$(VERSION)
+DEFS = /nologo /O2 /LD /W3 /wd4244 /wd4018 -DWITH_YAJL -DWITH_PCRE_JIT -DWIN32 -DWINNT -Dinline=APR_INLINE -DAP_DECLARE_STATIC -D_MBCS -D$(VERSION)
 
 DLL = ModSecurityIIS.dll
 

iis/build_dependencies.bat

@@ -7,15 +7,15 @@
 @set SOURCE_DIR=%USERPROFILE%\Downloads
 
 :: Dependencies
-@set CMAKE=cmake-3.12.4-win32-x86.zip
-@set PCRE=pcre-8.45.zip
-@set ZLIB=zlib-1.2.12.tar.gz
-@set LIBXML2=libxml2-2.9.14.tar.gz
-@set LUA=lua-5.3.6.tar.gz
-@set CURL=curl-7.83.1.zip
-@set APACHE_SRC=httpd-2.4.54.tar.gz
-@set APACHE_BIN32=httpd-2.4.54-win32-VS16.zip
-@set APACHE_BIN64=httpd-2.4.54-win64-VS16.zip
+@set CMAKE=cmake-4.2.3-windows-x86_64.zip
+@set PCRE=pcre2-10.47.zip
+@set ZLIB=zlib-1.3.1.tar.gz
+@set LIBXML2=libxml2-2.15.1.tar.gz
+@set LUA=lua-5.4.8.tar.gz
+@set CURL=curl-8.18.0.zip
+@set APACHE_SRC=httpd-2.4.66.tar.gz
+@set APACHE_BIN32=httpd-2.4.66-win32-VS16.zip
+@set APACHE_BIN64=httpd-2.4.66-win64-VS16.zip
 @set YAJL=yajl-2.1.0.zip
 @set SSDEEP=ssdeep-2.14.1.tar.gz
 @set SSDEEP_BIN=ssdeep-2.14.1.zip

iis/build_release.bat

@@ -32,7 +32,7 @@ copy "%OUTPUT_DIR%\libxml2.dll" "%AMD64%"
 copy "%OUTPUT_DIR%\lua5.1.dll" "%AMD64%"
 copy "%OUTPUT_DIR%\mlogc.exe" "%AMD64%"
 copy "%OUTPUT_DIR%\ModSecurityIIS.dll" "%AMD64%"
-copy "%OUTPUT_DIR%\pcre.dll" "%AMD64%"
+copy "%OUTPUT_DIR%\pcre2-8.dll" "%AMD64%"
 copy "%OUTPUT_DIR%\zlib1.dll" "%AMD64%"
 copy "%OUTPUT_DIR%\yajl.dll" "%AMD64%"
 copy "%OUTPUT_DIR%\fuzzy.dll" "%AMD64%"
@@ -51,7 +51,7 @@ copy "%OUTPUT_DIR%\libxml2.dll" "%X86%"
 copy "%OUTPUT_DIR%\lua5.1.dll" "%X86%"
 copy "%OUTPUT_DIR%\mlogc.exe" "%X86%"
 copy "%OUTPUT_DIR%\ModSecurityIIS.dll" "%X86%"
-copy "%OUTPUT_DIR%\pcre.dll" "%X86%"
+copy "%OUTPUT_DIR%\pcre2-8.dll" "%X86%"
 copy "%OUTPUT_DIR%\zlib1.dll" "%X86%"
 copy "%OUTPUT_DIR%\yajl.dll" "%X86%"
 copy "%OUTPUT_DIR%\fuzzy.dll" "%X86%"

iis/dependencies/build_pcre.bat

@@ -7,29 +7,22 @@ cd "%WORK_DIR%"
 7z.exe x "%SOURCE_DIR%\%PCRE%"
 set PCRE_DIR=%PCRE:~0,-4%
 
-move "%PCRE_DIR%" "pcre"
-
-@if "%PCRE_DIR%" == "pcre-8.40" (
-	Echo. && Echo "PCRE 8.40 found... trying to patch it to compile cleanly" 
-	::cscript /B /Nologo ../patch-pcre-8.40.vbs
-	cd "pcre"
-	cat CMakeLists.txt | sed "s/PCRE_STATIC_RUNTIME OFF CACHE BOOL/PCRE_STATIC_RUNTIME/g" > CMakeLists.txt.ops
-	move CMakeLists.txt CMakeLists.txt.old
-	move CMakeLists.txt.ops CMakeLists.txt
-	cd ..
-)
-
-cd "pcre"
-CMAKE -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_SHARED_LIBS=True
+move "%PCRE_DIR%" "pcre2"
+
+cd "pcre2"
+@echo Building PCRE2 with JIT support...
+CMAKE -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_SHARED_LIBS=True -DPCRE2_BUILD_PCRE2_8=ON -DPCRE2_SUPPORT_JIT=ON
 @if NOT (%ERRORLEVEL%) == (0) goto build_failed
 NMAKE
 @if NOT (%ERRORLEVEL%) == (0) goto build_failed
 cd "%WORK%"
 
-copy /y "%WORK_DIR%\pcre\pcre.dll" "%OUTPUT_DIR%"
-copy /y "%WORK_DIR%\pcre\pcre.pdb" "%OUTPUT_DIR%"
-copy /y "%WORK_DIR%\pcre\pcre.lib" "%OUTPUT_DIR%"
-copy /y "%WORK_DIR%\pcre\pcre.h.generic" "%WORK_DIR%\pcre\pcre.h"
+copy /y "%WORK_DIR%\pcre2\pcre2-8.dll" "%OUTPUT_DIR%"
+copy /y "%WORK_DIR%\pcre2\pcre2-8.pdb" "%OUTPUT_DIR%"
+copy /y "%WORK_DIR%\pcre2\pcre2-8.lib" "%OUTPUT_DIR%"
+@if not exist "%WORK_DIR%\pcre2\include" mkdir "%WORK_DIR%\pcre2\include"
+xcopy /y "%WORK_DIR%\pcre2\src\pcre2*.h" "%WORK_DIR%\pcre2\include\" >nul 2>&1
+copy /y "%WORK_DIR%\pcre2\pcre2.h" "%WORK_DIR%\pcre2\include\" >nul 2>&1
 echo "a"
 @exit /B 0
 

iis/download_files.bat

@@ -1,29 +1,29 @@
-@set CMAKE=cmake-3.12.4-win32-x86.zip	
-@set PCRE=pcre-8.41.zip
-@set ZLIB=zlib-1.2.11.tar.gz
-@set LIBXML2=libxml2-2.9.8.tar.gz
-@set LUA=lua-5.3.5.tar.gz
-@set CURL=curl-7.62.0.zip
-@set APACHE_SRC=httpd-2.4.37.tar.gz
-@set APACHE_BIN32=httpd-2.4.37-win32-VC11.zip
-@set APACHE_BIN64=httpd-2.4.37-win64-VC11.zip
+@set CMAKE=cmake-4.2.3-windows-x86_64.zip
+@set PCRE=pcre2-10.47.zip
+@set ZLIB=zlib-1.3.1.tar.gz
+@set LIBXML2=libxml2-2.15.1.tar.gz
+@set LUA=lua-5.4.8.tar.gz
+@set CURL=curl-8.18.0.zip
+@set APACHE_SRC=httpd-2.4.66.tar.gz
+@set APACHE_BIN32=httpd-2.4.66-win32-VS16.zip
+@set APACHE_BIN64=httpd-2.4.66-win64-VS16.zip
 @set YAJL=yajl-2.1.0.zip
-@set SSDEEP=ssdeep-2.13.tar.gz
-@set SSDEEP_BIN=ssdeep-2.13.zip
+@set SSDEEP=ssdeep-2.14.1.tar.gz
+@set SSDEEP_BIN=ssdeep-2.14.1.zip
 
 :: BITSAdmin refuses to download YAJL from GitHub URL
 :: @set YAJL_URL=https://github.com/lloyd/yajl/archive/%YAJL:~-9%
 @set YAJL_URL=http://http.debian.net/debian/pool/main/y/yajl/yajl_2.1.0.orig.tar.gz
 
-@set CMAKE_URL=https://cmake.org/files/v3.12/%CMAKE%  
-@set PCRE_URL=https://ftp.pcre.org/pub/pcre/%PCRE% 
-@set ZLIB_URL=https://zlib.net/%ZLIB%
-@set LIBXML2_URL=http://xmlsoft.org/sources/%LIBXML2% 
+@set CMAKE_URL=https://github.com/Kitware/CMake/releases/download/v4.2.3/%CMAKE%
+@set PCRE_URL=https://github.com/PCRE2Project/pcre2/releases/download/pcre2-10.47/%PCRE%
+@set ZLIB_URL=https://github.com/madler/zlib/releases/download/v1.3.1/%ZLIB%
+@set LIBXML2_URL=https://download.gnome.org/sources/libxml2/2.15/%LIBXML2%
 @set LUA_URL=https://www.lua.org/ftp/%LUA%
-@set CURL_URL=http://curl.askapache.com/download/%CURL% 
+@set CURL_URL=https://curl.se/download/%CURL%
 @set APACHE_SRC_URL=https://www.apache.org/dist/httpd/%APACHE_SRC%
-@set APACHE_BIN_URL=https://home.apache.org/~steffenal/VC11/binaries
-@set SSDEEP_URL=https://downloads.sourceforge.net/project/ssdeep/ssdeep-2.13
+@set APACHE_BIN_URL=https://www.apachelounge.com/download/VS16/binaries
+@set SSDEEP_URL=https://downloads.sourceforge.net/project/ssdeep/ssdeep-2.14.1
 
 bitsadmin.exe /transfer "Downloading dependencies..." %CMAKE_URL% %SOURCE_DIR%\%CMAKE% %PCRE_URL% %SOURCE_DIR%\%PCRE% %ZLIB_URL% %SOURCE_DIR%\%ZLIB% %LIBXML2_URL% %SOURCE_DIR%\%LIBXML2% %LUA_URL% %SOURCE_DIR%\%LUA% %CURL_URL% %SOURCE_DIR%\%CURL% %APACHE_SRC_URL% %SOURCE_DIR%\%APACHE_SRC% %APACHE_BIN_URL%/%APACHE_BIN32% %SOURCE_DIR%\%APACHE_BIN32% %APACHE_BIN_URL%/%APACHE_BIN64% %SOURCE_DIR%\%APACHE_BIN64% %YAJL_URL% %SOURCE_DIR%\%YAJL% %SSDEEP_URL%/%SSDEEP% %SOURCE_DIR%\%SSDEEP% %SSDEEP_URL%/%SSDEEP_BIN% %SOURCE_DIR%\%SSDEEP_BIN%