Engine benchmark tools by juliancnn · Pull Request #34918 · wazuh/wazuh

juliancnn · 2026-03-12T21:20:50Z

Description

Closes #34492
WIP

Proposed Changes

Results and Evidence

Manual tests with their corresponding evidence

Compilation without warnings on every supported platform
- Linux
- Windows
- MAC OS X
Log syntax and correct language review

Memory tests for Linux
- Coverity
- Valgrind (memcheck and descriptor leaks check)
- AddressSanitizer
Memory tests for Windows
- Coverity
- UMDH
Memory tests for macOS
- Leaks
- AddressSanitizer
Decoder/Rule tests (Wazuh v4.x)
- Added unit testing files ".ini"
- runtests.py executed without errors
Engine (Wazuh v5.x and above)
- Test run in parallel
- ASAN for test (utest/ctest)
- TSAN for test and wazuh-engine.
Wazuh server API/Framework
- Run API Integration Tests

Artifacts Affected

Configuration Changes

Tests Introduced

Review Checklist

Code changes reviewed
Relevant evidence provided
Tests cover the new functionality
Configuration changes documented
Developer documentation reflects the changes
Meets requirements and/or definition of done
No unresolved dependencies with other issues
...

LucioDonda

LGTM

Start of execution

(venv) ╭─root@93c2bb392f9a /workspaces/devContainer/wazuh/src/engine/test/acceptance_test ‹enhancement/engine-benchmark-tools●› 
╰─# ./acceptance_test.sh \
    --threads 1,2,4,8 \
    --time 60 \
    --rate 0 \
    --batch 100 \
    --input /workspaces/devContainer/testingEvents/test_logs/ \
    --grace 5 \
    --results ./results
[12:43:31] Pre-flight checks...
[12:43:31] Checking Python dependencies (/workspaces/devContainer/wazuh/src/engine/test/acceptance_test/requirements.txt)...
[12:43:31] All Python dependencies satisfied.
[12:43:31] Pre-flight checks passed.
[12:43:31] Generating system report -> ./results/system_report.txt
[12:43:31] System report saved.
[12:43:31] ============================================================
[12:43:31]   Engine Benchmark Suite
[12:43:31]   Threads to test: 1 2 4 8
[12:43:31]   Results dir:     ./results
[12:43:31] ============================================================
[12:43:31] 
[12:43:31] ============================================================
[12:43:31]   TEST: 1 orchestrator thread(s)
[12:43:31] ============================================================
[12:43:31] Manager not running.
[12:43:31] Removing stale KVDB lock files from engine...
[12:43:31] Removing stale KVDB lock files from queue...
[12:43:31] Starting analysisd with 1 orchestrator thread(s)...
[12:43:31] analysisd launched (PID 773359).
[12:43:31] Waiting for engine readiness (timeout 120s)...
[12:43:37] Engine ready (log detected).
[12:43:37] Verifying route 'cmsync_standard'...
[12:43:37] Route 'cmsync_standard' confirmed: {"status":"OK","route":{"name":"cmsync_standard","namespaceId":"cmsync_standard_5f3d","priority":1,"entry_status":"ENABLED","uptime":1}}
[12:43:37] Starting monitor -> ./results/monitor-1T.csv
[12:43:37] Monitor started (PID 773671).
[12:43:37] Grace period (5s) before benchmark...
2026-03-20 12:43:37,037 [INFO] PID file written: ./results/monitor-1T.pid (pid=773671)
2026-03-20 12:43:37,039 [INFO] Found process 'wazuh-manager-analysisd' with PID 771966
2026-03-20 12:43:37,039 [INFO] Monitoring PID 771966 (wazuh-manager-analysisd) every 1.0s -> ./results/monitor-1T.csv
2026-03-20 12:43:38,040 [INFO] cpu=1.0%  rss=143.7MB  vms=1998.9MB  fds=63  r_ops=1504  w_ops=415  r_bytes=0  w_bytes=516096  disk=0.00%
2026-03-20 12:43:39,041 [INFO] cpu=1.0%  rss=143.7MB  vms=1998.9MB  fds=63  r_ops=1504  w_ops=415  r_bytes=0  w_bytes=516096  disk=0.00%
2026-03-20 12:43:40,042 [INFO] cpu=1.0%  rss=143.7MB  vms=1998.9MB  fds=63  r_ops=1504  w_ops=415  r_bytes=0  w_bytes=516096  disk=0.00%
2026-03-20 12:43:41,043 [INFO] cpu=0.0%  rss=143.7MB  vms=1998.9MB  fds=63  r_ops=1504  w_ops=415  r_bytes=0  w_bytes=516096  disk=0.00%
[12:43:42] Running warmup benchmark (run 1/2)...
[12:43:42] Running benchmark: rate=0 time=60s batch=100 input=/workspaces/devContainer/testingEvents/test_logs/
[12:43:42]   output watched: /var/wazuh-manager/logs/alerts/alerts.json
[12:43:42]   csv: ./results/bench-1T-warmup.csv
2026-03-20 12:43:42,044 [INFO] cpu=1.0%  rss=143.7MB  vms=1998.9MB  fds=63  r_ops=1504  w_ops=415  r_bytes=0  w_bytes=516096  disk=0.00%
  access-list_input.txt  -> 31 lines
  access_input.txt  -> 3 lines
  access_transparency_input.txt  -> 1 lines
  activity_input.txt  -> 28 lines
  activitylogs-edgecases_input.txt  -> 1 lines
  activitylogs-raw_input.txt  -> 1 lines
  admin-application_input.txt  -> 9 lines
  admin-calendar_input.txt  -> 13 lines
  admin-chat_input.txt  -> 4 lines
  admin-chromeos_input.txt  -> 20 lines
  admin-contacts_input.txt  -> 1 lines
  admin-delegatedadmin_input.txt  -> 8 lines
  admin-docs_input.txt  -> 4 lines
  admin-domainlog_input.txt  -> 24 lines

Benchmark configuration
  Socket:            /var/wazuh-manager/queue/sockets/queue-http.sock
  Endpoint:          POST /events/enriched
  Target rate:       unlimited
  Send duration:     60 s
  Batch size:        100 events
  Input dir:         /workspaces/devContainer/testingEvents/test_logs/
  Output file:       /var/wazuh-manager/logs/alerts/alerts.json
  Truncate output:   true
  Events loaded:     11638
  Initial out lines: 0

=========================================================
                   BENCHMARK REPORT
=========================================================

Timing
  Sending phase:          60.00 s
  Drain phase:            33.00 s
  Total:                  93.00 s

Throughput
  Events sent:         23493300
  Events processed:      315392
  Events lost:         23177908  (98.66%)

  Avg send rate:      391547.64 EPS
  Avg process rate:     3391.29 EPS (over total time)

Per-second detail
     Sec    Sent (EPS)     Processed
  ------    ----------     ---------
       1        437700          3170
       2        404600          3342
       3        401900          3305
       4        398200          3300
       5        400800          3363
       6        396900          3288
       7        397600          3328
       8        398700          3295
       9        396300          3304
      10        393800          3304
      11        396800          3313
      12        392200          3299
      13        395200          3270
      14        394600          3261
      15        391900          3300
      16        394000          3283
      17        393900          3286
      18        392700          3301
      19        393700          3292
      20        388000          3256
      21        393000          3300
      22        389700          3245
      23        390700          3268
      24        391800          3249
      25        389700          3231
      26        390100          3215
      27        390700          3272
      28        387400          3271
      29        391100          3224
      30        386300          3277
      31        389900          3316
      32        388600          3259
      33        388300          3266
      34        391400          3238
      35        387800          3260
      36        389100          3275
      37        387800          3247
      38        389200          3255
      39        390700          3225
      40        384400          3282
      41        388400          3225
      42        386300          3261
      43        390100          3237
      44        390400          3275
      45        388000          3283
      46        389800          3262
      47        385800          3256
      48        404300          3254
      49        388200          3251
      50        384100          3167
      51        387400          3264
      52        385800          3274
      53        386900          3237
      54        382200          3273
      55        385200          3271
      56        385900          3260
      57        381300          3190
      58        388500          3233
      59        384700          3278
      60        382800          3211
      61             0          3932
      62             0          3993
      63             0          3972
      64             0          4014
      65             0          3958
      66             0          4034
      67             0          3970
      68             0          4013
      69             0          4072
      70             0          3909
      71             0          4037
      72             0          3953
      73             0          4041
      74             0          4018
      75             0          4035
      76             0          3946
      77             0          4036
      78             0          3971
      79             0          4035
      80             0          3919
      81             0          4017
      82             0          4012
      83             0          4020
      84             0          3919
      85             0          4008
      86             0          3978
      87             0          3963
      88             0          3963
      89             0          3982
      90             0          3675
      91             0             0
      92             0             0
      93             0             0

  Peak sent rate:        437700 EPS
  Peak process rate:       4072 EPS
  Avg sent (per-s):   252616.13 EPS
  Avg proc (per-s):     3391.31 EPS

=========================================================

CSV report written to ./results/bench-8T.csv (93 rows)
[13:05:09] Benchmark finished.
[13:05:09] Grace period (5s) after benchmark...
2026-03-20 13:05:10,507 [INFO] cpu=2.0%  rss=1198.4MB  vms=4627.3MB  fds=63  r_ops=9004  w_ops=643175  r_bytes=0  w_bytes=994070528  disk=0.10%
2026-03-20 13:05:11,508 [INFO] cpu=0.0%  rss=1198.4MB  vms=4627.3MB  fds=63  r_ops=9004  w_ops=643175  r_bytes=0  w_bytes=994070528  disk=0.10%
2026-03-20 13:05:12,509 [INFO] cpu=1.0%  rss=1198.4MB  vms=4627.3MB  fds=63  r_ops=9004  w_ops=643175  r_bytes=0  w_bytes=994070528  disk=0.10%
2026-03-20 13:05:13,510 [INFO] cpu=2.0%  rss=1198.4MB  vms=4627.3MB  fds=63  r_ops=9004  w_ops=643175  r_bytes=0  w_bytes=994070528  disk=0.10%
2026-03-20 13:05:14,511 [INFO] cpu=1.0%  rss=1198.4MB  vms=4627.3MB  fds=63  r_ops=9004  w_ops=643175  r_bytes=0  w_bytes=994070528  disk=0.10%
[13:05:14] Stopping monitor (PID 783639)...
2026-03-20 13:05:14,614 [INFO] Stop signal received — finishing current sample and exiting.
2026-03-20 13:05:15,512 [INFO] cpu=1.0%  rss=1198.4MB  vms=4627.3MB  fds=63  r_ops=9004  w_ops=643181  r_bytes=0  w_bytes=994074624  disk=0.10%
2026-03-20 13:05:15,512 [INFO] Monitor finished. CSV written to ./results/monitor-8T.csv
2026-03-20 13:05:15,512 [INFO] PID file removed: ./results/monitor-8T.pid
[13:05:15] Monitor stopped.
[13:05:15] Stopping analysisd...
wazuh-manager-clusterd not running...
wazuh-manager-modulesd not running...
wazuh-manager-monitord not running...
wazuh-manager-remoted not running...
Killing wazuh-manager-analysisd...
wazuh-manager-db not running...
wazuh-manager-authd not running...
wazuh-manager-apid not running...
Wazuh v5.0.0 Stopped
[13:05:17] analysisd stopped.
[13:05:17] Test with 8 thread(s) complete.
[13:05:17]   Monitor CSV: ./results/monitor-8T.csv
[13:05:17]   Bench CSV:   ./results/bench-8T.csv
[13:05:17] 
[13:05:17] ============================================================
[13:05:17]   All tests complete. Results in: ./results/
[13:05:17] ============================================================
total 416K
-rw-r--r-- 1 root root 6.5K Mar 20 12:51 bench-1T.csv
-rw-r--r-- 1 root root  38K Mar 20 12:51 bench-1T.log
-rw-r--r-- 1 root root 6.5K Mar 20 12:47 bench-1T-warmup.csv
-rw-r--r-- 1 root root  38K Mar 20 12:47 bench-1T-warmup.log
-rw-r--r-- 1 root root 4.5K Mar 20 12:56 bench-2T.csv
-rw-r--r-- 1 root root  31K Mar 20 12:56 bench-2T.log
-rw-r--r-- 1 root root 4.5K Mar 20 12:54 bench-2T-warmup.csv
-rw-r--r-- 1 root root  31K Mar 20 12:54 bench-2T-warmup.log
-rw-r--r-- 1 root root 3.4K Mar 20 13:01 bench-4T.csv
-rw-r--r-- 1 root root  28K Mar 20 13:01 bench-4T.log
-rw-r--r-- 1 root root 3.4K Mar 20 12:59 bench-4T-warmup.csv
-rw-r--r-- 1 root root  28K Mar 20 12:59 bench-4T-warmup.log
-rw-r--r-- 1 root root 2.9K Mar 20 13:05 bench-8T.csv
-rw-r--r-- 1 root root  26K Mar 20 13:05 bench-8T.log
-rw-r--r-- 1 root root 2.9K Mar 20 13:03 bench-8T-warmup.csv
-rw-r--r-- 1 root root  26K Mar 20 13:03 bench-8T-warmup.log
-rw-r--r-- 1 root root  32K Mar 20 12:51 monitor-1T.csv
-rw-r--r-- 1 root root  24K Mar 20 12:57 monitor-2T.csv
-rw-r--r-- 1 root root  18K Mar 20 13:01 monitor-4T.csv
-rw-r--r-- 1 root root  16K Mar 20 13:05 monitor-8T.csv
-rw-r--r-- 1 root root 1010 Mar 20 12:43 system_report.txt
[13:05:17] Cleanup triggered (exit code: 0)...

Details

This seems to have some issues on scaling

NahuFigueroa97

LGTM!

iNFRA

[vagrant@amzn2023 ~]$ lscpu
Architecture:                x86_64
  CPU op-mode(s):            32-bit, 64-bit
  Address sizes:             48 bits physical, 48 bits virtual
  Byte Order:                Little Endian
CPU(s):                      4
  On-line CPU(s) list:       0-3
Vendor ID:                   AuthenticAMD
  Model name:                AMD Ryzen 9 7900 12-Core Processor
    CPU family:              25
    Model:                   97
    Thread(s) per core:      1
    Core(s) per socket:      4
    Socket(s):               1
    Stepping:                2
    BogoMIPS:                7386.11
    Flags:                   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt rdtscp lm constant_tsc rep_
                             good nopl nonstop_tsc cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx rdrand hypervisor lahf_lm c
                             mp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch vmmcall fsgsbase bmi1 avx2 bmi2 invpcid rdseed clflushopt arat flush_l1d
Virtualization features:     
  Hypervisor vendor:         KVM
  Virtualization type:       full
Caches (sum of all):         
  L1d:                       128 KiB (4 instances)
  L1i:                       128 KiB (4 instances)
  L2:                        4 MiB (4 instances)
  L3:                        256 MiB (4 instances)
NUMA:                        
  NUMA node(s):              1
  NUMA node0 CPU(s):         0-3
Vulnerabilities:             
  Gather data sampling:      Not affected
  Indirect target selection: Not affected
  Itlb multihit:             Not affected
  L1tf:                      Not affected
  Mds:                       Not affected
  Meltdown:                  Not affected
  Mmio stale data:           Not affected
  Reg file data sampling:    Not affected
  Retbleed:                  Not affected
  Spec rstack overflow:      Mitigation; safe RET
  Spec store bypass:         Not affected
  Spectre v1:                Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:                Mitigation; Retpolines; STIBP disabled; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
  Srbds:                     Not affected
  Tsa:                       Vulnerable: Clear CPU buffers attempted, no microcode
  Tsx async abort:           Not affected
  Vmscape:                   Not affected
[vagrant@amzn2023 ~]$ 
free -h
               total        used        free      shared  buff/cache   available
Mem:           7.8Gi       129Mi       7.3Gi       8.0Mi       313Mi       7.4Gi
Swap:             0B          0B          0B
[vagrant@amzn2023 ~]$ cat /etc/os-release 
NAME="Amazon Linux"
VERSION="2023"
ID="amzn"
ID_LIKE="fedora"
VERSION_ID="2023"
PLATFORM_ID="platform:al2023"
PRETTY_NAME="Amazon Linux 2023.10.20260105"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2023"
HOME_URL="https://aws.amazon.com/linux/amazon-linux-2023/"
DOCUMENTATION_URL="https://docs.aws.amazon.com/linux/"
SUPPORT_URL="https://aws.amazon.com/premiumsupport/"
BUG_REPORT_URL="https://github.com/amazonlinux/amazon-linux-2023"
VENDOR_NAME="AWS"
VENDOR_URL="https://aws.amazon.com/"
SUPPORT_END="2029-06-30"

INSTALL AND INIT SCRIPT

[root@amzn2023 vagrant]# yum install ./wazuh-manager-5.0.0-latest.x86_64.rpm 
Last metadata expiration check: 0:26:37 ago on Fri Mar 20 13:56:50 2026.
Dependencies resolved.
========================================================================================================================================================================================
 Package                                       Architecture                           Version                                        Repository                                    Size
========================================================================================================================================================================================
Installing:
 wazuh-manager                                 x86_64                                 5.0.0-latest                                   @commandline                                 424 M

Transaction Summary
========================================================================================================================================================================================
Install  1 Package

Total size: 424 M
Installed size: 692 M
Is this ok [y/N]: y
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                1/1 
  Running scriptlet: wazuh-manager-5.0.0-latest.x86_64                                                                                                                              1/1 
  Installing       : wazuh-manager-5.0.0-latest.x86_64                                                                                                                              1/1 
  Running scriptlet: wazuh-manager-5.0.0-latest.x86_64                                                                                                                              1/1 
  Verifying        : wazuh-manager-5.0.0-latest.x86_64                                                                                                                              1/1 
========================================================================================================================================================================================
WARNING:
  A newer release of "Amazon Linux" is available.

  Available Versions:

  Version 2023.10.20260120:
    Run the following command to upgrade to 2023.10.20260120:

      dnf upgrade --releasever=2023.10.20260120

    Release notes:
     https://docs.aws.amazon.com/linux/al2023/release-notes/relnotes-2023.10.20260120.html

  Version 2023.10.20260202:
    Run the following command to upgrade to 2023.10.20260202:

      dnf upgrade --releasever=2023.10.20260202

    Release notes:
     https://docs.aws.amazon.com/linux/al2023/release-notes/relnotes-2023.10.20260202.html

  Version 2023.10.20260216:
    Run the following command to upgrade to 2023.10.20260216:

      dnf upgrade --releasever=2023.10.20260216

    Release notes:
     https://docs.aws.amazon.com/linux/al2023/release-notes/relnotes-2023.10.20260216.html

  Version 2023.10.20260302:
    Run the following command to upgrade to 2023.10.20260302:

      dnf upgrade --releasever=2023.10.20260302

    Release notes:
     https://docs.aws.amazon.com/linux/al2023/release-notes/relnotes-2023.10.20260302.html

========================================================================================================================================================================================

Installed:
  wazuh-manager-5.0.0-latest.x86_64                                                                                                                                                     

Complete!
Installed:
  dmidecode-1:3.6-1.amzn2023.0.1.x86_64     libxcrypt-compat-4.4.33-7.amzn2023.x86_64     python3-devel-3.9.25-1.amzn2023.0.3.x86_64     python3-pip-21.3.1-2.amzn2023.0.14.noarch    

Complete!
[14:41:04] curl already available (likely curl-minimal), skipping install.
[14:41:04] System packages installed.
[14:41:04] Downloading Go 1.23.6 for linux/amd64...
[14:41:08] Extracting Go to /usr/local/go...
[14:41:10] Go go version go1.23.6 linux/amd64 installed.
[14:41:10] Installing Python dependencies...
Requirement already satisfied: pip in /usr/lib/python3.9/site-packages (21.3.1)
Collecting pip
  Downloading pip-26.0.1-py3-none-any.whl (1.8 MB)
     |████████████████████████████████| 1.8 MB 2.6 MB/s            
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 21.3.1
Collecting matplotlib
  Downloading matplotlib-3.9.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.3 MB)
     |████████████████████████████████| 8.3 MB 2.7 MB/s            
Collecting seaborn
  Downloading seaborn-0.13.2-py3-none-any.whl (294 kB)
     |████████████████████████████████| 294 kB 25.2 MB/s            
Collecting pandas
  Downloading pandas-2.3.3-cp39-cp39-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (12.8 MB)
     |████████████████████████████████| 12.8 MB 26.8 MB/s            
Collecting psutil
  Downloading psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl (155 kB)
     |████████████████████████████████| 155 kB 23.0 MB/s            
Collecting numpy
  Downloading numpy-2.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (19.5 MB)
     |████████████████████████████████| 19.5 MB 11.3 MB/s            
Collecting pillow>=8
  Downloading pillow-11.3.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.6 MB)
     |████████████████████████████████| 6.6 MB 26.4 MB/s            
Collecting importlib-resources>=3.2.0
  Downloading importlib_resources-6.5.2-py3-none-any.whl (37 kB)
Collecting contourpy>=1.0.1
  Downloading contourpy-1.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (321 kB)
     |████████████████████████████████| 321 kB 30.1 MB/s            
Collecting kiwisolver>=1.3.1
  Downloading kiwisolver-1.4.7-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
     |████████████████████████████████| 1.6 MB 30.9 MB/s            
Collecting fonttools>=4.22.0
  Downloading fonttools-4.60.2-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.8 MB)
     |████████████████████████████████| 4.8 MB 22.3 MB/s            
Collecting packaging>=20.0
  Downloading packaging-26.0-py3-none-any.whl (74 kB)
     |████████████████████████████████| 74 kB 8.9 MB/s             
Collecting cycler>=0.10
  Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB)
Requirement already satisfied: python-dateutil>=2.7 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/vagrant/wazuh/src/engine/test/acceptance_test/requirements.txt (line 3)) (2.8.1)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/vagrant/wazuh/src/engine/test/acceptance_test/requirements.txt (line 3)) (2.4.7)
Requirement already satisfied: pytz>=2020.1 in /usr/lib/python3.9/site-packages (from pandas->-r /home/vagrant/wazuh/src/engine/test/acceptance_test/requirements.txt (line 5)) (2022.7.1)
Collecting tzdata>=2022.7
  Downloading tzdata-2025.3-py2.py3-none-any.whl (348 kB)
     |████████████████████████████████| 348 kB 25.4 MB/s            
Collecting python-dateutil>=2.7
  Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
     |████████████████████████████████| 229 kB 29.9 MB/s            
Collecting zipp>=3.1.0
  Downloading zipp-3.23.0-py3-none-any.whl (10 kB)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib->-r /home/vagrant/wazuh/src/engine/test/acceptance_test/requirements.txt (line 3)) (1.15.0)
Installing collected packages: zipp, numpy, tzdata, python-dateutil, pillow, packaging, kiwisolver, importlib-resources, fonttools, cycler, contourpy, pandas, matplotlib, seaborn, psutil
  Attempting uninstall: python-dateutil
    Found existing installation: python-dateutil 2.8.1
    Uninstalling python-dateutil-2.8.1:
      Successfully uninstalled python-dateutil-2.8.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
awscli 2.32.22 requires python-dateutil<=2.9.0,>=2.1, but you have python-dateutil 2.9.0.post0 which is incompatible.
Successfully installed contourpy-1.3.0 cycler-0.12.1 fonttools-4.60.2 importlib-resources-6.5.2 kiwisolver-1.4.7 matplotlib-3.9.4 numpy-2.0.2 packaging-26.0 pandas-2.3.3 pillow-11.3.0 psutil-7.2.2 python-dateutil-2.9.0.post0 seaborn-0.13.2 tzdata-2025.3 zipp-3.23.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[14:41:23] Python dependencies installed.
[14:41:23] Pre-compiling benchmark_tool.go (downloading Go modules & caching build)...
[14:41:29] Pre-compilation done.
[14:41:29] Verifying installation...
[14:41:29]   [OK] python3: Python 3.9.25
[14:41:29]   [OK] pip: pip 21.3.1 from /usr/lib/python3.9/site-packages/pip (python 3.9)
[14:41:29]   [OK] go: go version go1.23.6 linux/amd64
[14:41:29]   [OK] curl: curl 8.15.0 (x86_64-amazon-linux-gnu) libcurl/8.15.0 OpenSSL/3.2.2 zlib/1.2.11 libidn2/2.3.2 libpsl/0.21.5 nghttp2/1.59.0
[14:41:29]   [OK] pgrep available
[14:41:29]   [OK] dmidecode available
[14:41:29]   [OK] python3 -c 'import psutil'
[14:41:29]   [OK] python3 -c 'import matplotlib'
[14:41:30]   [OK] python3 -c 'import seaborn'
[14:41:30]   [OK] python3 -c 'import pandas'
[14:41:30]   [OK] python3 -c 'import numpy'
[14:41:30] All checks passed. Ready to run acceptance_test.sh.
[14:41:30] 
[14:41:30] ============================================================
[14:41:30]   Setup complete!
[14:41:30]   You can now run: ./acceptance_test.sh
[14:41:30] ============================================================

START MANAGER

[root@amzn2023 vagrant]# systemctl restart wazuh-manager.service 
[root@amzn2023 vagrant]# tail -f /var/wazuh-manager/logs/wazuh-manager.log
2026/03/20 15:11:38 wazuh-manager-modulesd:inventory-sync: INFO: Starting inventory_sync module.
2026/03/20 15:11:38 wazuh-manager-modulesd:task-manager: INFO: (8200): Module Task Manager started.
2026/03/20 15:11:38 wazuh-manager-modulesd:download: INFO: Module started.
2026/03/20 15:11:38 wazuh-manager-modulesd:database: INFO: Module started.
2026/03/20 15:11:38 wazuh-manager-modulesd:control: INFO: Starting control thread.
2026/03/20 15:11:38 wazuh-manager-analysisd: INFO: [IOC::Sync] Changes detected for IOC type 'connection', updating...
2026/03/20 15:11:38 wazuh-manager-modulesd:vulnerability-scanner: INFO: Schema validator initialized successfully from embedded resources.
2026/03/20 15:11:39 IndexerConnector: WARNING: No username and password found in the keystore, using default values.
2026/03/20 15:11:39 logger-helper: INFO: InventorySync session limit: 1000
2026/03/20 15:11:39 logger-helper: INFO: InventorySyncFacade started.
2026/03/20 15:11:48 wazuh-manager-monitord: INFO: wazuh: Manager started.
2026/03/20 15:11:48 wazuh-manager-analysisd: INFO: [IOC::Sync] Synchronized IOC type 'connection'
2026/03/20 15:11:48 wazuh-manager-analysisd: INFO: [IOC::Sync] Changes detected for IOC type 'url_full', updating...
2026/03/20 15:11:51 wazuh-manager-analysisd: INFO: [IOC::Sync] Synchronized IOC type 'url_full'
2026/03/20 15:11:51 wazuh-manager-analysisd: INFO: [IOC::Sync] Changes detected for IOC type 'url_domain', updating...
2026/03/20 15:11:54 wazuh-manager-analysisd: INFO: [IOC::Sync] Synchronized IOC type 'url_domain'
2026/03/20 15:11:54 wazuh-manager-analysisd: INFO: [IOC::Sync] Changes detected for IOC type 'hash_md5', updating...
2026/03/20 15:11:54 wazuh-manager-analysisd: INFO: [IOC::Sync] Synchronized IOC type 'hash_md5'
2026/03/20 15:11:54 wazuh-manager-analysisd: INFO: [IOC::Sync] Changes detected for IOC type 'hash_sha1', updating...
2026/03/20 15:11:55 wazuh-manager-analysisd: INFO: [IOC::Sync] Synchronized IOC type 'hash_sha1'
2026/03/20 15:11:55 wazuh-manager-analysisd: INFO: [IOC::Sync] Changes detected for IOC type 'hash_sha256', updating...
2026/03/20 15:11:55 wazuh-manager-analysisd: INFO: [IOC::Sync] Synchronized IOC type 'hash_sha256'

CHECK ROUTE

[root@amzn2023 vagrant]# url --unix-socket /var/wazuh-manager/queue/sockets/analysis             -X POST -H "Content-Type: application/json"             -d '{"name":"cmsync_standard"}'             http://localhost/router/route/get
{"status":"OK","route":{"name":"cmsync_standard","namespaceId":"cmsync_standard_7903","priority":1,"entry_status":"ENABLED","uptime":75}}[root@amzn2023 vagrant]#

RUN ANALYSIS

[15:39:27] Monitor stopped.
[15:39:27] Stopping analysisd...
wazuh-manager-clusterd not running...
wazuh-manager-modulesd not running...
wazuh-manager-monitord not running...
wazuh-manager-remoted not running...
Killing wazuh-manager-analysisd...
wazuh-manager-db not running...
wazuh-manager-authd not running...
wazuh-manager-apid not running...
Wazuh v5.0.0 Stopped
[15:39:28] analysisd stopped.
[15:39:28] Test with 4 thread(s) complete.
[15:39:28]   Monitor CSV: /home/vagrant/wazuh/src/engine/test/acceptance_test/results/monitor-4T.csv
[15:39:28]   Bench CSV:   /home/vagrant/wazuh/src/engine/test/acceptance_test/results/bench-4T.csv
[15:39:28] 
[15:39:28] ============================================================
[15:39:28]   All tests complete. Results in: /home/vagrant/wazuh/src/engine/test/acceptance_test/results/
[15:39:28] ============================================================
total 112K
-rw-r--r--. 1 root root 4.3K Mar 20 15:36 bench-4T-warmup.csv
-rw-r--r--. 1 root root  29K Mar 20 15:36 bench-4T-warmup.log
-rw-r--r--. 1 root root 4.2K Mar 20 15:39 bench-4T.csv
-rw-r--r--. 1 root root  29K Mar 20 15:39 bench-4T.log
-rw-r--r--. 1 root root  28K Mar 20 15:39 monitor-4T.csv
-rw-r--r--. 1 root root  963 Mar 20 15:34 system_report.txt
[15:39:28] Cleanup triggered (exit code: 0)...

Results

matigarciadev

LGTM!

System Report:

==========================================
  System Report – Benchmark Environment
==========================================

Date:           2026-03-20 15:20:48 UTC

--- Linux ---
Kernel:         6.17.0-19-generic
OS:             Ubuntu 22.04.5 LTS
Architecture:   x86_64

--- CPU ---
Model:          Intel(R) Core(TM) Ultra 7 155H
Cores:          22 (logical)
Sockets:        1
Cores per sock: 16
Threads per core: 2
Max MHz:        4800.0000
Current MHz:    
CPU cache:      24 MiB (1 instance)

--- RAM ---
Total:          46Gi
Available:      29Gi
Swap:           4.0Gi
RAM Speed:      N/A (dmidecode not available or not root)
RAM Type:       N/A

--- Test Parameters ---
Threads tested: 1 2 4 8
Bench duration: 60s
Target rate:    0 EPS (0=unlimited)
Batch size:     100
Grace period:   20s
Monitor interval: 1s
Input dir:      /workspaces/wazuh-5.x/test_logs/
Output watched: /var/wazuh-manager/logs/alerts/alerts.json
Route:          cmsync_standard
==========================================

juliancnn self-assigned this Mar 12, 2026

juliancnn requested review from a team as code owners March 12, 2026 22:32

juliancnn force-pushed the enhancement/engine-benchmark-tools branch 2 times, most recently from d86d970 to 6f35b36 Compare March 16, 2026 20:32

juliancnn added 15 commits March 19, 2026 15:44

feat: Delete old benchmark files

3cc1398

feat: engine benchmark base tool

0a96152

feat: Added cvs output to engine benchmark tool

75f0010

feat: Added a monitor process for benchmarks

3e003be

feat: Implement the acceptance_test for automatic benchmarks

c825f69

feat: Addeded graphs generator

511962a

feat: Added lock rocksdb free, traps and hardware stats

a858bc8

feat: Update accetance_test and readme documentacion

ab0cbe6

feat: Improve line counter

eb6099c

feat: Added time between bench

b6783e9

feat: Added grace time to benchmarks

2314cd2

feat: Added setup deps to benchmark

e7091d0

feat: Improve engine benchmark setup for amz 2023

fbec0ca

feat: Adapat setup dependencies for ubuntu

f26576b

feat: Support venv to isolated test

00b6ff2

juliancnn force-pushed the enhancement/engine-benchmark-tools branch from 6747c30 to 00b6ff2 Compare March 19, 2026 15:45

change: optimize fastqueue traits

8048fc8

LucioDonda approved these changes Mar 20, 2026

View reviewed changes

NahuFigueroa97 approved these changes Mar 20, 2026

View reviewed changes

matigarciadev approved these changes Mar 20, 2026

View reviewed changes

juliancnn merged commit 79d8dd5 into main Mar 20, 2026
68 of 69 checks passed

juliancnn deleted the enhancement/engine-benchmark-tools branch March 20, 2026 19:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Engine benchmark tools#34918

Engine benchmark tools#34918
juliancnn merged 16 commits intomainfrom
enhancement/engine-benchmark-tools

juliancnn commented Mar 12, 2026

Uh oh!

LucioDonda left a comment

Uh oh!

NahuFigueroa97 left a comment •

edited

Loading

Uh oh!

matigarciadev left a comment •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

juliancnn commented Mar 12, 2026

Description

Proposed Changes

Results and Evidence

Manual tests with their corresponding evidence

Artifacts Affected

Configuration Changes

Tests Introduced

Review Checklist

Uh oh!

LucioDonda left a comment

Choose a reason for hiding this comment

LGTM

Uh oh!

NahuFigueroa97 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

iNFRA

INSTALL AND INIT SCRIPT

START MANAGER

CHECK ROUTE

RUN ANALYSIS

Uh oh!

matigarciadev left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

LGTM!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

NahuFigueroa97 left a comment •

edited

Loading

matigarciadev left a comment •

edited

Loading