Merge pull request #239 from ReproNim/enh-qr-parse-optimization

Optimize `qr_parse` processing

Author: vmdocua

.ai/context.md

@@ -32,7 +32,7 @@ Main Python package with CLI tools and analysis utilities.
 **Key Modules:**
 - **cli/** - Command-line interface (Click-based with DYMGroup for suggestions)
   - `entrypoint.py` - Main CLI dispatcher
-  - `cmd_qr_parse.py` - Parse QR codes from `.mkv` videos (PARSE/INFO modes)
+  - `cmd_qr_parse.py` - Parse QR codes from `.mkv` videos (PARSE/INFO modes) (see [spec-qr-parse.md](.ai/spec-qr-parse.md))
   - `cmd_timesync_stimuli.py` - PsychoPy integration for QR/audio code generation
   - `cmd_detect_noscreen.py` - Detect no-signal/rainbow frames with fixup capabilities
   - `cmd_list_displays.py` - List available GUI displays (cross-platform)
@@ -43,7 +43,7 @@ Main Python package with CLI tools and analysis utilities.
   - `cmd_echo.py` - Simple echo command for testing
 
 - **qr/** - QR code processing and time synchronization
-  - `qr_parse.py` - Parse `.mkv` files, extract QR codes, audio codes, metadata → JSONL
+  - `qr_parse.py` - Parse `.mkv` files, extract QR codes, audio codes, metadata → JSONL (see [spec-qr-parse.md](.ai/spec-qr-parse.md))
   - `timesync_stimuli.py` - PsychoPy-based MRI/BIRCH/Magewell synchronization
   - `disp_mon.py` - Cross-platform display monitoring (Linux/macOS/Windows)
   - `psychopy.py` - PsychoPy framework integration utilities

.ai/spec-qr-parse.md

@@ -0,0 +1,106 @@
+# `qr-parse` Task List
+
+Tracks implementation progress against [spec-qr-parse.md](spec-qr-parse.md).
+
+---
+
+## CLI Options
+
+- [x] `PATH` argument — path to video file or directory
+- [x] `-m / --mode [PARSE|INFO]` — execution mode
+- [x] `-g / --grayscale [none|numpy|opencv]` — frame grayscale conversion method; default `cvtcolor`
+- [x] `-t / --std-threshold FLOAT` — grayscale std-deviation pre-filter; skip decode when std < threshold; disabled when ≤ 0; default `10.0`
+- [x] `-x / --scale FLOAT` — frame downscale factor `(0, 1]`; `1.0` = no resize; default `1.0`
+- [x] `-s / --skip INT` — frames to skip after each processed frame; `0` = every frame; default `0`
+- [x] `-q / --qr-decoder [none|opencv|pyzbar]` — QR backend; `none` skips decode; default `pyzbar`
+- [x] `-v / --video-decoder [opencv]` — video frame backend; only `opencv` supported now; placeholder for `ffmpeg`/`pyav`; default `opencv`
+- [x] `-Q / --qrdet` — enable qrdet-based frame pre-filter; default `False`
+- [x] `-M / --qrdet-model-size [n|s|m|l]` — qrdet model size; default `s`; only used when `--qrdet` is set
+- [x] `-W / --qr-decoder-workers INT` — worker threads for parallel QR decoding; `0`/`1` = sequential (streaming); `N > 1` = parallel (buffered, iframe-ordered)
+
+---
+
+## Core Logic
+
+### PARSE mode
+- [x] Read frames via `cv2.VideoCapture`
+- [x] Grayscale conversion (`np.mean` — current; candidate for `cv2.cvtColor` optimisation)
+- [x] QR detection via `pyzbar.decode`
+- [x] Deduplicate consecutive identical QR codes
+- [x] Output JSONL (`ParseSummary` + per-code records) to stdout
+- [x] Std-deviation pre-filter: compute grayscale std before decode, skip frame if below `--std-threshold`
+- [x] Replace `np.mean` grayscale with `cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)` (×10 speedup)
+- [x] Replace `np.std` with `cv2.meanStdDev` on grayscale frame (faster, same result)
+
+### INFO mode
+- [x] Enumerate `.mkv` files in directory (or single file)
+- [x] Output `InfoSummary` JSONL (path, duration, size, rate)
+
+---
+
+## Tests
+
+### CLI option handling
+- [x] `--grayscale none` — verify raw BGR frame is passed through without conversion
+- [x] `--grayscale numpy` — verify `np.mean(frame, axis=2)` path is taken
+- [x] `--grayscale opencv` — verify `cv2.cvtColor` path is taken (default)
+- [x] `--std-threshold 0` — verify pre-filter is disabled, all frames reach decoder
+- [x] `--std-threshold 40` — verify frames below threshold are skipped
+- [x] `--scale 0.5` — verify frames are resized before decode
+- [x] `--scale 1.0` — verify no resize is applied (default, no-op)
+- [x] `--skip 0` — verify every frame is processed
+- [x] `--skip 2` — verify only every 3rd frame is processed
+- [x] `--qr-decoder none` — verify decode is skipped, output still produced
+- [x] `--qr-decoder opencv` — verify `cv2.QRCodeDetector.detectAndDecode` is used
+- [x] `--qr-decoder pyzbar` — verify `pyzbar.decode` is used (default)
+- [x] `--video-decoder opencv` — verify `cv2.VideoCapture` is used (default)
+- [x] `--qrdet` — verify qrdet pre-filter is activated (mocked, no GPU needed)
+- [x] `--qrdet-model-size n/s/m/l` — verify correct model variant is loaded (mocked)
+- [x] `--qr-decoder-workers 0` — verify sequential path is taken (ThreadPoolExecutor not instantiated)
+- [x] `--qr-decoder-workers 1` — verify sequential path is taken (threshold is `> 1`)
+- [x] `--qr-decoder-workers 4` — verify ThreadPoolExecutor instantiated with `max_workers=4`
+- [x] `--qr-decoder-workers 4` — verify `_process_frame` called once per non-skipped frame
+- [x] `--qr-decoder-workers 4` — verify output records match sequential path (data, frame_start, time_start)
+
+### Integration
+- [ ] Combined `--grayscale cvtcolor --std-threshold 40 --skip 1` — verify all three interact correctly on a real video
+- [ ] `--qrdet --qr-decoder pyzbar` — verify qrdet pre-filter feeds into pyzbar decode
+- [ ] `--qr-decoder none --std-threshold 40` — verify std filter still runs even when decode is disabled
+- [ ] QR codes are detected and match expected output on reference test video
+
+### Coverage
+- [x] `qr_parse.py` ≥ 80% — achieved **93%**
+- [x] `cmd_qr_parse.py` ≥ 80% — achieved **100%**
+- [x] `get_video_time_info` edge cases: invalid filename, start-only filename, start ≥ end
+- [x] `_decode_qr_pyzbar` / `_decode_qr_opencv` found-code paths
+- [x] `_qr_state_machine` — two different QR codes in sequence; QR code at end of video
+- [x] `do_parse` — `summary_only=True`; `ignore_errors=True`; `cap.isOpened()=False`
+- [x] `do_info` / `do_info_file` — file, directory, invalid path
+- [x] `do_main` — path not found, invalid scale, invalid skip, INFO mode, unknown mode, PARSE mode success
+- [x] CLI (`cmd_qr_parse`) — basic invocation, `--mode INFO`, option forwarding, invalid path
+
+### Regression
+- [ ] Existing PARSE mode output unchanged when all new options are at their defaults
+- [ ] Existing INFO mode output unchanged
+
+---
+
+## Performance / Optimisation (from spec benchmarks)
+
+- [x] `cv2.cvtColor` grayscale (proposal 1) — 23.7 → 46.1 fps
+- [x] `cv2.meanStdDev` std deviation (proposal 2)
+- [x] Std pre-filter with `--std-threshold` (proposal 3)
+- [x] Optional frame downscaling `-x / --scale` (proposal 4)
+- [x] Parallel decoding via `ThreadPoolExecutor` + `-W / --qr-decoder-workers` (proposal 5)
+- [ ] GPU / ZXing decoder (proposal 6)
+
+---
+
+## Future: Video Decoder Backends
+
+Extend `-v / --video-decoder` with additional backends once `opencv` path is stable.
+
+- [ ] `ffmpeg` — drive frame extraction via `ffmpeg` subprocess or `ffmpeg-python` bindings; useful for formats/codecs OpenCV cannot handle
+- [ ] `pyav` — use `av` (PyAV) bindings for libavcodec/libavformat; lower overhead than subprocess, supports hardware-accelerated decode (VAAPI, NVDEC)
+- [ ] ? `decord` — GPU-accelerated video reader (`decord` package); designed for ML workloads, supports batch frame reads and CUDA tensors
+- [ ] Abstract `_open_video(ctx) -> iterator[frame]` API in `qr_parse.py` so backends are swappable without touching the main frame loop

.ai/task-qr-parse.md

@@ -27,12 +27,12 @@ PSYCHOPY_VENV_BIN=${PSYCHOPY_HOME}/.venv/bin
 
 # Install psychopy_linux_installer from GitHub
 echo "Install psychopy_linux_installer from GitHub..."
-git clone --branch v2.2.5 --depth 1 https://github.com/wieluk/psychopy_linux_installer/ /opt/psychopy-installer
+git clone --branch v2.2.7 --depth 1 https://github.com/wieluk/psychopy_linux_installer/ /opt/psychopy-installer
 cd /opt/psychopy-installer
 
 # Install PsychoPy via psychopy_linux_installer
 echo "Install PsychoPy v${PSYCHOPY_VERSION} via psychopy_linux_installer..."
-/opt/psychopy-installer/psychopy_linux_installer --install-dir=${PSYCHOPY_INSTALL_DIR} --venv-name=${PSYCHOPY_VENV_NAME} --psychopy-version=${PSYCHOPY_VERSION} --additional-packages=psychopy_bids==2025.1.2,psychopy-mri-emulator==0.0.2,con-duct==0.18.0 --python-version=${PYTHON_VERSION} --wxpython-version=4.2.3 --cleanup -v -f
+/opt/psychopy-installer/psychopy_linux_installer --install-dir=${PSYCHOPY_INSTALL_DIR} --venv-name=${PSYCHOPY_VENV_NAME} --psychopy-version=${PSYCHOPY_VERSION} --additional-packages=psychopy_bids==2025.1.2,psychopy-mri-emulator==0.0.2,con-duct==0.18.0 --python-version=${PYTHON_VERSION} --wxpython-version=4.2.3 --cleanup --log-level=debug -f
 # Create symlink to psychopy executable
 ln -sf "${PSYCHOPY_HOME}/start_psychopy" /usr/local/bin/psychopy
 

containers/repronim-reprostim/setup_container.sh

@@ -76,6 +76,13 @@ psychopy = [
   "psychopy",
   "psychopy-sounddevice",
 ]
+# GPU-accelerated QR pre-filter via qrdet (YOLOv8) + PyTorch
+# Note: install the CUDA-enabled torch variant manually if a GPU is available:
+#   pip install torch --index-url https://download.pytorch.org/whl/cu121
+gpu = [
+  "torch>=2.0.0",
+  "qrdet>=2.2",
+]
 all = [
   "reprostim[audio]",
   "reprostim[disp_mon]",

pyproject.toml

@@ -3,7 +3,7 @@
 # SPDX-License-Identifier: MIT
 
 import logging
-import os
+import time
 
 import click
 
@@ -18,36 +18,146 @@
 )
 @click.argument("path", type=click.Path(exists=True))
 @click.option(
+    "-m",
     "--mode",
     default="PARSE",
     type=click.Choice(["PARSE", "INFO"]),
+    show_default=True,
     help="Specify execution mode. Default is `PARSE`, "
     "normal execution. "
     "Use `INFO` to dump video file info like duration, "
     "bitrate, file size etc, (in this case "
     "`PATH` argument specifies video file or directory "
     "containing video files).",
 )
+@click.option(
+    "-g",
+    "--grayscale",
+    default="opencv",
+    type=click.Choice(["none", "numpy", "opencv"]),
+    show_default=True,
+    help="Grayscale conversion method applied to each frame before QR decoding. "
+    "`opencv` uses cv2.cvtColor (fast, recommended). "
+    "`numpy` uses np.mean (slow, legacy). "
+    "`none` passes raw frame as is — may cause errors with some decoders.",
+)
+@click.option(
+    "-x",
+    "--scale",
+    default=1.0,
+    type=click.FloatRange(min=0.0, min_open=True, max=1.0),
+    show_default=True,
+    help="Frame downscale factor in (0, 1]. At 0.5 frame area is reduced to 25%, "
+    "cutting decode cost. At 1.0 (default) no resize is applied.",
+)
+@click.option(
+    "-s",
+    "--skip",
+    default=0,
+    type=click.IntRange(min=0),
+    show_default=True,
+    help="Number of frames to skip after each processed frame. "
+    "0 = process every frame. 1 = process 1 of 2. 2 = process 1 of 3, etc.",
+)
+@click.option(
+    "-t",
+    "--std-threshold",
+    default=10.0,
+    type=float,
+    show_default=True,
+    help="Grayscale std-deviation pre-filter threshold. Frames with std dev below "
+    "this value are skipped before QR decode. Set to 0 or less to disable.",
+)
+@click.option(
+    "-q",
+    "--qr-decoder",
+    default="pyzbar",
+    type=click.Choice(["none", "opencv", "pyzbar"]),
+    show_default=True,
+    help="QR decoding backend. "
+    "`pyzbar` uses pyzbar.decode (default). "
+    "`opencv` uses cv2.QRCodeDetector.detectAndDecode. "
+    "`none` disables QR decoding entirely — useful for benchmarking.",
+)
+@click.option(
+    "-v",
+    "--video-decoder",
+    default="opencv",
+    type=click.Choice(["opencv"]),
+    show_default=True,
+    help="Video frame decoding backend. "
+    "Currently only `opencv` (cv2.VideoCapture) is supported. "
+    "Placeholder for future backends such as `ffmpeg` or `pyav`.",
+)
+@click.option(
+    "-Q",
+    "--qrdet",
+    is_flag=True,
+    default=False,
+    show_default=True,
+    help="Enable qrdet-based GPU frame pre-filter. "
+    "A YOLOv8 QR detector runs on each frame before the full QR decode; "
+    "frames with no detected QR region are skipped. "
+    "Requires `qrdet` and `torch` packages (pip install reprostim[gpu]).",
+)
+@click.option(
+    "-M",
+    "--qrdet-model-size",
+    default="s",
+    type=click.Choice(["n", "s", "m", "l"]),
+    show_default=True,
+    help="qrdet model size. "
+    "`n` (nano) is fastest; `s` (small) balances speed/accuracy (default); "
+    "`m` and `l` give higher accuracy at greater cost. "
+    "Only used when --qrdet is set.",
+)
+@click.option(
+    "-W",
+    "--qr-decoder-workers",
+    default=0,
+    type=click.IntRange(min=0),
+    show_default=True,
+    help="Number of worker threads for parallel QR decoding. "
+    "0 or 1 = sequential (default). N > 1 = parallel with N threads.",
+)
 @click.pass_context
-def qr_parse(ctx, path: str, mode: str):
+def qr_parse(
+    ctx,
+    path: str,
+    mode: str,
+    grayscale: str,
+    scale: float,
+    skip: int,
+    std_threshold: float,
+    qr_decoder: str,
+    video_decoder: str,
+    qrdet: bool,
+    qrdet_model_size: str,
+    qr_decoder_workers: int,
+):
     """Parse QR codes in captured videos."""
 
-    from ..qr.qr_parse import do_info, do_parse
+    from ..qr.qr_parse import do_main
 
     logger.debug("qr_parse(...)")
-    logger.debug(f"Working dir      : {os.getcwd()}")
-    logger.info(f"Video full path  : {path}")
-
-    if not os.path.exists(path):
-        logger.error(f"Path does not exist: {path}")
-        return 1
-
-    if mode == "PARSE":
-        for item in do_parse(path):
-            print(item.model_dump_json())
-    elif mode == "INFO":
-        for item in do_info(path):
-            print(item.model_dump_json())
-    else:
-        logger.error(f"Unknown mode: {mode}")
-    return 0
+
+    start_time_sec = time.time()
+
+    res = do_main(
+        path=path,
+        mode=mode,
+        grayscale=grayscale,
+        scale=scale,
+        skip=skip,
+        std_threshold=std_threshold,
+        qr_decoder=qr_decoder,
+        video_decoder=video_decoder,
+        qrdet=qrdet,
+        qrdet_model_size=qrdet_model_size,
+        qr_decoder_workers=qr_decoder_workers,
+        out_func=click.echo,
+    )
+
+    elapsed_sec = round(time.time() - start_time_sec, 1)
+    logger.debug(f"Command 'qr-parse' completed in {elapsed_sec} sec, exit code {res}")
+    return res

src/reprostim/cli/cmd_qr_parse.py

@@ -56,9 +56,8 @@ def main(ctx, log_level: str, log_format):
     logger.debug(f"main(...), command={ctx.invoked_subcommand}")
 
 
-from .cmd_bids_inject import bids_inject  # noqa: E402
-
 # Import all CLI commands
+from .cmd_bids_inject import bids_inject  # noqa: E402
 from .cmd_detect_noscreen import detect_noscreen  # noqa: E402
 from .cmd_echo import echo  # noqa: E402
 from .cmd_list_displays import list_displays  # noqa: E402