Skip to content

multiprocessing connection error #41

New issue
New issue
Open
Open
multiprocessing connection error#41
@YuanfengZhang

Description

@YuanfengZhang
YuanfengZhang
opened on Aug 7, 2025

CMD:

# cd to snakemake dir
cd /data/nanopore

THREADS="8"
EQA_ID="202401"
SAMPLE_ID="NP1_HF_1"

conda activate deepmod2
time python resources/DeepMod2/deepmod2 detect \
  --bam nanopore/bam_with_moves/${SAMPLE_ID}.bam \
  --input uploaded/20250719/${EQA_ID}_pod5_pass \
  --model transformer_r10.4.1_5khz_v5.0 \
  --file_type pod5 --seq_type dna --threads ${THREADS} \
  --ref ../custom_genomes/BL.fa \
  --output nanopore/deepmod2/ \
  --prefix {SAMPLE_ID}

ERROR:

2025-08-06 21:34:25.572372: Starting DeepMod2.
2025-08-06 21:34:25.860719: 
Command: python resources/DeepMod2/deepmod2 detect --bam nanopore/bam_with_moves/NP1_HF_1.bam --input uploaded/20250719/202401_pod5_pass --model transformer_r10.4.1_5khz_v5.0 --file_type pod5 --seq_type dna --threads 8 --ref ../custom_genomes/BL.fa --output nanopore/deepmod2/ --prefix {SAMPLE_ID}
/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/site-packages/ont_fast5_api/compression_settings.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
2025-08-06 21:34:28.674240: Starting Per Read Methylation Detection.
2025-08-06 21:34:30.391273: Getting motif positions from the reference.
2025-08-06 21:35:13.783261: Finished getting motif positions from the reference.
/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/site-packages/ont_fast5_api/compression_settings.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
2025-08-06 21:35:15.518431: Building BAM index.
/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/site-packages/ont_fast5_api/compression_settings.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/site-packages/ont_fast5_api/compression_settings.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
CUDA test: True True
/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/site-packages/ont_fast5_api/compression_settings.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
CUDA test: True True
/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/site-packages/ont_fast5_api/compression_settings.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
CUDA test: True True
/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/site-packages/ont_fast5_api/compression_settings.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
CUDA test: True True
/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/site-packages/ont_fast5_api/compression_settings.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
CUDA test: True True
/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/site-packages/ont_fast5_api/compression_settings.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
CUDA test: True True
/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/site-packages/ont_fast5_api/compression_settings.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
CUDA test: True True
2025-08-06 21:43:58.631146: Finished building BAM index.
2025-08-06 21:49:07.353975: Number of reads processed: 10090
2025-08-06 21:54:01.712547: Number of reads processed: 20137
2025-08-06 22:00:15.612669: Number of reads processed: 30036
2025-08-06 22:06:56.162713: Number of reads processed: 40085
2025-08-06 22:11:58.202230: Number of reads processed: 50043
2025-08-06 22:19:16.906254: Number of reads processed: 60014
2025-08-06 22:25:41.123003: Number of reads processed: 70012
2025-08-06 22:31:10.760211: Number of reads processed: 80005
2025-08-06 22:36:31.096983: Number of reads processed: 90143
2025-08-06 22:42:14.753482: Number of reads processed: 100050
2025-08-06 22:48:03.413977: Number of reads processed: 110125
2025-08-06 22:53:57.734186: Number of reads processed: 120165
2025-08-06 22:58:59.334226: Number of reads processed: 130184
2025-08-06 23:04:29.837810: Number of reads processed: 140085
2025-08-06 23:11:12.808394: Number of reads processed: 150052
2025-08-06 23:17:50.178974: Number of reads processed: 160079
2025-08-06 23:23:22.460501: Number of reads processed: 170054
2025-08-06 23:28:40.029061: Number of reads processed: 180003
Process Process-14:
Process Process-12:
Process Process-13:
Process Process-18:
Process Process-15:
Process Process-16:
Process Process-17:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zyf/eqa/resources/DeepMod2/src/detect.py", line 508, in process
    chunk=signal_Q.get(block=False, timeout=10)
  File "<string>", line 2, in get
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 831, in _callmethod
    kind, result = conn.recv()
                   ~~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 430, in _recv_bytes
    buf = self._recv(4)
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 395, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zyf/eqa/resources/DeepMod2/src/detect.py", line 508, in process
    chunk=signal_Q.get(block=False, timeout=10)
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "<string>", line 2, in get
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 831, in _callmethod
    kind, result = conn.recv()
                   ~~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 430, in _recv_bytes
    buf = self._recv(4)
  File "/home/zyf/eqa/resources/DeepMod2/src/detect.py", line 503, in process
    if (signal_Q.empty() and input_event.is_set()):
        ~~~~~~~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 399, in _recv
    raise EOFError
  File "<string>", line 2, in empty
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 831, in _callmethod
    kind, result = conn.recv()
                   ~~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 430, in _recv_bytes
    buf = self._recv(4)
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 395, in _recv
    chunk = read(handle, remaining)
EOFError
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zyf/eqa/resources/DeepMod2/src/detect.py", line 503, in process
    if (signal_Q.empty() and input_event.is_set()):
        ~~~~~~~~~~~~~~^^
  File "<string>", line 2, in empty
ConnectionResetError: [Errno 104] Connection reset by peer
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 831, in _callmethod
    kind, result = conn.recv()
                   ~~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 430, in _recv_bytes
    buf = self._recv(4)
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 395, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zyf/eqa/resources/DeepMod2/src/detect.py", line 503, in process
    if (signal_Q.empty() and input_event.is_set()):
        ~~~~~~~~~~~~~~^^
  File "<string>", line 2, in empty
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 831, in _callmethod
    kind, result = conn.recv()
                   ~~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 430, in _recv_bytes
    buf = self._recv(4)
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 399, in _recv
    raise EOFError
EOFError
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zyf/eqa/resources/DeepMod2/src/detect.py", line 503, in process
    if (signal_Q.empty() and input_event.is_set()):
                             ~~~~~~~~~~~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 1103, in is_set
    return self._callmethod('is_set')
           ~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 831, in _callmethod
    kind, result = conn.recv()
                   ~~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 430, in _recv_bytes
    buf = self._recv(4)
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 399, in _recv
    raise EOFError
EOFError
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zyf/eqa/resources/DeepMod2/src/detect.py", line 503, in process
    if (signal_Q.empty() and input_event.is_set()):
        ~~~~~~~~~~~~~~^^
  File "<string>", line 2, in empty
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 831, in _callmethod
    kind, result = conn.recv()
                   ~~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 430, in _recv_bytes
    buf = self._recv(4)
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 395, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer
Process Process-11:
Traceback (most recent call last):
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zyf/eqa/resources/DeepMod2/src/detect.py", line 195, in get_cpg_output
    if methylation_event.is_set() and output_Q.empty():
       ~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 1103, in is_set
    return self._callmethod('is_set')
           ~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 831, in _callmethod
    kind, result = conn.recv()
                   ~~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 430, in _recv_bytes
    buf = self._recv(4)
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 399, in _recv
    raise EOFError
EOFError
Process Process-10:
Traceback (most recent call last):
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zyf/eqa/resources/DeepMod2/src/detect.py", line 766, in get_input
    signal_Q.put(chunk)
    ~~~~~~~~~~~~^^^^^^^
  File "<string>", line 2, in put
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 831, in _callmethod
    kind, result = conn.recv()
                   ~~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 430, in _recv_bytes
    buf = self._recv(4)
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 399, in _recv
    raise EOFError
EOFError
2025-08-06 23:33:10.868783: Reading inputs complete.
Traceback (most recent call last):
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 823, in _callmethod
    conn = self._tls.connection
           ^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/zyf/eqa/resources/DeepMod2/deepmod2", line 172, in <module>
    detect.call_manager(params)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/home/zyf/eqa/resources/DeepMod2/src/detect.py", line 864, in call_manager
    methylation_event.set()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 1105, in set
    return self._callmethod('set')
           ~~~~~~~~~~~~~~~~^^^^^^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 827, in _callmethod
    self._connect()
    ~~~~~~~~~~~~~^^
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/managers.py", line 814, in _connect
    conn = self._Client(self._token.address, authkey=self._authkey)
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 519, in Client
    c = SocketClient(address)
  File "/home/zyf/miniforge3/envs/deepmod2/lib/python3.13/multiprocessing/connection.py", line 647, in SocketClient
    s.connect(address)
    ~~~~~~~~~^^^^^^^^^
ConnectionRefusedError: [Errno 111] Connection refused

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions