Skip to content

Problems with requesting youtube channel page #32

@rumca-js

Description

@rumca-js

Example code

    import httpmorph
    response =  httpmorph.get("https://www.youtube.com/channel/UCXuqSBlHAE6Xw-yeJA0Tunw", timeout=30)
    print(reponse.status_code)
Traceback (most recent call last):
  File "/home/hunter/WorkDir/git/crawler-buddy/manual_test.py", line 207, in <module>
    main()
  File "/home/hunter/WorkDir/git/crawler-buddy/manual_test.py", line 205, in main
    test_urls()
  File "/home/hunter/WorkDir/git/crawler-buddy/manual_test.py", line 198, in test_urls
    response =  httpmorph.get("https://www.youtube.com/channel/UCXuqSBlHAE6Xw-yeJA0Tunw", timeout=30)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hunter/.cache/pypoetry/virtualenvs/crawler-buddy-sKUXuEBu-py3.12/lib/python3.12/site-packages/httpmorph/_client_c.py", line 959, in get
    return get_default_session().get(url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hunter/.cache/pypoetry/virtualenvs/crawler-buddy-sKUXuEBu-py3.12/lib/python3.12/site-packages/httpmorph/_client_c.py", line 910, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hunter/.cache/pypoetry/virtualenvs/crawler-buddy-sKUXuEBu-py3.12/lib/python3.12/site-packages/httpmorph/_client_c.py", line 825, in request
    raise Timeout(error_msg)
httpmorph._client_c.Timeout: Request timed out

Proposition

  • introduce manual tests to your project
  • test pages like youtube.com, x.com, reddit.com, etc

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions