Fixed entire application. Should work as intended now. #612

leaskeg · 2024-07-06T23:59:22Z

Summary

This pull request addresses an issues where the repo is not working anymore. Tested locally and it works for me as it did before.

Changes

Updated the version number
Updated download_driver to support the new version of undetected_chromedriver
Converts major_version to an int so it doesnt cause an issue
ChromeOptions updated
Removed the loggingPrefs capability from the WebDriver initialization since it was causing an issue with the new version.
The loggingPrefs capability was causing an invalid argument: unrecognized capability error. Removing this capability resolves the issue and allows multiple workers to initialize correctly.
Updated Requirements.txt

Testing

Tested locally by running the script with multiple workers to ensure that the error no longer occurs and that the script functions as expected.

I think thats pretty much it. Works like before - tested with proxies from webshare

Please review and merge this to fix the application

… loggingPrefs and improved error handling. Working as of 07-07-2024 by Leaske

Should be its original values

should be its orignal value, this was used for testing

Removed proxies, was used for testing

leaskeg

@MShawon

AminDhouib · 2024-07-07T00:12:17Z

Appreciate the effort you put there, @leaskeg, though it seems like there are still some issues when I run it. It attempts to get the chrome driver automatically but fails to do so. I attached a screenshot of the error, including the written text.

PS C:\Users\amind\OneDrive\Desktop\Projects\YouTube-Viewer> py -3.10 .\youtube_viewer.py

Yb  dP  dP"Yb  88   88 888888 88   88 88""Yb 888888
 YbdP  dP   Yb 88   88   88   88   88 88__dP 88__
  8P   Yb   dP Y8   8P   88   Y8   8P 88""Yb 88""
 dP     YbodP  `YbodP'   88   `YbodP' 88oodP 888888

                        Yb    dP 88 888888 Yb        dP 888888 88""Yb
                         Yb  dP  88 88__    Yb  db  dP  88__   88__dP
                          YbdP   88 88""     YbdPYbdP   88""   88"Yb
                           YP    88 888888    YP  YP    888888 88  Yb


           [ GitHub : https://github.com/MShawon/YouTube-Viewer ]


+-------------------------- Version: 1.8.0 --------------------------+

Getting Chrome Driver...
Traceback (most recent call last):
  File "C:\Users\amind\OneDrive\Desktop\Projects\YouTube-Viewer\youtube_viewer.py", line 1003, in <module>
    osname, exe_name = download_driver(patched_drivers=patched_drivers)
  File "C:\Users\amind\OneDrive\Desktop\Projects\YouTube-Viewer\youtubeviewer\download_driver.py", line 112, in download_driver
    uc.install()
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 262, in install
    ChromeDriverManager(executable_path, target_version, *args, **kwargs).install()
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 192, in install
    self.fetch_chromedriver()
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 221, in fetch_chromedriver
    ver = self.get_release_version_number().vstring
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 211, in get_release_version_number
    return LooseVersion(urlopen(self.__class__.DL_BASE + path).read().decode())
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 525, in open
    response = meth(req, response)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 634, in http_response
    response = self.parent.error(
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 563, in error
    return self._call_chain(*args)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 496, in _call_chain
    result = func(*args)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

leaskeg · 2024-07-07T00:17:30Z

Appreciate the effort you put there, @leaskeg, though it seems like there are still some issues when I run it. It attempts to get the chrome driver automatically but fails to do so. I attached a screenshot of the error, including the written text.

PS C:\Users\amind\OneDrive\Desktop\Projects\YouTube-Viewer> py -3.10 .\youtube_viewer.py

Yb  dP  dP"Yb  88   88 888888 88   88 88""Yb 888888
 YbdP  dP   Yb 88   88   88   88   88 88__dP 88__
  8P   Yb   dP Y8   8P   88   Y8   8P 88""Yb 88""
 dP     YbodP  `YbodP'   88   `YbodP' 88oodP 888888

                        Yb    dP 88 888888 Yb        dP 888888 88""Yb
                         Yb  dP  88 88__    Yb  db  dP  88__   88__dP
                          YbdP   88 88""     YbdPYbdP   88""   88"Yb
                           YP    88 888888    YP  YP    888888 88  Yb


           [ GitHub : https://github.com/MShawon/YouTube-Viewer ]


+-------------------------- Version: 1.8.0 --------------------------+

Getting Chrome Driver...
Traceback (most recent call last):
  File "C:\Users\amind\OneDrive\Desktop\Projects\YouTube-Viewer\youtube_viewer.py", line 1003, in <module>
    osname, exe_name = download_driver(patched_drivers=patched_drivers)
  File "C:\Users\amind\OneDrive\Desktop\Projects\YouTube-Viewer\youtubeviewer\download_driver.py", line 112, in download_driver
    uc.install()
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 262, in install
    ChromeDriverManager(executable_path, target_version, *args, **kwargs).install()
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 192, in install
    self.fetch_chromedriver()
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 221, in fetch_chromedriver
    ver = self.get_release_version_number().vstring
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 211, in get_release_version_number
    return LooseVersion(urlopen(self.__class__.DL_BASE + path).read().decode())
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 525, in open
    response = meth(req, response)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 634, in http_response
    response = self.parent.error(
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 563, in error
    return self._call_chain(*args)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 496, in _call_chain
    result = func(*args)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

Could you attempt to download the chromedriver manually and add it to the folder? Much like below? I will attempt to work on the automatic version. But for now, this should be working. Fingers crossed.

…ouTube-Viewer into fix-entire-application

leaskeg · 2024-07-07T10:42:39Z

Hope that sorts the issues with getting chromedriver. Remember to follow MShawon's installation guide:

AminDhouib · 2024-07-07T16:33:39Z

I manually inserted a ChromeDriver and gave it a fixed path using driver_executable_path.
However, the script should support adding ChromeDrivers automatically.

For the time being, I installed ChromeDriver and called it chromedriver1.exe. Source for download:

Replaced the following lines at line 113:

    driver = uc.Chrome(
        driver_executable_path="./chromedriver1.exe",
        options=options,
        major_version=major_version,
    )

It works so far. But yeah, it should automatically download Chrome drivers seamlessly. I have other scripts currently using undetected Selenium that always download the latest Chrome driver correctly, so the issue lies with how it is implemented now.

Note I am using Windows.

leaskeg · 2024-07-07T16:50:20Z

I manually inserted a ChromeDriver and gave it a fixed path using driver_executable_path. However, the script should support adding ChromeDrivers automatically.

For the time being, I installed ChromeDriver and called it chromedriver1.exe. Source for download:

https://storage.googleapis.com/chrome-for-testing-public/126.0.6478.126/win64/chromedriver-win64.zip

https://googlechromelabs.github.io/chrome-for-testing/

Replaced the following lines at line 113:
    driver = uc.Chrome(
        driver_executable_path="./chromedriver1.exe",
        options=options,
        major_version=major_version,
    )
It works so far. But yeah, it should automatically download Chrome drivers seamlessly. I have other scripts currently using undetected Selenium that always download the latest Chrome driver correctly, so the issue lies with how it is implemented now.

Note I am using Windows.

I will attempt to make the automatic download work sometime next week :) Glad it works manually tho.

AminDhouib · 2024-07-07T17:57:47Z

Thanks, @leaskeg, for your efforts! And I appreciate what you have done to bring the project back to life.

I am curious about your current fixes. Are the views still active on your analytics?

leaskeg · 2024-07-07T19:59:26Z

Thanks, @leaskeg, for your efforts! And I appreciate what you have done to bring the project back to life.

I am curious about your current fixes. Are the views still active on your analytics?

They are actually! Tried with 100 views from webshares proxies. They counted.

knofte · 2024-07-07T20:32:03Z

leaskeg · 2024-07-07T22:21:27Z

@leaskeg And if I get " | HTTP | Random | Lat : 13.8196 | Lon : 100.04427 | TZ: Asia/Bangkok | Detected? : True " the detected = true, I guess it's because of proxies not because of the app? (linux)

I will get back to you on that. Its possible that it detects the browser as being automated which is not ideal, because youtube would be able to detect the views as bots. But that would indicate a problem with the undetected-chromedriver im using. I will look it over!

leaskeg · 2024-07-07T22:48:42Z

I have fixed the detection issue. I will attempt to fix the automatic chromedriver mentioned above before releasing. And thank you for all the feedbacks. It really helps, you can only test so much locally 👍

…y download

leaskeg · 2024-07-08T12:40:43Z

This version should hopefully fix the issues people are mentioning here in the comments. I tested locally and it downloads the chromedriver automatically, and its also undetected. Test it out and let me know

knofte · 2024-07-08T13:32:46Z

It works, ( undetected False) however it started triggering a few other issues :)

inside chrome window:
"Check if there is a typo in user-agent=mozilla"

I'll continue test though :)

leaskeg · 2024-07-08T17:54:20Z

This is the analytics while i was simply testing a few things. Seems to work as intended. Will look into the errors above.

zaid404 · 2024-08-18T06:36:48Z

Hi @leaskeg,

Thanks!
some log..
!./chromedriver --version
!google-chrome-stable --version
!./patched_drivers/chromedriver_0 --version
!lsb_release -a

ChromeDriver 126.0.6478.126 (d36ace6122e0a59570e258d82441395206d60e1c-refs/branch-heads/6478@{#1591})
Google Chrome for Testing 126.0.6478.126
ChromeDriver 126.0.6478.126 (d36ace6122e0a59570e258d82441395206d60e1c-refs/branch-heads/6478@{#1591})
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy

pefman · 2025-01-18T09:39:01Z

@leaskeg Cheers for helping out!

i downloaded your pr, but im still getting errors.

Getting Chrome Driver... Traceback (most recent call last): File "/home/pefman/YouTube-Viewer/youtube_viewer.py", line 961, in <module> osname, exe_name, driver_path = download_driver(patched_drivers=patched_drivers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pefman/YouTube-Viewer/youtubeviewer/download_driver.py", line 111, in download_driver driver = uc.Chrome(version_main=major_version, options=options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pefman/.local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 246, in __init__ patcher.auto() File "/home/pefman/.local/lib/python3.11/site-packages/undetected_chromedriver/patcher.py", line 130, in auto release = self.fetch_release_number() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pefman/.local/lib/python3.11/site-packages/undetected_chromedriver/patcher.py", line 151, in fetch_release_number return LooseVersion(urlopen(self.url_repo + path).read().decode()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/urllib/request.py", line 216, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/urllib/request.py", line 525, in open response = meth(req, response) ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/urllib/request.py", line 634, in http_response response = self.parent.error( ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/urllib/request.py", line 563, in error return self._call_chain(*args) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/urllib/request.py", line 496, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.11/urllib/request.py", line 643, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found

leaskeg · 2025-01-19T22:05:51Z

Hello, i will return to this project tomorrow. Have been busy at work. Will look at all the above issues, and get it updated to the new chrome/python version also.

- Supports newer Python versions (up to 3.13) - No need for setuptools<59 restriction - Improved proxy handling - Better error handling - More stable driver management - Fixed Linux chromedriver permission issues (chmod 755) - Fixed time module imports for better compatibility - Updated ChromeDriver download mechanism for latest Chrome versions - Improved version detection and compatibility checks - Added better error messages for driver installation - Version updated to 1.8.3 with all fixes included - Reduced config creation wait time from 20s to 10s for better UX - Standardized log message formatting for better readability - Improved cleanup process for graceful program termination - Fixed connection retry errors during driver cleanup - Added proper sleep delays between cleanup operations - Enhanced driver shutdown sequence (close -> quit -> kill) - Improved ad detection and skipping mechanism - Added robust YouTube consent popup handling - Enhanced popup management for better automation reliability - Fixed video playback issues after ad/consent handling - Added Chrome version caching to avoid repeated manual input - Automated Chrome version detection and storage - Improved startup time by saving browser configuration

leaskeg · 2025-02-13T12:57:53Z

That took way longer than expected. One thing led to another, and I had to redo a lot of the ad/consent detection. Here is the entire overhaul I ended up doing.

You'll find the latest commit above this comment (Major Improvements)

Version 1.9.0 - Major Improvements and Fixes:

Core Functionality:

Fixed macOS detection in download_driver by changing sys.platform == "darwin" to sys.platform.startswith("darwin") for consistency with Linux/Windows detection
Supports newer Python versions (up to 3.13)
Removed setuptools<59 restriction
Improved proxy handling and validation

Driver Management:

More stable ChromeDriver management
Fixed Linux chromedriver permission issues (chmod 755)
Updated ChromeDriver download mechanism for the latest Chrome versions
Improved version detection and compatibility checks
Added Chrome version caching to avoid repeated manual input
Automated Chrome version detection and storage

Performance & Stability:

Fixed time module imports for better compatibility
Reduced config creation wait time from 20s to 10s for better UX
Improved cleanup process for graceful program termination
Fixed connection retry errors during driver cleanup
Added proper sleep delays between cleanup operations
Enhanced driver shutdown sequence (close -> quit -> kill)

YouTube Interaction:

Improved ad detection and skipping mechanism
Added robust YouTube consent popup handling
Enhanced popup management for better automation reliability
Fixed video playback issues after ad/consent handling

Note: While I've tested these changes extensively on Windows, I haven't been able to test the functionality on macOS/Linux systems. Would appreciate it if someone could verify the macOS/Linux functionality, particularly the ChromeDriver installation and management. (It should download/install automatically once you run the program)

All changes have been tested with the latest Chrome and Python 3.x versions.

pefman · 2025-02-19T12:52:17Z

im sorry but still getting errors on linux
Getting Chrome Driver... ChromeDriver not found, downloading... Downloading ChromeDriver from: https://storage.googleapis.com/chrome-for-testing-public/133.0.6943.53/linux64/chromedriver-linux64.zip Alternative driver download failed: [Errno 2] No such file or directory: 'chromedriver-win64/chromedriver'

And on windows...
Getting Chrome Driver... Using saved Chrome version: Error detecting Chrome version: Failed to detect Google Chrome version. Ensure Google Chrome is installed and accessible.

leaskeg · 2025-02-19T15:56:57Z

im sorry but still getting errors on linux Getting Chrome Driver... ChromeDriver not found, downloading... Downloading ChromeDriver from: https://storage.googleapis.com/chrome-for-testing-public/133.0.6943.53/linux64/chromedriver-linux64.zip Alternative driver download failed: [Errno 2] No such file or directory: 'chromedriver-win64/chromedriver'

And on windows... Getting Chrome Driver... Using saved Chrome version: Error detecting Chrome version: Failed to detect Google Chrome version. Ensure Google Chrome is installed and accessible.

That was to be expected since i did most the linux stuff based on a previous project i worked on. I will look into it! If you have any other errors let me know also.

leaskeg and others added 5 commits July 7, 2024 01:51

Fixed chromedriver not working with the new version. Fixed issue with…

dce5940

… loggingPrefs and improved error handling. Working as of 07-07-2024 by Leaske

Update search.txt

081729b

Should be its original values

Update search.txt

67caeb7

Update urls.txt

b5bdf1c

should be its orignal value, this was used for testing

Update proxy.txt

2ee4a8d

Removed proxies, was used for testing

leaskeg commented Jul 7, 2024

View reviewed changes

leaskeg changed the title ~~Fixed chromedriver not working with the new version. Fixed issue with…~~ Fixed entire application. Should work as intended now. Jul 7, 2024

leaskeg added 4 commits July 7, 2024 12:32

updated requirements.txt - completely forgot -.-

5245907

Merge branch 'fix-entire-application' of https://github.com/leaskeg/Y…

0e16c8e

…ouTube-Viewer into fix-entire-application

Updated

9bf61fb

Updated Requirements

c0521ba

Undetected chromedriver now works correctly, should also automaticall…

77f51e0

…y download

leaskeg force-pushed the fix-entire-application branch from 90e51ba to cbf2d77 Compare February 13, 2025 13:33

Uh oh!

Fixed entire application. Should work as intended now. #612

Are you sure you want to change the base?

Fixed entire application. Should work as intended now. #612

Uh oh!

Conversation

leaskeg commented Jul 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Testing

Uh oh!

leaskeg left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AminDhouib commented Jul 7, 2024

Uh oh!

leaskeg commented Jul 7, 2024

Uh oh!

leaskeg commented Jul 7, 2024

Uh oh!

AminDhouib commented Jul 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

leaskeg commented Jul 7, 2024

Uh oh!

AminDhouib commented Jul 7, 2024

Uh oh!

leaskeg commented Jul 7, 2024

Uh oh!

knofte commented Jul 7, 2024

Uh oh!

leaskeg commented Jul 7, 2024

Uh oh!

leaskeg commented Jul 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

leaskeg commented Jul 8, 2024

Uh oh!

knofte commented Jul 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

leaskeg commented Jul 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zaid404 commented Aug 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pefman commented Jan 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

leaskeg commented Jan 19, 2025

Uh oh!

leaskeg commented Feb 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pefman commented Feb 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

leaskeg commented Feb 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

leaskeg commented Jul 6, 2024 •

edited

Loading

leaskeg left a comment •

edited

Loading

AminDhouib commented Jul 7, 2024 •

edited

Loading

leaskeg commented Jul 7, 2024 •

edited

Loading

knofte commented Jul 8, 2024 •

edited

Loading

leaskeg commented Jul 8, 2024 •

edited

Loading

zaid404 commented Aug 18, 2024 •

edited

Loading

pefman commented Jan 18, 2025 •

edited

Loading

leaskeg commented Feb 13, 2025 •

edited

Loading

pefman commented Feb 19, 2025 •

edited

Loading