RuntimeError: Tried 5 times to log in; all failed due to xfinity rejecting logins from automation engines

[apippin]

Bug Report

It appears that the xfinity login page has started rejecting authentication attempts by automation engines like selenium or python based session requests. Even though I am using a valid username and password, and can login from the same computer and browser into my xfinity account, xfinity returns an invalid username and password error when attempting to do the same thing via this script.

Version

xfinity-usage 3.0.1 https://github.com/jantman/xfinity-usage

Installation Method

pip install xfinity-usage

Supporting Software Versions

Python 3.7.5
Virtualenv 15.1.0
selenium==3.141.0
Distributor ID: Ubuntu
Description: Ubuntu 18.04.4 LTS
Release: 18.04
Codename: bionic

Actual Output

xfinity-usage -b chrome-headless -vv

2020-07-09 21:16:47,721 [DEBUG xfinity_usage.py:114 - root.__init__() ] Getting browser instance...
2020-07-09 21:16:47,721 [DEBUG xfinity_usage.py:120 - root.run() ] Getting page...
2020-07-09 21:16:47,721 [DEBUG xfinity_usage.py:400 - root.get_browser() ] getting Chrome browser (local) with --headless
2020-07-09 21:16:48,729 [DEBUG connectionpool.py:230 - urllib3.connectionpool._new_conn() ] Starting new HTTP connection (1): 127.0.0.1:46023
2020-07-09 21:16:48,941 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session HTTP/1.1" 200 683
2020-07-09 21:16:49,045 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/window/rect HTTP/1.1" 200 49
2020-07-09 21:16:49,045 [DEBUG xfinity_usage.py:424 - root.get_browser() ] returning browser
2020-07-09 21:16:49,045 [INFO xfinity_usage.py:371 - root.get() ] GET https://customer.xfinity.com/#/devices
2020-07-09 21:16:54,294 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/url HTTP/1.1" 200 14
2020-07-09 21:16:54,299 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/url HTTP/1.1" 200 50
2020-07-09 21:16:54,302 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/execute/sync HTTP/1.1" 200 20
2020-07-09 21:16:54,313 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/source HTTP/1.1" 200 125486
2020-07-09 21:16:54,428 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/screenshot HTTP/1.1" 200 62468
2020-07-09 21:16:54,431 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/url HTTP/1.1" 200 50
2020-07-09 21:16:54,431 [DEBUG xfinity_usage.py:350 - root.do_screenshot() ] Screenshot: /naspool/www/pippins.net/homeassistant/config/comcast/1.png of: https://customer.xfinity.com/#/devices
2020-07-09 21:16:54,440 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element HTTP/1.1" 404 254
2020-07-09 21:16:54,950 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element HTTP/1.1" 404 254
2020-07-09 21:16:55,458 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element HTTP/1.1" 404 254
2020-07-09 21:16:59,415 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element HTTP/1.1" 200 88
2020-07-09 21:16:59,415 [INFO xfinity_usage.py:222 - root.get_usage_page() ] Not logged in; logging in now
2020-07-09 21:16:59,565 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/screenshot HTTP/1.1" 200 371764
2020-07-09 21:16:59,569 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/url HTTP/1.1" 200 445
2020-07-09 21:16:59,570 [DEBUG xfinity_usage.py:350 - root.do_screenshot() ] Screenshot: /naspool/www/pippins.net/homeassistant/config/comcast/2.png of: https://login.xfinity.com/login?r=comcast.net&s=oauth&continue=https%3A%2F%2Foauth.xfinity.com%2Foauth%2Fauthorize%3Fclient_id%3Dmy-account-web%26prompt%3Dlogin%26redirect_uri%3Dhttps%253A%252F%252Fcustomer.xfinity.com%252Foauth%252Fcallback%26response_type%3Dcode%26state%3Dhttps%253A%252F%252Fcustomer.xfinity.com%252F%2523%252Fdevices%26response%3D1&client_id=my-account-web&reqId=319dbee5-ed03-46de-94f4-c65ef9fe5621&forceAuthn=1
2020-07-09 21:16:59,570 [INFO xfinity_usage.py:232 - root.get_usage_page() ] Trying to login...
2020-07-09 21:16:59,571 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/url HTTP/1.1" 200 445
2020-07-09 21:16:59,572 [INFO xfinity_usage.py:148 - root.do_login() ] Logging in (https://login.xfinity.com/login?r=comcast.net&s=oauth&continue=https%3A%2F%2Foauth.xfinity.com%2Foauth%2Fauthorize%3Fclient_id%3Dmy-account-web%26prompt%3Dlogin%26redirect_uri%3Dhttps%253A%252F%252Fcustomer.xfinity.com%252Foauth%252Fcallback%26response_type%3Dcode%26state%3Dhttps%253A%252F%252Fcustomer.xfinity.com%252F%2523%252Fdevices%26response%3D1&client_id=my-account-web&reqId=319dbee5-ed03-46de-94f4-c65ef9fe5621&forceAuthn=1)
2020-07-09 21:16:59,574 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/execute/sync HTTP/1.1" 200 20
2020-07-09 21:16:59,577 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/source HTTP/1.1" 200 18554
2020-07-09 21:16:59,716 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/screenshot HTTP/1.1" 200 371764
2020-07-09 21:16:59,721 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/url HTTP/1.1" 200 445
2020-07-09 21:16:59,721 [DEBUG xfinity_usage.py:350 - root.do_screenshot() ] Screenshot: /naspool/www/pippins.net/homeassistant/config/comcast/3.png of: https://login.xfinity.com/login?r=comcast.net&s=oauth&continue=https%3A%2F%2Foauth.xfinity.com%2Foauth%2Fauthorize%3Fclient_id%3Dmy-account-web%26prompt%3Dlogin%26redirect_uri%3Dhttps%253A%252F%252Fcustomer.xfinity.com%252Foauth%252Fcallback%26response_type%3Dcode%26state%3Dhttps%253A%252F%252Fcustomer.xfinity.com%252F%2523%252Fdevices%26response%3D1&client_id=my-account-web&reqId=319dbee5-ed03-46de-94f4-c65ef9fe5621&forceAuthn=1
2020-07-09 21:16:59,731 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element HTTP/1.1" 404 269
2020-07-09 21:16:59,735 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/source HTTP/1.1" 200 18704
2020-07-09 21:16:59,742 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element HTTP/1.1" 200 88
2020-07-09 21:16:59,765 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element/4b5200b3-e114-4dc8-8493-98963d1a4523/clear HTTP/1.1" 200 14
2020-07-09 21:16:59,824 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element/4b5200b3-e114-4dc8-8493-98963d1a4523/value HTTP/1.1" 200 14
2020-07-09 21:16:59,833 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element HTTP/1.1" 200 88
2020-07-09 21:16:59,840 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/element/17a790b4-1b4d-4ed3-831a-effb31efd213/selected HTTP/1.1" 200 15
2020-07-09 21:16:59,840 [DEBUG xfinity_usage.py:172 - root.do_login() ] Clicking "Remember Me"
2020-07-09 21:16:59,849 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element HTTP/1.1" 200 88
2020-07-09 21:16:59,934 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element/f16bcce8-7cc4-490c-85d0-9d2fe843c43d/click HTTP/1.1" 200 14
2020-07-09 21:16:59,943 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element HTTP/1.1" 200 88
2020-07-09 21:16:59,954 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element HTTP/1.1" 200 88
2020-07-09 21:16:59,979 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element/dad54416-cb56-4cdd-b722-c41980283ad8/clear HTTP/1.1" 200 14
2020-07-09 21:17:00,026 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element/dad54416-cb56-4cdd-b722-c41980283ad8/value HTTP/1.1" 200 14
2020-07-09 21:17:00,026 [DEBUG xfinity_usage.py:196 - root.do_login() ] Clicking Sign In button
2020-07-09 21:17:00,029 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/url HTTP/1.1" 200 445
2020-07-09 21:17:00,185 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/screenshot HTTP/1.1" 200 373924
2020-07-09 21:17:00,190 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/url HTTP/1.1" 200 445
2020-07-09 21:17:00,190 [DEBUG xfinity_usage.py:350 - root.do_screenshot() ] Screenshot: /naspool/www/pippins.net/homeassistant/config/comcast/4.png of: https://login.xfinity.com/login?r=comcast.net&s=oauth&continue=https%3A%2F%2Foauth.xfinity.com%2Foauth%2Fauthorize%3Fclient_id%3Dmy-account-web%26prompt%3Dlogin%26redirect_uri%3Dhttps%253A%252F%252Fcustomer.xfinity.com%252Foauth%252Fcallback%26response_type%3Dcode%26state%3Dhttps%253A%252F%252Fcustomer.xfinity.com%252F%2523%252Fdevices%26response%3D1&client_id=my-account-web&reqId=319dbee5-ed03-46de-94f4-c65ef9fe5621&forceAuthn=1
2020-07-09 21:17:01,916 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/element/447370ca-4d5d-4844-9971-8a49543f8921/click HTTP/1.1" 200 14
2020-07-09 21:17:02,061 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/screenshot HTTP/1.1" 200 378668
2020-07-09 21:17:02,088 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/url HTTP/1.1" 200 43
2020-07-09 21:17:02,089 [DEBUG xfinity_usage.py:350 - root.do_screenshot() ] Screenshot: /naspool/www/pippins.net/homeassistant/config/comcast/5.png of: https://login.xfinity.com/login
2020-07-09 21:17:02,090 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/url HTTP/1.1" 200 43
2020-07-09 21:17:02,092 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "POST /session/591402fdfa1ec4b3fee526b2dcbed3cd/execute/sync HTTP/1.1" 200 20
2020-07-09 21:17:02,096 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/source HTTP/1.1" 200 21094
2020-07-09 21:17:02,246 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/screenshot HTTP/1.1" 200 378668
2020-07-09 21:17:02,250 [DEBUG connectionpool.py:442 - urllib3.connectionpool._make_request() ] http://127.0.0.1:46023 "GET /session/591402fdfa1ec4b3fee526b2dcbed3cd/url HTTP/1.1" 200 43
2020-07-09 21:17:02,250 [DEBUG xfinity_usage.py:350 - root.do_screenshot() ] Screenshot: /naspool/www/pippins.net/homeassistant/config/comcast/6.png of: https://login.xfinity.com/login
2020-07-09 21:17:02,250 [INFO xfinity_usage.py:371 - root.get() ] GET https://customer.xfinity.com/#/devices
...
Traceback (most recent call last):
  File "/naspool/www/pippins.net/homeassistant/lib/python3.7/site-packages/xfinity_usage/xfinity_usage.py", line 234, in get_usage_page
    self.get_usage_page(count=(count + 1))
  File "/naspool/www/pippins.net/homeassistant/lib/python3.7/site-packages/xfinity_usage/xfinity_usage.py", line 230, in get_usage_page
    raise RuntimeError("Tried 5 times to log in; all failed.")
RuntimeError: Tried 5 times to log in; all failed.

Expected Output

I expect the script to run and return either the text output of the usage or json file of the usage from xfinity.

Testing Assistance

I've tested that selenium is being blocked by xfinity, the root of the issue, but launching each supported browser via the -b option supported by this script in selenium and typing in the same username and password and verifying that xfinity rejects them as invalid.

The good news is that I've also tested a fix to the script for firefox (and added a firefox-headless mode) that sets the following preferences that enables xfinity to allow the login to work through selenium using firefox for firefox-headless:

            profile.set_preference('devtools.jsonview.enabled', False)
            profile.set_preference("dom.webdriver.enabled", False)
            profile.set_preference('useAutomationExtension', False)

Here's a patchset, that if applied, fixes the issue and adds support for a new browser option, firefox-headless:

diff --git a/xfinity_usage/xfinity_usage.py b/xfinity_usage/xfinity_usage.py
index 91ad57d..da2db14 100755
--- a/xfinity_usage/xfinity_usage.py
+++ b/xfinity_usage/xfinity_usage.py
@@ -387,12 +387,20 @@ class XfinityUsage(object):
 
     def get_browser(self):
         """get a webdriver browser instance """
-        if self.browser_name == 'firefox':
+        if 'firefox' in self.browser_name:
             logger.debug("getting Firefox browser (local)")
             if 'DISPLAY' not in os.environ:
                 logger.debug("exporting DISPLAY=:0")
                 os.environ['DISPLAY'] = ":0"
-            browser = webdriver.Firefox()
+            if 'headless' in self.browser_name:
+                os.environ['MOZ_HEADLESS'] = "1"
+            options = webdriver.FirefoxOptions()
+            profile = webdriver.FirefoxProfile()
+            profile.set_preference('devtools.jsonview.enabled', False)
+            profile.set_preference("dom.webdriver.enabled", False)
+            profile.set_preference('useAutomationExtension', False)
+            profile.update_preferences()
+            browser = webdriver.Firefox(firefox_profile=profile, firefox_options=options)
         elif self.browser_name == 'chrome':
             logger.debug("getting Chrome browser (local)")
             browser = webdriver.Chrome()
@@ -594,7 +602,7 @@ def parse_args(argv):
                    type=str,
                    default=os.path.realpath('xfinity_usage_cookies.json'),
                    help='File to save cookies in')
-    browsers = ['phantomjs', 'firefox', 'chrome', 'chrome-headless']
+    browsers = ['phantomjs', 'firefox', 'firefox-headless', 'chrome', 'chrome-headless']
     p.add_argument('-b', '--browser', dest='browser_name', type=str,
                    default='phantomjs', choices=browsers,
                    help='Browser name/type to use')

With the patch applied above, the script is able to login to xfinity and get the data usage properly. Note I've only patched firefox and firefox-headless. I tried to fix chrome and chrome-headless, but wasn't able to figure out the magic to get it to work like I do for firefox above.

[mtdoyle]

Thanks, I applied your changes and it's working for me as well.

[chuccck]

thanks @apippin ! that patch was a lifesaver! worked for me but i had to manually change the browser to firefox-headless from phantomjs in lines 84/85. I had to do this with chrome too when i had it working with chrome. I also downloaded geckodriver.exe into my script folder

def __init__(self, username, password, debug=False, cookie_file='cookies.json', browser_name='firefox-headless'):

[tbabula]

I am fairly new to Docker and deployed container from https://hub.docker.com/r/billimek/comcastusage-for-influxdb/ Just as you, once I analyzed docker logs, I got exception due to login rejection 5 times by Comcast despite using correct username, password and image screenshot pulled from exceptions error output showed it was on correct login page.

[tbabula]

It seems to fail for me even with the changes mentioned by @apippin. It seems to fail to login 5 times for headless Firefox now too. It seems Comcast made changes that break automation bots from polling pages after login. Docker images need complete rebuild, they have a lot of dependencies on old upstream repositories set years ago such as phantomjs which is now deprecated.

[jantman]

@tbabula Unfortunately, I haven't had Xfinity service in over 2 years. @billimek was tentatively going to take over this project, as without xfinity service, I have no way of testing it. There's not really a whole lot I can say aside from the fact that if Jeff is no longer actively using this, it may once again be in need of a new maintainer. Apologies.

[tbabula]

Thanks for response. As you mentioned in the blog, the screen-scrapper may break anytime due to site redesign or element ID changes. I just noticed when recently I tried to login to Comcast via app, that they locked my account for password reset even though I used my correct username/password in the config file used by script for scrapping.

…

On Aug 14, 2020, at 11:28 AM, Jason Antman ***@***.***> wrote: @tbabula <https://github.com/tbabula> Unfortunately, I haven't had Xfinity service in over 2 years. @billimek <https://github.com/billimek> was tentatively going to take over this project, as without xfinity service, I have no way of testing it. There's not really a whole lot I can say aside from the fact that if Jeff is no longer actively using this, it may once again be in need of a new maintainer. Apologies. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#30 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AM22SQEOL4VEC7PV7NJFMA3SAV65PANCNFSM4OWGEVMA>.

[chuccck]

Thanks for response. As you mentioned in the blog, the screen-scrapper may break anytime due to site redesign or element ID changes. I just noticed when recently I tried to login to Comcast via app, that they locked my account for password reset even though I used my correct username/password in the config file used by script for scrapping.
…
On Aug 14, 2020, at 11:28 AM, Jason Antman @.***> wrote: @tbabula https://github.com/tbabula Unfortunately, I haven't had Xfinity service in over 2 years. @billimek https://github.com/billimek was tentatively going to take over this project, as without xfinity service, I have no way of testing it. There's not really a whole lot I can say aside from the fact that if Jeff is no longer actively using this, it may once again be in need of a new maintainer. Apologies. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#30 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM22SQEOL4VEC7PV7NJFMA3SAV65PANCNFSM4OWGEVMA.

I am still having success with the patches suggested earlier. The one other change I realized I made was to edit line 85 to browser_name='firefox-headless' from browser_name='phantomjs'

[tbabula]

Thanks for response. As you mentioned in the blog, the screen-scrapper may break anytime due to site redesign or element ID changes. I just noticed when recently I tried to login to Comcast via app, that they locked my account for password reset even though I used my correct username/password in the config file used by script for scrapping.
…
On Aug 14, 2020, at 11:28 AM, Jason Antman @.***> wrote: @tbabula https://github.com/tbabula Unfortunately, I haven't had Xfinity service in over 2 years. @billimek https://github.com/billimek was tentatively going to take over this project, as without xfinity service, I have no way of testing it. There's not really a whole lot I can say aside from the fact that if Jeff is no longer actively using this, it may once again be in need of a new maintainer. Apologies. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#30 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM22SQEOL4VEC7PV7NJFMA3SAV65PANCNFSM4OWGEVMA.

I am still having success with the patches suggested earlier. The one other change I realized I made was to edit line 85 to browser_name='firefox-headless' from browser_name='phantomjs'

Yeah I had reflected that change in the line as well. But also just before patching was running in docker the original script to see it workflows in order to see how to patch rebuilt docker which seems Comcast's website IDS/IPS responded by locking out my login credentials for password reset due to phantomjs automation login. And of course there are couple more things to change than just the script. There are different ways to install gecokdriver for example. For example, using pip3 install webdriver install method did not work for me. So downloaded the gecokdriver directly and deployed to /usr/local/bin.

Then if you want to deploy on the docker, using billimek/comcastUsage-for-influxDB repository, you have to think how to rebuilt his source image which involves installing some extra prerequisite dependencies that if its not handled correctly, will break install even if xfinity_usage.py script is correctly patched.

[TomRoush]

I created an updated Docker image that uses firefox-headless as its browser: https://hub.docker.com/r/tomroush/comcastusage-for-influxdb. I've been running it in Kubernetes for a couple days. @apippin are you planning to make a pull request with your changes?

[indiana500]

I am still having success with the patches suggested earlier. The one other change I realized I made was to edit line 85 to browser_name='firefox-headless' from browser_name='phantomjs'

If you had to change line 85, then the there must be an assignment missing when you created the instance of the class. That line is just providing a default and wherever you actually create the instance should provide an assignment for the browser you want to use. For instance, the example on the front page of this project has this line:

u = XfinityUsage(os.environ['XFINITY_USER'], os.environ['XFINITY_PASSWORD'], browser_name='chrome-headless')

you need that browser_name= assignement in there.

I tried setting up something myself recently that wasn't working and I couldn't figure out why until I stumbled across this project, and this issue. After pulling xfinity-usage, and making the changes suggested by apippin above, it's been running fine for the last week. I'm not using Docker though.

[apippin]

thanks @apippin ! that patch was a lifesaver! worked for me but i had to manually change the browser to firefox-headless from phantomjs in lines 84/85. I had to do this with chrome too when i had it working with chrome. I also downloaded geckodriver.exe into my script folder

def __init__(self, username, password, debug=False, cookie_file='cookies.json', browser_name='firefox-headless'):

I'm glad it is working for you. I'm surprised the command line -b option (like -b firefox-headless) didn't work for you to change the browser? I do agree that changing the default to be firefox-headless does make sense as phantomjs doesn't work at all.

[apippin]

I created an updated Docker image that uses firefox-headless as its browser: https://hub.docker.com/r/tomroush/comcastusage-for-influxdb. I've been running it in Kubernetes for a couple days. @apippin are you planning to make a pull request with your changes?

Great suggestion. I just pushed up a pull request that encapsulates my changes (#32).