Enhance Proxy Handling: Use ProxyManager for HTTP and SOCKSProxyManager for SOCKS Proxies

[mcflem06]

Summary:
Users can now pass a proxy URL directly to functions such as fetch_url and fetch_response, as well as the buffered download functions. This enhancement enables more granular control over the proxy settings on a per-request basis, rather than relying solely on global or environment variables.

Changes:

Enhanced Proxy Parameter Support:
Users can now pass a proxy parameter directly to key functions (fetch_url, fetch_response, and the buffered download functions). This provides greater flexibility by allowing per-request proxy configuration rather than relying solely on environment variables.

pycurl Support:
pycurl branch in _send_pycurl_request has been updated to handle proxies as well. The function now sets the PRE_PROXY option based on the provided proxy parameter or falls back to the global PROXY_URL if none is passed.

Conditional Manager Selection:
The create_pool function now inspects the proxy URL. If it starts with "socks", it imports and uses SOCKSProxyManager; otherwise, it falls back to ProxyManager for HTTP proxies.

Logging Improvements:
Added logging to indicate which proxy manager is being used and the effective proxy value.

Code Refactoring:
Minor refactoring for clarity and to ensure that the effective proxy value is consistently used in pool creation.

Motivation:
Pulling data from news sites is an easy way to get your local IP banned. This updated enables a user to pass 'proxy' into various functions, and ensure the correct proxy manager is used based on the proxy URL, thereby allowing proper support for both HTTP and SOCKS proxies.

Testing:

• Verified that the updated code correctly creates a pool using ProxyManager when an HTTP proxy is provided.
• Confirmed that requests succeed without errors and that the logging output reflects the correct manager being used.
• Additional tests can be performed with SOCKS proxies to ensure full coverage.

[adbar]

@mcflem06 This is an interesting feature but could you please make sure the tests pass?

[mcflem06]

Yep, let me take a look at the tests

[adbar]

@mcflem06 Are you still working on it?