ytfetcher.config package

Submodules

ytfetcher.config.http_config module

class ytfetcher.config.http_config.HTTPConfig(timeout: float | None = None, headers: dict | None = None)

Bases: object

Configuration object for HTTP client settings.

This class provides a structured way to configure HTTP-related options such as timeouts and headers when making network requests. It ensures that headers are valid and assigns default, realistic browser-like headers if none are provided.

timeout

Timeout configuration for HTTP requests.

Type:

httpx.Timeout

headers

Dictionary of HTTP headers to be used in requests.

Type:

dict

ytfetcher.config.logging_config module

ytfetcher.config.logging_config.enable_default_config(level=20)

Simple method for enabling basic logging. :param level: Log level. Default to INFO

Module contents

class ytfetcher.config.GenericProxyConfig(http_url: str | None = None, https_url: str | None = None)

Bases: ProxyConfig

This proxy config can be used to set up any generic HTTP/HTTPS/SOCKS proxy. As it the requests library is used under the hood, you can follow the requests documentation to get more detailed information on how to set up proxies: https://requests.readthedocs.io/en/latest/user/advanced/#proxies

If only an HTTP or an HTTPS proxy is provided, it will be used for both types of connections. However, you will have to provide at least one of the two.

to_requests_dict() → RequestsProxyConfigDict

Turns this proxy config into the Dict that is expected by the requests library. More information on this can be found in the official requests documentation: https://requests.readthedocs.io/en/latest/user/advanced/#proxies

class ytfetcher.config.HTTPConfig(timeout: float | None = None, headers: dict | None = None)

Bases: object

Configuration object for HTTP client settings.

This class provides a structured way to configure HTTP-related options such as timeouts and headers when making network requests. It ensures that headers are valid and assigns default, realistic browser-like headers if none are provided.

timeout

Timeout configuration for HTTP requests.

Type:

httpx.Timeout

headers

Dictionary of HTTP headers to be used in requests.

Type:

dict

class ytfetcher.config.ProxyConfig

Bases: ABC

The base class for all proxy configs. Anything can be a proxy config, as longs as it can be turned into a RequestsProxyConfigDict by calling to_requests_dict.

property prevent_keeping_connections_alive: bool

If you are using rotating proxies, it can be useful to prevent the HTTP client from keeping TCP connections alive, as your IP won’t be rotated on every request, if your connection stays open.

property retries_when_blocked: int

Defines how many times we should retry if a request is blocked. When using rotating residential proxies with a large IP pool it can make sense to retry a couple of times when a blocked IP is encountered, since a retry will trigger an IP rotation and the next IP might not be blocked.

abstractmethod to_requests_dict() → RequestsProxyConfigDict

Turns this proxy config into the Dict that is expected by the requests library. More information on this can be found in the official requests documentation: https://requests.readthedocs.io/en/latest/user/advanced/#proxies

class ytfetcher.config.WebshareProxyConfig(proxy_username: str, proxy_password: str, filter_ip_locations: List[str] | None = None, retries_when_blocked: int = 10, domain_name: str = 'p.webshare.io', proxy_port: int = 80)

Bases: GenericProxyConfig

Webshare is a provider offering rotating residential proxies, which is the most reliable way to work around being blocked by YouTube.

If you don’t have a Webshare account yet, you will have to create one at https://www.webshare.io/?referral_code=w0xno53eb50g and purchase a “Residential” proxy package that suits your workload, to be able to use this proxy config (make sure NOT to purchase “Proxy Server” or “Static Residential”!).

Once you have created an account you only need the “Proxy Username” and “Proxy Password” that you can find in your Webshare settings at https://dashboard.webshare.io/proxy/settings to set up this config class, which will take care of setting up your proxies as needed, by defaulting to rotating proxies.

Note that referral links are used here and any purchases made through these links will support this Open Source project, which is very much appreciated! :) However, you can of course integrate your own proxy solution by using the GenericProxyConfig class, if that’s what you prefer.

DEFAULT_DOMAIN_NAME = 'p.webshare.io'

DEFAULT_PORT = 80

property http_url: str

property https_url: str

property prevent_keeping_connections_alive: bool

If you are using rotating proxies, it can be useful to prevent the HTTP client from keeping TCP connections alive, as your IP won’t be rotated on every request, if your connection stays open.

property retries_when_blocked: int

Defines how many times we should retry if a request is blocked. When using rotating residential proxies with a large IP pool it can make sense to retry a couple of times when a blocked IP is encountered, since a retry will trigger an IP rotation and the next IP might not be blocked.

property url: str

ytfetcher.config.enable_default_config(level=20)

Simple method for enabling basic logging. :param level: Log level. Default to INFO