Skip to content

RequestError: The "msecs" argument must be of type number. Received undefined. When used with socks-proxy-agent. #483

@Demak

Description

@Demak

I need to use a socks5 proxy with custom parameters for crawler requests, I am passing it to the crawler via "agent" got setting:

const agent = new SocksProxyAgent(
	'socks5://proxyurl', {
		...
	}
);

const transport = new Crawler({
	...
	agent: {
		https: agent
	},
	...
});

This works with got, but with crawler request returns an error:
Crawler WARN RequestError: The "msecs" argument must be of type number. Received undefined occurred on "url"

After some debugging, I have found that error occures inside SocksProxyAgent object when it tries to set connection timeout, https://github.com/TooTallNate/proxy-agents/blob/b923f5363875d8fd56477094d5ffc562840a4b76/packages/socks-proxy-agent/src/index.ts#L183 which by default is set to null in constructor, so in code above agent.timeout is null. When preparing got options for request crawler's function cleanObject from utils

export const cleanObject = (obj: Record<string, unknown>): Record<string, unknown> => {
strips that setting and when agent runs its code it's undefined instead, which produces an error since its code expects either number or null https://github.com/TooTallNate/proxy-agents/blob/b923f5363875d8fd56477094d5ffc562840a4b76/packages/socks-proxy-agent/src/index.ts#L182.

I am not sure why Crawler needs to clean up nulls from its options, but in this case it breaks custom https agent.
Setting timeout option for SocksProxyAgent to integer can be used as workaround, but Crawler should probably not clean up custom request agents.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions