rabbitmq_auth_backend_http fails to connect to a hostname that only has DNS AAAA (IPv6) records

[graorane]

Describe the bug

Hi

I am using RabbitMQ on IpV6 only host. Details as below
RabbitMQ version : 3.13.2
Erlang version : 26.2.5.2
Host : Windows Server 2022 Datacenter (But I guess Windows version is not playing any role in this issue.)

As mentioned, the host is IpV6 only. Hence I have set following variable before starting RabbitMQ

RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS=-kernel inetrc 'C:/erl_inetrc' -proto_dist inet6_tcp

erl_inetrc file content as below
{inet6,true}.

RabbitMQ is configured to authenticate user using rabbitmq_auth_backend_http

{rabbitmq_auth_backend_http,
[{http_method, post},
{user_path, "https://winhost:xxxxx/user"},
{vhost_path, "https://winhost:xxxxx/vhost"},
{resource_path, "https://winhost:xxxxx/resource"},
{topic_path, "https://winhost:xxxxx/topic"},
{ssl_options, [{cacertfile, "C:\ca.pem"},
{verify, verify_peeer},
{versions, ['tlsv1.3']},
{middlebox_comp_mode, false},
{depth, 5}
]}
]},

Host name "winhost" gets resolved to ipv6 address. No IpV4 address.

It can be seen that RabbitMQ is up and running. I can see that it is accepting AMQP connections on Ipv6 interface. But when it tries to connect to "winhost" to authenticate user, it is failing with below error.

2025-01-27 00:00:34.695000-08:00 [info] <0.5433.0> accepting AMQP connection <0.5433.0> ([xxxx:xxx:xxxx:xxxx::xxx]:62052 -> [xxxx:xxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx]:5672)
2025-01-27 00:00:34.701000-08:00 [error] <0.5433.0> Error on AMQP connection <0.5433.0> ([xxxx:xxx:xxxx:xxxx::xxx]:62052 -> [xxxx:xxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx]:5672, state: starting):
2025-01-27 00:00:34.701000-08:00 [error] <0.5433.0> PLAIN login refused: rabbit_auth_backend_http failed authenticating xxxxxxxx: {failed_connect,
2025-01-27 00:00:34.701000-08:00 [error] <0.5433.0> [{to_address,
2025-01-27 00:00:34.701000-08:00 [error] <0.5433.0> {"winhost",
2025-01-27 00:00:34.701000-08:00 [error] <0.5433.0> xxxxx}},
2025-01-27 00:00:34.701000-08:00 [error] <0.5433.0> {inet,
2025-01-27 00:00:34.701000-08:00 [error] <0.5433.0> [inet],
2025-01-27 00:00:34.701000-08:00 [error] <0.5433.0>

Reproduction steps

Refer description.

Expected behavior

Authenticate should work even if http uri getting resolved ipv6 host.

Additional context

No response

[michaelklishin]

@graorane RabbitMQ 3.13 is out of community support.

I don't see why the HTTP client used underneath would not use with IPv6 addresses. There are no special "IPv6 settings" in the client.

failed_connect can mean a broad range of potential connectivity issues. It's on you to demonstrates the evidence (e.g. with a traffic capture). We have several example applications that can be used with said plugin.

You configure the client to use TLS v1.3 which won't work with TLS v1.2 servers, as explained in the docs.

In addition, troubleshooting TLS connections is a whole different universe of possible scenarios, and our Community Support Policy explicitly states that we will not troubleshoot TLS for you. You must provide clear evidence with a detailed set of reproducible steps (e.g. a git repository with containers and a set of specific steps to follow).

[michaelklishin]

The underlying's HTTP client docs say

The client supports IPv6 as long as the underlying mechanisms also do so.

where the underlying mechanism is the DNS, TCP and IP implementations. Not only they do support IPv6,
there are many networking settings available,
including an option to force the DNS resolver use AAAA records (IPv6 addresses):

{inet6, true}.

This file is mentioned in the Networking guide.

A Functional Example

Using the following Spring Boot example, an /etc/hosts file that has this entry:

::1             localhost localhost6

and this rabbitmq.conf file:

log.file.level = debug
log.console.level = debug

# auth_backends.1 = cache
# auth_cache.cached_backend = http

auth_backends.1 = http

auth_http.http_method   = post
auth_http.user_path     = http://localhost6:8080/auth/user
auth_http.vhost_path    = http://localhost6:8080/auth/vhost
auth_http.resource_path = http://localhost6:8080/auth/resource
auth_http.topic_path    = http://localhost6:8080/auth/topic

I run the node like so:

RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS="-kernel inetrc '/path/to/inetrc'" RABBITMQ_CONFIG_FILE='/path/to/auth_backend_http_spring.conf' rabbitmq-server

I can successfully connect using a Ruby client REPL using the default credentials, the only ones supported by the example Spring Boot app. Both RabbitMQ node and Spring Boot logs report a successful connection:

2025-01-30T11:58:11.809-05:00  INFO 96826 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2025-01-30T11:58:11.810-05:00  INFO 96826 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 1 ms
2025-01-30T12:05:08.643-05:00  INFO 96826 --- [nio-8080-exec-2] c.r.examples.AuthBackendHttpController   : Successfully authenticated user guest
2025-01-30T12:05:08.652-05:00  INFO 96826 --- [nio-8080-exec-3] c.r.examples.AuthBackendHttpController   : Checking vhost access with VirtualHostCheck{username='guest', vhost='/'}
2025-01-30T12:06:05.719-05:00  INFO 96826 --- [nio-8080-exec-4] c.r.examples.AuthBackendHttpController   : Successfully authenticated user guest
2025-01-30T12:06:05.724-05:00  INFO 96826 --- [nio-8080-exec-5] c.r.examples.AuthBackendHttpController   : Checking vhost access with VirtualHostCheck{username='guest', vhost='/'}
2025-01-30T12:13:52.538-05:00  INFO 96826 --- [nio-8080-exec-9] c.r.examples.AuthBackendHttpController   : Successfully authenticated user guest
2025-01-30T12:13:52.544-05:00  INFO 96826 --- [io-8080-exec-10] c.r.examples.AuthBackendHttpController   : Checking vhost access with VirtualHostCheck{username='guest', vhost='/'}

2025-01-30 12:18:34.594911-05:00 [debug] <0.785.0> auth_backend_http: POST http://localhost6:8080/auth/user
2025-01-30 12:18:34.594981-05:00 [debug] <0.785.0> auth_backend_http: request timeout: infinity, connection timeout: infinity
2025-01-30 12:18:34.652104-05:00 [debug] <0.785.0> auth_backend_http: response code is 200, body: "allow administrator management"
2025-01-30 12:18:34.652158-05:00 [debug] <0.785.0> User 'guest' authenticated successfully by backend rabbit_auth_backend_http
2025-01-30 12:18:34.652571-05:00 [debug] <0.785.0> auth_backend_http: POST http://localhost6:8080/auth/vhost
2025-01-30 12:18:34.652595-05:00 [debug] <0.785.0> auth_backend_http: request timeout: infinity, connection timeout: infinity
2025-01-30 12:18:34.653818-05:00 [debug] <0.785.0> auth_backend_http: response code is 200, body: "allow"
2025-01-30 12:18:34.653951-05:00 [info] <0.785.0> connection 127.0.0.1:61543 -> 127.0.0.1:5672: user 'guest' authenticated and granted access to vhost '/'
2025-01-30 12:18:36.659702-05:00 [info] <0.785.0> closing AMQP connection (127.0.0.1:61543 -> 127.0.0.1:5672, vhost: '/', user: 'guest', duration: '2s')

To verify that my inetrc file indeed as an effect:

rabbitmq-diagnostics environment | grep inetrc
# =>      {inetrc,'/path/to/inetrc'},

[gabrielmancini]

but i can't reach ipv6 service,
and maybe this is way:

(rabbit@48e7292b377748)19> inet:get_rc().
[{inet6,true},
 {dnssec_ok,undefined},
 {resolv_conf,"/etc/resolv.conf"},
 {hosts_file,"/etc/hosts"},
 {lookup,[native]}]
(rabbit@48e7292b377748)20> httpc:get_options(all).
{ok,[{proxy,{undefined,[]}},
     {https_proxy,{undefined,[]}},
     {pipeline_timeout,0},
     {max_pipeline_length,2},
     {max_keep_alive_length,5},
     {keep_alive_timeout,120000},
     {max_sessions,2},
     {cookies,disabled},
     {verbose,false},
     {ipfamily,inet},
     {ip,default},
     {port,default},
     {socket_opts,[]},
     {unix_socket,undefined}]}
(rabbit@48e7292b377748)21>

the client dont get the ipfamily in my case my backend app is on other machine on fly.io and the network is configured to IPV6

i have the same problem:

2026-01-03T00:33:08Z app[48e7292b377748] iad [info]2026-01-03 00:33:08.621410+00:00 [error] <0.1285.0> MQTT connection failed: access refused for user 'device':rabbit_auth_backend_http failed authenticating device: {failed_connect,
2026-01-03T00:33:08Z app[48e7292b377748] iad [info]2026-01-03 00:33:08.621410+00:00 [error] <0.1285.0>                                                                                                                  [{to_address,
2026-01-03T00:33:08Z app[48e7292b377748] iad [info]2026-01-03 00:33:08.621410+00:00 [error] <0.1285.0>                                                                                                                    {"dev-umbrella.internal",
2026-01-03T00:33:08Z app[48e7292b377748] iad [info]2026-01-03 00:33:08.621410+00:00 [error] <0.1285.0>                                                                                                                     8080}},
2026-01-03T00:33:08Z app[48e7292b377748] iad [info]2026-01-03 00:33:08.621410+00:00 [error] <0.1285.0>                                                                                                                   {inet,
2026-01-03T00:33:08Z app[48e7292b377748] iad [info]2026-01-03 00:33:08.621410+00:00 [error] <0.1285.0>                                                                                                                    [inet],
2026-01-03T00:33:08Z app[48e7292b377748] iad [info]2026-01-03 00:33:08.621410+00:00 [error] <0.1285.0>                                                                                                                    nxdomain}]}
2026-01-03T00:33:11Z app[48e7292b377748] iad [info]2026-01-03 00:33:11.622401+00:00 [error] <0.1285.0> Rejected MQTT connection 172.16.3.218:45736 -> 172.19.3.218:1883 with Connect Reason Code 134

my docker file:

FROM rabbitmq:4.2.2-management-alpine

RUN apk add --no-cache curl erlang27

ARG RABBITMQ_DEFAULT_USER RABBITMQ_DEFAULT_PASS RABBITMQ_DEFAULT_VHOST

COPY rabbitmq.conf /etc/rabbitmq/rabbitmq.conf
COPY enabled_plugins /etc/rabbitmq/enabled_plugins
COPY definitions.json /etc/rabbitmq/definitions.json
COPY erl_inetrc /etc/rabbitmq/erl_inetrc

ENV RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS="-kernel inetrc '/etc/rabbitmq/erl_inetrc' -proto_dist inet6_tcp"
ENV RABBITMQ_CTL_ERL_ARGS="-proto_dist inet6_tcp"
# ENV RABBITMQ_USE_LONGNAME=true
ENV PATH=$PATH:/opt/rabbitmq/sbin

erl_inetrc:

{inet6, true}.

rabbitmq.conf

listeners.tcp.default = :::5672
management.tcp.port = 8080
management.tcp.ip = ::
vm_memory_high_watermark.relative = 0.9

auth_backends.1             = rabbit_auth_backend_internal
auth_backends.2             = rabbit_auth_backend_http
auth_http.http_method       = post
auth_http.user_path         = http://dev-umbrella.internal:8080/device/auth/user
auth_http.vhost_path        = http://dev-umbrella.internal:8080/device/auth/vhost
auth_http.resource_path     = http://dev-umbrella.internal:8080/device/auth/resource
auth_http.topic_path        = http://dev-umbrella.internal:8080/device/auth/topic

auth_http.connection_timeout = 5000
auth_http.request_timeout    = 10000

load_definitions = /etc/rabbitmq/definitions.json


mqtt.allow_anonymous = false
anonymous_login_user = none

mqtt.exchange = mqtt.ingest
mqtt.message_interceptors.incoming.set_client_id_annotation.enabled = true
mqtt.max_session_expiry_interval_seconds = 86400

enabled_plugins:

[rabbitmq_management,rabbitmq_prometheus,rabbitmq_mqtt,rabbitmq_web_mqtt,rabbitmq_shovel_management,rabbitmq_shovel,rabbitmq_auth_backend_http,rabbitmq_event_exchange,rabbitmq_federation,rabbitmq_federation_management].

anyone have any gotcha? i kind desperate here

[michaelklishin]

Our team really does not appreciate when existing discussions are hijacked for new questions.

@gabrielmancini nxdomain is UNIX speak for "the domain name cannot be resolved" (on the RabbitMQ node).