MG-2126 - Update messaging with mProxy TLS by nyagamunene · Pull Request #2206 · absmach/supermq

nyagamunene · 2024-04-26T11:13:05Z

What type of PR is this?

This is a feature: It updates Magistrala messaging with mProxy with TLS.

What does this do?

It updates Magistrala messaging with mProxy with TLS and mTLS.

Which issue(s) does this PR fix/relate to?

Related Issue Feature: Update Magistrala messaging with mProxy with TLS #2126
Resolves Feature: Update Magistrala messaging with mProxy with TLS #2126

Have you included tests for your changes?

Yes

Did you document any new/modified feature?

Yes

Notes

arvindh123

When I start with make run getting to below error.

GRPC_MTLS=
GRPC_TLS=
docker compose -f docker/docker-compose.yml --env-file docker/.env -p arvindh123_magistrala_git_  up 
WARN[0000] The "MG_MQTT_WS_ADAPTER_KEY_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_WS_ADAPTER_CERT_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_WS_ADAPTER_CLIENT_CA_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_ADAPTER_CERT_VERIFICATION_METHODS" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_WS_ADAPTER_PREFIX_PATH" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_WS_ADAPTER_OCSP_RESPONDER_URL" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_ADAPTER_CLIENT_CA_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHOD" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_WS_ADAPTER_SERVER_CA_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_MQTT_ADAPTER_OCSP_RESPONDER_URL" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_HTTP_ADAPTER_CLIENT_CA_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_HTTP_ADAPTER_CERT_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_HTTP_ADAPTER_KEY_FILE" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_HTTP_ADAPTER_CERT_VERIFICATION_METHODS" variable is not set. Defaulting to a blank string. 
WARN[0000] The "MG_HTTP_ADAPTER_SERVER_CA_FILE" variable is not set. Defaulting to a blank string. 
validating /home/arvindh123/magistrala/docker/docker-compose.yml: services.mqtt-adapter.volumes array items[3,4] must be unique
make: *** [Makefile:240: run] Error 15

arvindh123 · 2024-05-02T08:32:33Z

docker/docker-compose.yml

+      MG_MQTT_WS_ADAPTER_SERVER_CA_FILE: ${MG_MQTT_WS_ADAPTER_SERVER_CA_FILE}
+      MG_MQTT_WS_ADAPTER_PREFIX_PATH: ${MG_MQTT_WS_ADAPTER_PREFIX_PATH}
+      MG_MQTT_WS_ADAPTER_CLIENT_CA_FILE: ${MG_MQTT_WS_ADAPTER_CLIENT_CA_FILE}
+      MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHOD: ${MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHOD}


Suggested change

MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHOD: ${MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHOD}

MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHODS: ${MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHODS}

WashingtonKK · 2024-05-02T10:52:53Z

docker/docker-compose.yml

    networks:
      - magistrala-base-net
    volumes:
+      - ./ssl/certs/ca.key:/etc/ssl/certs/ca.key


Have this as an env variable. Sth like: - ./ssl/certs/ca.key:${MG_MQTT_CA_CRT_KEY}

Do the same for the others.

WashingtonKK · 2024-05-02T11:39:21Z

cmd/mqtt/main.go

 		log.Fatalf("failed to load %s configuration : %s", svcName, err)
 	}
-
+	log.Println(cfg)


Remove the debug log

Remove @nyagamunene

arvindh123

I have tried other way for loading certifcate, but I could not find better way than approaches which i have mentioned in the suggestions

arvindh123 · 2024-05-02T12:52:03Z

docker/.env

+# MG_MQTT_WS_ADAPTER_CLIENT_CA_FILE=etc/ssl/certs/ca.crt
+# MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHODS=ocsp
+# MG_MQTT_WS_ADAPTER_OCSP_RESPONDER_URL=http://localhost:8080/ocsp
+


Lets have env name for mqtt websocket as MG_MQTT_ADAPTER_WS to align with previous version of env naming.
Then In .env we need to provide the actual certificates location. not the docker location.

Suggested change

## MPROXY MQTT WS

MG_MQTT_ADAPTER_WS_ADDRESS=:8080

MG_MQTT_ADAPTER_WS_TARGET=ws://${MG_MQTT_BROKER_TYPE}:8080/mqtt

MG_MQTT_ADAPTER_WS_PREFIX_PATH=/mqtt

# MG_MQTT_ADAPTER_WS_CERT_FILE=./ssl/certs/magistrala-server.crt

# MG_MQTT_ADAPTER_WS_KEY_FILE=./ssl/certs/magistrala-server.key

# MG_MQTT_ADAPTER_WS_SERVER_CA_FILE=./ssl/certs/ca.crt

# MG_MQTT_ADAPTER_WS_CLIENT_CA_FILE=./ssl/certs/ca.crt

# MG_MQTT_ADAPTER_WS_CERT_VERIFICATION_METHODS=ocsp

# MG_MQTT_ADAPTER_WS_OCSP_RESPONDER_URL=http://localhost:8080/ocsp

arvindh123 · 2024-05-02T12:57:10Z

docker/.env

+# MG_MQTT_WS_ADAPTER_CLIENT_CA_FILE=etc/ssl/certs/ca.crt
+# MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHODS=ocsp
+# MG_MQTT_WS_ADAPTER_OCSP_RESPONDER_URL=http://localhost:8080/ocsp
+


We should provide the certificate location , not the docker location
Same for HTTP Adapter also. Please change in HTTP Adapter env values to certificates location

Suggested change

MG_MQTT_ADAPTER_CERT_FILE=./ssl/magistrala-server.crt

MG_MQTT_ADAPTER_KEY_FILE=./ssl/magistrala-server.key

MG_MQTT_ADAPTER_SERVER_CA_FILE=./ssl/certs/ca.crt

MG_MQTT_ADAPTER_CLIENT_CA_FILE=./ssl/certs/ca.crt

arvindh123 · 2024-05-02T14:13:58Z

docker/docker-compose.yml

+      - ./ssl/certs/magistrala-server.key:/etc/ssl/certs/magistrala-server.key
+      - ./ssl/certs/thing.crt:/etc/ssl/certs/thing.crt
+      - ./ssl/certs/thing.key:/etc/ssl/certs/thing.key
      # Things gRPC mTLS client certificates


We load env cert path to const path, because in .env the actual location of certificates will be given and here we will load the const path with conditionally.
If the path given is not given , a dummy path will be loaded. but the env varabile of contaienr is not set.

Suggested change

# Things gRPC mTLS client certificates

# MQTT Adapter certificates

- type: bind

source: ${MG_MQTT_ADAPTER_CERT_FILE:-./ssl/certs/dummy/server_cert}

target: /magistrala-mqtt-adapter${MG_MQTT_ADAPTER_CERT_FILE:+.crt}

read_only: true

bind:

create_host_path: true

- type: bind

source: ${MG_MQTT_ADAPTER_KEY_FILE:-./ssl/certs/dummy/server_key}

target: /magistrala-mqtt-adapter${MG_MQTT_ADAPTER_KEY_FILE:+.key}

read_only: true

bind:

create_host_path: true

- type: bind

source: ${MG_MQTT_ADAPTER_SERVER_CA_FILE:-./ssl/certs/dummy/server_ca}

target: /magistrala-ca${MG_MQTT_ADAPTER_SERVER_CA_FILE:+.crt}

read_only: true

bind:

create_host_path: true

- type: bind

source: ${MG_MQTT_ADAPTER_CLIENT_CA_FILE:-./ssl/certs/dummy/client_ca}

target: /magistrala-client-ca${MG_MQTT_ADAPTER_CLIENT_CA_FILE:+.key}

read_only: true

bind:

create_host_path: true

rodneyosodo

Have a look at https://github.com/absmach/magistrala/pull/2212 there are some concepts overlapping

cmd/http/main.go

cmd/mqtt/main.go

http/api/endpoint_test.go

pkg/sdk/go/message_test.go

cmd/http/main.go

rodneyosodo · 2024-05-07T10:51:12Z

go.mod

 	github.com/0x6flab/namegenerator v1.3.1
 	github.com/absmach/callhome v0.14.0
-	github.com/absmach/mproxy v0.4.2
+	github.com/absmach/mproxy v0.4.3-0.20240430090627-27dad4c91c6c


update mproxy

rodneyosodo · 2024-05-07T10:53:06Z

cmd/http/main.go

 const (
 	svcName        = "http_adapter"
-	envPrefix      = "MG_HTTP_ADAPTER_"
+	envPrefixHTTP  = "MG_HTTP_ADAPTER_"


Revert to using envPrefix

Suggested change

envPrefixHTTP = "MG_HTTP_ADAPTER_"

envPrefix = "MG_HTTP_ADAPTER_"

rodneyosodo · 2024-05-07T10:56:33Z

docker/.env

 MG_THINGS_AUTH_GRPC_SERVER_CERT=${GRPC_MTLS:+./ssl/certs/things-grpc-server.crt}${GRPC_TLS:+./ssl/certs/things-grpc-server.crt}
 MG_THINGS_AUTH_GRPC_SERVER_KEY=${GRPC_MTLS:+./ssl/certs/things-grpc-server.key}${GRPC_TLS:+./ssl/certs/things-grpc-server.key}
 MG_THINGS_AUTH_GRPC_SERVER_CA_CERTS=${GRPC_MTLS:+./ssl/certs/ca.crt}${GRPC_TLS:+./ssl/certs/ca.crt}
+MG_THINGS_AUTH_GRPC_ADDRESS=test:7000


This is not the correct URL

arvindh123

Please update the nginx config at here https://github.com/absmach/magistrala/blob/main/docker/nginx/nginx-key.conf#L192-L209

# MQTT
stream {
   include snippets/stream_access_log.conf;

    # Include single-node or multiple-node (cluster) upstream
    # Configure upstream with available nginx loading blancing strageies https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/
    include snippets/mqtt-upstream.conf;

    server {
        listen ${MG_NGINX_MQTT_PORT};
        listen [::]:${MG_NGINX_MQTT_PORT};
        listen ${MG_NGINX_MQTTS_PORT} ;
        listen [::]:${MG_NGINX_MQTTS_PORT} ;

        # include snippets/ssl.conf;

        proxy_pass mqtt_cluster;
    }
}

arvindh123 · 2024-05-14T08:17:55Z

docker/docker-compose.yml

+      MG_MQTT_ADAPTER_WS_KEY_FILE: ${MG_MQTT_ADAPTER_WS_KEY_FILE:+/magistrala-mqtt-adapter.key}
+      MG_MQTT_ADAPTER_WS_SERVER_CA_FILE: ${MG_MQTT_ADAPTER_WS_SERVER_CA_FILE:+/magistrala-ca.crt}
+      MG_MQTT_ADAPTER_WS_CLIENT_CA_FILE: ${MG_MQTT_ADAPTER_WS_CLIENT_CA_FILE:+/magistrala-client-ca.key}
+      MG_MQTT_ADAPTER_WS_CERT_VERIFICATION_METHODS: ${MG_MQTT_ADAPTER_WS_CERT_VERIFICATION_METHODS:-}


Suggested change

MG_MQTT_ADAPTER_WS_CERT_VERIFICATION_METHODS: ${MG_MQTT_ADAPTER_WS_CERT_VERIFICATION_METHODS:-}

MG_MQTT_ADAPTER_WS_CERT_FILE: ${MG_MQTT_ADAPTER_WS_CERT_FILE:+/magistrala-mqtt-adapter-ws.crt}

MG_MQTT_ADAPTER_WS_KEY_FILE: ${MG_MQTT_ADAPTER_WS_KEY_FILE:+/magistrala-mqtt-adapter-ws.key}

MG_MQTT_ADAPTER_WS_SERVER_CA_FILE: ${MG_MQTT_ADAPTER_WS_SERVER_CA_FILE:+/magistrala-ca-ws.crt}

MG_MQTT_ADAPTER_WS_CLIENT_CA_FILE: ${MG_MQTT_ADAPTER_WS_CLIENT_CA_FILE:+/magistrala-client-ca-ws.key}

arvindh123 · 2024-05-14T09:38:32Z

docker/ssl/certs/thing.crt

+-----BEGIN CERTIFICATE-----
+MIIEaDCCA1ACFAeBgoEhMA0RBjK9wPoCUZMOJqt0MA0GCSqGSIb3DQEBCwUAMHUx
+IjAgBgNVBAMMGU1hZ2lzdHJhbGFfU2VsZl9TaWduZWRfQ0ExEzARBgNVBAoMCk1h
+Z2lzdHJhbGExFjAUBgNVBAsMDW1hZ2lzdHJhbGFfY2ExIjAgBgkqhkiG9w0BCQEW
+E2luZm9AbWFnaXN0cmFsYS5jb20wHhcNMjQwNDMwMTExOTM4WhcNMjYwNDMwMTEx
+OTM4WjBsMRgwFgYDVQQDDA88VEhJTkdfU0VDUkVUPiAxEzARBgNVBAoMCk1hZ2lz
+dHJhbGExFzAVBgNVBAsMDm1hZ2lzdHJhbGFfY3J0MSIwIAYJKoZIhvcNAQkBFhNp
+bmZvQG1hZ2lzdHJhbGEuY29tMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKC
+AgEAvdpn5W14xTNDtcxy8RjGCpQ5cHR1wfILN57Mwx4nv+YNrwmW8mn3Bn2+w0rT
+lkNelYSgA7KlkO9aayRyDKuPJYb26rMITnizasxiTl+0PFnEv1E5KhzvoCM7lern
+0/bSoeW04tM+vGYWZq+LWO7s5xE1veefwGiYqZjYBdltCdtvID8zgc8OrgB7ZztP
+1jRHhpssEBHQK0dZeWaHpHnfeHK2fYc4ih0fW6Xvr+ziIbCY8popCu5Y3xU18E7b
+pgs3mFpdytWar968aR0U3dYi3f5vWoq+BOu3UL67nqkGg0dHtdPnBlu4zyQ4TxAr
+KUDQwsdd4HotvYNl09d6MvCMygtTT1mfTiScStHuwvWIiU5mI8PnFeYMaLmScKgj
+bA812ak9jClCFE8gEgKWW35z9Bv0e/ZjHqX+98HRfw7S9icWTcoaUwOcbbnb2enx
+niWpQBZzgz0J2NJn5MxCPFUzUn1WCqlWC6FGfVve3CLPCJoiF77IVSZJ+rNrwyxW
+LcefQigGOuZ7NLg5Y7O2LD/xXxPkNlVail8oklaQk1c/bHd6YPiG0/9uKz2NcCL9
+txFl6errRfPl1c/5ACvG69FYXADSvMZ8pPesTGLAag7JGj8pxLyLFYaLnjlbvleO
+RtHCnIKcwYW8AjYiuuRBtDiP7W61CiuqpT12QZCTvPSerg0CAwEAATANBgkqhkiG
+9w0BAQsFAAOCAQEATZfofPEARNP+Hysv6bWKTsFr8HUOWE2OgEPEOMHnIQ5zL9RV
+Lyt1sY62DjDXibirnM55F8cM1gBrKSrfelshClTNRxKc2/wsnZwjOepLb5xZ14Z2
+zdEJKS2IOtdQkxV8uDfu2wdBTOC9AyhIdDITSUwD5M7TqSegemIceZ8QU07rNqJ
+AqYTrsqVVks9+b50ggjkYmEjhiukfzpidwNRax//k5/gRXNJBCvn6oIXlbB0wgpY
+Zqomsxby3t2c71YN/edd5y3kaM3FN21stPkqVFEM/SQTNctiw6ZQcMIgyvqZn3YR
+U8NaiWc7xDTKY7R/U5SZBkuqPpBVeDKoCFruvg==
+-----END CERTIFICATE-----


Remove this file

Suggested change

-----BEGIN CERTIFICATE-----

MIIEaDCCA1ACFAeBgoEhMA0RBjK9wPoCUZMOJqt0MA0GCSqGSIb3DQEBCwUAMHUx

IjAgBgNVBAMMGU1hZ2lzdHJhbGFfU2VsZl9TaWduZWRfQ0ExEzARBgNVBAoMCk1h

Z2lzdHJhbGExFjAUBgNVBAsMDW1hZ2lzdHJhbGFfY2ExIjAgBgkqhkiG9w0BCQEW

E2luZm9AbWFnaXN0cmFsYS5jb20wHhcNMjQwNDMwMTExOTM4WhcNMjYwNDMwMTEx

OTM4WjBsMRgwFgYDVQQDDA88VEhJTkdfU0VDUkVUPiAxEzARBgNVBAoMCk1hZ2lz

dHJhbGExFzAVBgNVBAsMDm1hZ2lzdHJhbGFfY3J0MSIwIAYJKoZIhvcNAQkBFhNp

bmZvQG1hZ2lzdHJhbGEuY29tMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKC

AgEAvdpn5W14xTNDtcxy8RjGCpQ5cHR1wfILN57Mwx4nv+YNrwmW8mn3Bn2+w0rT

lkNelYSgA7KlkO9aayRyDKuPJYb26rMITnizasxiTl+0PFnEv1E5KhzvoCM7lern

0/bSoeW04tM+vGYWZq+LWO7s5xE1veefwGiYqZjYBdltCdtvID8zgc8OrgB7ZztP

1jRHhpssEBHQK0dZeWaHpHnfeHK2fYc4ih0fW6Xvr+ziIbCY8popCu5Y3xU18E7b

pgs3mFpdytWar968aR0U3dYi3f5vWoq+BOu3UL67nqkGg0dHtdPnBlu4zyQ4TxAr

KUDQwsdd4HotvYNl09d6MvCMygtTT1mfTiScStHuwvWIiU5mI8PnFeYMaLmScKgj

bA812ak9jClCFE8gEgKWW35z9Bv0e/ZjHqX+98HRfw7S9icWTcoaUwOcbbnb2enx

niWpQBZzgz0J2NJn5MxCPFUzUn1WCqlWC6FGfVve3CLPCJoiF77IVSZJ+rNrwyxW

LcefQigGOuZ7NLg5Y7O2LD/xXxPkNlVail8oklaQk1c/bHd6YPiG0/9uKz2NcCL9

txFl6errRfPl1c/5ACvG69FYXADSvMZ8pPesTGLAag7JGj8pxLyLFYaLnjlbvleO

RtHCnIKcwYW8AjYiuuRBtDiP7W61CiuqpT12QZCTvPSerg0CAwEAATANBgkqhkiG

9w0BAQsFAAOCAQEATZfofPEARNP+Hysv6bWKTsFr8HUOWE2OgEPEOMHnIQ5zL9RV

Lyt1sY62DjDXibirnM55F8cM1gBrKSrfelshClTNRxKc2/wsnZwjOepLb5xZ14Z2

+zdEJKS2IOtdQkxV8uDfu2wdBTOC9AyhIdDITSUwD5M7TqSegemIceZ8QU07rNqJ

AqYTrsqVVks9+b50ggjkYmEjhiukfzpidwNRax//k5/gRXNJBCvn6oIXlbB0wgpY

Zqomsxby3t2c71YN/edd5y3kaM3FN21stPkqVFEM/SQTNctiw6ZQcMIgyvqZn3YR

U8NaiWc7xDTKY7R/U5SZBkuqPpBVeDKoCFruvg==

-----END CERTIFICATE-----

arvindh123 · 2024-05-14T09:38:51Z

docker/ssl/certs/thing.key

+-----BEGIN PRIVATE KEY-----
+MIIJQQIBADANBgkqhkiG9w0BAQEFAASCCSswggknAgEAAoICAQC92mflbXjFM0O1
+zHLxGMYKlDlwdHXB8gs3nszDHie/5g2vCZbyafcGfb7DStOWQ16VhKADsqWQ71pr
+JHIMq48lhvbqswhOeLNqzGJOX7Q8WcS/UTkqHO+gIzuV6ufT9tKh5bTi0z68ZhZm
+r4tY7uznETW955/AaJipmNgF2W0J228gPzOBzw6uAHtnO0/WNEeGmywQEdArR1l5
+Zoeked94crZ9hziKHR9bpe+v7OIhsJjymikK7ljfFTXwTtumCzeYWl3K1Zqv3rxp
+HRTd1iLd/m9air4E67dQvrueqQaDR0e10+cGW7jPJDhPECspQNDCx13gei29g2XT
+13oy8IzKC1NPWZ9OJJxK0e7C9YiJTmYjw+cV5gxouZJwqCNsDzXZqT2MKUIUTyAS
+ApZbfnP0G/R79mMepf73wdF/DtL2JxZNyhpTA5xtudvZ6fGeJalAFnODPQnY0mfk
+zEI8VTNSfVYKqVYLoUZ9W97cIs8ImiIXvshVJkn6s2vDLFYtx59CKAY65ns0uDlj
+s7YsP/FfE+Q2VVqKXyiSVpCTVz9sd3pg+IbT/24rPY1wIv23EWXp6utF8+XVz/kA
+K8br0VhcANK8xnyk96xMYsBqDskaPynEvIsVhoueOVu+V45G0cKcgpzBhbwCNiK6
+5EG0OI/tbrUKK6qlPXZBkJO89J6uDQIDAQABAoICAFwCCdH9cMcZJhpn/9hqKigc
+V43xUtRwShFli1SQO7Fbe4w1uIq44feqXxXkIJC1C5SolqfUHzsTVJ2SJxeO2SvV
+sbpik4b8Je9/J7FV3lJVKu2RHoQTS+ouyQTJuBpYhxHg8f3vZHsHepm/P2gVOoht
+Jb9oDlrAFiQzJ5YGHjrltoBJQNwLDdEz+Jo5+2ITKgrJDtym1Vz0Eh+smdYTHkMQ
+TPKe4r7EiFrxGy0GPz0WibkhDvz2L+nFLMPOY9KfowzZ1pSjWnkbF/JVHrS3v2QY
+l7yoepiF5+xg0yp8PeTHWbB8Bn1lXDY/Fm7ypggzGjkbR5qve5VZpbK5vcYMIB5B
+FSVERwNho8d665uWF9aFw56OxSFSWewb3ddN4FY5Efo/0dGtUt02HyFvuVLI5xbP
+rEvZHGvrHyqzY9uqtHTdEgJgYfz05DoxRoXEgktQrdo01UxJwh/Xc0Rsh6Z/I0mF
+3GWdqOodysrm+9PI5WLGDZmu3e6dXysmhjE0oMwQiAbb43LJwpR1XBPfc08vhEat
+YMnafd4dsxbAPyTqLPzTg+2ovkA5VLWDYarUDNdzEvrcUalEUA6wTEN1J8WITjGQ
+8t4Fqfkv669ZEYQNG3p9VP3Wm3WtUluh3vDEPWI06QMnnGLrI1M+FJXtQlbgb+aG
+kUxwTr2GvpHg0ISwsaV9AoIBAQDfSw/l4iYSIf9YFD1uFoCIiHu2VWiXXWhHElC7
+31mFBNoP7TQ/J6u7Bp9nody77ycJZpVYm0DPqn5tHH+gXopS/oEK7Kpxgdaqhu28
+J4NVGVu4P5FiUye0Fza/96TtaOkxPa4bg+bRMDN35XGUYE0KwW4WPMk+20IhcmP2
+sn2uSFLqhjCDnuVxCK8kZGt7cEtIFEvI6eckhTcD764n5okL+Aj/RqyuiAN4y7QQ
+EiepayoFYXS1gFATecGOTqTpeWGuKThQPgY6t3OLGc1b6WpZt13yxv/TLA/rg20l
+67f8c054mkUdzwtwavmHNmTltDb4cocJB8k66Oy8rFV9EcRLAoIBAQDZqWzDXQRo
+mYLR2qmgBH/PJFmRVOb/lj7DKtiuW8CjCASOuvGQnU1FSsWHoO5TyetpuXhXabsy
+uz4wT4soloTyqIRfAEjt6obEI/LN2THiGYdpiA9cvtods6LOQFzhzLYRx79lUGic
+CwfvivgcoRROsQRZFSunnRQ7uBpDl3JpYGsR3BlFjPzMbdkd3M1saysLZ0TN8H3d
+jqmd6V8LZka94ulXjC+KcPxId5AhySvdBrgyXSzdPo80RaHbYu2S7UDjiF/GkIVd
+jpO6aNZ4HmQAwk8YKkUKPN16QpgEdNQuSulYl6CsMY2dnqPCdJlh8NCMjkUfkFSS
+DhqpiUwZ1PAHAoIBAFAzUrN75bavlVoacvH+jQ39D4ou5REw0Ipubqb8EtgvJO+B
+gDmK0yiquUhsn660uC2tNckMglvUXfJswP5l1oL2vHyjApkgTG0VZ4AkJQzEvefQ
+lTRer60EhXqvxQIso003XTiuyJGsIWBX/dKQQDuzvAmqrZGxCPCgeH0cIP4fJgDs
+T7KGuNw/hUCp/NzqOW/bMlkoggfg/SChzsiwCdtNrFWcGdvaKH0TlUoMa0hzoDUs
+itHCE4DbQ4UMhDkDiu0WCw7vKCegypRUoSj5DmTI8qqHhU1gzxsFKofrQTsgYzf4
+GQka2332PketZtIGR5Q29n9s138cOCOQNmhkoD8CggEAc5TNRZyyR50k46z6oBdj
+iyqhSVRNafFtmJO7BjOAwtHyIZzRw9pT+vSumcvyYpn5oWW0qq5hkpntdxf2LHJp
+XLXxvMTY9UWxlJwGRhsMqySlTHvMtJJAHr+SN3XkHJ9pG133m6QsOJh++UvRXtqG
+fxbu2SMt0E7QPLoRcE0OmzhxAsXewZkRcLjwLzRd/TGXZPHMjpMjMzCbj1gqeVIx
+RIonIK83W4uN3ClpZxd9bFwyGcRym3Wp87b6MRaCU6Aa8fgr67Q4raqVdcpQAO8a
+aXDFMW7YxIv3jAGq2cr0FTUAXmP+FRInQkjIau0huel3xPuTZyRsVDabH+XhfGIf
+pwKCAQB1CNtcspjw6UMpxVMCsUtcjD2/ze0ipgfisp4ICVdfY2ZaEu5XOP8ZrvqA
+Adc+51uayhXvTM9qH8nK6thAQkHvwi/JyTOsuQfqqdaRbH6ywX8qWm2p+1BsHntP
+4xuaPp4ViEPJwopsn2MDrvcjvpARSgHB42s5m9plnt33gIk1yf2C0RIXiPwMS8eu
+jQrznv2ieXhS1GWE46V+J66Q/m/nojSi1jQQEBLNFvvQc8eMZdiFwGR9uS/GRBPH
+St3HhEffsmT191Shnuvk0y//CpQLF4PeH0UqephH3qVsCJP6gXJcZH6pUgLoDfmI
+/YOhmC7Rw0vBzAcqqblj25mXwfcb
+-----END PRIVATE KEY-----


Remove this file

Suggested change

-----BEGIN PRIVATE KEY-----

MIIJQQIBADANBgkqhkiG9w0BAQEFAASCCSswggknAgEAAoICAQC92mflbXjFM0O1

zHLxGMYKlDlwdHXB8gs3nszDHie/5g2vCZbyafcGfb7DStOWQ16VhKADsqWQ71pr

JHIMq48lhvbqswhOeLNqzGJOX7Q8WcS/UTkqHO+gIzuV6ufT9tKh5bTi0z68ZhZm

r4tY7uznETW955/AaJipmNgF2W0J228gPzOBzw6uAHtnO0/WNEeGmywQEdArR1l5

Zoeked94crZ9hziKHR9bpe+v7OIhsJjymikK7ljfFTXwTtumCzeYWl3K1Zqv3rxp

HRTd1iLd/m9air4E67dQvrueqQaDR0e10+cGW7jPJDhPECspQNDCx13gei29g2XT

13oy8IzKC1NPWZ9OJJxK0e7C9YiJTmYjw+cV5gxouZJwqCNsDzXZqT2MKUIUTyAS

ApZbfnP0G/R79mMepf73wdF/DtL2JxZNyhpTA5xtudvZ6fGeJalAFnODPQnY0mfk

zEI8VTNSfVYKqVYLoUZ9W97cIs8ImiIXvshVJkn6s2vDLFYtx59CKAY65ns0uDlj

s7YsP/FfE+Q2VVqKXyiSVpCTVz9sd3pg+IbT/24rPY1wIv23EWXp6utF8+XVz/kA

K8br0VhcANK8xnyk96xMYsBqDskaPynEvIsVhoueOVu+V45G0cKcgpzBhbwCNiK6

5EG0OI/tbrUKK6qlPXZBkJO89J6uDQIDAQABAoICAFwCCdH9cMcZJhpn/9hqKigc

V43xUtRwShFli1SQO7Fbe4w1uIq44feqXxXkIJC1C5SolqfUHzsTVJ2SJxeO2SvV

sbpik4b8Je9/J7FV3lJVKu2RHoQTS+ouyQTJuBpYhxHg8f3vZHsHepm/P2gVOoht

Jb9oDlrAFiQzJ5YGHjrltoBJQNwLDdEz+Jo5+2ITKgrJDtym1Vz0Eh+smdYTHkMQ

TPKe4r7EiFrxGy0GPz0WibkhDvz2L+nFLMPOY9KfowzZ1pSjWnkbF/JVHrS3v2QY

l7yoepiF5+xg0yp8PeTHWbB8Bn1lXDY/Fm7ypggzGjkbR5qve5VZpbK5vcYMIB5B

FSVERwNho8d665uWF9aFw56OxSFSWewb3ddN4FY5Efo/0dGtUt02HyFvuVLI5xbP

rEvZHGvrHyqzY9uqtHTdEgJgYfz05DoxRoXEgktQrdo01UxJwh/Xc0Rsh6Z/I0mF

3GWdqOodysrm+9PI5WLGDZmu3e6dXysmhjE0oMwQiAbb43LJwpR1XBPfc08vhEat

YMnafd4dsxbAPyTqLPzTg+2ovkA5VLWDYarUDNdzEvrcUalEUA6wTEN1J8WITjGQ

8t4Fqfkv669ZEYQNG3p9VP3Wm3WtUluh3vDEPWI06QMnnGLrI1M+FJXtQlbgb+aG

kUxwTr2GvpHg0ISwsaV9AoIBAQDfSw/l4iYSIf9YFD1uFoCIiHu2VWiXXWhHElC7

31mFBNoP7TQ/J6u7Bp9nody77ycJZpVYm0DPqn5tHH+gXopS/oEK7Kpxgdaqhu28

J4NVGVu4P5FiUye0Fza/96TtaOkxPa4bg+bRMDN35XGUYE0KwW4WPMk+20IhcmP2

sn2uSFLqhjCDnuVxCK8kZGt7cEtIFEvI6eckhTcD764n5okL+Aj/RqyuiAN4y7QQ

EiepayoFYXS1gFATecGOTqTpeWGuKThQPgY6t3OLGc1b6WpZt13yxv/TLA/rg20l

67f8c054mkUdzwtwavmHNmTltDb4cocJB8k66Oy8rFV9EcRLAoIBAQDZqWzDXQRo

mYLR2qmgBH/PJFmRVOb/lj7DKtiuW8CjCASOuvGQnU1FSsWHoO5TyetpuXhXabsy

uz4wT4soloTyqIRfAEjt6obEI/LN2THiGYdpiA9cvtods6LOQFzhzLYRx79lUGic

CwfvivgcoRROsQRZFSunnRQ7uBpDl3JpYGsR3BlFjPzMbdkd3M1saysLZ0TN8H3d

jqmd6V8LZka94ulXjC+KcPxId5AhySvdBrgyXSzdPo80RaHbYu2S7UDjiF/GkIVd

jpO6aNZ4HmQAwk8YKkUKPN16QpgEdNQuSulYl6CsMY2dnqPCdJlh8NCMjkUfkFSS

DhqpiUwZ1PAHAoIBAFAzUrN75bavlVoacvH+jQ39D4ou5REw0Ipubqb8EtgvJO+B

gDmK0yiquUhsn660uC2tNckMglvUXfJswP5l1oL2vHyjApkgTG0VZ4AkJQzEvefQ

lTRer60EhXqvxQIso003XTiuyJGsIWBX/dKQQDuzvAmqrZGxCPCgeH0cIP4fJgDs

T7KGuNw/hUCp/NzqOW/bMlkoggfg/SChzsiwCdtNrFWcGdvaKH0TlUoMa0hzoDUs

itHCE4DbQ4UMhDkDiu0WCw7vKCegypRUoSj5DmTI8qqHhU1gzxsFKofrQTsgYzf4

GQka2332PketZtIGR5Q29n9s138cOCOQNmhkoD8CggEAc5TNRZyyR50k46z6oBdj

iyqhSVRNafFtmJO7BjOAwtHyIZzRw9pT+vSumcvyYpn5oWW0qq5hkpntdxf2LHJp

XLXxvMTY9UWxlJwGRhsMqySlTHvMtJJAHr+SN3XkHJ9pG133m6QsOJh++UvRXtqG

fxbu2SMt0E7QPLoRcE0OmzhxAsXewZkRcLjwLzRd/TGXZPHMjpMjMzCbj1gqeVIx

RIonIK83W4uN3ClpZxd9bFwyGcRym3Wp87b6MRaCU6Aa8fgr67Q4raqVdcpQAO8a

aXDFMW7YxIv3jAGq2cr0FTUAXmP+FRInQkjIau0huel3xPuTZyRsVDabH+XhfGIf

pwKCAQB1CNtcspjw6UMpxVMCsUtcjD2/ze0ipgfisp4ICVdfY2ZaEu5XOP8ZrvqA

Adc+51uayhXvTM9qH8nK6thAQkHvwi/JyTOsuQfqqdaRbH6ywX8qWm2p+1BsHntP

4xuaPp4ViEPJwopsn2MDrvcjvpARSgHB42s5m9plnt33gIk1yf2C0RIXiPwMS8eu

jQrznv2ieXhS1GWE46V+J66Q/m/nojSi1jQQEBLNFvvQc8eMZdiFwGR9uS/GRBPH

St3HhEffsmT191Shnuvk0y//CpQLF4PeH0UqephH3qVsCJP6gXJcZH6pUgLoDfmI

/YOhmC7Rw0vBzAcqqblj25mXwfcb

-----END PRIVATE KEY-----

rodneyosodo · 2024-05-17T11:22:11Z

cmd/http/main.go

 	defSvcHTTPPort = "80"
 	targetHTTPPort = "81"
-	targetHTTPHost = "http://localhost"
 )


Revert the change and have it as a default tarrgetHTTPHost

rodneyosodo · 2024-05-17T11:22:36Z

cmd/http/main.go

-	address := fmt.Sprintf("%s:%s", "", cfg.Port)
-	target := fmt.Sprintf("%s:%s", targetHTTPHost, targetHTTPPort)
-	mp, err := mproxy.NewProxy(address, target, sessionHandler, logger)
+	httpConfig, err := mproxy.NewConfig(env.Options{Prefix: envPrefix})


Specify the default Target to be used

rodneyosodo · 2024-05-17T11:24:07Z

cmd/mqtt/main.go

 		log.Fatalf("failed to load %s configuration : %s", svcName, err)
 	}
-
+	log.Println(cfg)


Remove @nyagamunene

rodneyosodo · 2024-05-22T12:28:55Z

cmd/http/main.go


 func proxyHTTP(ctx context.Context, cfg server.Config, logger *slog.Logger, sessionHandler session.Handler) error {
-	config := mproxy.Config{
+	httpConfig := mproxy.Config{


Let's change to something like to be able to load TLS configs too

envPrefixMProxy = "MG_HTTP_ADAPTER_MPROXY_" httpConfig, err := mproxy.NewConfig(env.Options{Prefix: envPrefixMProxy}) if err != nil { return err } if httpConfig.Address == "" { httpConfig.Address = fmt.Sprintf("%s:%s", "", cfg.Port) } if httpConfig.Target == "" { httpConfig.Target = fmt.Sprintf("%s:%s", targetHTTPHost, targetHTTPPort) }

This applies to all adapters using mproxy and update the environment variables accordingly

rodneyosodo · 2024-05-22T12:29:48Z

cmd/http/main.go

+	go func() {
+		errCh <- mp.Listen(ctx)
+	}()
+	logger.Info(fmt.Sprintf("%s service https server listening at %s:%s with TLS cert %s and key %s", svcName, cfg.Host, cfg.Port, cfg.CertFile, cfg.KeyFile))


change this message to be different it tls config is not loaded

rodneyosodo · 2024-05-22T12:30:19Z

docker/.env

+MG_HTTP_ADAPTER_ADDRESS=:80
+MG_HTTP_ADAPTER_PREFIX_PATH=/
+MG_HTTP_ADAPTER_TARGET=http://localhost:81
+# MG_HTTP_ADAPTER_CERT_FILE=./ssl/certs/magistrala-server.crt


Why do we have them commented out?

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

dborovcanin · 2024-12-04T11:11:43Z

@nyagamunene What's the status of this PR?

nyagamunene self-assigned this Apr 26, 2024

nyagamunene force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch 3 times, most recently from 547e1ad to 52d8828 Compare April 30, 2024 17:37

arvindh123 reviewed May 2, 2024

View reviewed changes

arvindh123 requested changes May 2, 2024

View reviewed changes

WashingtonKK reviewed May 2, 2024

View reviewed changes

arvindh123 requested changes May 2, 2024

View reviewed changes

nyagamunene marked this pull request as ready for review May 2, 2024 14:27

rodneyosodo requested changes May 3, 2024

View reviewed changes

rodneyosodo requested changes May 5, 2024

View reviewed changes

cmd/http/main.go Show resolved Hide resolved

rodneyosodo requested changes May 7, 2024

View reviewed changes

nyagamunene requested a review from rodneyosodo May 7, 2024 11:24

nyagamunene force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch 2 times, most recently from dec4dae to 8c479cc Compare May 9, 2024 08:36

dborovcanin changed the title ~~MG-2126-Update Magistrala messaging with mproxy with TLS~~ MG-2126 - Update Magistrala messaging with mProxy with TLS May 13, 2024

nyagamunene changed the title ~~MG-2126 - Update Magistrala messaging with mProxy with TLS~~ MG-2126 - Update messaging with mProxy TLS May 13, 2024

nyagamunene force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch from 8c479cc to e991b72 Compare May 13, 2024 12:02

arvindh123 requested changes May 14, 2024

View reviewed changes

dborovcanin force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch from e991b72 to 898644d Compare May 14, 2024 11:44

nyagamunene force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch 2 times, most recently from a67df6e to 4db5f11 Compare May 15, 2024 08:09

dborovcanin mentioned this pull request May 15, 2024

Refactor CoAP adapter #1144

Open

rodneyosodo requested changes May 17, 2024

View reviewed changes

nyagamunene force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch 2 times, most recently from eda7ad5 to cf69257 Compare May 21, 2024 11:45

rodneyosodo requested a review from felixgateru May 22, 2024 10:14

rodneyosodo requested changes May 22, 2024

View reviewed changes

nyagamunene force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch from cf69257 to 41e527b Compare May 23, 2024 09:46

nyagamunene added 27 commits June 26, 2024 19:25

Added interpolation in docker compose

0eb54f6

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

TLS Testing

fe01854

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

TLS Testing

a892198

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

TLS Configuration changed

4485fdd

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Updated documentation

592b4cb

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Removed panic

43b3a0a

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Upgraded Mproxy

2a8bcd0

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Updated the test

7a0ebec

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Linted code

182a89c

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Go mod tidy

e70f7f6

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Changed all services to v11

31487ae

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Undo change in make file

7f40d4c

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Refactored the variables

6c006ba

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Refactored the environment variables

5cd72f2

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Reduced redundancy

c6b4e50

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Reverted the env and main.go files

cbe5210

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

linted code

4e35322

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Updated nginx config file

ef8cdef

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

changed http configuration

1d2c0d7

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

updated http

4b8cdf2

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

updated http config settings

ccb1770

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Updated env file

0b27cf9

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Corrected logging message

57a0dd1

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Corrected logging message

29a2a99

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Updated the env file

ed0cf3b

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Updated the env file

227db9c

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

Updated docker compose file

c3225b7

Signed-off-by: nyagamunene <stevenyaga2014@gmail.com>

nyagamunene force-pushed the MG-2126-UpdateMessagingWithMproxyTLS branch from fb94b8a to c3225b7 Compare June 26, 2024 16:25

dborovcanin requested a review from a team as a code owner December 29, 2025 09:41

	MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHOD: ${MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHOD}
	MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHODS: ${MG_MQTT_WS_ADAPTER_CERT_VERIFICATION_METHODS}

+## MPROXY MQTT WS
+MG_MQTT_ADAPTER_WS_ADDRESS=:8080
+MG_MQTT_ADAPTER_WS_TARGET=ws://${MG_MQTT_BROKER_TYPE}:8080/mqtt
+MG_MQTT_ADAPTER_WS_PREFIX_PATH=/mqtt
+# MG_MQTT_ADAPTER_WS_CERT_FILE=./ssl/certs/magistrala-server.crt
+# MG_MQTT_ADAPTER_WS_KEY_FILE=./ssl/certs/magistrala-server.key
+# MG_MQTT_ADAPTER_WS_SERVER_CA_FILE=./ssl/certs/ca.crt
+# MG_MQTT_ADAPTER_WS_CLIENT_CA_FILE=./ssl/certs/ca.crt
+# MG_MQTT_ADAPTER_WS_CERT_VERIFICATION_METHODS=ocsp
+# MG_MQTT_ADAPTER_WS_OCSP_RESPONDER_URL=http://localhost:8080/ocsp

+MG_MQTT_ADAPTER_CERT_FILE=./ssl/magistrala-server.crt
+MG_MQTT_ADAPTER_KEY_FILE=./ssl/magistrala-server.key
+MG_MQTT_ADAPTER_SERVER_CA_FILE=./ssl/certs/ca.crt
+MG_MQTT_ADAPTER_CLIENT_CA_FILE=./ssl/certs/ca.crt

-      # Things gRPC mTLS client certificates
+      # MQTT Adapter certificates
+      - type: bind
+        source: ${MG_MQTT_ADAPTER_CERT_FILE:-./ssl/certs/dummy/server_cert}
+        target: /magistrala-mqtt-adapter${MG_MQTT_ADAPTER_CERT_FILE:+.crt}
+        read_only: true
+        bind:
+          create_host_path: true
+      - type: bind
+        source: ${MG_MQTT_ADAPTER_KEY_FILE:-./ssl/certs/dummy/server_key}
+        target: /magistrala-mqtt-adapter${MG_MQTT_ADAPTER_KEY_FILE:+.key}
+        read_only: true
+        bind:
+          create_host_path: true
+      - type: bind
+        source: ${MG_MQTT_ADAPTER_SERVER_CA_FILE:-./ssl/certs/dummy/server_ca}
+        target: /magistrala-ca${MG_MQTT_ADAPTER_SERVER_CA_FILE:+.crt}
+        read_only: true
+        bind:
+          create_host_path: true
+      - type: bind
+        source: ${MG_MQTT_ADAPTER_CLIENT_CA_FILE:-./ssl/certs/dummy/client_ca}
+        target: /magistrala-client-ca${MG_MQTT_ADAPTER_CLIENT_CA_FILE:+.key}
+        read_only: true
+        bind:
+          create_host_path: true

	envPrefixHTTP = "MG_HTTP_ADAPTER_"
	envPrefix = "MG_HTTP_ADAPTER_"

-      MG_MQTT_ADAPTER_WS_CERT_VERIFICATION_METHODS: ${MG_MQTT_ADAPTER_WS_CERT_VERIFICATION_METHODS:-}
+      MG_MQTT_ADAPTER_WS_CERT_FILE: ${MG_MQTT_ADAPTER_WS_CERT_FILE:+/magistrala-mqtt-adapter-ws.crt}
+      MG_MQTT_ADAPTER_WS_KEY_FILE: ${MG_MQTT_ADAPTER_WS_KEY_FILE:+/magistrala-mqtt-adapter-ws.key}
+      MG_MQTT_ADAPTER_WS_SERVER_CA_FILE: ${MG_MQTT_ADAPTER_WS_SERVER_CA_FILE:+/magistrala-ca-ws.crt}
+      MG_MQTT_ADAPTER_WS_CLIENT_CA_FILE: ${MG_MQTT_ADAPTER_WS_CLIENT_CA_FILE:+/magistrala-client-ca-ws.key}

Conversation

nyagamunene commented Apr 26, 2024

What type of PR is this?

What does this do?

Which issue(s) does this PR fix/relate to?

Have you included tests for your changes?

Did you document any new/modified feature?

Notes

Uh oh!

arvindh123 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arvindh123 May 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arvindh123 left a comment

Choose a reason for hiding this comment

Uh oh!

arvindh123 May 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rodneyosodo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arvindh123 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dborovcanin commented Dec 4, 2024

Uh oh!

Reviewers

Assignees

Labels

arvindh123 left a comment •

edited

Loading

arvindh123 May 2, 2024 •

edited

Loading

arvindh123 May 2, 2024 •

edited

Loading