Skip to content

Regular Expressions in Data Conversion returns the Match String instead of the Group String #2064

@Maxim-Voelker

Description

@Maxim-Voelker

Describe the bug
When a MQTT Message is sent to a Gateway and it gets to the Data Conversion in the Data Mapping of the Connector, the Regular Expression intended to get the Device Name and Profile from the topic of the Message returns the Match to the Regular Expression instead of the Group. This also goes against what is outlined in the "Regular expression for topic examples" found in ThingsBoard.
See "To Reproduce" and "Screenshots"

Your Server Environment

  • own setup
    • Deployment: monolith
    • Deployment type: docker-compose
    • ThingsBoard Version: 4.3.0
    • ThingsBoard Gateway Version: 3.8.0
    • Community Edition
    • Ubuntu 24.04 LTS

Your Client Environment
Desktop:

  • OS: Ubuntu 24.04 LTS
  • Firefox 147.0.3 64 bit

Your Device

Emulated Device Data sent to a Gateway with a MQTT Connector

To Reproduce
Steps to reproduce the behavior:

  1. Go to your Gateway
  2. Click on your Gateway
  3. Click on Connectors Configuration
  4. Click on your Connector
  5. Click on Data Mapping
  6. Click on your Data Mapping
  7. Set Topic Filter to "/devices/AM123/mytype/data"
  8. Scroll down to Data Conversion
  9. Set both Sources to Topic
  10. Set the Name Expression to "/devices/([^/]+)/mytype/data"
  11. Set the Profile Expression to "/devices/[A-Z0-9]+/([^/]+)/data"
  12. Apply the Data Mapping
  13. Send this MQTT Message to the Connector
    "mosquitto_pub -h localhost -p 1884 -t /devices/AM123/mytype/data -m '{"Testdata":{"Testdata":"Testvalue"}"
    The Gateway generates the Device using the Name and Profile filtered from the Topic using the Regular Expressions.
    The Name however becomes: "/devices/AM123/mytype/data"and the Profile also becomes: "/devices/AM123/mytype/data"

Expected behavior
The Name should actually be "AM123" and the Profile should be "mytype".
This is also outlined in Thingsboard itself, see Screenshots.

Screenshots
Here the Tutorial provided in ThingsBoard itself:
Image

Here the Topic Filter:
Image

Here the Data Conversion:
Image

Here the MQTT Message:
mosquitto_pub -h localhost -p 1884 -t /devices/AM123/mytype/data -m '{"values":{"testvalue":10}}'

And here the Device created by the Gateway:
Image

As we can see it used the Regular Expression Match as the Name and Profile instead of the Regular Expression Group.

Additional context
So essentially, the Regular Expression returns the Match instead of the Group:

string: "/devices/AM123/mytype/data"

Regular Expression: "/devices/([^/]+)/mytype/data"

Match: "/devices/AM123/mytype/data"

Group: "AM123"

Expected Output: "AM123"

Actual Output: "/devices/AM123/mytype/data"

I therefore believe that ThingsBoard mistakenly returns the Match instead of the Group.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions