You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/cdp/cdp_activation/cdp_data_export.md
+39Lines changed: 39 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -133,3 +133,42 @@ Finally, you can specify the audiences you wish to include.
133
133
!!! note "CDP requests"
134
134
135
135
All CDP requests are logged in with `debug` severity.
136
+
137
+
### [[= product_name_base =]] Messenger support for large batches of data
138
+
139
+
CDP uses [[= product_name_base =]] Messenger to process incoming data from [Raptor](www.raptorservices.com/).
140
+
This approach improves performance and reliability when processing large amounts of CDP user records.
141
+
For more information, see [Background tasks: How it works](background_tasks.md#how-it-works).
142
+
143
+
By using Messenger while working with large batches of data, requests are queued instead of being processed synchronously:
144
+
145
+
- queuing items starts automatically once a certain number of actions is reached (below this number, items are processed in a single request, using the standard synchronous behavior)
146
+
- every single data is recorded in the database
147
+
- a background worker retrieves records from the queue, processing them one by one or in batches, depending on the [Messenger](https://symfony.com/doc/current/messenger.html) configuration
148
+
- processing happens at set intervals to avoid timeouts and keep the system stable
149
+
150
+
Follow the [database update procedure](https://doc.ibexa.co/en/latest/update_and_migration/from_5.0/update_from_5.0/#database-update) to proceed.
151
+
152
+
1\. Make sure that the transport layer is [defined properly](background_tasks.md#configure-package) in [[= product_name_base =]] Messenger configuration.
153
+
154
+
2\. Add `bulk_async_threshold` setting in the `config/packages/ibexa_cdp.yaml` configuration:
155
+
156
+
```bash
157
+
ibexa_cdp:
158
+
bulk_async_threshold: 100 # Default: 100 items
159
+
```
160
+
161
+
Available options:
162
+
163
+
-`bulk_async_threshold` (integer, default: 100) - minimum number of items required to trigger asynchronous processing
164
+
- below threshold - items are processed immediately in a single request, using the standard synchronous behavior
165
+
- at/above threshold - items are automatically dispatched to the asynchronous queue for background processing
166
+
167
+
3\. Make sure that the [worker starts](background_tasks.md#start-worker) together with the application to watch the transport bus:
## [[= product_name_base =]] Messenger support for large batches of data
71
-
72
-
CDP uses [[= product_name_base =]] Messenger to process data.
73
-
This approach improves performance and reliability when processing large amounts of CDP user records.
74
-
For more information, see [Background tasks: How it works](background_tasks.md#how-it-works).
75
-
76
-
By using Messenger while working with large batches of data, requests are queued instead of being processed synchronously:
77
-
78
-
- queuing items starts automatically once a certain number of actions is reached (below this number, items are processed in a single request, using the standard synchronous behavior)
79
-
- every single data is recorded in the database
80
-
- a background worker retrieves records from the queue, processing them one by one or in batches, depending on the [Messenger](https://symfony.com/doc/current/messenger.html) configuration
81
-
- processing happens at set intervals to avoid timeouts and keep the system stable
82
-
83
-
Messenger requires a dedicated database table.
84
-
This table is not created automatically and must be added manually using a database update script.
85
-
86
-
Follow the [database update procedure](https://doc.ibexa.co/en/latest/update_and_migration/from_5.0/update_from_5.0/#database-update) to proceed.
87
-
88
-
1\. Make sure that the transport layer is [defined properly](background_tasks.md#configure-package) in [[= product_name_base =]] Messenger configuration.
89
-
90
-
2\. Add `bulk_async_threshold` setting in the `config/packages/ibexa_cdp.yaml` configuration:
91
-
92
-
``` bash
93
-
ibexa_cdp:
94
-
bulk_async_threshold: 100 # Default: 100 items
95
-
```
96
-
97
-
Available options:
98
-
99
-
- `bulk_async_threshold` (integer, default: 100) - minimum number of items required to trigger asynchronous processing
100
-
- below threshold - items are processed immediately in a single request, using the standard synchronous behavior
101
-
- at/above threshold - items are automatically dispatched to the asynchronous queue for background processing
102
-
103
-
3\. Make sure that the [worker starts](background_tasks.md#start-worker) together with the application to watch the transport bus:
For more information about deploying the Messenger to production, see [Symfony documentation]([[= symfony_doc =]]/messenger.html#deploying-to-production).
0 commit comments