Skip to content

Commit 62ad3be

Browse files
messenger support in CDP described
1 parent 73ba39e commit 62ad3be

File tree

1 file changed

+44
-1
lines changed

1 file changed

+44
-1
lines changed

docs/cdp/cdp_data_export_schedule.md

Lines changed: 44 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,4 +65,47 @@ php bin/console ibexa:cdp:stream-product-data --help
6565

6666
```bash
6767
php bin/console ibexa:cdp:stream-content-data --help
68-
```
68+
```
69+
70+
## [[= product_name_base =]] Messenger support for large batches of data
71+
72+
CDP uses [[= product_name_base =]] Messenger to process data.
73+
This approach improves performance and reliability when processing large amounts of CDP user records.
74+
For more information, see [Background tasks: How it works](background_tasks.md#how-it-works).
75+
76+
By using Messenger while working with large batches of data, requests are queued instead of being processed synchronously:
77+
78+
- queuing items starts automatically once a certain number of actions is reached (below this number, items are processed in a single request, using the standard synchronous behavior)
79+
- every single data is recorded in the database
80+
- a background worker retrieves records from the queue, processing them one by one or in batches, depending on the [Messenger](https://symfony.com/doc/current/messenger.html) configuration
81+
- processing happens at set intervals to avoid timeouts and keep the system stable
82+
83+
Messenger requires a dedicated database table.
84+
This table is not created automatically and must be added manually using a database update script.
85+
86+
Follow the [database update procedure](https://doc.ibexa.co/en/latest/update_and_migration/from_5.0/update_from_5.0/#database-update) to proceed.
87+
88+
1\. Make sure that the transport layer is [defined properly](background_tasks.md#configure-package) in [[= product_name_base =]] Messenger configuration.
89+
90+
2\. Add `bulk_async_threshold` setting in the `config/packages/ibexa_cdp.yaml` configuration:
91+
92+
``` bash
93+
ibexa_cdp:
94+
bulk_async_threshold: 100 # Default: 100 items
95+
```
96+
97+
Available options:
98+
99+
- `bulk_async_threshold` (integer, default: 100) - minimum number of items required to trigger asynchronous processing
100+
- below threshold - items are processed immediately in a single request, using the standard synchronous behavior
101+
- at/above threshold - items are automatically dispatched to the asynchronous queue for background processing
102+
103+
3\. Make sure that the [worker starts](background_tasks.md#start-worker) together with the application to watch the transport bus:
104+
105+
``` bash
106+
php bin/console messenger:consume ibexa.messenger.transport --bus=ibexa.messenger.bus
107+
```
108+
109+
!!! note "Deploying Symfony Messenger"
110+
111+
For more information about deploying the Messenger to production, see [Symfony documentation]([[= symfony_doc =]]/messenger.html#deploying-to-production).

0 commit comments

Comments
 (0)