Group delete event 2

[jesperhodge]

Description

When an EnterpriseGroup gets deleted, we want to send a message to the Openedx Event Bus that is read by enterprise-access to delete the corresponding PolicyGroupAssociation.

Deployment instructions

When rolling this out with edx-platform, delete some enterprisegroup on stage to confirm there are no errors. The Event Bus signal will not be sent without the edx-internal config, but it's important that there's no error even without that config.

Links

Internal 2U ticket: https://2u-internal.atlassian.net/browse/ENT-10440

Dependencies

This depends on openedx/openedx-events#504
and on #2431, which is planned to be merged on August 11.

To Dos

• Wait for chore: Upgrade Python requirements #2431
• Bump version and changelog

Implemented in this PR as of 06/11/25:

• Producer for EventBus message is defined
• EventBus message is defined as schema in corresponding PR in openedx-events
• Defined a django signal for enterprise group deletion that triggers the producer to send an eventbus message

Setup Instructions for Kafka for Enterprise Group Deleted Event

Follow this for testing and for local development. It's a number of steps, but if you follow this closely it should not be too hard to set up.

Reference Setup

Let's follow a reference setup to first ensure that Kafka is working properly on your
local machine. For this, set up enterprise-access and enterprise-subsidy repos. Their readmes
contain instructions on how to set up Kafka in docker.

Follow the instructions to run your kafka docker container, make sure that the Confluence control center is running on localhost:9021/clusters/, and produce / consume the ping test event described in the readmes. Verify that the event shows up in the consumer logs and in the Confluence control center in the dev-enterprise-core topic. (You need to open the topic page -> messages first and then emit the event afterwards in order to see it, as when you open the page / refresh it removes previous logs.)

The docker container seems to not stay online indefinitely, it may have an auto-shutdown or something. So you may have to re-run make dev.up.with-events sometimes from the enterprise-subsidy and enterprise-access folders.

Edx-Enterprise / LMS Config

edx-enterprise is a library installed in LMS. As such, to get Kafka to work, you need to configure it locally in LMS. So paste this code into your edx-platform/lms/envs/private.py (create if the file doesn't exist):

## Depending on your setup, the two lines below may need to be commented out or not
# from .devstack import INSTALLED_APPS, LOGGING
# INSTALLED_APPS += ['edx_event_bus_kafka']

EVENT_BUS_PRODUCER = 'edx_event_bus_kafka.create_producer'
EVENT_BUS_CONSUMER = 'edx_event_bus_kafka.KafkaEventConsumer'
EVENT_BUS_TOPIC_PREFIX = 'dev'
EVENT_BUS_KAFKA_SCHEMA_REGISTRY_URL = 'http://edx.devstack.schema-registry:8081'
EVENT_BUS_KAFKA_BOOTSTRAP_SERVERS = 'edx.devstack.kafka:29092'

EVENT_BUS_PRODUCER_CONFIG = {
    'org.openedx.enterprise.enterprise_group.deleted.v1': {
        'enterprise-core': {
            'event_key_field': 'enterprise_group.uuid',
            'enabled': True,
        },
    },
}

Install local libraries

To run your edx-enterprise consumer, LMS depends on edx_enterprise for the consumer code
and openedx_events for the event schema. So you need the code from both PRs locally.

• Make sure to clone both repos locally into your edx /src folder if you haven't already
• Check out the branches from the PRs
• go to devstack -> make lms-shell
• pip install -e /edx/src/edx-enterprise
• pip install openedx-events==10.4.0 (or newer)

You will also need to install two other libraries as dependencies into your lms-shell:

• pip install edx-event-bus-kafka
• pip install pip install confluent-kafka[avro,schema-registry]

Generate Avro schemas if necessary

Only necessary if you make changes to openedx-events:

If you're in the middle of development and have made any changes to the signals or data in openedx-events, or added a new signal to openedx-events, you need to regenerate the avro schema by running the generate_avro_schemas <your-signal-name> management command that can be found in openedx-signals.

Test Signals locally

• Make sure your Confluent control center is running by visiting localhost:9021/clusters/. If not, go to enterprise-access and enterprise-subsidy folders and just do make dev.up.with-events again.
• Let's do a first test to see if the events work from LMS side in general:
• Go to lms-shell
• Run ./manage.py lms produce_enterprise_ping_event.
• This should log a 200 and "Message delivered to Kafka event bus", and you should see the message pop up on the Confluent topic page (you need to emit the event after you open that page for it to show up).
• If this doesn't work as expected, double check all the steps above and get this to work before continuing. Make sure you have the libraries installed in lms-shell, the right private.py config, Kafka container running, etc. Perhaps double check that the test signal between enterprise-access and enterprise-subsidy works, to narrow down at which step things went wrong.
• Now, let's test the signal you created in openedx-events. To do this, you can run:
  manage.py lms produce_event --signal openedx_events.enterprise.signals.ENTERPRISE_GROUP_DELETED --topic enterprise-core --key-field enterprise_group.uuid --data '{"enterprise_group": { "uuid": "d509a63b-eb11-4dc8-8b84-4688c7335110" }}'.
• This is checking that things work on the openedx-events side of things, and does not interact with the consumer in edx-enterprise.
• Lastly, let's test the consumer together with this. I created a test command for this: ./manage.py lms produce_event_bus_group_delete_test_event
• That calls the actual producer code. It uses some mock uuid though as data that I hardcoded into the producer function - that will need to be replaced with proper code.
• If this works, you are all set and can go from here. Next up is implementing the To Dos defined in the JIRA task.

Merge checklist:

• Any new requirements are in the right place (do not manually modify the requirements/*.txt files)
  • base.in if needed in production but edx-platform doesn't install it
  • test-master.in if edx-platform pins it, with a matching version
  • make upgrade && make requirements have been run to regenerate requirements
• make static has been run to update webpack bundling if any static content was updated
• ./manage.py makemigrations has been run
  • Checkout the Database Migration Confluence page for helpful tips on creating migrations.
  • Note: This must be run if you modified any models.
    • It may or may not make a migration depending on exactly what you modified, but it should still be run.
  • This should be run from either a venv with all the lms/edx-enterprise requirements installed or if you checked out edx-enterprise into the src directory used by lms, you can run this command through an lms shell.
    • It would be ./manage.py lms makemigrations in the shell.
• Version bumped
• Changelog record added
• Translations updated (see docs/internationalization.rst but also this isn't blocking for merge atm)

Post merge:

• Tag pushed and a new version released
  • Note: Assets will be added automatically. You just need to provide a tag (should match your version number) and title and description.
• After versioned build finishes in GitHub Actions, verify version has been pushed to PyPI
  • Each step in the release build has a condition flag that checks if the rest of the steps are done and if so will deploy to PyPi.
    (so basically once your build finishes, after maybe a minute you should see the new version in PyPi automatically (on refresh))
• PR created in edx-platform to upgrade dependencies (including edx-enterprise)
  • Trigger the 'Upgrade one Python dependency' action against master in edx-platform with new version number to generate version bump PR
  • This must be done after the version is visible in PyPi as make upgrade in edx-platform will look for the latest version in PyPi.
  • Note: the edx-enterprise constraint in edx-platform must also be bumped to the latest version in PyPi.

[katrinan029]

Implementation looks good to me. Some minor details:

• include change log and bump version
• since openedx-events is on 10.4.0 https://pypi.org/project/openedx-events/, will this cause an issue with the ENTERPRISE_GROUP_DELETED signal?

[jesperhodge]

Implementation looks good to me. Some minor details:

• include change log and bump version
• since openedx-events is on 10.4.0 https://pypi.org/project/openedx-events/, will this cause an issue with the ENTERPRISE_GROUP_DELETED signal?

@katrinan029 10.4.0 is the right version where I added it, so this works fine after the branch has been rebased to include Hamzah's upgrade request that sets edx-enterprise's openedx-events version to 10.4.0

[jesperhodge]

I manually tested this and can confirm:

When lacking Kafka / Event Bus settings, this code will not produce an uncaught exception.

Thus, we can merge this PR as-is without depending on external config.
FYI @katrinan029

[jesperhodge]

This is on the correct version of openedx-events now. @katrinan029

[jesperhodge]

@katrinan029 Here's the video of the event working, and two screenshots for the contents of the event message.

This requires some explanation:

• I have logged into a Django shell in LMS.
• In the terminal, I delete an EnterpriseGroup.
• The second screen I show is the Kafka control center where you can see the EventBus.
• I show that the topic is correct: dev-enterprise-groups-lifecycle
• You can see the incoming message after the group deletion
• In the two screenshots, you can see the key for the message - kind of a message id - that's set to the group uuid, and you can see the data the message holds, which is "enterprise_group" keyword and its uuid.
• This shows that when I delete an EnterpriseGroup, an event will be automatically sent to the EventBus, and it's picked up in the EventBus in the correct topic.

Screen.Recording.2025-08-12.at.5.28.04.PM.mov