Thoughts on long-term architecture and scalability

[dmamontov]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
• Please do not modify this template :) and fill in all the required fields.

Content

Thank you for the great work — among the solutions currently available on the market, Dify feels like one of the strongest and most well-thought-out ones.

That said, I’d like to share an idea and open a discussion. At the moment, Dify is gradually evolving into a fairly large monolith (excluding separate services like plugins, etc.). In the long run, this could make it harder to introduce new functionality, increase code coupling, and potentially lead to more bugs.

This is a subjective opinion (no offense intended), but Python, for a project of this scale, may eventually become a bottleneck in terms of performance and resource usage. Similarly, Celery does not always scale efficiently under higher loads.

Based on this, an idea came up: to consider gradually extracting parts of the core functionality into separate microservices, for example, implemented in Go. This approach could bring multiple benefits — improved performance, more predictable behavior under load, and the opportunity to refactor existing bottlenecks in a more isolated and controlled way.

Personally, I’m willing to get involved in this effort at an implementation level, and I can also bring in a couple of additional engineers if needed. The goal would be to help make Dify even more robust and competitive.

P.S. I have some thoughts on the architecture and hands-on development experience — happy to share more details if this direction resonates with the team.