How to clean up database when file system is broken?

[lebouquetin]

Note: this is not a bug but an suggestion query

In tracim, we manage file uploads using depot. It works very well actaully.

For some reasons, we try to work with a database which has not the file system part of data, so depot is broken - and it's expected.

What we try to do is to clean the database according to missing files.

Example of what I mean:

for user_file in session.query(UserFile):
    if user_file.uploaded_content.depot.is_invalid():
        session.delete(userf_file)

Unfortunately the query fails because depot fails (because it can't find files on file system) .

What is the right way to manage this case? What I expect is to be able to get the depot file uuid without to try loading data from files then manually check file existence, but it requires to write raw SQL or to use another model mapping.

Maybe there is a more clever solution?

cc @Skylsmoi

[amol-]

I'm probably late to this, but if the issue is that Depot crashes on the delete operation, have you tried disabling session synchronization during the delete query? https://docs.sqlalchemy.org/en/21/orm/queryguide/dml.html#selecting-a-synchronization-strategy

Also, delete operation in depot are meant to succeed if the file doesn't already exist, what's the error you are seeing?