When location gets unhidden by publish-by-date-workflow cache isn't purged

[homarx]

ezworkflowenginebundle is installed under ezp6.
A workflow is configured for publishing content by publish date.
When content is published with publish date in future the content gets visibility "hidden". When publish date is reached and workflow is resumed the visibility of the content is set to "visible" in database (ezcontentobject_tree table). In the backend the content is still labeled as "hidden". Moreover the content isn't visible in frontend until cache is purged manually.

[gggeek]

Hello.

Sorry, but I am a bit swamped with work at the moment.
My main dev (and prod) environments are still based on eZPublish 5, so the only testing that the codebase sees on ezplatform is via the automated testing on Travis (and from bundle users of course).
I will try to set aside some time this weekend to take a look.

Thinking about quick-and-dirty workarounds: if you use the migration-bundle version 4.5.0, it is easy to make a call to a SF service as part of a migration/workflow step. Maybe it would be easy to add as extra step just after the "unhide" one a call to "clear cache for this content" ?

[homarx]

Hello Gaetano,

thanks for your feedback.

I tried to set up a further workflow step for clearing cache. As I am not sure how to pass the current location Id as argument I decided to test the purgeAll method first.

The configuration of this step looks like:

-
    type: service
    mode: call
    service: ezpublish.http_cache.purge_client.local
    method: purgeAll

As I put some debug logging in purgeAll method of the LocalPurgeClient I am sure the method is triggered.

Unfortunatly the content is still labeled as "hidden" in backend and is not visible in frontend.

[gggeek]

Back after some testing:

1. I installed ezplatform 1.13
2. set up a 'hide until date' workflow - exactly the same as found in the Resources/Docs example
3. publish an article => workflow does not get triggered at all... (no idea why tbh!)
4. change vhost to use sf 'dev' env by default
5. publish an article => workflow does get triggered. Article is indeed hidden
6. wait until date, resume workflow by running php app/console kaliop:workflows:resume -n => get correct cli message
7. go back to admin, hit F5 (old habit...) => whole app reloads => article is visible

I see 2 things so far:
a) the workflow does not get triggered when running in 'prod' sf env
b) I have no clue about how the editorial interface loads/caches the view of the article. In eZ4 there was the 'clear view cache' button, but not any more...

[gggeek]

ps: executed the same test, still in 'dev' mode. At step 7, instead of reloading the whole page, I just navigated away from the article view and back to it => again I saw the article as visible...

[gggeek]

pps: figured out I had to clear the Sf config cache for the workflow to effectively kick in for 'prod' env.

Tested again, using 'prod', and what I get now, editing using 'prod' env, at step 5 is:

• as soon as obj is published, it is displayed as visible in the Admin Interface, even though it is probably in fact hidden
• if I navigate away from it and come back, I see it as hidden => which makes me think this is a caching/timing problem in the design of the Admin Interface
• otoh after running the workflow "resume" step, if I do the same in the A.I. (navigate away then navigate back to obj) => I see it as visible

Note that all this testing was done in an env. where there is no Varnish enabled.

My take: it seems that the workflow engine works as intended, but the way it interacts with the Admin Interface is less than ideal, because apparently of the caching layers.
I wonder if in your case the problems might come from some misconfiguration of the cache - either the persistence one (stash/memcache) or the http one (sf/varnish)...
We could dig a bit to find out which existing eZ services can be used to force the clearing of either of those, of course, but as the workflow engine uses the standard eZ repository calls to execute its operations, it seems strange that we would have to manually add extra 'cache clear' calls.
Otoh it is quite easy to create a custom controller that does not set the proper x-location-id when the content is displayed in the Frontend, and thus does not get automatically cleared when a content is hidden/displayed (or maybe even simply sets the http caching headers so that the browser keeps the content in its own cache).
What I would need to know to help troubleshoot your case, esp. related to the Frontend is:
a. have a copy of the workflow yml
b. info about the setup: dev/prod, varnish, memcache ?
c. info about the controller that displays the content for the frontend: what kind of caching headers does it send ?
d. caching headers received by the browser when looking at that content (or, I imagine, its parent)

Note2: i generally use a 'patched' http-cache-cleaner service that logs any purge calls sent to Varnish. This helps a lot when having to troubleshoot cache expiration problems.

Note3: also remember to run your cronjobs using the same Sf env as your web interfaces. Having the web interface running as 'prod' with Varnish on and executing cli commands as 'dev' is a mistake I sometimes do with the result that the http cache does not get cleared...

[homarx]

First of all: Thank you for your research!

We did some more debugging.

env=prod

• publish an article, workflow gets triggered as expected
• in Platform UI the article is hidden, in frontend the article is hidden
• run resume -n --env=prod cronjob
• workflow gets triggered, in database article is visible, in frontend and Platform UI the article is still hidden

env=dev

• publish an article, workflow gets triggered as expected
• in Platform UI the article is hidden, in frontend the article is hidden
• run resume -n --env=dev cronjob
• workflow gets triggered, in database article is visible, in frontend and Platform UI the article is visible

We use the workflow yml as found in Resources/doc/Examples/, we only changed the content type
We are testing this bundle, no memchache or varnish is used in this case
We run the website in both dev and prod env (see above), but the prod env is the one we wanted to use in further installations

Cache headers of the parent node:
env=prod: Cache-Control: must-revalidate, no-cache, private
env=dev: Cache-Control: public, s-maxage=60

We tested this with diferrent controller, querytypes and criteria in the frontend. We also tried different content types. We always run into this in env=prod.

We debugged our folder-controller an the menu-controller and recognized some different or strange behaviour:
The object is avaible, eg we are able to display the object name in the frontend menu (even in prod env), but all data in the content (attributes) are not available. We did a loop in the twig template to display all visible children in the parent folder. The object count is correct, but the content attributes are still hidden.

[gggeek]

Hello

I imagine that you already tested this extensively, but:

am I correct in understanding that, for the prod env, after the workflow has finished, you refreshing in the browser the view of the parent node (to see if the node subject to the workflow does pop up)?
And that that view comes with a Cache-Control: public, s-maxage=60 http header?

If so, did you try the following:

• before workflow finishes: access view of parent node: /the/parent/node
• after workflow finishes: access view of parent node and make sure that any browser cache or proxy are bypassed: /the/parent/node?xxx=ok

This would rule out the case where the effect that you are seeing comes actually from a browser or reverse proxy cache (for the record: I generally always deploy using Varnish in front of eZ, and to avoid this kind of situations set up Varnish to cache the responses from eZ but always strip out any caching headers from the responses that it forwards to end users)

[gggeek]

PS: I will try to reproduce his by setting up 'parent-folder' and 'menu' controllers that both display 'article' contents. Which exact version of eZ are you running ?

[homarx]

am I correct in understanding that, for the prod env, after the workflow has finished, you refreshing in the browser the view of the parent node (to see if the node subject to the workflow does pop up)?
And that that view comes with a Cache-Control: public, s-maxage=60 http header?

Yes, the view of the parent node is refreshed in the browser.
No, in prod env the cache header is: Cache-Control:must-revalidate, no-cache, private.

The view of the parent node is definitely not affected by any browser or reverse proxy cache, as the content of the dropdown menu changes (the content which is published by the workflow appears as sub menu item).

We are running eZ kernel v6.13.1.2.