Guest parser brittleness (Infinite loop or memory error parsing PHP with guest parsers)

[b4n]

CTags version: master (b11e39a)

When parsing combined.php from Geany's test suite, using ctags -o- --extras=+g combined.php we get an infinite loop.

Bissecting gives me f72c896 as the faulty commit -- oops, not really a good news for debugging this 😓 …

[b4n]

This can be triggered with smaller inputs, e.g.:

<?= $title ?>

or

<!-- the bug doesn't trigger on the first line with this example -->
<title><?= $title ?></title>

[b4n]

@masatake FWIW, fcab90a fixes the issue for me and doesn't seem to introduce any new one, but I'll leave this to you for now because you understand this area better than I do, and you authored most of f72c896 which replaced the aforementioned commit.

Given how subtle problematic code is (e.g. the second example above works fine if on the first line), I'm kind of wondering if ungetc buffer doesn't play a role here. Or maybe not and it's just problematic offsets past the first line?

[b4n]

OK actually the issue with combined.php was due to an actual bug in the PHP parser, resulting in improper coordination with the HTML parser. Fix is in #4376.

However, this still shows that some guest parser setups (like HTML ↔ PHP) is fragile and "dangerous". So again, what about verifying the promise areas before handling them? We do this in Geany with geany/geany@1b8cb9d.

Additionally, we have geany/geany@4f57234 to verify promise ranges are valid. This is actually useful in a very real case: make Units/parser-php.r/wp-guest.d use CR/LF endings, and it then fails to parse (see #4377). Yet again, fcab90a fixes it it seems.

If one or both of those checks is something we want, I can open a PR (I have versions of those rebased on top of master).

[techee]

If one or both of those checks is something we want, I can open a PR (I have versions of those rebased on top of master).

The ctags PR already exists in #4266

[b4n]

@masatake I merged #4377 so there's an actual test case for this

[masatake]

@b4n Thank you. Sorry for my inactivity. I will catch up.

[b4n]

@masatake don't worry none, we can wait and handle some stuff ourselves :)