-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
Describe the bug
If a bridge
- accepts either long URLs or
- has many parameters
(or both) the generated cache key is longer than 250 characters and memcached fails to write that key to the cache.
To Reproduce
Steps to reproduce the behavior:
- Use
ARDMediathekBridge - Generate feed for the series "Kunst + Krempel" w/ the URL
https://www.ardmediathek.de/sendung/kunst-krempel/Y3JpZDovL2JyLmRlL2Jyb2FkY2FzdFNlcmllcy9icm9hZGNhc3RTZXJpZXM6L2JyZGUvZmVybnNlaGVuL2JheWVyaXNjaGVzLWZlcm5zZWhlbi9zZW5kdW5nZW4va3Vuc3QtdW5kLWtyZW1wZWw - See error in server logs
Expected behavior
No failing
Additional context
Server log entry (XXX = redaction):
2026/01/31 18:06:19 [error] 929582#929582: *687395 FastCGI sent in stderr: "PHP message: [2026-01-31 18:06:19] rssbridge.WARNING Failed to store an item in memcached {
"key": "http_{\"action\":\"display\",\"bridge\":\"ARDMediathekBridge\",\"path\":\"https:\\/\\/www.ardmediathek.de\\/sendung\\/kunst-krempel\\/Y3JpZDovL2JyLmRlL2Jyb2FkY2FzdFNlcmllcy9icm9hZGNhc3RTZXJpZXM6L2JyZGUvZmVybnNlaGVuL2JheWVyaXNjaGVzLWZlcm5zZWhlbi9zZW5kdW5nZW4va3Vuc3QtdW5kLWtyZW1wZWw\",\"format\":\"Html\"}",
}" while reading response header from upstream, client: XXX.XXX.XXX.XXX, server: rss-bridge.XXX.XXX, request: "GET /?action=display&bridge=ARDMediathekBridge&path=https%3A%2F%2Fwww.ardmediathek.de%2Fsendung%2Fkunst-krempel%2FY3JpZDovL2JyLmRlL2Jyb2FkY2FzdFNlcmllcy9icm9hZGNhc3RTZXJpZXM6L2JyZGUvZmVybnNlaGVuL2JheWVyaXNjaGVzLWZlcm5zZWhlbi9zZW5kdW5nZW4va3Vuc3QtdW5kLWtyZW1wZWw&format=Html HTTP/2.0", upstream: "fastcgi://unix:/var/run/1-XXX-rss-bridge.XXX.XXX-php-fpm.socket:", host: "rss-bridge.XXX.XXX", referrer: "https://rss-bridge.XXX.XXX/"
As you can see, the key length is 305 characters.
The ARDMediathekBridge is just an example (bridge).
The memcached docs say "Currently the length limit of a key is set at 250 characters […]"
Similar to the problem in #4876, the current (cache) key generation IMHO doesn't respect that which is why the memcached usage w/ RSS-Bridge can sometimes cause such errors.
The same is true for bridges with many parameters and there is already the comment TODO: might want to remove som params from query in CacheMiddleware.php#L23 (, IMHO regarding this).
For memcached I adopted my solution from #4876 that way (and use it at caches/MemcachedCache.php in production ATM):
[…]
public function get(string $key, $default = null)
{
$key = $this->createCacheKey($key);
[…]
}
public function set(string $key, $value, $ttl = null): void
{
$key = $this->createCacheKey($key);
[…]
}
public function delete(string $key): void
{
$key = $this->createCacheKey($key);
$this->conn->delete($key);
}
[…]
private function createCacheKey($key)
{
// See https://github.com/RSS-Bridge/rss-bridge/issues/4876
//return str_replace(' ', '_', $key);
// New problem: too long keys (> 250 characters not allowed)
// 1. Preserve the key prefix, only use the json request (see https://github.com/RSS-Bridge/rss-bridge/blob/master/actions/DisplayAction.php#L50)
$prefix = 'http_';
//$request = json_decode(substr($key, strlen($prefix)), true);
$request = json_decode(trim($key, $prefix), true);
if (is_array($request)) {
/**
// 2. Long version: only hash the longest value in request array …
// Is it worth the effort? – Maybe …
// But this has the problem, if there are more bridge paramaters that exceed the max 250 chars after hashing the longest value … :/
$mapped = array_map('strlen', $request);
$flipped = array_flip($mapped);
$longest = $flipped[max($mapped)];
// Hashes: https://ideone.com/embed/0iwuGn (from https://www.php.net/manual/en/function.hash.php#122785)
//$request[$longest] = hash('sha512', $request[$longest]); // 128 bytes
//$request[$longest] = hash('haval256,3', $request[$longest]); // 64 bytes
$request[$longest] = hash('md5', $request[$longest]); // 32 bytes
$key = $prefix . json_encode($request);
*/
// 2. Short version: hash the whole request …
$key = $prefix . hash('sha512', json_encode($request));
}
return $key;
}For a system wide solution to handle the $key/$cacheKey I'll refer to a/the more generic approach, mentioned (at the end) in #4876 .
IMHO there should be clarification which cache key prefixes are needed and which hash algorithms should/can be used (to ensure unique cache keys) …
I'm not very familiar ATM and I'm no expert. Since two days I'm only trying to script a new bridge and stumbled over the all the problems …
BR (and sry for the noise)