Skip to content

No mechanism exists to clear all magento cache entries without flushing the storage #33122

Open
@convenient

Description

@convenient

Preconditions (*)

  1. Any version of magento, these tests ran on 2.4.2
  2. Use redis as cache backend

Summary

If you want to clean your whole magento cache and you have your redis cache on a database of its own, you should be fine to "flush cache storage" or cache:flush. The other options miss an appreciable portion of the cache entries.

  • Admin "Flush Magento Cache" ⚠️
  • CLI ./bin/magento cache:clean ⚠️
  • Admin "Flush Cache Storage" ✅
  • CLI ./bin/magento cache:flush

The issue

There is no way to exclusively clear all the magento cache without flushing the full cache storage, which can affect other processes using the same storage.

From the documentation

When you pass no arguments to cache:clean it iterates through each cache type and cleans the entries with that tag. A user could reasonably assume that this means that all magento cache data has been removed however this is not the case.

Every time you save something to cache using the magento framework an additional tag (MAGE) is added on top of the tags you provide, this helps to identify all magento cache entries.

This is the code responsible

/**
* Enforce marking with a tag
*
* {@inheritdoc}
*/
public function save($data, $identifier, array $tags = [], $lifeTime = null)
{
$tags[] = $this->getTag();
return parent::save($data, $identifier, $tags, $lifeTime);
}

This means the tags behave like

 // tags are MAGE
$cache->save('data_here', 'lukeridentifier1', []);

// tags are CONFIG, MAGE
$cache->save('data_here', 'lukeridentifier2', ['CONFIG']); 

When we run cache:clean with no arguments it iterates through each defined cache type, but not MAGE. This means that entries with no cache tags explicitly defined default to only having the MAGE tag, which means they'll never get cleaned.

I have created a little script which outputs the cache identifiers that have only MAGE tags do not have any tags against them.

// filename is cache-without-tag.php 
<?php
use Magento\Framework\App\Bootstrap;
require __DIR__ . '/app/bootstrap.php';

$bootstrap = Bootstrap::create(BP, $_SERVER);
$obj = $bootstrap->getObjectManager();

/** @var \Magento\Framework\App\CacheInterface $cache */
$cache = $obj->get(\Magento\Framework\App\CacheInterface::class);
/** @var \Magento\Framework\Cache\Core $lowLevelCache */
$lowLevelCache = $cache->getFrontend()->getLowLevelFrontend();
/** @var \Magento\Framework\App\Cache\TypeListInterface $cacheTypeList */
$cacheTypeList = $obj->get(\Magento\Framework\App\Cache\TypeListInterface::class);
/** @var \Magento\Framework\App\Cache\Manager $cacheManager */
$cacheManager = $obj->get(\Magento\Framework\App\Cache\Manager::class);

$reflector = new \ReflectionObject($cacheTypeList);
$getTypeInstance = $reflector->getMethod('_getTypeInstance');
$getTypeInstance->setAccessible(true);

$mageOnlyCacheEntries = $lowLevelCache->getIdsMatchingTags(['MAGE']);
$mageOnlyCacheEntries = array_combine($mageOnlyCacheEntries, $mageOnlyCacheEntries);

// getAvailableTypes is the mechanism used when running cache:clean
foreach ($cacheManager->getAvailableTypes() as $type) {
    $typeInstance = $getTypeInstance->invoke($cacheTypeList, $type);
    $tag = $typeInstance->getTag();
    $cacheEntriesForType = $lowLevelCache->getIdsMatchingTags([$tag]);
    foreach ($cacheEntriesForType as $cacheEntryForType) {
        if (isset($mageOnlyCacheEntries[$cacheEntryForType])) {
            unset($mageOnlyCacheEntries[$cacheEntryForType]);
        }
    }
}
$mageOnlyCacheEntries = array_values($mageOnlyCacheEntries);
sort($mageOnlyCacheEntries);
foreach ($mageOnlyCacheEntries as $mageOnlyCacheEntry) {
    echo $mageOnlyCacheEntry . PHP_EOL;
}

Some sample output from a production website looks like below, this is only the last few entries there are approx 7000 entries on this site.

MAGEPLAZA_NOTIFICATIONS_LASTCHECK
MAGEWORX_OFFERS_NOTIFICATIONS_LASTCHECK
MAGEWORX_UPDATES_NOTIFICATIONS_LASTCHECK
MIRASVIT_CORE_SERVICE_COMPATIBILITYSERVICE
SWATCH_ATTRIBUTE_LIST
THEMEADMINHTML_MAGENTO_BACKEND
THEME_BY_ID_2
THEME_BY_ID_4
THEME_BY_ID_5

We can never clean these values without flushing the full cache storage

Replication

Save something to cache without specifying any tags, this will default to only the MAGE tag

<?php
use Magento\Framework\App\Bootstrap;
require __DIR__ . '/app/bootstrap.php';

$bootstrap = Bootstrap::create(BP, $_SERVER);
$obj = $bootstrap->getObjectManager();

/** @var \Magento\Framework\App\CacheInterface $cache */
$cache = $obj->get(\Magento\Framework\App\CacheInterface::class);
$cache->save('data_here', 'someidentifier', []);

Clean the cache
php bin/magento cache:clean

Load the value from the cache, see it still exists despite being a magento cache which we would think maybe was cleaned

<?php
use Magento\Framework\App\Bootstrap;
require __DIR__ . '/app/bootstrap.php';

$bootstrap = Bootstrap::create(BP, $_SERVER);
$obj = $bootstrap->getObjectManager();

/** @var \Magento\Framework\App\CacheInterface $cache */
$cache = $obj->get(\Magento\Framework\App\CacheInterface::class);
echo $cache->load('someidentifier') . PHP_EOL;

This is quite a trivial example, but there are a bunch of places in the core that do this and a bunch of third party modules operate this way. For example

private function saveThemeToCache(\Magento\Theme\Model\Theme $theme, $cacheId)
{
$themeData = $this->serializer->serialize($theme->toArray());
$this->cache->save($themeData, $cacheId);
}

Expected result (*)

  1. I would have some way of actually clearing all magento cache entries without flushing the the full cache storage, as this "might affect other processes applications that are using the same storage" as described in the documentation

Actual result (*)

  1. There is no way of clearing only the magento entries. You have to use flush which can affect other apps using the same storage

Please provide Severity assessment for the Issue as Reporter. This information will help during Confirmation and Issue triage processes.

  • Severity: S0 - Affects critical data or functionality and leaves users without workaround.
  • Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
  • Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
  • Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
  • Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Area: PerformanceIssue: ConfirmedGate 3 Passed. Manual verification of the issue completed. Issue is confirmedPriority: P2A defect with this priority could have functionality issues which are not to expectations.Progress: ready for devReported on 2.4.2Indicates original Magento version for the Issue report.Reproduced on 2.4.xThe issue has been reproduced on latest 2.4-develop branchTriage: Dev.ExperienceIssue related to Developer Experience and needs help with Triage to Confirm or Reject it

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions