My open edX site runs juniper.3 and I have found the size of memcached continues increasing these days.
I spent some time in analyzing contents in memcached and found most keys start with course_structure:xxxx, which are cache for course structure.
In Django settings, the expire time for course_structure cache was set to 7200 seconds, but seems that it is not working as a lot of very old keys still exist. I looked into CourseStructureCache code in common/lib/xmodule/xmodule/modulestore/split_mongo/mongo_connection.py and found below code snippet:
def set(self, key, structure, course_context=None): """Given a structure, will pickle, compress, and write to cache.""" if self.cache is None: return None with TIMER.timer("CourseStructureCache.set", course_context) as tagger: pickled_data = pickle.dumps(structure, 4) # Protocol can't be incremented until cache is cleared tagger.measure('uncompressed_size', len(pickled_data)) # 1 = Fastest (slightly larger results) compressed_pickled_data = zlib.compress(pickled_data, 1) tagger.measure('compressed_size', len(compressed_pickled_data)) # Stuctures are immutable, so we set a timeout of "never" self.cache.set(key, compressed_pickled_data, None)
There are comments "# Stuctures are immutable, so we set a timeout of “never”, does this mean in current open edX implementation, course structure will always stay in cache as long as it is created? If there are many courses in the platform, course structure may occupy too much cache space.
Has anyone faced similar problem and had a solution to this? I have also posted this problem in open edx forum.