« Back Issue #503
  • 2023-03-01 15:37 Reported by padiarushi
  • 2023-05-23 03:16 Assigned to padiarushi
Status: Assigned
Priority: Medium

Improve Cache in Yioop

padiarushi 2023-03-01 15:37

Create a module to evaluate the performance of cache implmentations

-Change implementation of FileCache such that it acts as proxy to actual cache implementation -Moved implementation of Marker's algorithm to cache package -Added cache wrapper class to evaluate the performance of the cache. -Added test cases to test cases to FileCache

Testing cache - 1. Login to Yioop 2. Open Configure 3. Click Test Info 4. Click FileCacheTest

This will run all unit test cases for File Cache. All test cases should be working.

padiarushi 2023-03-14 01:58
padiarushi 2023-03-21 13:08
padiarushi 2023-04-11 14:54

Combined patches

changes.patch

padiarushi 2023-04-18 14:50

created patch with linux shell

cacheImprove.patch

padiarushi 2023-04-23 15:08

Fixed author and added docs fixed php 8.2 deprecation warning

cacheImprovement.patch

administrator 2023-04-23 18:19

Hi,

Thanks for your patch. After applying it, and going to the unit tests. The FileCacheTest gives a lot of PHP warnings and deprecations (see below). The addTestCase also fails. Can you fix these issues and make a new patch. Also, be sure to test using PHP 8.2.

Best, Chris

Warning: mkdir(): No such file or directory in /Applications/MAMP/htdocs/git/yioop/src/library/cache/MarkersCache.php on line 77

Warning: chmod(): No such file or directory in /Applications/MAMP/htdocs/git/yioop/src/library/cache/MarkersCache.php on line 78

PHP OTHER ERROR Creation of dynamic property seekquarry\yioop\library\cache\MarkersCache::$web_site is deprecated at line 83 in /Applications/MAMP/htdocs/git/yioop/src/library/cache/MarkersCache.php

 in seekquarry\yioop\library\cache\MarkersCache->__construct, line 61 in /Applications/MAMP/htdocs/git/yioop/src/library/FileCache.php 
 called from seekquarry\yioop\library\FileCache->__construct, line 51 in /Applications/MAMP/htdocs/git/yioop/tests/FileCacheTest.php 
 called from seekquarry\yioop\tests\FileCacheTest->setUp, line 83 in /Applications/MAMP/htdocs/git/yioop/src/library/UnitTest.php 
 called from seekquarry\yioop\library\UnitTest->run, line 191 in /Applications/MAMP/htdocs/git/yioop/src/controllers/TestsController.php 
 called from seekquarry\yioop\controllers\TestsController->runTest, line 102 in /Applications/MAMP/htdocs/git/yioop/src/controllers/TestsController.php 

...

WARNING: mkdir(): No such file or directory at line 151 in /Applications/MAMP/htdocs/git/yioop/src/library/cache/MarkersCache.php

 in mkdir, line 151 in /Applications/MAMP/htdocs/git/yioop/src/library/cache/MarkersCache.php 
 called from seekquarry\yioop\library\cache\MarkersCache->set, line 92 in /Applications/MAMP/htdocs/git/yioop/src/library/cache/CacheMetricsWrapper.php 
 called from seekquarry\yioop\library\cache\CacheMetricsWrapper->set, line 83 in /Applications/MAMP/htdocs/git/yioop/src/library/FileCache.php 
 called from seekquarry\yioop\library\FileCache->set, line 65 in /Applications/MAMP/htdocs/git/yioop/tests/FileCacheTest.php 
 called from seekquarry\yioop\tests\FileCacheTest->addTestCase, line 86 in /Applications/MAMP/htdocs/git/yioop/src/library/UnitTest.php 

...

WARNING: chmod(): No such file or directory at line 152 in /Applications/MAMP/htdocs/git/yioop/src/library/cache/MarkersCache.php

 in chmod, line 152 in /Applications/MAMP/htdocs/git/yioop/src/library/cache/MarkersCache.php 
 called from seekquarry\yioop\library\cache\MarkersCache->set, line 92 in /Applications/MAMP/htdocs/git/yioop/src/library/cache/CacheMetricsWrapper.php 
 called from seekquarry\yioop\library\cache\CacheMetricsWrapper->set, line 83 in /Applications/MAMP/htdocs/git/yioop/src/library/FileCache.php 
 called from seekquarry\yioop\library\FileCache->set, line 65 in /Applications/MAMP/htdocs/git/yioop/tests/FileCacheTest.php 
 called from seekquarry\yioop\tests\FileCacheTest->addTestCase, line 86 in /Applications/MAMP/htdocs/git/yioop/src/library/UnitTest.php 

...

WARNING: mkdir(): No such file or directory at line 203 in /Applications/MAMP/htdocs/git/yioop/src/library/cache/MarkersCache.php

 in mkdir, line 203 in /Applications/MAMP/htdocs/git/yioop/src/library/cache/MarkersCache.php 
 called from seekquarry\yioop\library\cache\MarkersCache->updateCache, line 155 in /Applications/MAMP/htdocs/git/yioop/src/library/cache/MarkersCache.php 
 called from seekquarry\yioop\library\cache\MarkersCache->set, line 92 in /Applications/MAMP/htdocs/git/yioop/src/library/cache/CacheMetricsWrapper.php 
 called from seekquarry\yioop\library\cache\CacheMetricsWrapper->set, line 83 in /Applications/MAMP/htdocs/git/yioop/src/library/FileCache.php 
 called from seekquarry\yioop\library\FileCache->set, line 65 in /Applications/MAMP/htdocs/git/yioop/tests/FileCacheTest.php 

...

padiarushi 2023-04-27 04:19

1. Fixed: mkdir,chmod issue. Issue was appearing directory did not exist 2. Fixed: Dynamic property issue 3. Fixed: addTestCase was failing because of 1st issue

improvedCache.patch

administrator 2023-04-27 11:24

You forgot to add the UI for the dropdown to choose the Caching mechanism

padiarushi 2023-04-27 15:45

Could you please check if this patch is okay? Currently this patch has only english language lables, is there any tool to generate label for different languages?

improvedCache-2.patch

administrator 2023-04-27 18:20

Here are some comments on your latest patch: (1) You only have two choices of cache: Marker's and None. What happened to the other ones you developed? The

   whole point of the drop down is to allow the admin to select from all your new cache systems.

(2) Don't use apostrophes in localized string. I.e., don't have a localized string like Marker's (3) Filecache Algorithm --> Cache Type: (4) Currently, your patch crashes fatally on my system because I don't have the FILECACHE constant yet in

    my Profile.php. Your code needs to upgrade gracefully. At a minimum, you should have
   a check like: C\nsdefined("FILECACHE") && C\FILECACHE !== "None"

(5) Don't use "None" add new constants to library/CrawlConstants as needed for the cache types.

padiarushi 2023-05-23 03:13

Running instructions - 1. Please make sure StaticCacheRefreshJob media job is run after selecting Static-Dynamic cache or Static-Topic-Dynamic cache 2. Please make sure CacheTopicTrainer executable is run at least once before use Static-Topic-Dynamic cache. This will store paramters for k-means model

ImprovedCache.patch does not contain dataset.csv file required for training topic model and populating static cache ImprovedCacheDataset.patch contains dataset.csv (100MB) file as well (There is 5 MB limit on patch upload hence this patch was not uploaded) Sending dataset file seperately on email. Please add dataset.csv file at location "src\data\"

ImprovedCache-3.patch

X