From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on gnuweeb.org X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gnuweeb.org; s=default; t=1683629226; bh=3v7Vwrp5R9kK+F8LpNAfQSwkXOf/Glla2huh+/pCJ3k=; h=From:To:Cc:Subject:Date; b=Lvl8eYGQ49Nl40yx6IWvwdQLdgcWb8PBzzSY2I1RsaJlDnkApoieM0YbeCoIp8TtP +3CUFOavIrqyBSgtsJkOWeN6TexV4uHuyljQX6a8gCl3knAsW6l0s30vdH3VwXgSDW bk3wbu67+3qSOLBd/0UilkyAqJ+yIcjigoR6XOaid2ax0J7S/hBeSRy+6hIl2v/m4U KUniLCFedz2GOjhvjzLJiyK5jQVEvXRgt+tavnKCDFugwnupVFeqk0SwJr9oenJp0I D5I0EjPDXv7CUaDKCZjsfZULjOiLoMujiowHgRmcnP+4FkWg6dMsGEGBpkKZgwWwL4 7rVKVS3wpTr7g== Received: from localhost.localdomain (unknown [128.199.192.202]) by gnuweeb.org (Postfix) with ESMTPSA id 002EE245C58; Tue, 9 May 2023 17:47:04 +0700 (WIB) From: Ammar Faizi To: GNU/Weeb FB Team Cc: Ammar Faizi , GNU/Weeb Mailing List , Michael William Jonathan Subject: [PATCH fb v1 0/6] Introducing cache for the Facebook scraper Date: Tue, 9 May 2023 17:46:52 +0700 Message-Id: <20230509104658.70953-1-ammarfaizi2@gnuweeb.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Hi, This series introduce a cache mechanism to speed up the web API performance. It's very useful to reduce the pain when developing an app that uses the API. It also greatly reduces the number of requests to the same endpoint that happens in a short period of time. There are 6 patches in this series: Patch #1: Introduce `getCache()` and `setCache()`. A preparation patch to implement better caching mechanism. All methods that need cache will call these functions. Patch #2: Replace old cache mechanism in `getTimelineYears()`. Simplify the caching mechanism and make the `getTimelineYears()` cache private to itself. This also means that the endpoint "action=getTimelineYears" will utilize the cache. Patch #3, #4: Implement cache in `getTimelinePosts()` and `getPost()`. Make short repeated calls fast. Patch #5: Introduce `clearExpiredCaches()`. When a cache is expired, it won't be deleted unless getCache() with the corresponding key is invoked. Introduce a new function to scan for expired caches and delete them. Patch #6: Create cron.php to clear cache. Allow the server to clear expired caches via a small PHP script, cron.php. Periodically calling clearExpiredCaches() will delete old expired caches, it saves storage space. Signed-off-by: Ammar Faizi --- The following changes since commit 0d5e59e00359e165778a81f80122bb522f8edb0f: Merge branch 'rewrite_url' (Facebook Onion rewrite support) (2023-05-03 18:46:47 +0700) are available in the Git repository at: https://gitlab.torproject.org/ammarfaizi2/Facebook.git dev.cache for you to fetch changes up to d30f2dad8ca761b5a9c8de32ea48adbbdd201d03: fb: web: Create cron.php to clear cache (2023-05-09 17:33:12 +0700) ---------------------------------------------------------------- Ammar Faizi (6): fb: Introduce `getCache()` and `setCache()` functions fb: Post: Replace old cache mechanism in `getTimelineYears()` fb: Post: Implement cache in `getPost()` fb: Post: Implement cache in `getTimelinePosts()` fb: cache: Introduce `clearExpiredCaches()` fb: web: Create cron.php to clear cache src/Facebook/Facebook.php | 99 ++++++++++++++++++++++++++++++---------- src/Facebook/Methods/Post.php | 74 ++++++++++-------------------- web/cron.php | 9 ++++ 3 files changed, 108 insertions(+), 74 deletions(-) create mode 100644 web/cron.php base-commit: 0d5e59e00359e165778a81f80122bb522f8edb0f -- Ammar Faizi