From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on gnuweeb.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NO_DNS_FOR_FROM,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 Received: from localhost.localdomain (unknown [101.128.125.100]) by gnuweeb.org (Postfix) with ESMTPSA id 0FB2B8060C; Tue, 20 Sep 2022 13:48:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gnuweeb.org; s=default; t=1663681710; bh=t2SwQ+ZCYL5qg4anRZ6jMF0Tj5B9BpTofY4XLhovIhc=; h=From:To:Cc:Subject:Date:From; b=pMzOCR4Y30ywVuBVCLrUc5qkE7nULtzjWgkwqWkRFwOirnTX9gTDi6t3U6s4x9X03 r7qluFxqMZm2IkZDYqTjLpvbR9DKSMyzGRPW5CEZ9P2kyOOLMGe9NLutK7L9TT3Zpk y74eyg57zQ6d92ywanigZmxVWaI+iTD2JlXVqFdP+3hN4G8qS1qHTof5QMxg05K6oI ZlQD/sK5/SGt9iWuH0qvZx0iQCKkA37yPXB8zvJiQGGVTkVp/SiRynY13BO8pqTC/M HAvhRH6/btHP5UvTQtLBA7SHuAMFCAooVKjxKfs9HQM5A5ZKV4u2hNSFvxFQevoPPt FVhtdvM0/y1Qw== From: Muhammad Rizki To: Ammar Faizi Cc: Muhammad Rizki , Alviro Iskandar Setiawan , GNU/Weeb Mailing List Subject: [RFC PATCH v4 00/17] Refactor Telegram & initial work Discord Date: Tue, 20 Sep 2022 20:47:55 +0700 Message-Id: <20220920134812.331-1-kiizuha@gnuweeb.org> X-Mailer: git-send-email 2.34.1.windows.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Hi sir, This series, Moving the Telegram bot source code to it's specific directory because there will be a Discord bot so I must to split them. Refactor the Telegram bot database method to split between insert, get, and delete to make it more clean and maintainable. Also, move the atom & utility source code to it's specific directory to make it reusable functions to reduce code duplication. Make initial work for Discord bot, it's still undone but it can run. I just prepared for the event handlers such as on_error handler and on_ready handler. on_error is to ignore error when any user in the Discord channel typing a word that doesn't exist in the basic commands bot like `ComandNotFound: "w"`. on_ready is to display a bot information when it's successfuly run the bot, also prepared for the database functions too, the methods pattern just like the Telegram bot, also combine the tables in db.sql. There are 17 patches in this series: - Patch 1 is to split the Telegram bot source code to it's specific dir - Patch 2 is to refactor Telegram database method to split between funcs - Patch 3 is to rename some functions to make it more readable - Patch 4 is to add ignore file for any names that have .env extension - Patch 5 is to move the Telegram bot runner script outside it's dir - Patch 6 is to rename mailer directory and listener file - Patch 7 is to move scraper and utility to it's specific directory - Patch 8 is to commit initial work for Discord bot - Patch 9 is to add a print success run in the on_ready event handler - Patch 10 is to add an error handler in the on_error event handler - Patch 11 is to move db.sql to the workdir and combine db tables - Patch 12 is to add database tables for Discord bot in the db.sql file - Patch 13 is to make an initial Discord bot database instance class - Patch 14 is to add save_atom() function in database insertion dir - Patch 15 is to add save_broadcast() function in database insertion dir - Patch 16 is to add save_discord_mail() func in database insertion dir - Patch 17 is to add save_email() func in database insertion dir How to use: 1. Execute db.sql file in the daemon directory 2. Setup .env file, the example is there with suffix .example, this file name must remove the suffix name .example 3. Setup the config.py in each bot directory such as dscord & telegram, the example is there with suffix .example & the file name must remove suffix name .example 4. Run `pip3 install -r requirements.txt` in each bot directory 5. STORAGE_DIR env value must `storage` to make it work fine 6. Run the bot by `python3 dc.py` or `python3 tg.py` Conclusion: Only change in commit title and description, no other code change. This should be works fine. Request: Please test it first so I can do more revision in a review, with this, I can continue this progress faster, but, if you want this project there is no need to rush that's okay, as long as you increase the time for making this project, thanks. ## Changelog v3 -> v4 - Change commits title & description grammar v2 -> v3 - Change commit description for commit 07/17 v1 -> v2 - Remove the ignore file in .gitignore while the file is not added yet. - Change prefix name each commits from `[]` to `:` - Change commit description for commit 05/17 Signed-off-by: Muhammad Rizki --- Muhammad Rizki (17): telegram: Move the Telegram bot source code telegram: Refactor Telegram bot database method telegram: Rename some functions in scraper/bot.py Add ignore file for .env telegram: Refactor the Telegram bot telegram: Renames in telegram mailer directory Move scraper and utility file discord: Initial work Discord bot discord: Add success run notice on_ready event discord: Add error handler on events Move db.sql to combine database with Discord and Telegram discord: Add database tables for Discord bot discord: Add initial Discord bot database instance discord: Add save_atom() in database insertion discord: Add save_broadcast() in database insertion discord: Add save_discord_mail() in database insertion discord: Add save_email() in database insertion .gitignore | 4 +- daemon/{scraper => atom}/__init__.py | 3 +- daemon/{scraper => atom}/scraper.py | 0 daemon/{scraper => atom}/utils.py | 0 daemon/db.sql | 74 +++++- daemon/dc.py | 20 ++ daemon/discord.env.example | 11 + daemon/dscord/config.py.example | 13 ++ daemon/dscord/database/__init__.py | 6 + daemon/dscord/database/core.py | 17 ++ daemon/dscord/database/methods/__init__.py | 12 + .../database/methods/insertion/__init__.py | 18 ++ .../database/methods/insertion/insert_atom.py | 27 +++ .../methods/insertion/insert_broadcast.py | 42 ++++ .../methods/insertion/insert_discord.py | 14 ++ .../methods/insertion/insert_email.py | 20 ++ daemon/dscord/gnuweeb/__init__.py | 6 + daemon/dscord/gnuweeb/client.py | 28 +++ daemon/dscord/gnuweeb/plugins/__init__.py | 17 ++ .../dscord/gnuweeb/plugins/events/__init__.py | 13 ++ .../dscord/gnuweeb/plugins/events/on_error.py | 17 ++ .../dscord/gnuweeb/plugins/events/on_ready.py | 21 ++ daemon/scraper/db.py | 217 ------------------ daemon/{.env.example => telegram.env.example} | 3 +- daemon/{ => telegram}/config.py.example | 0 daemon/telegram/database/__init__.py | 7 + daemon/telegram/database/core.py | 20 ++ daemon/telegram/database/methods/__init__.py | 17 ++ .../database/methods/deletion/__init__.py | 14 ++ .../database/methods/deletion/delet_atom.py | 15 ++ .../methods/deletion/delete_broadcast.py | 15 ++ .../database/methods/getter/__init__.py | 18 ++ .../database/methods/getter/get_atom_urls.py | 21 ++ .../methods/getter/get_broadcast_chats.py | 21 ++ .../database/methods/getter/get_email_id.py | 62 +++++ .../methods/getter/get_telegram_reply.py | 33 +++ .../database/methods/insertion/__init__.py | 18 ++ .../database/methods/insertion/insert_atom.py | 27 +++ .../methods/insertion/insert_broadcast.py | 56 +++++ .../methods/insertion/insert_email.py | 27 +++ .../methods/insertion/insert_telegram.py | 21 ++ daemon/telegram/mailer/__init__.py | 8 + .../bot.py => telegram/mailer/listener.py} | 31 +-- daemon/{ => telegram}/packages/__init__.py | 0 daemon/{ => telegram}/packages/client.py | 6 +- daemon/{ => telegram}/packages/decorator.py | 0 .../packages/plugins/callbacks/del_atom.py | 6 +- .../packages/plugins/callbacks/del_chat.py | 6 +- .../packages/plugins/commands/debugger.py | 2 +- .../packages/plugins/commands/manage_atom.py | 8 +- .../plugins/commands/manage_broadcast.py | 8 +- .../packages/plugins/commands/scrape.py | 8 +- daemon/{ => telegram}/requirements.txt | 0 daemon/{ => telegram}/storage/.gitignore | 0 daemon/{run.py => tg.py} | 14 +- 55 files changed, 818 insertions(+), 274 deletions(-) rename daemon/{scraper => atom}/__init__.py (80%) rename daemon/{scraper => atom}/scraper.py (100%) rename daemon/{scraper => atom}/utils.py (100%) create mode 100644 daemon/dc.py create mode 100644 daemon/discord.env.example create mode 100644 daemon/dscord/config.py.example create mode 100644 daemon/dscord/database/__init__.py create mode 100644 daemon/dscord/database/core.py create mode 100644 daemon/dscord/database/methods/__init__.py create mode 100644 daemon/dscord/database/methods/insertion/__init__.py create mode 100644 daemon/dscord/database/methods/insertion/insert_atom.py create mode 100644 daemon/dscord/database/methods/insertion/insert_broadcast.py create mode 100644 daemon/dscord/database/methods/insertion/insert_discord.py create mode 100644 daemon/dscord/database/methods/insertion/insert_email.py create mode 100644 daemon/dscord/gnuweeb/__init__.py create mode 100644 daemon/dscord/gnuweeb/client.py create mode 100644 daemon/dscord/gnuweeb/plugins/__init__.py create mode 100644 daemon/dscord/gnuweeb/plugins/events/__init__.py create mode 100644 daemon/dscord/gnuweeb/plugins/events/on_error.py create mode 100644 daemon/dscord/gnuweeb/plugins/events/on_ready.py delete mode 100644 daemon/scraper/db.py rename daemon/{.env.example => telegram.env.example} (82%) rename daemon/{ => telegram}/config.py.example (100%) create mode 100644 daemon/telegram/database/__init__.py create mode 100644 daemon/telegram/database/core.py create mode 100644 daemon/telegram/database/methods/__init__.py create mode 100644 daemon/telegram/database/methods/deletion/__init__.py create mode 100644 daemon/telegram/database/methods/deletion/delet_atom.py create mode 100644 daemon/telegram/database/methods/deletion/delete_broadcast.py create mode 100644 daemon/telegram/database/methods/getter/__init__.py create mode 100644 daemon/telegram/database/methods/getter/get_atom_urls.py create mode 100644 daemon/telegram/database/methods/getter/get_broadcast_chats.py create mode 100644 daemon/telegram/database/methods/getter/get_email_id.py create mode 100644 daemon/telegram/database/methods/getter/get_telegram_reply.py create mode 100644 daemon/telegram/database/methods/insertion/__init__.py create mode 100644 daemon/telegram/database/methods/insertion/insert_atom.py create mode 100644 daemon/telegram/database/methods/insertion/insert_broadcast.py create mode 100644 daemon/telegram/database/methods/insertion/insert_email.py create mode 100644 daemon/telegram/database/methods/insertion/insert_telegram.py create mode 100644 daemon/telegram/mailer/__init__.py rename daemon/{scraper/bot.py => telegram/mailer/listener.py} (77%) rename daemon/{ => telegram}/packages/__init__.py (100%) rename daemon/{ => telegram}/packages/client.py (95%) rename daemon/{ => telegram}/packages/decorator.py (100%) rename daemon/{ => telegram}/packages/plugins/callbacks/del_atom.py (88%) rename daemon/{ => telegram}/packages/plugins/callbacks/del_chat.py (90%) rename daemon/{ => telegram}/packages/plugins/commands/debugger.py (97%) rename daemon/{ => telegram}/packages/plugins/commands/manage_atom.py (91%) rename daemon/{ => telegram}/packages/plugins/commands/manage_broadcast.py (92%) rename daemon/{ => telegram}/packages/plugins/commands/scrape.py (91%) rename daemon/{ => telegram}/requirements.txt (100%) rename daemon/{ => telegram}/storage/.gitignore (100%) rename daemon/{run.py => tg.py} (78%) base-commit: 2582d7e5225d47a01f606808fc71e5e6aa7cb153 -- Muhammad Rizki