* [PATCH v4 0/9] Fix some bugs and add some features @ 2022-10-22 6:51 Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 1/9] discord: Add send_text_mail_interaction() Muhammad Rizki ` (9 more replies) 0 siblings, 10 replies; 11+ messages in thread From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw) To: Ammar Faizi Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List Hi sir, This is v4 revision of fix some bugs and add some features. This series is to fix This series is to fix some bugs, improve some codes, add some new features and add enumeration. These bugs should have been fixed now, and the email file attachments should have been removed after all attachments have been sent. Known bugs: 1. Email payload extraction result become unicode if the email payload contain non-UTF8 characters like chinese, japanese, and similar like that. 2. remove_patch() doesn't remove all file attachments properly. Improvements: 1. Improve remove_patch() to make it all file attachments removed after sending them. 2. Fix `/lore {raw atom url}` to add a "telegram" onto the create_template() platform parameter. 3. Improvement on extract_body() to ignore whenever the email contain text/html content-type and use the email library function to get the header value. New features: 1. Add send_text_mail_interaction() for the `/lore {raw atom url}` slash command. 2. Add send_patch_mail_interaction() for the `/lore {raw atom url}` slash command. 3. Add `/lore {raw atom url}` slash command. 4. Platform class enumeration There are 9 patches in this series: - Patch 1 is to add send_text_mail_interaction() - Patch 2 is to add send_patch_mail_interaction() - Patch 3 is to add `/lore` slash command - Patch 4 is to improve remove_patch() code to make it more stable - Patch 5 is to add manage_payload() for manage email payload extraction - Patch 6 is to fix the Telegram `/lore` command - Patch 7 is to improve extract_body() to ignore text/html content-type - Patch 8 is to add initial Platform class enumeration - Patch 9 is to use the created Platform class enumeration How to use: 1. Execute the db.sql file in the daemon directory, 2. Setup .env file, the example is there with suffix .example, this file name must remove the suffix name .example, 3. Set up the config.py in each bot directory, such as dscord and telegram. The example is there with suffix .example & the file name must remove suffix name .example, 4. Run `pip3 install -r requirements.txt` in each bot directory, 5. STORAGE_DIR env value must `storage` to make it work fine, 6. Run the bot by `python3 dc.py` or `python3 tg.py`. Tested and works, what do you think with this patch? ## Changelog v3 -> v4: - Fix the decode error `invalid start byte` - Rename manage_payload() to get_decoded_payload() - Use charset to decode the payload - Use the email library function to get the header value v2 -> v3: - Add Platform class enumeration - Use the old one of fix_utf8_char() instead - Add Content-Transfer-Encoding handler in manage_payload() v1 -> v2: - Remove re.sub() in the fix_utf8_char() when set unescape to True - Add if statement logic to ignore text/html in the extract_body() Signed-off-by: Muhammad Rizki <[email protected]> --- Muhammad Rizki (9): discord: Add send_text_mail_interaction() discord: Add send_patch_mail_interaction() discord: Add get lore mail slash command atom: Improve remove_patch() atom: add get_decoded_payload() telegram: Fix get lore command atom: Improve extract_body() enum: Add Platform enumeration enum: Use the created Platform class enumeration daemon/atom/utils.py | 61 ++++++++++++------- daemon/dscord/gnuweeb/client.py | 26 +++++++- .../plugins/slash_commands/__init__.py | 2 + .../plugins/slash_commands/get_lore_mail.py | 40 ++++++++++++ daemon/dscord/mailer/listener.py | 7 ++- daemon/enums/__init__.py | 1 + daemon/enums/base.py | 9 +++ daemon/enums/platform.py | 7 +++ daemon/telegram/mailer/listener.py | 7 +-- daemon/telegram/packages/client.py | 3 +- .../packages/plugins/commands/scrape.py | 7 +-- 11 files changed, 136 insertions(+), 34 deletions(-) create mode 100644 daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py create mode 100644 daemon/enums/__init__.py create mode 100644 daemon/enums/base.py create mode 100644 daemon/enums/platform.py base-commit: d9b20dab81202b93f48d5365ad680796e5839d80 -- Muhammad Rizki ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v4 1/9] discord: Add send_text_mail_interaction() 2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki @ 2022-10-22 6:51 ` Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 2/9] discord: Add send_patch_mail_interaction() Muhammad Rizki ` (8 subsequent siblings) 9 siblings, 0 replies; 11+ messages in thread From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw) To: Ammar Faizi Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List Add send_text_mail_interaction() for the `/atom get {url}` to get the specific lore email message from an URL in future use. Signed-off-by: Muhammad Rizki <[email protected]> --- daemon/dscord/gnuweeb/client.py | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/daemon/dscord/gnuweeb/client.py b/daemon/dscord/gnuweeb/client.py index 03a1b8c..15a6eee 100644 --- a/daemon/dscord/gnuweeb/client.py +++ b/daemon/dscord/gnuweeb/client.py @@ -4,6 +4,7 @@ # import discord +from discord import Interaction from discord.ext import commands from discord import Intents from dscord.config import ACTIVITY_NAME @@ -76,3 +77,11 @@ class GWClient(commands.Bot): utils.remove_patch(tmp) return m + + + async def send_text_mail_interaction(self, i: "Interaction", + text: str, url: str = None): + return await i.response.send_message( + content=text, + view=models.FullMessageBtn(url) + ) -- Muhammad Rizki ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 2/9] discord: Add send_patch_mail_interaction() 2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 1/9] discord: Add send_text_mail_interaction() Muhammad Rizki @ 2022-10-22 6:51 ` Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 3/9] discord: Add get lore mail slash command Muhammad Rizki ` (7 subsequent siblings) 9 siblings, 0 replies; 11+ messages in thread From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw) To: Ammar Faizi Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List This function is just the same as send_text_mail_interaction(), the different between them is send_patch_mail_interaction() for sending a lore email message with patch file attachment. Signed-off-by: Muhammad Rizki <[email protected]> --- daemon/dscord/gnuweeb/client.py | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/daemon/dscord/gnuweeb/client.py b/daemon/dscord/gnuweeb/client.py index 15a6eee..b921f7d 100644 --- a/daemon/dscord/gnuweeb/client.py +++ b/daemon/dscord/gnuweeb/client.py @@ -85,3 +85,17 @@ class GWClient(commands.Bot): content=text, view=models.FullMessageBtn(url) ) + + + async def send_patch_mail_interaction(self, mail, i: "Interaction", + text: str, url: str = None): + tmp, doc, caption, url = utils.prepare_patch( + mail, text, url, "discord" + ) + m = await i.response.send_message( + content=caption, + file=discord.File(doc), + view=models.FullMessageBtn(url) + ) + utils.remove_patch(tmp) + return m -- Muhammad Rizki ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 3/9] discord: Add get lore mail slash command 2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 1/9] discord: Add send_text_mail_interaction() Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 2/9] discord: Add send_patch_mail_interaction() Muhammad Rizki @ 2022-10-22 6:51 ` Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 4/9] atom: Improve remove_patch() Muhammad Rizki ` (6 subsequent siblings) 9 siblings, 0 replies; 11+ messages in thread From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw) To: Ammar Faizi Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List Add a `/lore {url}` slash command to get the specific lore email message from the raw atom URL. Signed-off-by: Muhammad Rizki <[email protected]> --- .../plugins/slash_commands/__init__.py | 2 + .../plugins/slash_commands/get_lore_mail.py | 39 +++++++++++++++++++ 2 files changed, 41 insertions(+) create mode 100644 daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py diff --git a/daemon/dscord/gnuweeb/plugins/slash_commands/__init__.py b/daemon/dscord/gnuweeb/plugins/slash_commands/__init__.py index a6d913c..126af45 100644 --- a/daemon/dscord/gnuweeb/plugins/slash_commands/__init__.py +++ b/daemon/dscord/gnuweeb/plugins/slash_commands/__init__.py @@ -5,9 +5,11 @@ from .manage_atom import ManageAtomSC from .manage_broadcast import ManageBroadcastSC +from .get_lore_mail import GetLoreSC class SlashCommands( + GetLoreSC, ManageAtomSC, ManageBroadcastSC ): pass diff --git a/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py b/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py new file mode 100644 index 0000000..2d55d16 --- /dev/null +++ b/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py @@ -0,0 +1,39 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Copyright (C) 2022 Muhammad Rizki <[email protected]> +# + +import asyncio +import discord +from discord.ext import commands +from discord import Interaction +from discord import app_commands + +from atom import utils +from atom import Scraper + + +class GetLoreSC(commands.Cog): + def __init__(self, bot) -> None: + self.bot = bot + + + @app_commands.command( + name="lore", + description="Get lore email from raw email URL." + ) + @app_commands.describe(url="Raw lore email URL") + async def get_lore(self, i: "Interaction", url: str): + s = Scraper() + mail = await s.get_email_from_url(url) + text, _, is_patch = utils.create_template(mail, "discord") + + if is_patch: + m = await self.bot.send_patch_mail_interaction( + mail=mail, i=i, text=text, url=url + ) + else: + text = "#ml\n" + text + m = await self.bot.send_text_mail_interaction( + i=i, text=text, url=url + ) -- Muhammad Rizki ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 4/9] atom: Improve remove_patch() 2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki ` (2 preceding siblings ...) 2022-10-22 6:51 ` [PATCH v4 3/9] discord: Add get lore mail slash command Muhammad Rizki @ 2022-10-22 6:51 ` Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 5/9] atom: add get_decoded_payload() Muhammad Rizki ` (5 subsequent siblings) 9 siblings, 0 replies; 11+ messages in thread From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw) To: Ammar Faizi Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List Improvement on remove_patch(). So, after send all the email attachments then remove them. On the previous patch, this function can't remove them properly. Signed-off-by: Muhammad Rizki <[email protected]> --- daemon/atom/utils.py | 10 +++++++--- daemon/dscord/mailer/listener.py | 4 ++-- daemon/telegram/mailer/listener.py | 3 +-- daemon/telegram/packages/plugins/commands/scrape.py | 3 +-- 4 files changed, 11 insertions(+), 9 deletions(-) diff --git a/daemon/atom/utils.py b/daemon/atom/utils.py index a30d5cb..f554f6f 100644 --- a/daemon/atom/utils.py +++ b/daemon/atom/utils.py @@ -6,7 +6,7 @@ from pyrogram.types import Chat, InlineKeyboardMarkup, InlineKeyboardButton from email.message import Message -from typing import Dict +from typing import Dict, Union from slugify import slugify import hashlib import uuid @@ -238,8 +238,12 @@ def prepare_patch(mail, text, url, platform: str): return tmp, file, caption, url -def remove_patch(tmp): - shutil.rmtree(tmp) +def remove_patch(tmp: Union[str, list]): + if isinstance(tmp, str): + return shutil.rmtree(tmp) + + for d,_ in tmp: + shutil.rmtree(d) def fix_utf8_char(text: str, html_escape: bool = True): diff --git a/daemon/dscord/mailer/listener.py b/daemon/dscord/mailer/listener.py index a280a58..cc0a9f7 100644 --- a/daemon/dscord/mailer/listener.py +++ b/daemon/dscord/mailer/listener.py @@ -125,10 +125,10 @@ class Listener: for d, f in files: await m.reply(file=File(f"{d}/{f}")) - if files.index((d,f)) == len(files)-1: - utils.remove_patch(d) await asyncio.sleep(1) + utils.remove_patch(files) + return True diff --git a/daemon/telegram/mailer/listener.py b/daemon/telegram/mailer/listener.py index 208aed0..08feddd 100644 --- a/daemon/telegram/mailer/listener.py +++ b/daemon/telegram/mailer/listener.py @@ -118,8 +118,7 @@ class Bot(): await m.reply_document(f"{d}/{f}", file_name=f) await asyncio.sleep(1) - if files: - shutil.rmtree(str(files[0][0])) + utils.remove_patch(files) return True diff --git a/daemon/telegram/packages/plugins/commands/scrape.py b/daemon/telegram/packages/plugins/commands/scrape.py index d4d10a9..52ddb0b 100644 --- a/daemon/telegram/packages/plugins/commands/scrape.py +++ b/daemon/telegram/packages/plugins/commands/scrape.py @@ -53,5 +53,4 @@ async def scrap_email(c: DaemonClient, m: Message): await m.reply_document(f"{d}/{f}", file_name=f) await asyncio.sleep(1) - if files: - shutil.rmtree(str(files[0][0])) + utils.remove_patch(files) -- Muhammad Rizki ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 5/9] atom: add get_decoded_payload() 2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki ` (3 preceding siblings ...) 2022-10-22 6:51 ` [PATCH v4 4/9] atom: Improve remove_patch() Muhammad Rizki @ 2022-10-22 6:51 ` Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 6/9] telegram: Fix get lore command Muhammad Rizki ` (4 subsequent siblings) 9 siblings, 0 replies; 11+ messages in thread From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw) To: Ammar Faizi Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List Add get_decoded_payload() to handle the email decoding to utf-8. This include a non-UTF8 character, base64 decoding, and quoted-printable decoding. Signed-off-by: Muhammad Rizki <[email protected]> --- daemon/atom/utils.py | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/daemon/atom/utils.py b/daemon/atom/utils.py index f554f6f..deff99d 100644 --- a/daemon/atom/utils.py +++ b/daemon/atom/utils.py @@ -8,6 +8,7 @@ from pyrogram.types import Chat, InlineKeyboardMarkup, InlineKeyboardButton from email.message import Message from typing import Dict, Union from slugify import slugify +from base64 import b64decode import hashlib import uuid import os @@ -15,6 +16,7 @@ import re import shutil import httpx import html +import quopri def get_email_msg_id(mail): @@ -136,7 +138,7 @@ def gen_temp(name: str, platform: str): def extract_body(thread: Message, platform: str): if not thread.is_multipart(): - p = thread.get_payload(decode=True).decode(errors='replace') + p = get_decoded_payload(thread) if platform == "discord": p = quote_reply(p) @@ -253,6 +255,20 @@ def fix_utf8_char(text: str, html_escape: bool = True): return t +def get_decoded_payload(payload: Message): + p = str(payload.get_payload()) + tf_encode = payload.get("Content-Transfer-Encoding") + charset = payload.get_content_charset("utf-8") + + if tf_encode == "base64": + return b64decode(p).decode(charset) + if tf_encode == "quoted-printable": + quobyte = quopri.decodestring(p.encode()) + return quobyte.decode(charset) + + return p.encode().decode(charset, errors="replace") + + EMAIL_MSG_ID_PATTERN = r"<([^\<\>]+)>" def extract_email_msg_id(msg_id): ret = re.search(EMAIL_MSG_ID_PATTERN, msg_id) -- Muhammad Rizki ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 6/9] telegram: Fix get lore command 2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki ` (4 preceding siblings ...) 2022-10-22 6:51 ` [PATCH v4 5/9] atom: add get_decoded_payload() Muhammad Rizki @ 2022-10-22 6:51 ` Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 7/9] atom: Improve extract_body() Muhammad Rizki ` (3 subsequent siblings) 9 siblings, 0 replies; 11+ messages in thread From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw) To: Ammar Faizi Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List Fix the `/lore {raw lore url}` to add "telegram" on the create_template() platform parameter. Signed-off-by: Muhammad Rizki <[email protected]> --- daemon/telegram/packages/plugins/commands/scrape.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daemon/telegram/packages/plugins/commands/scrape.py b/daemon/telegram/packages/plugins/commands/scrape.py index 52ddb0b..860e993 100644 --- a/daemon/telegram/packages/plugins/commands/scrape.py +++ b/daemon/telegram/packages/plugins/commands/scrape.py @@ -37,7 +37,7 @@ async def scrap_email(c: DaemonClient, m: Message): s = Scraper() mail = await s.get_email_from_url(url) - text, files, is_patch = utils.create_template(mail) + text, files, is_patch = utils.create_template(mail, "telegram") if is_patch: m = await c.send_patch_email( -- Muhammad Rizki ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 7/9] atom: Improve extract_body() 2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki ` (5 preceding siblings ...) 2022-10-22 6:51 ` [PATCH v4 6/9] telegram: Fix get lore command Muhammad Rizki @ 2022-10-22 6:51 ` Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 8/9] enum: Add Platform enumeration Muhammad Rizki ` (2 subsequent siblings) 9 siblings, 0 replies; 11+ messages in thread From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw) To: Ammar Faizi Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List Add an if statement to ignore whenever the email payload containing a text/html content-type header. v1 -> v2: - Simplify the usage to get the email header by using the library get function itself. Signed-off-by: Muhammad Rizki <[email protected]> Reviewed-by: Alviro Iskandar Setiawan <[email protected]> --- daemon/atom/utils.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/daemon/atom/utils.py b/daemon/atom/utils.py index deff99d..4383b03 100644 --- a/daemon/atom/utils.py +++ b/daemon/atom/utils.py @@ -151,10 +151,13 @@ def extract_body(thread: Message, platform: str): fname = p.get_filename() payload = p.get_payload(decode=True) + if p.get_content_type() == "text/html": + continue + if not payload: continue - if 'inline' in [p.get('content-disposition')] or not bool(fname): + if p.get_content_disposition() == 'inline' or not bool(fname): ret += f"{payload.decode(errors='replace')}\n".lstrip() continue -- Muhammad Rizki ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 8/9] enum: Add Platform enumeration 2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki ` (6 preceding siblings ...) 2022-10-22 6:51 ` [PATCH v4 7/9] atom: Improve extract_body() Muhammad Rizki @ 2022-10-22 6:51 ` Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 9/9] enum: Use the created Platform class enumeration Muhammad Rizki 2022-10-22 7:44 ` [PATCH v4 0/9] Fix some bugs and add some features Ammar Faizi 9 siblings, 0 replies; 11+ messages in thread From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw) To: Ammar Faizi Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List Add a Platform enum class to use it for enumeration instead of a string. Signed-off-by: Muhammad Rizki <[email protected]> Acked-by: Alviro Iskandar Setiawan <[email protected]> --- daemon/enums/__init__.py | 1 + daemon/enums/base.py | 9 +++++++++ daemon/enums/platform.py | 7 +++++++ 3 files changed, 17 insertions(+) create mode 100644 daemon/enums/__init__.py create mode 100644 daemon/enums/base.py create mode 100644 daemon/enums/platform.py diff --git a/daemon/enums/__init__.py b/daemon/enums/__init__.py new file mode 100644 index 0000000..efe9c79 --- /dev/null +++ b/daemon/enums/__init__.py @@ -0,0 +1 @@ +from .platform import Platform diff --git a/daemon/enums/base.py b/daemon/enums/base.py new file mode 100644 index 0000000..526aba5 --- /dev/null +++ b/daemon/enums/base.py @@ -0,0 +1,9 @@ +import enum + + +class EnumBase(enum.Enum): + def _generate_next_value_(self, *args): + return self.lower() + + def __repr__(self): + return f"enums.{self}" diff --git a/daemon/enums/platform.py b/daemon/enums/platform.py new file mode 100644 index 0000000..638d8dd --- /dev/null +++ b/daemon/enums/platform.py @@ -0,0 +1,7 @@ +import enum +from .base import EnumBase + + +class Platform(EnumBase): + DISCORD = enum.auto() + TELEGRAM = enum.auto() -- Muhammad Rizki ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 9/9] enum: Use the created Platform class enumeration 2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki ` (7 preceding siblings ...) 2022-10-22 6:51 ` [PATCH v4 8/9] enum: Add Platform enumeration Muhammad Rizki @ 2022-10-22 6:51 ` Muhammad Rizki 2022-10-22 7:44 ` [PATCH v4 0/9] Fix some bugs and add some features Ammar Faizi 9 siblings, 0 replies; 11+ messages in thread From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw) To: Ammar Faizi Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List Use the Platform class enumeration to make it more practice, avoid getting typo, type-hinted suggestion to easily manageable-maintainable. Signed-off-by: Muhammad Rizki <[email protected]> --- daemon/atom/utils.py | 28 ++++++++----------- daemon/dscord/gnuweeb/client.py | 5 ++-- .../plugins/slash_commands/get_lore_mail.py | 3 +- daemon/dscord/mailer/listener.py | 3 +- daemon/telegram/mailer/listener.py | 4 +-- daemon/telegram/packages/client.py | 3 +- .../packages/plugins/commands/scrape.py | 4 +-- 7 files changed, 25 insertions(+), 25 deletions(-) diff --git a/daemon/atom/utils.py b/daemon/atom/utils.py index 4383b03..03453b4 100644 --- a/daemon/atom/utils.py +++ b/daemon/atom/utils.py @@ -4,6 +4,8 @@ # Copyright (C) 2022 Ammar Faizi <[email protected]> # +from enums import Platform + from pyrogram.types import Chat, InlineKeyboardMarkup, InlineKeyboardButton from email.message import Message from typing import Dict, Union @@ -115,19 +117,13 @@ def consruct_to_n_cc(to: list, cc: list): return ret -def gen_temp(name: str, platform: str): - platform = platform.lower() - plt_ls = ["telegram", "discord"] - - if platform not in plt_ls: - t = f"Platform {platform} is not found, " - t += f"only {', '.join(plt_ls)} is available" - raise ValueError(f"Platform {platform} is not found") - +def gen_temp(name: str, platform: Platform): + platform: str = platform.value md5 = hashlib.md5(name.encode()).hexdigest() store_dir = os.getenv("STORAGE_DIR", "storage") platform = platform.replace("discord", "dscord") path = f"{platform}/{store_dir}/{md5}" + try: os.mkdir(path) except FileExistsError: @@ -136,11 +132,11 @@ def gen_temp(name: str, platform: str): return path -def extract_body(thread: Message, platform: str): +def extract_body(thread: Message, platform: Platform): if not thread.is_multipart(): p = get_decoded_payload(thread) - if platform == "discord": + if platform is Platform.DISCORD: p = quote_reply(p) return f"{p}\n".lstrip(), [] @@ -183,12 +179,12 @@ def __is_patch(subject, content): return True -def create_template(thread: Message, platform: str, to=None, cc=None): +def create_template(thread: Message, platform: Platform, to=None, cc=None): if not to: to = extract_list("to", thread) if not cc: cc = extract_list("cc", thread) - if platform == "telegram": + if platform is Platform.TELEGRAM: substr = 4000 border = f"\n<code>{'-'*72}</code>" else: @@ -211,13 +207,13 @@ def create_template(thread: Message, platform: str, to=None, cc=None): if len(ret) >= substr: ret = ret[:substr] + "..." - ret = fix_utf8_char(ret, platform == "telegram") + ret = fix_utf8_char(ret, platform is Platform.TELEGRAM) ret += border return ret, files, is_patch -def prepare_patch(mail, text, url, platform: str): +def prepare_patch(mail, text, url, platform: Platform): tmp = gen_temp(url, platform) fnm = str(mail.get("subject")) sch = re.search(PATCH_PATTERN, fnm, re.IGNORECASE) @@ -237,7 +233,7 @@ def prepare_patch(mail, text, url, platform: str): f.write(bytes(text, encoding="utf8")) caption = "#patch #ml" - if platform == "telegram": + if platform is Platform.TELEGRAM: caption += fix_utf8_char("\n" + cap, True) return tmp, file, caption, url diff --git a/daemon/dscord/gnuweeb/client.py b/daemon/dscord/gnuweeb/client.py index b921f7d..82858c5 100644 --- a/daemon/dscord/gnuweeb/client.py +++ b/daemon/dscord/gnuweeb/client.py @@ -13,6 +13,7 @@ from typing import Union from . import filters from . import models from atom import utils +from enums import Platform from dscord.database import DB @@ -59,7 +60,7 @@ class GWClient(commands.Bot): reply_to: Union[int, None] = None, url: str = None): print("[send_patch_email]") tmp, doc, caption, url = utils.prepare_patch( - mail, text, url, "discord" + mail, text, url, Platform.DISCORD ) channel = self.get_channel(chat_id) @@ -90,7 +91,7 @@ class GWClient(commands.Bot): async def send_patch_mail_interaction(self, mail, i: "Interaction", text: str, url: str = None): tmp, doc, caption, url = utils.prepare_patch( - mail, text, url, "discord" + mail, text, url, Platform.DISCORD ) m = await i.response.send_message( content=caption, diff --git a/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py b/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py index 2d55d16..0c67b8c 100644 --- a/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py +++ b/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py @@ -11,6 +11,7 @@ from discord import app_commands from atom import utils from atom import Scraper +from enums import Platform class GetLoreSC(commands.Cog): @@ -26,7 +27,7 @@ class GetLoreSC(commands.Cog): async def get_lore(self, i: "Interaction", url: str): s = Scraper() mail = await s.get_email_from_url(url) - text, _, is_patch = utils.create_template(mail, "discord") + text, _, is_patch = utils.create_template(mail, Platform.DISCORD) if is_patch: m = await self.bot.send_patch_mail_interaction( diff --git a/daemon/dscord/mailer/listener.py b/daemon/dscord/mailer/listener.py index cc0a9f7..d986fbd 100644 --- a/daemon/dscord/mailer/listener.py +++ b/daemon/dscord/mailer/listener.py @@ -14,6 +14,7 @@ from discord import Message from dscord.gnuweeb import GWClient from atom.scraper import Scraper from atom import utils +from enums import Platform class Mutexes: @@ -107,7 +108,7 @@ class Listener: # return False - text, files, is_patch = utils.create_template(mail, "discord") + text, files, is_patch = utils.create_template(mail, Platform.DISCORD) reply_to = self.get_discord_reply(mail, dc_chat_id) url = str(re.sub(r"/raw$", "", url)) diff --git a/daemon/telegram/mailer/listener.py b/daemon/telegram/mailer/listener.py index 08feddd..1c92f23 100644 --- a/daemon/telegram/mailer/listener.py +++ b/daemon/telegram/mailer/listener.py @@ -9,8 +9,8 @@ from apscheduler.schedulers.asyncio import AsyncIOScheduler from telegram.packages import DaemonClient from atom import Scraper from atom import utils +from enums import Platform import asyncio -import shutil import re import traceback @@ -99,7 +99,7 @@ class Bot(): # return False - text, files, is_patch = utils.create_template(mail, "telegram") + text, files, is_patch = utils.create_template(mail, Platform.TELEGRAM) reply_to = self.get_reply(mail, tg_chat_id) url = str(re.sub(r"/raw$", "", url)) diff --git a/daemon/telegram/packages/client.py b/daemon/telegram/packages/client.py index 686e5ef..c971ea1 100644 --- a/daemon/telegram/packages/client.py +++ b/daemon/telegram/packages/client.py @@ -9,6 +9,7 @@ from pyrogram.types import Message, InlineKeyboardMarkup, InlineKeyboardButton from typing import Union from email.message import Message from atom import utils +from enums import Platform from telegram.database import DB from .decorator import handle_flood @@ -57,7 +58,7 @@ class DaemonClient(Client): ) -> Message: print("[send_patch_email]") tmp, doc, caption, url = utils.prepare_patch( - mail, text, url, "telegram" + mail, text, url, Platform.TELEGRAM ) m = await self.send_document( chat_id=chat_id, diff --git a/daemon/telegram/packages/plugins/commands/scrape.py b/daemon/telegram/packages/plugins/commands/scrape.py index 860e993..29cc8ad 100644 --- a/daemon/telegram/packages/plugins/commands/scrape.py +++ b/daemon/telegram/packages/plugins/commands/scrape.py @@ -9,8 +9,8 @@ from pyrogram import filters from telegram.packages import DaemonClient from atom import Scraper from atom import utils +from enums import Platform from telegram import config -import shutil import re import asyncio @@ -37,7 +37,7 @@ async def scrap_email(c: DaemonClient, m: Message): s = Scraper() mail = await s.get_email_from_url(url) - text, files, is_patch = utils.create_template(mail, "telegram") + text, files, is_patch = utils.create_template(mail, Platform.TELEGRAM) if is_patch: m = await c.send_patch_email( -- Muhammad Rizki ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v4 0/9] Fix some bugs and add some features 2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki ` (8 preceding siblings ...) 2022-10-22 6:51 ` [PATCH v4 9/9] enum: Use the created Platform class enumeration Muhammad Rizki @ 2022-10-22 7:44 ` Ammar Faizi 9 siblings, 0 replies; 11+ messages in thread From: Ammar Faizi @ 2022-10-22 7:44 UTC (permalink / raw) To: Muhammad Rizki Cc: Ammar Faizi, Alviro Iskandar Setiawan, GNU/Weeb Mailing List On Sat, 22 Oct 2022 13:51:40 +0700, Muhammad Rizki wrote: > This is v4 revision of fix some bugs and add some features. > This series is to fix > > This series is to fix some bugs, improve some codes, add some new > features and add enumeration. These bugs should have been fixed now, > and the email file attachments should have been removed after all > attachments have been sent. > > [...] Applied, thanks! [1/9] discord: Add send_text_mail_interaction() commit: d5aa7ce0be3681ebc976e5190bd3346d857d35ee [2/9] discord: Add send_patch_mail_interaction() commit: 6042615e846d4e62379aa1a1592444d7c072bfe9 [3/9] discord: Add get lore mail slash command commit: acd051cc10465c89f7c741fae65f3d5de558b30c [4/9] atom: Improve remove_patch() commit: 1afda802c0f2f7e6311cd0b9ef3a739ce4dcf179 [5/9] atom: add get_decoded_payload() commit: 0e5194f41e33914fc914e7a6300615ac3e39f1c5 [6/9] telegram: Fix get lore command commit: 4b24c92ef238a8c0b629c7fef616bfd690fae81a [7/9] atom: Improve extract_body() commit: d64c49ac8c40e5bf83eea6d04fe88813b19ac506 [8/9] enum: Add Platform enumeration commit: 7896dacc2a5f29bbc8aa72a23068b68a2b847a72 [9/9] enum: Use the created Platform class enumeration commit: 3c1fc79dcdc222a4cf8c594a4c7e352cd9103bed Best regards, -- Ammar Faizi ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2022-10-22 7:44 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 1/9] discord: Add send_text_mail_interaction() Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 2/9] discord: Add send_patch_mail_interaction() Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 3/9] discord: Add get lore mail slash command Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 4/9] atom: Improve remove_patch() Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 5/9] atom: add get_decoded_payload() Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 6/9] telegram: Fix get lore command Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 7/9] atom: Improve extract_body() Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 8/9] enum: Add Platform enumeration Muhammad Rizki 2022-10-22 6:51 ` [PATCH v4 9/9] enum: Use the created Platform class enumeration Muhammad Rizki 2022-10-22 7:44 ` [PATCH v4 0/9] Fix some bugs and add some features Ammar Faizi
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox