* [PATCH v4 0/9] Fix some bugs and add some features
@ 2022-10-22 6:51 Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 1/9] discord: Add send_text_mail_interaction() Muhammad Rizki
` (9 more replies)
0 siblings, 10 replies; 11+ messages in thread
From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Hi sir,
This is v4 revision of fix some bugs and add some features.
This series is to fix
This series is to fix some bugs, improve some codes, add some new
features and add enumeration. These bugs should have been fixed now,
and the email file attachments should have been removed after all
attachments have been sent.
Known bugs:
1. Email payload extraction result become unicode if the email payload
contain non-UTF8 characters like chinese, japanese, and similar
like that.
2. remove_patch() doesn't remove all file attachments properly.
Improvements:
1. Improve remove_patch() to make it all file attachments removed
after sending them.
2. Fix `/lore {raw atom url}` to add a "telegram" onto the
create_template() platform parameter.
3. Improvement on extract_body() to ignore whenever the email contain
text/html content-type and use the email library function to get the
header value.
New features:
1. Add send_text_mail_interaction() for the `/lore {raw atom url}` slash
command.
2. Add send_patch_mail_interaction() for the `/lore {raw atom url}`
slash command.
3. Add `/lore {raw atom url}` slash command.
4. Platform class enumeration
There are 9 patches in this series:
- Patch 1 is to add send_text_mail_interaction()
- Patch 2 is to add send_patch_mail_interaction()
- Patch 3 is to add `/lore` slash command
- Patch 4 is to improve remove_patch() code to make it more stable
- Patch 5 is to add manage_payload() for manage email payload extraction
- Patch 6 is to fix the Telegram `/lore` command
- Patch 7 is to improve extract_body() to ignore text/html content-type
- Patch 8 is to add initial Platform class enumeration
- Patch 9 is to use the created Platform class enumeration
How to use:
1. Execute the db.sql file in the daemon directory,
2. Setup .env file, the example is there with suffix .example, this
file name must remove the suffix name .example,
3. Set up the config.py in each bot directory, such as dscord and
telegram. The example is there with suffix .example & the file name
must remove suffix name .example,
4. Run `pip3 install -r requirements.txt` in each bot directory,
5. STORAGE_DIR env value must `storage` to make it work fine,
6. Run the bot by `python3 dc.py` or `python3 tg.py`.
Tested and works, what do you think with this patch?
## Changelog
v3 -> v4:
- Fix the decode error `invalid start byte`
- Rename manage_payload() to get_decoded_payload()
- Use charset to decode the payload
- Use the email library function to get the header value
v2 -> v3:
- Add Platform class enumeration
- Use the old one of fix_utf8_char() instead
- Add Content-Transfer-Encoding handler in manage_payload()
v1 -> v2:
- Remove re.sub() in the fix_utf8_char() when set unescape to True
- Add if statement logic to ignore text/html in the extract_body()
Signed-off-by: Muhammad Rizki <[email protected]>
---
Muhammad Rizki (9):
discord: Add send_text_mail_interaction()
discord: Add send_patch_mail_interaction()
discord: Add get lore mail slash command
atom: Improve remove_patch()
atom: add get_decoded_payload()
telegram: Fix get lore command
atom: Improve extract_body()
enum: Add Platform enumeration
enum: Use the created Platform class enumeration
daemon/atom/utils.py | 61 ++++++++++++-------
daemon/dscord/gnuweeb/client.py | 26 +++++++-
.../plugins/slash_commands/__init__.py | 2 +
.../plugins/slash_commands/get_lore_mail.py | 40 ++++++++++++
daemon/dscord/mailer/listener.py | 7 ++-
daemon/enums/__init__.py | 1 +
daemon/enums/base.py | 9 +++
daemon/enums/platform.py | 7 +++
daemon/telegram/mailer/listener.py | 7 +--
daemon/telegram/packages/client.py | 3 +-
.../packages/plugins/commands/scrape.py | 7 +--
11 files changed, 136 insertions(+), 34 deletions(-)
create mode 100644 daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py
create mode 100644 daemon/enums/__init__.py
create mode 100644 daemon/enums/base.py
create mode 100644 daemon/enums/platform.py
base-commit: d9b20dab81202b93f48d5365ad680796e5839d80
--
Muhammad Rizki
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v4 1/9] discord: Add send_text_mail_interaction()
2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki
@ 2022-10-22 6:51 ` Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 2/9] discord: Add send_patch_mail_interaction() Muhammad Rizki
` (8 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Add send_text_mail_interaction() for the `/atom get {url}` to get the
specific lore email message from an URL in future use.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/dscord/gnuweeb/client.py | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/daemon/dscord/gnuweeb/client.py b/daemon/dscord/gnuweeb/client.py
index 03a1b8c..15a6eee 100644
--- a/daemon/dscord/gnuweeb/client.py
+++ b/daemon/dscord/gnuweeb/client.py
@@ -4,6 +4,7 @@
#
import discord
+from discord import Interaction
from discord.ext import commands
from discord import Intents
from dscord.config import ACTIVITY_NAME
@@ -76,3 +77,11 @@ class GWClient(commands.Bot):
utils.remove_patch(tmp)
return m
+
+
+ async def send_text_mail_interaction(self, i: "Interaction",
+ text: str, url: str = None):
+ return await i.response.send_message(
+ content=text,
+ view=models.FullMessageBtn(url)
+ )
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 2/9] discord: Add send_patch_mail_interaction()
2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 1/9] discord: Add send_text_mail_interaction() Muhammad Rizki
@ 2022-10-22 6:51 ` Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 3/9] discord: Add get lore mail slash command Muhammad Rizki
` (7 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
This function is just the same as send_text_mail_interaction(), the
different between them is send_patch_mail_interaction() for sending a
lore email message with patch file attachment.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/dscord/gnuweeb/client.py | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/daemon/dscord/gnuweeb/client.py b/daemon/dscord/gnuweeb/client.py
index 15a6eee..b921f7d 100644
--- a/daemon/dscord/gnuweeb/client.py
+++ b/daemon/dscord/gnuweeb/client.py
@@ -85,3 +85,17 @@ class GWClient(commands.Bot):
content=text,
view=models.FullMessageBtn(url)
)
+
+
+ async def send_patch_mail_interaction(self, mail, i: "Interaction",
+ text: str, url: str = None):
+ tmp, doc, caption, url = utils.prepare_patch(
+ mail, text, url, "discord"
+ )
+ m = await i.response.send_message(
+ content=caption,
+ file=discord.File(doc),
+ view=models.FullMessageBtn(url)
+ )
+ utils.remove_patch(tmp)
+ return m
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 3/9] discord: Add get lore mail slash command
2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 1/9] discord: Add send_text_mail_interaction() Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 2/9] discord: Add send_patch_mail_interaction() Muhammad Rizki
@ 2022-10-22 6:51 ` Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 4/9] atom: Improve remove_patch() Muhammad Rizki
` (6 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Add a `/lore {url}` slash command to get the specific lore email message
from the raw atom URL.
Signed-off-by: Muhammad Rizki <[email protected]>
---
.../plugins/slash_commands/__init__.py | 2 +
.../plugins/slash_commands/get_lore_mail.py | 39 +++++++++++++++++++
2 files changed, 41 insertions(+)
create mode 100644 daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py
diff --git a/daemon/dscord/gnuweeb/plugins/slash_commands/__init__.py b/daemon/dscord/gnuweeb/plugins/slash_commands/__init__.py
index a6d913c..126af45 100644
--- a/daemon/dscord/gnuweeb/plugins/slash_commands/__init__.py
+++ b/daemon/dscord/gnuweeb/plugins/slash_commands/__init__.py
@@ -5,9 +5,11 @@
from .manage_atom import ManageAtomSC
from .manage_broadcast import ManageBroadcastSC
+from .get_lore_mail import GetLoreSC
class SlashCommands(
+ GetLoreSC,
ManageAtomSC,
ManageBroadcastSC
): pass
diff --git a/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py b/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py
new file mode 100644
index 0000000..2d55d16
--- /dev/null
+++ b/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py
@@ -0,0 +1,39 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+import asyncio
+import discord
+from discord.ext import commands
+from discord import Interaction
+from discord import app_commands
+
+from atom import utils
+from atom import Scraper
+
+
+class GetLoreSC(commands.Cog):
+ def __init__(self, bot) -> None:
+ self.bot = bot
+
+
+ @app_commands.command(
+ name="lore",
+ description="Get lore email from raw email URL."
+ )
+ @app_commands.describe(url="Raw lore email URL")
+ async def get_lore(self, i: "Interaction", url: str):
+ s = Scraper()
+ mail = await s.get_email_from_url(url)
+ text, _, is_patch = utils.create_template(mail, "discord")
+
+ if is_patch:
+ m = await self.bot.send_patch_mail_interaction(
+ mail=mail, i=i, text=text, url=url
+ )
+ else:
+ text = "#ml\n" + text
+ m = await self.bot.send_text_mail_interaction(
+ i=i, text=text, url=url
+ )
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 4/9] atom: Improve remove_patch()
2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki
` (2 preceding siblings ...)
2022-10-22 6:51 ` [PATCH v4 3/9] discord: Add get lore mail slash command Muhammad Rizki
@ 2022-10-22 6:51 ` Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 5/9] atom: add get_decoded_payload() Muhammad Rizki
` (5 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Improvement on remove_patch(). So, after send all the email attachments
then remove them. On the previous patch, this function can't remove
them properly.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/atom/utils.py | 10 +++++++---
daemon/dscord/mailer/listener.py | 4 ++--
daemon/telegram/mailer/listener.py | 3 +--
daemon/telegram/packages/plugins/commands/scrape.py | 3 +--
4 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/daemon/atom/utils.py b/daemon/atom/utils.py
index a30d5cb..f554f6f 100644
--- a/daemon/atom/utils.py
+++ b/daemon/atom/utils.py
@@ -6,7 +6,7 @@
from pyrogram.types import Chat, InlineKeyboardMarkup, InlineKeyboardButton
from email.message import Message
-from typing import Dict
+from typing import Dict, Union
from slugify import slugify
import hashlib
import uuid
@@ -238,8 +238,12 @@ def prepare_patch(mail, text, url, platform: str):
return tmp, file, caption, url
-def remove_patch(tmp):
- shutil.rmtree(tmp)
+def remove_patch(tmp: Union[str, list]):
+ if isinstance(tmp, str):
+ return shutil.rmtree(tmp)
+
+ for d,_ in tmp:
+ shutil.rmtree(d)
def fix_utf8_char(text: str, html_escape: bool = True):
diff --git a/daemon/dscord/mailer/listener.py b/daemon/dscord/mailer/listener.py
index a280a58..cc0a9f7 100644
--- a/daemon/dscord/mailer/listener.py
+++ b/daemon/dscord/mailer/listener.py
@@ -125,10 +125,10 @@ class Listener:
for d, f in files:
await m.reply(file=File(f"{d}/{f}"))
- if files.index((d,f)) == len(files)-1:
- utils.remove_patch(d)
await asyncio.sleep(1)
+ utils.remove_patch(files)
+
return True
diff --git a/daemon/telegram/mailer/listener.py b/daemon/telegram/mailer/listener.py
index 208aed0..08feddd 100644
--- a/daemon/telegram/mailer/listener.py
+++ b/daemon/telegram/mailer/listener.py
@@ -118,8 +118,7 @@ class Bot():
await m.reply_document(f"{d}/{f}", file_name=f)
await asyncio.sleep(1)
- if files:
- shutil.rmtree(str(files[0][0]))
+ utils.remove_patch(files)
return True
diff --git a/daemon/telegram/packages/plugins/commands/scrape.py b/daemon/telegram/packages/plugins/commands/scrape.py
index d4d10a9..52ddb0b 100644
--- a/daemon/telegram/packages/plugins/commands/scrape.py
+++ b/daemon/telegram/packages/plugins/commands/scrape.py
@@ -53,5 +53,4 @@ async def scrap_email(c: DaemonClient, m: Message):
await m.reply_document(f"{d}/{f}", file_name=f)
await asyncio.sleep(1)
- if files:
- shutil.rmtree(str(files[0][0]))
+ utils.remove_patch(files)
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 5/9] atom: add get_decoded_payload()
2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki
` (3 preceding siblings ...)
2022-10-22 6:51 ` [PATCH v4 4/9] atom: Improve remove_patch() Muhammad Rizki
@ 2022-10-22 6:51 ` Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 6/9] telegram: Fix get lore command Muhammad Rizki
` (4 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Add get_decoded_payload() to handle the email decoding to utf-8. This
include a non-UTF8 character, base64 decoding, and quoted-printable
decoding.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/atom/utils.py | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/daemon/atom/utils.py b/daemon/atom/utils.py
index f554f6f..deff99d 100644
--- a/daemon/atom/utils.py
+++ b/daemon/atom/utils.py
@@ -8,6 +8,7 @@ from pyrogram.types import Chat, InlineKeyboardMarkup, InlineKeyboardButton
from email.message import Message
from typing import Dict, Union
from slugify import slugify
+from base64 import b64decode
import hashlib
import uuid
import os
@@ -15,6 +16,7 @@ import re
import shutil
import httpx
import html
+import quopri
def get_email_msg_id(mail):
@@ -136,7 +138,7 @@ def gen_temp(name: str, platform: str):
def extract_body(thread: Message, platform: str):
if not thread.is_multipart():
- p = thread.get_payload(decode=True).decode(errors='replace')
+ p = get_decoded_payload(thread)
if platform == "discord":
p = quote_reply(p)
@@ -253,6 +255,20 @@ def fix_utf8_char(text: str, html_escape: bool = True):
return t
+def get_decoded_payload(payload: Message):
+ p = str(payload.get_payload())
+ tf_encode = payload.get("Content-Transfer-Encoding")
+ charset = payload.get_content_charset("utf-8")
+
+ if tf_encode == "base64":
+ return b64decode(p).decode(charset)
+ if tf_encode == "quoted-printable":
+ quobyte = quopri.decodestring(p.encode())
+ return quobyte.decode(charset)
+
+ return p.encode().decode(charset, errors="replace")
+
+
EMAIL_MSG_ID_PATTERN = r"<([^\<\>]+)>"
def extract_email_msg_id(msg_id):
ret = re.search(EMAIL_MSG_ID_PATTERN, msg_id)
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 6/9] telegram: Fix get lore command
2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki
` (4 preceding siblings ...)
2022-10-22 6:51 ` [PATCH v4 5/9] atom: add get_decoded_payload() Muhammad Rizki
@ 2022-10-22 6:51 ` Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 7/9] atom: Improve extract_body() Muhammad Rizki
` (3 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Fix the `/lore {raw lore url}` to add "telegram" on the
create_template() platform parameter.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/telegram/packages/plugins/commands/scrape.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/daemon/telegram/packages/plugins/commands/scrape.py b/daemon/telegram/packages/plugins/commands/scrape.py
index 52ddb0b..860e993 100644
--- a/daemon/telegram/packages/plugins/commands/scrape.py
+++ b/daemon/telegram/packages/plugins/commands/scrape.py
@@ -37,7 +37,7 @@ async def scrap_email(c: DaemonClient, m: Message):
s = Scraper()
mail = await s.get_email_from_url(url)
- text, files, is_patch = utils.create_template(mail)
+ text, files, is_patch = utils.create_template(mail, "telegram")
if is_patch:
m = await c.send_patch_email(
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 7/9] atom: Improve extract_body()
2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki
` (5 preceding siblings ...)
2022-10-22 6:51 ` [PATCH v4 6/9] telegram: Fix get lore command Muhammad Rizki
@ 2022-10-22 6:51 ` Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 8/9] enum: Add Platform enumeration Muhammad Rizki
` (2 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Add an if statement to ignore whenever the email payload containing a
text/html content-type header.
v1 -> v2:
- Simplify the usage to get the email header by using the library get
function itself.
Signed-off-by: Muhammad Rizki <[email protected]>
Reviewed-by: Alviro Iskandar Setiawan <[email protected]>
---
daemon/atom/utils.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/daemon/atom/utils.py b/daemon/atom/utils.py
index deff99d..4383b03 100644
--- a/daemon/atom/utils.py
+++ b/daemon/atom/utils.py
@@ -151,10 +151,13 @@ def extract_body(thread: Message, platform: str):
fname = p.get_filename()
payload = p.get_payload(decode=True)
+ if p.get_content_type() == "text/html":
+ continue
+
if not payload:
continue
- if 'inline' in [p.get('content-disposition')] or not bool(fname):
+ if p.get_content_disposition() == 'inline' or not bool(fname):
ret += f"{payload.decode(errors='replace')}\n".lstrip()
continue
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 8/9] enum: Add Platform enumeration
2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki
` (6 preceding siblings ...)
2022-10-22 6:51 ` [PATCH v4 7/9] atom: Improve extract_body() Muhammad Rizki
@ 2022-10-22 6:51 ` Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 9/9] enum: Use the created Platform class enumeration Muhammad Rizki
2022-10-22 7:44 ` [PATCH v4 0/9] Fix some bugs and add some features Ammar Faizi
9 siblings, 0 replies; 11+ messages in thread
From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Add a Platform enum class to use it for enumeration instead of a string.
Signed-off-by: Muhammad Rizki <[email protected]>
Acked-by: Alviro Iskandar Setiawan <[email protected]>
---
daemon/enums/__init__.py | 1 +
daemon/enums/base.py | 9 +++++++++
daemon/enums/platform.py | 7 +++++++
3 files changed, 17 insertions(+)
create mode 100644 daemon/enums/__init__.py
create mode 100644 daemon/enums/base.py
create mode 100644 daemon/enums/platform.py
diff --git a/daemon/enums/__init__.py b/daemon/enums/__init__.py
new file mode 100644
index 0000000..efe9c79
--- /dev/null
+++ b/daemon/enums/__init__.py
@@ -0,0 +1 @@
+from .platform import Platform
diff --git a/daemon/enums/base.py b/daemon/enums/base.py
new file mode 100644
index 0000000..526aba5
--- /dev/null
+++ b/daemon/enums/base.py
@@ -0,0 +1,9 @@
+import enum
+
+
+class EnumBase(enum.Enum):
+ def _generate_next_value_(self, *args):
+ return self.lower()
+
+ def __repr__(self):
+ return f"enums.{self}"
diff --git a/daemon/enums/platform.py b/daemon/enums/platform.py
new file mode 100644
index 0000000..638d8dd
--- /dev/null
+++ b/daemon/enums/platform.py
@@ -0,0 +1,7 @@
+import enum
+from .base import EnumBase
+
+
+class Platform(EnumBase):
+ DISCORD = enum.auto()
+ TELEGRAM = enum.auto()
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v4 9/9] enum: Use the created Platform class enumeration
2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki
` (7 preceding siblings ...)
2022-10-22 6:51 ` [PATCH v4 8/9] enum: Add Platform enumeration Muhammad Rizki
@ 2022-10-22 6:51 ` Muhammad Rizki
2022-10-22 7:44 ` [PATCH v4 0/9] Fix some bugs and add some features Ammar Faizi
9 siblings, 0 replies; 11+ messages in thread
From: Muhammad Rizki @ 2022-10-22 6:51 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Use the Platform class enumeration to make it more practice, avoid
getting typo, type-hinted suggestion to easily manageable-maintainable.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/atom/utils.py | 28 ++++++++-----------
daemon/dscord/gnuweeb/client.py | 5 ++--
.../plugins/slash_commands/get_lore_mail.py | 3 +-
daemon/dscord/mailer/listener.py | 3 +-
daemon/telegram/mailer/listener.py | 4 +--
daemon/telegram/packages/client.py | 3 +-
.../packages/plugins/commands/scrape.py | 4 +--
7 files changed, 25 insertions(+), 25 deletions(-)
diff --git a/daemon/atom/utils.py b/daemon/atom/utils.py
index 4383b03..03453b4 100644
--- a/daemon/atom/utils.py
+++ b/daemon/atom/utils.py
@@ -4,6 +4,8 @@
# Copyright (C) 2022 Ammar Faizi <[email protected]>
#
+from enums import Platform
+
from pyrogram.types import Chat, InlineKeyboardMarkup, InlineKeyboardButton
from email.message import Message
from typing import Dict, Union
@@ -115,19 +117,13 @@ def consruct_to_n_cc(to: list, cc: list):
return ret
-def gen_temp(name: str, platform: str):
- platform = platform.lower()
- plt_ls = ["telegram", "discord"]
-
- if platform not in plt_ls:
- t = f"Platform {platform} is not found, "
- t += f"only {', '.join(plt_ls)} is available"
- raise ValueError(f"Platform {platform} is not found")
-
+def gen_temp(name: str, platform: Platform):
+ platform: str = platform.value
md5 = hashlib.md5(name.encode()).hexdigest()
store_dir = os.getenv("STORAGE_DIR", "storage")
platform = platform.replace("discord", "dscord")
path = f"{platform}/{store_dir}/{md5}"
+
try:
os.mkdir(path)
except FileExistsError:
@@ -136,11 +132,11 @@ def gen_temp(name: str, platform: str):
return path
-def extract_body(thread: Message, platform: str):
+def extract_body(thread: Message, platform: Platform):
if not thread.is_multipart():
p = get_decoded_payload(thread)
- if platform == "discord":
+ if platform is Platform.DISCORD:
p = quote_reply(p)
return f"{p}\n".lstrip(), []
@@ -183,12 +179,12 @@ def __is_patch(subject, content):
return True
-def create_template(thread: Message, platform: str, to=None, cc=None):
+def create_template(thread: Message, platform: Platform, to=None, cc=None):
if not to:
to = extract_list("to", thread)
if not cc:
cc = extract_list("cc", thread)
- if platform == "telegram":
+ if platform is Platform.TELEGRAM:
substr = 4000
border = f"\n<code>{'-'*72}</code>"
else:
@@ -211,13 +207,13 @@ def create_template(thread: Message, platform: str, to=None, cc=None):
if len(ret) >= substr:
ret = ret[:substr] + "..."
- ret = fix_utf8_char(ret, platform == "telegram")
+ ret = fix_utf8_char(ret, platform is Platform.TELEGRAM)
ret += border
return ret, files, is_patch
-def prepare_patch(mail, text, url, platform: str):
+def prepare_patch(mail, text, url, platform: Platform):
tmp = gen_temp(url, platform)
fnm = str(mail.get("subject"))
sch = re.search(PATCH_PATTERN, fnm, re.IGNORECASE)
@@ -237,7 +233,7 @@ def prepare_patch(mail, text, url, platform: str):
f.write(bytes(text, encoding="utf8"))
caption = "#patch #ml"
- if platform == "telegram":
+ if platform is Platform.TELEGRAM:
caption += fix_utf8_char("\n" + cap, True)
return tmp, file, caption, url
diff --git a/daemon/dscord/gnuweeb/client.py b/daemon/dscord/gnuweeb/client.py
index b921f7d..82858c5 100644
--- a/daemon/dscord/gnuweeb/client.py
+++ b/daemon/dscord/gnuweeb/client.py
@@ -13,6 +13,7 @@ from typing import Union
from . import filters
from . import models
from atom import utils
+from enums import Platform
from dscord.database import DB
@@ -59,7 +60,7 @@ class GWClient(commands.Bot):
reply_to: Union[int, None] = None, url: str = None):
print("[send_patch_email]")
tmp, doc, caption, url = utils.prepare_patch(
- mail, text, url, "discord"
+ mail, text, url, Platform.DISCORD
)
channel = self.get_channel(chat_id)
@@ -90,7 +91,7 @@ class GWClient(commands.Bot):
async def send_patch_mail_interaction(self, mail, i: "Interaction",
text: str, url: str = None):
tmp, doc, caption, url = utils.prepare_patch(
- mail, text, url, "discord"
+ mail, text, url, Platform.DISCORD
)
m = await i.response.send_message(
content=caption,
diff --git a/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py b/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py
index 2d55d16..0c67b8c 100644
--- a/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py
+++ b/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py
@@ -11,6 +11,7 @@ from discord import app_commands
from atom import utils
from atom import Scraper
+from enums import Platform
class GetLoreSC(commands.Cog):
@@ -26,7 +27,7 @@ class GetLoreSC(commands.Cog):
async def get_lore(self, i: "Interaction", url: str):
s = Scraper()
mail = await s.get_email_from_url(url)
- text, _, is_patch = utils.create_template(mail, "discord")
+ text, _, is_patch = utils.create_template(mail, Platform.DISCORD)
if is_patch:
m = await self.bot.send_patch_mail_interaction(
diff --git a/daemon/dscord/mailer/listener.py b/daemon/dscord/mailer/listener.py
index cc0a9f7..d986fbd 100644
--- a/daemon/dscord/mailer/listener.py
+++ b/daemon/dscord/mailer/listener.py
@@ -14,6 +14,7 @@ from discord import Message
from dscord.gnuweeb import GWClient
from atom.scraper import Scraper
from atom import utils
+from enums import Platform
class Mutexes:
@@ -107,7 +108,7 @@ class Listener:
#
return False
- text, files, is_patch = utils.create_template(mail, "discord")
+ text, files, is_patch = utils.create_template(mail, Platform.DISCORD)
reply_to = self.get_discord_reply(mail, dc_chat_id)
url = str(re.sub(r"/raw$", "", url))
diff --git a/daemon/telegram/mailer/listener.py b/daemon/telegram/mailer/listener.py
index 08feddd..1c92f23 100644
--- a/daemon/telegram/mailer/listener.py
+++ b/daemon/telegram/mailer/listener.py
@@ -9,8 +9,8 @@ from apscheduler.schedulers.asyncio import AsyncIOScheduler
from telegram.packages import DaemonClient
from atom import Scraper
from atom import utils
+from enums import Platform
import asyncio
-import shutil
import re
import traceback
@@ -99,7 +99,7 @@ class Bot():
#
return False
- text, files, is_patch = utils.create_template(mail, "telegram")
+ text, files, is_patch = utils.create_template(mail, Platform.TELEGRAM)
reply_to = self.get_reply(mail, tg_chat_id)
url = str(re.sub(r"/raw$", "", url))
diff --git a/daemon/telegram/packages/client.py b/daemon/telegram/packages/client.py
index 686e5ef..c971ea1 100644
--- a/daemon/telegram/packages/client.py
+++ b/daemon/telegram/packages/client.py
@@ -9,6 +9,7 @@ from pyrogram.types import Message, InlineKeyboardMarkup, InlineKeyboardButton
from typing import Union
from email.message import Message
from atom import utils
+from enums import Platform
from telegram.database import DB
from .decorator import handle_flood
@@ -57,7 +58,7 @@ class DaemonClient(Client):
) -> Message:
print("[send_patch_email]")
tmp, doc, caption, url = utils.prepare_patch(
- mail, text, url, "telegram"
+ mail, text, url, Platform.TELEGRAM
)
m = await self.send_document(
chat_id=chat_id,
diff --git a/daemon/telegram/packages/plugins/commands/scrape.py b/daemon/telegram/packages/plugins/commands/scrape.py
index 860e993..29cc8ad 100644
--- a/daemon/telegram/packages/plugins/commands/scrape.py
+++ b/daemon/telegram/packages/plugins/commands/scrape.py
@@ -9,8 +9,8 @@ from pyrogram import filters
from telegram.packages import DaemonClient
from atom import Scraper
from atom import utils
+from enums import Platform
from telegram import config
-import shutil
import re
import asyncio
@@ -37,7 +37,7 @@ async def scrap_email(c: DaemonClient, m: Message):
s = Scraper()
mail = await s.get_email_from_url(url)
- text, files, is_patch = utils.create_template(mail, "telegram")
+ text, files, is_patch = utils.create_template(mail, Platform.TELEGRAM)
if is_patch:
m = await c.send_patch_email(
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v4 0/9] Fix some bugs and add some features
2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki
` (8 preceding siblings ...)
2022-10-22 6:51 ` [PATCH v4 9/9] enum: Use the created Platform class enumeration Muhammad Rizki
@ 2022-10-22 7:44 ` Ammar Faizi
9 siblings, 0 replies; 11+ messages in thread
From: Ammar Faizi @ 2022-10-22 7:44 UTC (permalink / raw)
To: Muhammad Rizki
Cc: Ammar Faizi, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
On Sat, 22 Oct 2022 13:51:40 +0700, Muhammad Rizki wrote:
> This is v4 revision of fix some bugs and add some features.
> This series is to fix
>
> This series is to fix some bugs, improve some codes, add some new
> features and add enumeration. These bugs should have been fixed now,
> and the email file attachments should have been removed after all
> attachments have been sent.
>
> [...]
Applied, thanks!
[1/9] discord: Add send_text_mail_interaction()
commit: d5aa7ce0be3681ebc976e5190bd3346d857d35ee
[2/9] discord: Add send_patch_mail_interaction()
commit: 6042615e846d4e62379aa1a1592444d7c072bfe9
[3/9] discord: Add get lore mail slash command
commit: acd051cc10465c89f7c741fae65f3d5de558b30c
[4/9] atom: Improve remove_patch()
commit: 1afda802c0f2f7e6311cd0b9ef3a739ce4dcf179
[5/9] atom: add get_decoded_payload()
commit: 0e5194f41e33914fc914e7a6300615ac3e39f1c5
[6/9] telegram: Fix get lore command
commit: 4b24c92ef238a8c0b629c7fef616bfd690fae81a
[7/9] atom: Improve extract_body()
commit: d64c49ac8c40e5bf83eea6d04fe88813b19ac506
[8/9] enum: Add Platform enumeration
commit: 7896dacc2a5f29bbc8aa72a23068b68a2b847a72
[9/9] enum: Use the created Platform class enumeration
commit: 3c1fc79dcdc222a4cf8c594a4c7e352cd9103bed
Best regards,
--
Ammar Faizi
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2022-10-22 7:44 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-10-22 6:51 [PATCH v4 0/9] Fix some bugs and add some features Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 1/9] discord: Add send_text_mail_interaction() Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 2/9] discord: Add send_patch_mail_interaction() Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 3/9] discord: Add get lore mail slash command Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 4/9] atom: Improve remove_patch() Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 5/9] atom: add get_decoded_payload() Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 6/9] telegram: Fix get lore command Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 7/9] atom: Improve extract_body() Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 8/9] enum: Add Platform enumeration Muhammad Rizki
2022-10-22 6:51 ` [PATCH v4 9/9] enum: Use the created Platform class enumeration Muhammad Rizki
2022-10-22 7:44 ` [PATCH v4 0/9] Fix some bugs and add some features Ammar Faizi
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox