* [PATCH v1 00/11] Plan to redesign code
@ 2022-07-18 11:19 Muhammad Rizki
2022-07-18 11:19 ` [PATCH v1 01/11] Fix __send_patch_msg function parameter Muhammad Rizki
` (12 more replies)
0 siblings, 13 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-07-18 11:19 UTC (permalink / raw)
To: Ammar Faizi; +Cc: Muhammad Rizki, GNU/Weeb Mailing List
Hi sir,
In this series I want to redesign codes to make it clear and clean.
I've inherit the DaemonClient with the Pyrogram's Client, so with
this method we can create our custom functions, such as
send email message and patch message in the DaemonClient.
There 10 patches in this series:
- Patch 1 is to fix the lack of __send_patch_msg function parameter
- Patch 2 is just add required imports and remove unused imports
- Patch 3 is to add default temporary directory name
- Patch 4 is to move Telegram bot session file
- Patch 5 is to add traceback to get the error detail while debugging
- Patch 6 is to create send email and patch msg in the DaemonClient
- Patch 7 is just remove the ____send_patch_msg
- Patch 8 is to move utility functions to the utils file
- Patch 9 is to create fix_utf8_chars function
- Patch 10 is just cleaning some codes to make it looks more practice
Please give it a test, thanks.
Signed-off-by: Muhammad Rizki <[email protected]>
---
Ammar Faizi (1):
daemon: Fix raw lore URL on the inline keyboard button
Muhammad Rizki (10):
Fix __send_patch_msg function parameter
Fix import problem
Add default temporary directory
Move the Telegram bot session into the storage directory
Add traceback to get the error detail
Re-design send email message to Telegram
Move ____send_patch_msg
Move prepare for patch and clean up patch functions
Create fix_utf8_chars function
Recode some codes
daemon/packages/__init__.py | 1 +
daemon/packages/client.py | 59 ++++++++++++++++++++++++++++++
daemon/packages/plugins/scrape.py | 1 +
daemon/run.py | 8 ++--
daemon/scraper/bot.py | 61 +++++++------------------------
daemon/scraper/scraper.py | 4 --
daemon/scraper/utils.py | 50 +++++++++++++++++++++----
7 files changed, 121 insertions(+), 63 deletions(-)
create mode 100644 daemon/packages/__init__.py
create mode 100644 daemon/packages/client.py
base-commit: 339d992bdd77f19e67d0d9a30e3ac6eef1e4035a
--
Muhammad Rizki
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v1 01/11] Fix __send_patch_msg function parameter
2022-07-18 11:19 [PATCH v1 00/11] Plan to redesign code Muhammad Rizki
@ 2022-07-18 11:19 ` Muhammad Rizki
2022-07-18 12:45 ` Ammar Faizi
2022-07-18 11:19 ` [PATCH v1 02/11] Fix import problem Muhammad Rizki
` (11 subsequent siblings)
12 siblings, 1 reply; 20+ messages in thread
From: Muhammad Rizki @ 2022-07-18 11:19 UTC (permalink / raw)
To: Ammar Faizi; +Cc: Muhammad Rizki, GNU/Weeb Mailing List
Lack of reply_to parameter in __send_patch_msg function.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/scraper/bot.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/daemon/scraper/bot.py b/daemon/scraper/bot.py
index 7746eb8..9b95d50 100644
--- a/daemon/scraper/bot.py
+++ b/daemon/scraper/bot.py
@@ -157,7 +157,7 @@ class Bot():
return self.db.get_tg_reply_to(reply_to, tg_chat_id)
- async def __send_patch_msg(self, mail, tg_chat_id, text, url):
+ async def __send_patch_msg(self, mail, tg_chat_id, reply_to, text, url):
print("[__send_patch_msg]")
tmp, fnm, caption, url = Bot.prepare_send_patch(mail, text, url)
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v1 02/11] Fix import problem
2022-07-18 11:19 [PATCH v1 00/11] Plan to redesign code Muhammad Rizki
2022-07-18 11:19 ` [PATCH v1 01/11] Fix __send_patch_msg function parameter Muhammad Rizki
@ 2022-07-18 11:19 ` Muhammad Rizki
2022-07-18 11:19 ` [PATCH v1 03/11] Add default temporary directory Muhammad Rizki
` (10 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-07-18 11:19 UTC (permalink / raw)
To: Ammar Faizi; +Cc: Muhammad Rizki, GNU/Weeb Mailing List
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/packages/plugins/scrape.py | 1 +
daemon/run.py | 2 --
daemon/scraper/bot.py | 1 -
daemon/scraper/scraper.py | 4 ----
4 files changed, 1 insertion(+), 7 deletions(-)
diff --git a/daemon/packages/plugins/scrape.py b/daemon/packages/plugins/scrape.py
index c9da71a..1698c6d 100644
--- a/daemon/packages/plugins/scrape.py
+++ b/daemon/packages/plugins/scrape.py
@@ -15,6 +15,7 @@ from scraper import utils
from scraper import Bot
import shutil
import re
+import asyncio
#
diff --git a/daemon/run.py b/daemon/run.py
index ef1d8f4..0a650cd 100644
--- a/daemon/run.py
+++ b/daemon/run.py
@@ -5,14 +5,12 @@
#
from apscheduler.schedulers.asyncio import AsyncIOScheduler
-from email.message import Message
from scraper import BotMutexes
from dotenv import load_dotenv
from mysql import connector
from pyrogram import Client
from scraper import Scraper
from scraper import Bot
-import asyncio
import os
diff --git a/daemon/scraper/bot.py b/daemon/scraper/bot.py
index 9b95d50..c62c554 100644
--- a/daemon/scraper/bot.py
+++ b/daemon/scraper/bot.py
@@ -13,7 +13,6 @@ from scraper import Scraper
from pyrogram import enums
from . import utils
from .db import Db
-import xmltodict
import pyrogram
import asyncio
import shutil
diff --git a/daemon/scraper/scraper.py b/daemon/scraper/scraper.py
index b70df31..2d5942b 100644
--- a/daemon/scraper/scraper.py
+++ b/daemon/scraper/scraper.py
@@ -5,14 +5,10 @@
#
from typing import Dict, List
-from .db import Db
import email.policy
import xmltodict
-import operator
-import asyncio
import httpx
import email
-import json
class Scraper():
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v1 03/11] Add default temporary directory
2022-07-18 11:19 [PATCH v1 00/11] Plan to redesign code Muhammad Rizki
2022-07-18 11:19 ` [PATCH v1 01/11] Fix __send_patch_msg function parameter Muhammad Rizki
2022-07-18 11:19 ` [PATCH v1 02/11] Fix import problem Muhammad Rizki
@ 2022-07-18 11:19 ` Muhammad Rizki
2022-07-18 11:31 ` Ammar Faizi
2022-07-18 11:20 ` [PATCH v1 04/11] Move the Telegram bot session into the storage directory Muhammad Rizki
` (9 subsequent siblings)
12 siblings, 1 reply; 20+ messages in thread
From: Muhammad Rizki @ 2022-07-18 11:19 UTC (permalink / raw)
To: Ammar Faizi; +Cc: Muhammad Rizki, GNU/Weeb Mailing List
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/scraper/utils.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/daemon/scraper/utils.py b/daemon/scraper/utils.py
index 9334b82..10fe956 100644
--- a/daemon/scraper/utils.py
+++ b/daemon/scraper/utils.py
@@ -110,7 +110,7 @@ def consruct_to_n_cc(to: list, cc: list):
def gen_temp(name: str):
md5 = hashlib.md5(name.encode()).hexdigest()
- ret = os.getenv("STORAGE_DIR") + "/" + md5
+ ret = os.getenv("STORAGE_DIR", "storage") + "/" + md5
try:
os.mkdir(ret)
except FileExistsError:
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v1 04/11] Move the Telegram bot session into the storage directory
2022-07-18 11:19 [PATCH v1 00/11] Plan to redesign code Muhammad Rizki
` (2 preceding siblings ...)
2022-07-18 11:19 ` [PATCH v1 03/11] Add default temporary directory Muhammad Rizki
@ 2022-07-18 11:20 ` Muhammad Rizki
2022-07-18 11:20 ` [PATCH v1 05/11] daemon: Fix raw lore URL on the inline keyboard button Muhammad Rizki
` (8 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-07-18 11:20 UTC (permalink / raw)
To: Ammar Faizi; +Cc: Muhammad Rizki, GNU/Weeb Mailing List
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/run.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/daemon/run.py b/daemon/run.py
index 0a650cd..83b2cdb 100644
--- a/daemon/run.py
+++ b/daemon/run.py
@@ -18,7 +18,7 @@ def main():
load_dotenv()
client = Client(
- "EmailScraper",
+ "storage/EmailScraper",
api_id=int(os.getenv("API_ID")),
api_hash=os.getenv("API_HASH"),
bot_token=os.getenv("BOT_TOKEN"),
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v1 05/11] daemon: Fix raw lore URL on the inline keyboard button
2022-07-18 11:19 [PATCH v1 00/11] Plan to redesign code Muhammad Rizki
` (3 preceding siblings ...)
2022-07-18 11:20 ` [PATCH v1 04/11] Move the Telegram bot session into the storage directory Muhammad Rizki
@ 2022-07-18 11:20 ` Muhammad Rizki
2022-07-18 11:20 ` [PATCH v1 06/11] Add traceback to get the error detail Muhammad Rizki
` (7 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-07-18 11:20 UTC (permalink / raw)
To: Ammar Faizi; +Cc: GNU/Weeb Mailing List, Muhammad Rizki
From: Ammar Faizi <[email protected]>
The current url contains "/raw" at the end of the link. This will give
us the raw email. As this button is supposed to be read by users, we
shouldn't give the raw version. Remove "/raw" at the end of URL before
sending it to Telegram to make it more user friendly.
Signed-off-by: Ammar Faizi <[email protected]>
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/scraper/bot.py | 1 +
1 file changed, 1 insertion(+)
diff --git a/daemon/scraper/bot.py b/daemon/scraper/bot.py
index c62c554..04caab6 100644
--- a/daemon/scraper/bot.py
+++ b/daemon/scraper/bot.py
@@ -115,6 +115,7 @@ class Bot():
text, files, is_patch = utils.create_template(mail)
reply_to = self.get_tg_reply_to(mail, tg_chat_id)
+ url = str(re.sub(r"/raw$", "", url))
if is_patch:
m = await self.__send_patch_msg(mail, tg_chat_id,
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v1 06/11] Add traceback to get the error detail
2022-07-18 11:19 [PATCH v1 00/11] Plan to redesign code Muhammad Rizki
` (4 preceding siblings ...)
2022-07-18 11:20 ` [PATCH v1 05/11] daemon: Fix raw lore URL on the inline keyboard button Muhammad Rizki
@ 2022-07-18 11:20 ` Muhammad Rizki
2022-07-18 11:27 ` Ammar Faizi
2022-07-18 11:20 ` [PATCH v1 07/11] Re-design send email message to Telegram Muhammad Rizki
` (6 subsequent siblings)
12 siblings, 1 reply; 20+ messages in thread
From: Muhammad Rizki @ 2022-07-18 11:20 UTC (permalink / raw)
To: Ammar Faizi; +Cc: Muhammad Rizki, GNU/Weeb Mailing List
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/scraper/bot.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/daemon/scraper/bot.py b/daemon/scraper/bot.py
index 04caab6..2392b61 100644
--- a/daemon/scraper/bot.py
+++ b/daemon/scraper/bot.py
@@ -17,6 +17,7 @@ import pyrogram
import asyncio
import shutil
import re
+import traceback
class BotMutexes():
@@ -63,8 +64,8 @@ class Bot():
for url in self.ATOM_URLS:
try:
await self.__handle_atom_url(url)
- except Exception as e:
- print(f"[__run]: Error: {e}")
+ except:
+ print(traceback.format_exc())
if not self.isRunnerFixed:
self.isRunnerFixed = True
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v1 07/11] Re-design send email message to Telegram
2022-07-18 11:19 [PATCH v1 00/11] Plan to redesign code Muhammad Rizki
` (5 preceding siblings ...)
2022-07-18 11:20 ` [PATCH v1 06/11] Add traceback to get the error detail Muhammad Rizki
@ 2022-07-18 11:20 ` Muhammad Rizki
2022-07-18 12:49 ` Ammar Faizi
2022-07-18 11:20 ` [PATCH v1 08/11] Move ____send_patch_msg Muhammad Rizki
` (5 subsequent siblings)
12 siblings, 1 reply; 20+ messages in thread
From: Muhammad Rizki @ 2022-07-18 11:20 UTC (permalink / raw)
To: Ammar Faizi; +Cc: Muhammad Rizki, GNU/Weeb Mailing List
We want to separate send message function and inherit with the Pyrogram
Client, so it should makes the code clean and clear.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/packages/__init__.py | 1 +
daemon/packages/client.py | 59 +++++++++++++++++++++++++++++++++++++
daemon/run.py | 4 +--
daemon/scraper/bot.py | 35 ++++++----------------
4 files changed, 71 insertions(+), 28 deletions(-)
create mode 100644 daemon/packages/__init__.py
create mode 100644 daemon/packages/client.py
diff --git a/daemon/packages/__init__.py b/daemon/packages/__init__.py
new file mode 100644
index 0000000..efef9ae
--- /dev/null
+++ b/daemon/packages/__init__.py
@@ -0,0 +1 @@
+from .client import DaemonClient
diff --git a/daemon/packages/client.py b/daemon/packages/client.py
new file mode 100644
index 0000000..fd5b5ec
--- /dev/null
+++ b/daemon/packages/client.py
@@ -0,0 +1,59 @@
+from pyrogram import Client
+from pyrogram.enums import ParseMode
+from pyrogram.types import Message, InlineKeyboardMarkup, InlineKeyboardButton
+from typing import Union, BinaryIO
+
+
+class DaemonClient(Client):
+ def __init__(self, name: str, api_id: int,
+ api_hash: str, **kwargs):
+ super().__init__(name, api_id,
+ api_hash, **kwargs)
+
+
+ async def send_text_email(
+ self,
+ chat_id: Union[int, str],
+ text: str,
+ reply_to: int,
+ url: str = None,
+ parse_mode: ParseMode = ParseMode.HTML
+ ) -> Message:
+ print("[send_text_email]")
+ return await self.send_message(
+ chat_id=chat_id,
+ text=text,
+ reply_to_message_id=reply_to,
+ parse_mode=parse_mode,
+ reply_markup=InlineKeyboardMarkup([
+ [InlineKeyboardButton(
+ "See the full message",
+ url=url
+ )]
+ ])
+ )
+
+
+ async def send_patch_email(
+ self,
+ chat_id: Union[int, str],
+ doc: Union[str, BinaryIO],
+ caption: str,
+ reply_to: int,
+ url: str = None,
+ parse_mode: ParseMode = ParseMode.HTML
+ ) -> Message:
+ print("[send_patch_email]")
+ return await self.send_document(
+ chat_id=chat_id,
+ document=doc,
+ caption=caption,
+ reply_to_message_id=reply_to,
+ parse_mode=parse_mode,
+ reply_markup=InlineKeyboardMarkup([
+ [InlineKeyboardButton(
+ "See the full message",
+ url=url
+ )]
+ ])
+ )
diff --git a/daemon/run.py b/daemon/run.py
index 83b2cdb..1151ccd 100644
--- a/daemon/run.py
+++ b/daemon/run.py
@@ -8,7 +8,7 @@ from apscheduler.schedulers.asyncio import AsyncIOScheduler
from scraper import BotMutexes
from dotenv import load_dotenv
from mysql import connector
-from pyrogram import Client
+from packages import DaemonClient
from scraper import Scraper
from scraper import Bot
import os
@@ -17,7 +17,7 @@ import os
def main():
load_dotenv()
- client = Client(
+ client = DaemonClient(
"storage/EmailScraper",
api_id=int(os.getenv("API_ID")),
api_hash=os.getenv("API_HASH"),
diff --git a/daemon/scraper/bot.py b/daemon/scraper/bot.py
index 2392b61..93e633a 100644
--- a/daemon/scraper/bot.py
+++ b/daemon/scraper/bot.py
@@ -8,7 +8,7 @@ from apscheduler.schedulers.asyncio import AsyncIOScheduler
from pyrogram.types import InlineKeyboardMarkup
from pyrogram.types import InlineKeyboardButton
from slugify import slugify
-from pyrogram import Client
+from packages import DaemonClient
from scraper import Scraper
from pyrogram import enums
from . import utils
@@ -32,12 +32,11 @@ class Bot():
]
TG_CHAT_IDS = [
- -1001394203410,
- -1001673279485,
+ "kiizuah"
]
- def __init__(self, client: Client, sched: AsyncIOScheduler,
+ def __init__(self, client: DaemonClient, sched: AsyncIOScheduler,
scraper: Scraper, mutexes: BotMutexes, conn):
self.client = client
self.sched = sched
@@ -123,8 +122,8 @@ class Bot():
reply_to, text, url)
else:
text = "#ml\n" + text
- m = await self.__send_text_msg(tg_chat_id, reply_to,
- text, url)
+ m = await self.__send_text_msg(tg_chat_id, text,
+ reply_to, url)
self.db.insert_telegram(email_id, m.chat.id, m.id)
for d, f in files:
@@ -161,10 +160,10 @@ class Bot():
async def __send_patch_msg(self, mail, tg_chat_id, reply_to, text, url):
print("[__send_patch_msg]")
- tmp, fnm, caption, url = Bot.prepare_send_patch(mail, text, url)
+ tmp, doc, caption, url = Bot.prepare_send_patch(mail, text, url)
ret = await self.__handle_telegram_floodwait(
- self.____send_patch_msg,
- *[tg_chat_id, reply_to, fnm, caption, url]
+ self.client.send_patch_email,
+ *[tg_chat_id, doc, caption, reply_to, url]
)
Bot.clean_up_after_send_patch(tmp)
return ret
@@ -207,7 +206,7 @@ class Bot():
async def __send_text_msg(self, *args):
return await self.__handle_telegram_floodwait(
- self.____send_text_msg,
+ self.client.send_text_email,
*args
)
@@ -251,19 +250,3 @@ class Bot():
)]
])
)
-
-
- async def ____send_text_msg(self, tg_chat_id, reply_to, text, url):
- print("[__send_text_msg]")
- return await self.client.send_message(
- tg_chat_id,
- text,
- reply_to_message_id=reply_to,
- parse_mode=enums.ParseMode.HTML,
- reply_markup=InlineKeyboardMarkup([
- [InlineKeyboardButton(
- "See the full message",
- url=url
- )]
- ])
- )
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v1 08/11] Move ____send_patch_msg
2022-07-18 11:19 [PATCH v1 00/11] Plan to redesign code Muhammad Rizki
` (6 preceding siblings ...)
2022-07-18 11:20 ` [PATCH v1 07/11] Re-design send email message to Telegram Muhammad Rizki
@ 2022-07-18 11:20 ` Muhammad Rizki
2022-07-18 11:20 ` [PATCH v1 09/11] Move prepare for patch and clean up patch functions Muhammad Rizki
` (4 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-07-18 11:20 UTC (permalink / raw)
To: Ammar Faizi; +Cc: Muhammad Rizki, GNU/Weeb Mailing List
I want to move the ____send_patch_msg into the DaemonClient, so it much
cleaner if we separate the send email message into the DaemonClient.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/scraper/bot.py | 17 -----------------
1 file changed, 17 deletions(-)
diff --git a/daemon/scraper/bot.py b/daemon/scraper/bot.py
index 93e633a..4903fef 100644
--- a/daemon/scraper/bot.py
+++ b/daemon/scraper/bot.py
@@ -233,20 +233,3 @@ class Bot():
n = int(x.group(1))
print(f"[____handle_telegram_floodwait]: Sleeping for {n} seconds due to Telegram limit")
await asyncio.sleep(n)
-
-
- async def ____send_patch_msg(self, tg_chat_id, reply_to, fnm, caption,
- url):
- return await self.client.send_document(
- tg_chat_id,
- fnm,
- caption=caption,
- reply_to_message_id=reply_to,
- parse_mode=enums.ParseMode.HTML,
- reply_markup=InlineKeyboardMarkup([
- [InlineKeyboardButton(
- "See the full message",
- url=url
- )]
- ])
- )
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v1 09/11] Move prepare for patch and clean up patch functions
2022-07-18 11:19 [PATCH v1 00/11] Plan to redesign code Muhammad Rizki
` (7 preceding siblings ...)
2022-07-18 11:20 ` [PATCH v1 08/11] Move ____send_patch_msg Muhammad Rizki
@ 2022-07-18 11:20 ` Muhammad Rizki
2022-07-18 11:20 ` [PATCH v1 10/11] Create fix_utf8_chars function Muhammad Rizki
` (3 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-07-18 11:20 UTC (permalink / raw)
To: Ammar Faizi; +Cc: Muhammad Rizki, GNU/Weeb Mailing List
I want these function is separate into the utils or utility file, I'm
the type of person who likes to separate the utility function into the
utility file.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/scraper/utils.py | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/daemon/scraper/utils.py b/daemon/scraper/utils.py
index 10fe956..5335fe2 100644
--- a/daemon/scraper/utils.py
+++ b/daemon/scraper/utils.py
@@ -6,10 +6,12 @@
from email.message import Message
from typing import Dict
+from slugify import slugify
import hashlib
import uuid
import os
import re
+import shutil
def get_email_msg_id(mail):
@@ -190,6 +192,39 @@ def create_template(thread: Message, to=None, cc=None):
return ret, files, is_patch
+def prepare_send_patch(mail, text, url):
+ tmp = gen_temp(url)
+ fnm = str(mail.get("subject"))
+ sch = re.search(PATCH_PATTERN, fnm, re.IGNORECASE)
+
+ nr_patch = sch.group(1)
+ if not nr_patch:
+ nr_patch = 1
+ else:
+ nr_patch = int(nr_patch)
+
+ num = "%04d" % nr_patch
+ fnm = slugify(sch.group(3)).replace("_", "-")
+ file = f"{tmp}/{num}-{fnm}.patch"
+ cap = text.split("\n\n")[0]
+
+ with open(file, "wb") as f:
+ f.write(bytes(text, encoding="utf8"))
+
+ caption = (
+ "#patch #ml\n" +
+ cap.rstrip()
+ .replace("<", "<")
+ .replace(">",">")
+ .replace("�"," ")
+ )
+ return tmp, file, caption, url
+
+
+def clean_up_after_send_patch(tmp):
+ shutil.rmtree(tmp)
+
+
EMAIL_MSG_ID_PATTERN = r"<([^\<\>]+)>"
def extract_email_msg_id(msg_id):
ret = re.search(EMAIL_MSG_ID_PATTERN, msg_id)
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v1 10/11] Create fix_utf8_chars function
2022-07-18 11:19 [PATCH v1 00/11] Plan to redesign code Muhammad Rizki
` (8 preceding siblings ...)
2022-07-18 11:20 ` [PATCH v1 09/11] Move prepare for patch and clean up patch functions Muhammad Rizki
@ 2022-07-18 11:20 ` Muhammad Rizki
2022-07-18 11:20 ` [PATCH v1 11/11] Recode some codes Muhammad Rizki
` (2 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-07-18 11:20 UTC (permalink / raw)
To: Ammar Faizi; +Cc: Muhammad Rizki, GNU/Weeb Mailing List
Some of the codes are repeated, so I create this function to make it
cleaner.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/scraper/utils.py | 19 +++++++++++--------
1 file changed, 11 insertions(+), 8 deletions(-)
diff --git a/daemon/scraper/utils.py b/daemon/scraper/utils.py
index 5335fe2..3f0d254 100644
--- a/daemon/scraper/utils.py
+++ b/daemon/scraper/utils.py
@@ -183,10 +183,7 @@ def create_template(thread: Message, to=None, cc=None):
ret = ret[:4000] + "..."
ret = (
- ret.rstrip()
- .replace("<", "<")
- .replace(">",">")
- .replace("�"," ")
+ fix_utf8_char(ret)
) + "\n<code>------------------------------------------------------------------------</code>"
return ret, files, is_patch
@@ -213,10 +210,7 @@ def prepare_send_patch(mail, text, url):
caption = (
"#patch #ml\n" +
- cap.rstrip()
- .replace("<", "<")
- .replace(">",">")
- .replace("�"," ")
+ fix_utf8_char(cap)
)
return tmp, file, caption, url
@@ -225,6 +219,15 @@ def clean_up_after_send_patch(tmp):
shutil.rmtree(tmp)
+def fix_utf8_char(text: str):
+ return (
+ text.rstrip()
+ .replace("<", "<")
+ .replace(">",">")
+ .replace("�"," ")
+ )
+
+
EMAIL_MSG_ID_PATTERN = r"<([^\<\>]+)>"
def extract_email_msg_id(msg_id):
ret = re.search(EMAIL_MSG_ID_PATTERN, msg_id)
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v1 11/11] Recode some codes
2022-07-18 11:19 [PATCH v1 00/11] Plan to redesign code Muhammad Rizki
` (9 preceding siblings ...)
2022-07-18 11:20 ` [PATCH v1 10/11] Create fix_utf8_chars function Muhammad Rizki
@ 2022-07-18 11:20 ` Muhammad Rizki
2022-07-18 11:33 ` Ammar Faizi
2022-07-18 11:28 ` [PATCH v1 00/11] Plan to redesign code Ammar Faizi
2022-07-18 11:34 ` Ammar Faizi
12 siblings, 1 reply; 20+ messages in thread
From: Muhammad Rizki @ 2022-07-18 11:20 UTC (permalink / raw)
To: Ammar Faizi; +Cc: Muhammad Rizki, GNU/Weeb Mailing List
Just recode some codes, I think it much practice and clean (?)
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/scraper/utils.py | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/daemon/scraper/utils.py b/daemon/scraper/utils.py
index 3f0d254..5496483 100644
--- a/daemon/scraper/utils.py
+++ b/daemon/scraper/utils.py
@@ -182,9 +182,8 @@ def create_template(thread: Message, to=None, cc=None):
if len(ret) >= 4000:
ret = ret[:4000] + "..."
- ret = (
- fix_utf8_char(ret)
- ) + "\n<code>------------------------------------------------------------------------</code>"
+ ret = fix_utf8_char(ret)
+ ret += "\n<code>------------------------------------------------------------------------</code>"
return ret, files, is_patch
@@ -208,10 +207,9 @@ def prepare_send_patch(mail, text, url):
with open(file, "wb") as f:
f.write(bytes(text, encoding="utf8"))
- caption = (
- "#patch #ml\n" +
- fix_utf8_char(cap)
- )
+ caption = "#patch #ml\n"
+ caption += fix_utf8_char(cap)
+
return tmp, file, caption, url
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v1 06/11] Add traceback to get the error detail
2022-07-18 11:20 ` [PATCH v1 06/11] Add traceback to get the error detail Muhammad Rizki
@ 2022-07-18 11:27 ` Ammar Faizi
2022-07-18 12:24 ` Kiizuha
0 siblings, 1 reply; 20+ messages in thread
From: Ammar Faizi @ 2022-07-18 11:27 UTC (permalink / raw)
To: Muhammad Rizki; +Cc: GNU/Weeb Mailing List
On 7/18/22 6:20 PM, Muhammad Rizki wrote:
> Signed-off-by: Muhammad Rizki <[email protected]>
> ---
> daemon/scraper/bot.py | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
Please give the reason of why do you make changes in this patch.
That patch is not trivial as it has a lengthy discussion background
story. It's something that needs an explanation. Not an empty body
like that.
A good commit message should explain "WHY" such a change is
necessary.
Maybe you can write it like this:
daemon: Use traceback.format_exc() to get the error detail
`print(f"[__run]: Error: {e}")` doesn't give enough information for
debugging because it doesn't show the traceback, files, line numbers,
etc. It only shows the error message raised by an exception. This is
severely lacking and very bad for debugging experience.
Use `traceback.format_exc()` instead to get a better log because it
shows a complete traceback of the error.
Signed-off-by: Your Name <[email protected]>
Improve it yourself if needed.
--
Ammar Faizi
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v1 00/11] Plan to redesign code
2022-07-18 11:19 [PATCH v1 00/11] Plan to redesign code Muhammad Rizki
` (10 preceding siblings ...)
2022-07-18 11:20 ` [PATCH v1 11/11] Recode some codes Muhammad Rizki
@ 2022-07-18 11:28 ` Ammar Faizi
2022-07-18 11:34 ` Ammar Faizi
12 siblings, 0 replies; 20+ messages in thread
From: Ammar Faizi @ 2022-07-18 11:28 UTC (permalink / raw)
To: Muhammad Rizki; +Cc: GNU/Weeb Mailing List
On 7/18/22 6:19 PM, Muhammad Rizki wrote:
> There 10 patches in this series:
That's 11 patches :-)
--
Ammar Faizi
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v1 03/11] Add default temporary directory
2022-07-18 11:19 ` [PATCH v1 03/11] Add default temporary directory Muhammad Rizki
@ 2022-07-18 11:31 ` Ammar Faizi
0 siblings, 0 replies; 20+ messages in thread
From: Ammar Faizi @ 2022-07-18 11:31 UTC (permalink / raw)
To: Muhammad Rizki; +Cc: GNU/Weeb Mailing List
On 7/18/22 6:19 PM, Muhammad Rizki wrote:
> Signed-off-by: Muhammad Rizki <[email protected]>
> ---
> daemon/scraper/utils.py | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
The same thing, missing a reason. The reason is simple, we want to
avoid error when "STORAGE_DIR" env is not defined, so you put a
default value here.
--
Ammar Faizi
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v1 11/11] Recode some codes
2022-07-18 11:20 ` [PATCH v1 11/11] Recode some codes Muhammad Rizki
@ 2022-07-18 11:33 ` Ammar Faizi
0 siblings, 0 replies; 20+ messages in thread
From: Ammar Faizi @ 2022-07-18 11:33 UTC (permalink / raw)
To: Muhammad Rizki; +Cc: GNU/Weeb Mailing List
On 7/18/22 6:20 PM, Muhammad Rizki wrote:
> Just recode some codes, I think it much practice and clean (?)
Yes.
--
Ammar Faizi
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v1 00/11] Plan to redesign code
2022-07-18 11:19 [PATCH v1 00/11] Plan to redesign code Muhammad Rizki
` (11 preceding siblings ...)
2022-07-18 11:28 ` [PATCH v1 00/11] Plan to redesign code Ammar Faizi
@ 2022-07-18 11:34 ` Ammar Faizi
12 siblings, 0 replies; 20+ messages in thread
From: Ammar Faizi @ 2022-07-18 11:34 UTC (permalink / raw)
To: Muhammad Rizki; +Cc: GNU/Weeb Mailing List
On 7/18/22 6:19 PM, Muhammad Rizki wrote:
> Hi sir,
>
> In this series I want to redesign codes to make it clear and clean.
> I've inherit the DaemonClient with the Pyrogram's Client, so with
> this method we can create our custom functions, such as
> send email message and patch message in the DaemonClient.
Will take a look into this further, later...
--
Ammar Faizi
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v1 06/11] Add traceback to get the error detail
2022-07-18 11:27 ` Ammar Faizi
@ 2022-07-18 12:24 ` Kiizuha
0 siblings, 0 replies; 20+ messages in thread
From: Kiizuha @ 2022-07-18 12:24 UTC (permalink / raw)
To: Ammar Faizi; +Cc: GNU/Weeb Mailing List
>
> Please give the reason of why do you make changes in this patch.
>
> That patch is not trivial as it has a lengthy discussion background
> story. It's something that needs an explanation. Not an empty body
> like that.
>
> A good commit message should explain "WHY" such a change is
> necessary.
>
> Maybe you can write it like this:
>
> daemon: Use traceback.format_exc() to get the error detail
>
> `print(f"[__run]: Error: {e}")` doesn't give enough information for
> debugging because it doesn't show the traceback, files, line numbers,
> etc. It only shows the error message raised by an exception. This is
> severely lacking and very bad for debugging experience.
>
> Use `traceback.format_exc()` instead to get a better log because it
> shows a complete traceback of the error.
>
> Signed-off-by: Your Name <[email protected]>
>
> Improve it yourself if needed.
>
Ok, I will do it. I just forget to check the git log and it doesn't
have a description to explain WHY, I will improve any commit in the
future. Thanks for reminding me.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v1 01/11] Fix __send_patch_msg function parameter
2022-07-18 11:19 ` [PATCH v1 01/11] Fix __send_patch_msg function parameter Muhammad Rizki
@ 2022-07-18 12:45 ` Ammar Faizi
0 siblings, 0 replies; 20+ messages in thread
From: Ammar Faizi @ 2022-07-18 12:45 UTC (permalink / raw)
To: Muhammad Rizki; +Cc: GNU/Weeb Mailing List
On 7/18/22 6:19 PM, Muhammad Rizki wrote:
> - async def __send_patch_msg(self, mail, tg_chat_id, text, url):
> + async def __send_patch_msg(self, mail, tg_chat_id, reply_to, text, url):
> print("[__send_patch_msg]")
>
^ There is a trailing whitespace here.
> tmp, fnm, caption, url = Bot.prepare_send_patch(mail, text, url)
--
Ammar Faizi
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v1 07/11] Re-design send email message to Telegram
2022-07-18 11:20 ` [PATCH v1 07/11] Re-design send email message to Telegram Muhammad Rizki
@ 2022-07-18 12:49 ` Ammar Faizi
0 siblings, 0 replies; 20+ messages in thread
From: Ammar Faizi @ 2022-07-18 12:49 UTC (permalink / raw)
To: Muhammad Rizki; +Cc: GNU/Weeb Mailing List
On 7/18/22 6:20 PM, Muhammad Rizki wrote:
> +class DaemonClient(Client):
> + def __init__(self, name: str, api_id: int,
> + api_hash: str, **kwargs):
> + super().__init__(name, api_id,
> + api_hash, **kwargs)
> +
> +
> + async def send_text_email(
> + self,
> + chat_id: Union[int, str],
> + text: str,
> + reply_to: int,
> + url: str = None,
> + parse_mode: ParseMode = ParseMode.HTML
> + ) -> Message:
> + print("[send_text_email]")
> + return await self.send_message(
> + chat_id=chat_id,
> + text=text,
> + reply_to_message_id=reply_to,
> + parse_mode=parse_mode,
> + reply_markup=InlineKeyboardMarkup([
> + [InlineKeyboardButton(
> + "See the full message",
> + url=url
> + )]
> + ])
> + )
[...]
> + print("[send_patch_email]")
> + return await self.send_document(
> + chat_id=chat_id,
> + document=doc,
> + caption=caption,
> + reply_to_message_id=reply_to,
> + parse_mode=parse_mode,
> + reply_markup=InlineKeyboardMarkup([
Please always use tabs to start the indentation.
--
Ammar Faizi
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2022-07-18 12:49 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-07-18 11:19 [PATCH v1 00/11] Plan to redesign code Muhammad Rizki
2022-07-18 11:19 ` [PATCH v1 01/11] Fix __send_patch_msg function parameter Muhammad Rizki
2022-07-18 12:45 ` Ammar Faizi
2022-07-18 11:19 ` [PATCH v1 02/11] Fix import problem Muhammad Rizki
2022-07-18 11:19 ` [PATCH v1 03/11] Add default temporary directory Muhammad Rizki
2022-07-18 11:31 ` Ammar Faizi
2022-07-18 11:20 ` [PATCH v1 04/11] Move the Telegram bot session into the storage directory Muhammad Rizki
2022-07-18 11:20 ` [PATCH v1 05/11] daemon: Fix raw lore URL on the inline keyboard button Muhammad Rizki
2022-07-18 11:20 ` [PATCH v1 06/11] Add traceback to get the error detail Muhammad Rizki
2022-07-18 11:27 ` Ammar Faizi
2022-07-18 12:24 ` Kiizuha
2022-07-18 11:20 ` [PATCH v1 07/11] Re-design send email message to Telegram Muhammad Rizki
2022-07-18 12:49 ` Ammar Faizi
2022-07-18 11:20 ` [PATCH v1 08/11] Move ____send_patch_msg Muhammad Rizki
2022-07-18 11:20 ` [PATCH v1 09/11] Move prepare for patch and clean up patch functions Muhammad Rizki
2022-07-18 11:20 ` [PATCH v1 10/11] Create fix_utf8_chars function Muhammad Rizki
2022-07-18 11:20 ` [PATCH v1 11/11] Recode some codes Muhammad Rizki
2022-07-18 11:33 ` Ammar Faizi
2022-07-18 11:28 ` [PATCH v1 00/11] Plan to redesign code Ammar Faizi
2022-07-18 11:34 ` Ammar Faizi
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox