* [PATCH v2 01/17] telegram: Simplify code to get DB_PORT from env
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 02/17] discord: " Muhammad Rizki
` (17 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Using `int(os.environ.get("DB_PORT", 3306))` is the best practice to get
the DB_PORT rather than using if statement. This change simplifies the
code.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/tg.py | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/daemon/tg.py b/daemon/tg.py
index 9f36e54..6b2ec20 100644
--- a/daemon/tg.py
+++ b/daemon/tg.py
@@ -22,12 +22,6 @@ def main():
logger = BotLogger()
logger.init()
- port = os.getenv("DB_PORT")
- if not port:
- port = 3306
- else:
- port = int(port)
-
client = DaemonTelegram(
"telegram/storage/EmailScraper",
api_id=int(os.getenv("API_ID")),
@@ -37,7 +31,7 @@ def main():
conn=connector.connect(
host=os.getenv("DB_HOST"),
user=os.getenv("DB_USER"),
- port=port,
+ port=int(os.environ.get("DB_PORT", 3306)),
password=os.getenv("DB_PASS"),
database=os.getenv("DB_NAME")
),
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 02/17] discord: Simplify code to get DB_PORT from env
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 01/17] telegram: Simplify code to get DB_PORT from env Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 03/17] telegram: logger: Add a telegram.logger.conf Muhammad Rizki
` (16 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Using `int(os.environ.get("DB_PORT", 3306))` is the best practice to get
the DB_PORT rather than using if statement. This change simplifies the
code.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/dc.py | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/daemon/dc.py b/daemon/dc.py
index a9d5ebd..7d34837 100644
--- a/daemon/dc.py
+++ b/daemon/dc.py
@@ -29,17 +29,11 @@ def main():
logger = BotLogger(Platform.DISCORD)
logger.init()
- port = os.getenv("DB_PORT")
- if not port:
- port = 3306
- else:
- port = int(port)
-
client = GWClient(
db_conn=connector.connect(
host=os.getenv("DB_HOST"),
user=os.getenv("DB_USER"),
- port=port,
+ port=int(os.environ.get("DB_PORT", 3306)),
password=os.getenv("DB_PASS"),
database=os.getenv("DB_NAME")
),
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 03/17] telegram: logger: Add a telegram.logger.conf
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 01/17] telegram: Simplify code to get DB_PORT from env Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 02/17] discord: " Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 04/17] discord: logger: Add a discord.logger.conf Muhammad Rizki
` (15 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Add a logger configuration file for improved configuration, debugging,
and management. This method is cleaner than before and the BotLogger()
will be removed in the future.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/telegram/telegram.logger.conf | 36 ++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)
create mode 100644 daemon/telegram/telegram.logger.conf
diff --git a/daemon/telegram/telegram.logger.conf b/daemon/telegram/telegram.logger.conf
new file mode 100644
index 0000000..d0c0de3
--- /dev/null
+++ b/daemon/telegram/telegram.logger.conf
@@ -0,0 +1,36 @@
+[loggers]
+keys=root,telegram
+
+[handlers]
+keys=consoleHandler,fileHandler
+
+[formatters]
+keys=sampleFormatter,dictFormatter
+
+[logger_root]
+level=INFO
+handlers=consoleHandler,fileHandler
+
+[logger_telegram]
+level=INFO
+handlers=consoleHandler
+qualname=telegram
+propagate=0
+
+[handler_consoleHandler]
+class=StreamHandler
+level=INFO
+formatter=sampleFormatter
+args=(sys.stdout,)
+
+[handler_fileHandler]
+class=FileHandler
+level=WARNING
+formatter=dictFormatter
+args=("telegram/storage/telegram.log",)
+
+[formatter_sampleFormatter]
+format=%(asctime)s - %(name)s: %(funcName)s - %(levelname)s - %(message)s
+
+[formatter_dictFormatter]
+format={"time": "%(asctime)s", "func": "%(name)s: %(funcName)s", "path": "%(pathname)s", "level": "%(levelname)s", "msg": "%(message)s"}
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 04/17] discord: logger: Add a discord.logger.conf
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (2 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 03/17] telegram: logger: Add a telegram.logger.conf Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 05/17] telegram: logger: Initialize the configuration for the Telegram logger Muhammad Rizki
` (14 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Add a logger configuration file for improved configuration, debugging,
and management. This method is cleaner than before and the BotLogger()
will be removed in the future.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/dscord/discord.logger.conf | 36 +++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)
create mode 100644 daemon/dscord/discord.logger.conf
diff --git a/daemon/dscord/discord.logger.conf b/daemon/dscord/discord.logger.conf
new file mode 100644
index 0000000..97b5e11
--- /dev/null
+++ b/daemon/dscord/discord.logger.conf
@@ -0,0 +1,36 @@
+[loggers]
+keys=root,dscord
+
+[handlers]
+keys=consoleHandler,fileHandler
+
+[formatters]
+keys=sampleFormatter,dictFormatter
+
+[logger_root]
+level=INFO
+handlers=consoleHandler,fileHandler
+
+[logger_dscord]
+level=INFO
+handlers=consoleHandler
+qualname=dscord
+propagate=0
+
+[handler_consoleHandler]
+class=StreamHandler
+level=INFO
+formatter=sampleFormatter
+args=(sys.stdout,)
+
+[handler_fileHandler]
+class=FileHandler
+level=WARNING
+formatter=dictFormatter
+args=("dscord/storage/discord.log",)
+
+[formatter_sampleFormatter]
+format=%(asctime)s - %(name)s: %(funcName)s - %(levelname)s - %(message)s
+
+[formatter_dictFormatter]
+format={"time": "%(asctime)s", "func": "%(name)s: %(funcName)s", "path": "%(pathname)s", "level": "%(levelname)s", "msg": "%(message)s"}
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 05/17] telegram: logger: Initialize the configuration for the Telegram logger.
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (3 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 04/17] discord: logger: Add a discord.logger.conf Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 06/17] discord: logger: Initialize the configuration for the Discord logger Muhammad Rizki
` (13 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
This commit adds the configuration for the logger and sets the logger
level of other loggers, such as pyrogram, to the WARNING level. Changing
the level is to disable some of their log messages, so that only our bot
logs to the console.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/tg.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/daemon/tg.py b/daemon/tg.py
index 6b2ec20..26f8374 100644
--- a/daemon/tg.py
+++ b/daemon/tg.py
@@ -14,6 +14,8 @@ from telegram.mailer import BotMutexes
from telegram.mailer import Bot
from logger import BotLogger
import os
+import logging
+import logging.config
def main():
@@ -22,6 +24,10 @@ def main():
logger = BotLogger()
logger.init()
+ logging.config.fileConfig("telegram/telegram.logger.conf")
+ logging.getLogger("apscheduler").setLevel(logging.WARNING)
+ logging.getLogger("pyrogram").setLevel(logging.WARNING)
+
client = DaemonTelegram(
"telegram/storage/EmailScraper",
api_id=int(os.getenv("API_ID")),
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 06/17] discord: logger: Initialize the configuration for the Discord logger.
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (4 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 05/17] telegram: logger: Initialize the configuration for the Telegram logger Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 07/17] telegram: fix: Fix the type annoations for the decorator Muhammad Rizki
` (12 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
This commit adds the configuration for the logger and sets the logger
level of other loggers, such as pyrogram, to the WARNING level. Changing
the level is to disable some of their log messages, so that only our bot
logs to the console.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/dc.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/daemon/dc.py b/daemon/dc.py
index 7d34837..f4eeefb 100644
--- a/daemon/dc.py
+++ b/daemon/dc.py
@@ -4,6 +4,8 @@
#
import os
+import logging
+import logging.config
from dotenv import load_dotenv
from mysql import connector
from apscheduler.schedulers.asyncio import AsyncIOScheduler
@@ -29,6 +31,10 @@ def main():
logger = BotLogger(Platform.DISCORD)
logger.init()
+ logging.config.fileConfig("dscord/discord.logger.conf")
+ logging.getLogger("apscheduler").setLevel(logging.WARNING)
+ logging.getLogger("discord").setLevel(logging.WARNING)
+
client = GWClient(
db_conn=connector.connect(
host=os.getenv("DB_HOST"),
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 07/17] telegram: fix: Fix the type annoations for the decorator
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (5 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 06/17] discord: logger: Initialize the configuration for the Discord logger Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 08/17] discord: cleanup: Remove some unnecessary comments Muhammad Rizki
` (11 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
In the previous commit, I created a TODO to fix the typing. In this
commit, I have fixed the decorator's typing.
Fixes: ba02aa3376a4 ("daemon: Add typing in decorator")
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/telegram/packages/decorator.py | 17 ++++++++---------
1 file changed, 8 insertions(+), 9 deletions(-)
diff --git a/daemon/telegram/packages/decorator.py b/daemon/telegram/packages/decorator.py
index efcfc38..6bfd8bc 100644
--- a/daemon/telegram/packages/decorator.py
+++ b/daemon/telegram/packages/decorator.py
@@ -4,25 +4,24 @@
#
from pyrogram.errors.exceptions.flood_420 import FloodWait
-from pyrogram.types import Message
-from typing import Any, Callable, TypeVar
+from typing import Any, Callable, TypeVar, Coroutine
+from typing_extensions import ParamSpec, ParamSpecArgs, ParamSpecKwargs
from functools import wraps
import re
import asyncio
__all__ = ["handle_flood"]
-T = TypeVar("T", bound=Message)
+T = TypeVar("T")
+P = ParamSpec("P")
-#
-# TODO(Muhammad Rizki): Add more typing for @handle_flood
-#
-def handle_flood(func: Callable[[T], T]) -> Callable[[T], T]:
+
+def handle_flood(func: Callable[P, Coroutine[Any,Any,T]]) -> Callable[P, Coroutine[Any,Any,T]]:
@wraps(func)
- async def callback(*args: Any) -> Any:
+ async def callback(*args: ParamSpecArgs, **kwargs: ParamSpecKwargs) -> T:
while True:
try:
- return await func(*args)
+ return await func(*args, **kwargs)
except FloodWait as e:
# Calling logger attr from the DaemonTelegram() class
logger = args[0].logger
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 08/17] discord: cleanup: Remove some unnecessary comments
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (6 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 07/17] telegram: fix: Fix the type annoations for the decorator Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 09/17] discord: fix: Fix the type annotations for the decorator Muhammad Rizki
` (10 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/dscord/gnuweeb/filters.py | 3 ---
1 file changed, 3 deletions(-)
diff --git a/daemon/dscord/gnuweeb/filters.py b/daemon/dscord/gnuweeb/filters.py
index 1fdb70d..b342c7a 100644
--- a/daemon/dscord/gnuweeb/filters.py
+++ b/daemon/dscord/gnuweeb/filters.py
@@ -4,16 +4,13 @@
#
-# built-in/dev package imports
import asyncio
from typing import Any
from functools import wraps
-# Discord imports
import discord
from discord import Interaction
-# gnuweeb package import
from dscord import config
from logger import BotLogger
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 09/17] discord: fix: Fix the type annotations for the decorator
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (7 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 08/17] discord: cleanup: Remove some unnecessary comments Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 10/17] discord: typing: Add return type annotations Muhammad Rizki
` (9 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Previously, these decorators did not use type annotations. In this
commit, I have implemented them.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/dscord/gnuweeb/filters.py | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/daemon/dscord/gnuweeb/filters.py b/daemon/dscord/gnuweeb/filters.py
index b342c7a..28b492c 100644
--- a/daemon/dscord/gnuweeb/filters.py
+++ b/daemon/dscord/gnuweeb/filters.py
@@ -5,7 +5,8 @@
import asyncio
-from typing import Any
+from typing import Any, Callable, TypeVar, Coroutine
+from typing_extensions import ParamSpec, ParamSpecArgs, ParamSpecKwargs
from functools import wraps
import discord
@@ -15,9 +16,13 @@ from dscord import config
from logger import BotLogger
-def lore_admin(func):
+T = TypeVar("T")
+P = ParamSpec("P")
+
+
+def lore_admin(func: Callable[P, Coroutine[Any,Any,T]]) -> Callable[P, Coroutine[Any,Any,T]]:
@wraps(func)
- async def callback(*args: Any, **kwargs: Any) -> Any:
+ async def callback(*args: ParamSpecArgs, **kwargs: ParamSpecKwargs) -> T:
i: "Interaction" = args[1]
user_roles = [role.id for role in i.user.roles]
@@ -33,12 +38,12 @@ def lore_admin(func):
return callback
-def wait_on_limit(func):
+def wait_on_limit(func: Callable[P, Coroutine[Any,Any,T]]) -> Callable[P, Coroutine[Any,Any,T]]:
@wraps(func)
- async def callback(*args: Any) -> Any:
+ async def callback(*args: ParamSpecArgs, **kwargs: ParamSpecKwargs) -> T:
while True:
try:
- return await func(*args)
+ return await func(*args, **kwargs)
except discord.errors.RateLimited as e:
# Calling logger attr from the GWClient() class
logger = args[0].logger
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 10/17] discord: typing: Add return type annotations
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (8 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 09/17] discord: fix: Fix the type annotations for the decorator Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 11/17] telegram: Implement DaemonException() and report_err() in scrape.py Muhammad Rizki
` (8 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Add return type annotations for send_text_email() & send_patch_email().
This improves code readability and helps with understanding the type of
object returned by these functions.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/dscord/gnuweeb/client.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/daemon/dscord/gnuweeb/client.py b/daemon/dscord/gnuweeb/client.py
index 78ec3f9..7d6433b 100644
--- a/daemon/dscord/gnuweeb/client.py
+++ b/daemon/dscord/gnuweeb/client.py
@@ -4,7 +4,7 @@
#
import discord
-from discord import Interaction
+from discord import Interaction, Message
from discord.ext import commands
from discord import Intents
from typing import Union
@@ -60,7 +60,7 @@ class GWClient(commands.Bot):
@filters.wait_on_limit
async def send_text_email(self, guild_id: int, chat_id: int, text: str,
- reply_to: Union[int, None] = None, url: str = None):
+ reply_to: Union[int, None] = None, url: str = None) -> Message:
self.logger.debug("[send_text_email]")
channel = self.get_channel(chat_id)
@@ -77,7 +77,7 @@ class GWClient(commands.Bot):
@filters.wait_on_limit
async def send_patch_email(self, mail, guild_id: int, chat_id: int, text: str,
- reply_to: Union[int, None] = None, url: str = None):
+ reply_to: Union[int, None] = None, url: str = None) -> Message:
self.logger.debug("[send_patch_email]")
tmp, doc, caption, url = utils.prepare_patch(
mail, text, url, Platform.DISCORD
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 11/17] telegram: Implement DaemonException() and report_err() in scrape.py
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (9 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 10/17] discord: typing: Add return type annotations Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 12/17] utils: fix: Fix charset issue for get_decoded_payload() Muhammad Rizki
` (7 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
In the last series, I didn't care so much to look into the Telegram's
scrape.py command. I've implemented them same as in the listener.py.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/telegram/packages/plugins/commands/scrape.py | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/daemon/telegram/packages/plugins/commands/scrape.py b/daemon/telegram/packages/plugins/commands/scrape.py
index 89581b6..ddcf7f1 100644
--- a/daemon/telegram/packages/plugins/commands/scrape.py
+++ b/daemon/telegram/packages/plugins/commands/scrape.py
@@ -11,6 +11,7 @@ from atom import Scraper
from atom import utils
from enums import Platform
from telegram import config
+from exceptions import DaemonException
import re
import asyncio
@@ -41,9 +42,11 @@ async def scrap_email(c: DaemonTelegram, m: Message):
mail = await s.get_email_from_url(url)
text, files, is_patch = utils.create_template(mail, Platform.TELEGRAM)
except:
- exc_str = utils.catch_err()
- c.logger.warning(exc_str)
- await c.send_log_file(url)
+ e = DaemonException()
+ e.set_thread_url(url)
+ e.set_message(utils.catch_err())
+ await c.report_err(e)
+ return
if is_patch:
m = await c.send_patch_email(
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 12/17] utils: fix: Fix charset issue for get_decoded_payload()
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (10 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 11/17] telegram: Implement DaemonException() and report_err() in scrape.py Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 13/17] telegram: logger: Refactor all logging method Muhammad Rizki
` (6 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Fix the `LookupError` that is thrown by the `decode()` function. The
`unknown-8bit` charset is not found in the built-in `decode()` function
in Python, so we must manually fix it by using an if statement and
decoding using the `ascii` charset while forcing errors using
`surrogateescape`.
Sample: https://lore.kernel.org/all/[email protected]/raw
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/atom/utils.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/daemon/atom/utils.py b/daemon/atom/utils.py
index 96a10a0..73c8978 100644
--- a/daemon/atom/utils.py
+++ b/daemon/atom/utils.py
@@ -323,6 +323,9 @@ def get_decoded_payload(payload: Message):
return payload.get_payload(decode=True) \
.decode(errors="replace")
+ if charset == "unknown-8bit":
+ return p.encode().decode("utf-8", "surrogateescape")
+
return p.encode().decode(charset, errors="replace")
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 13/17] telegram: logger: Refactor all logging method
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (11 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 12/17] utils: fix: Fix charset issue for get_decoded_payload() Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 14/17] discord: Implement DaemonException and report_err in get_lore_mail.py Muhammad Rizki
` (5 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
We use a simple logging method by using file config and replace all
custom logger method to built-in logging method. This method is more
cleaner and less code wasteful.
The reason why we should name the logger like:
`logging.getLogger("telegram")`
Because, we need to identify which bot platform that currently running.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/telegram/mailer/listener.py | 15 +++++++++------
daemon/telegram/packages/client.py | 21 +++++++++++++--------
daemon/telegram/packages/decorator.py | 14 +++++++-------
daemon/tg.py | 6 ------
4 files changed, 29 insertions(+), 27 deletions(-)
diff --git a/daemon/telegram/mailer/listener.py b/daemon/telegram/mailer/listener.py
index 2e8eeda..b34c74d 100644
--- a/daemon/telegram/mailer/listener.py
+++ b/daemon/telegram/mailer/listener.py
@@ -15,6 +15,10 @@ from atom import utils
from enums import Platform
import asyncio
import re
+import logging
+
+
+log = logging.getLogger("telegram")
class BotMutexes():
@@ -30,7 +34,6 @@ class Bot():
self.scraper = scraper
self.mutexes = mutexes
self.db = client.db
- self.logger = client.logger
self.isRunnerFixed = False
@@ -57,8 +60,8 @@ class Bot():
# TODO(ammarfaizi2):
# Ideally, we also want to log and report this situation.
#
- self.logger.error(f"Database error: {str(e)}")
- self.logger.info("Reconnecting to the database...")
+ log.error(f"Database error: {str(e)}")
+ log.info("Reconnecting to the database...")
#
# Don't do this too often to avoid reconnect burst.
@@ -71,7 +74,7 @@ class Bot():
async def __run(self):
- self.logger.info("Running...")
+ log.info("Running...")
url = None
try:
for url in self.db.get_atom_urls():
@@ -128,14 +131,14 @@ class Bot():
if not email_msg_id:
md = "email_msg_id not detected, skipping malformed email"
- self.logger.debug(md)
+ log.debug(md)
return False
email_id = self.__mail_id_from_db(email_msg_id,
tg_chat_id)
if not email_id:
md = f"Skipping {email_id} because has already been sent to Telegram"
- self.logger.debug(md)
+ log.debug(md)
return False
text, files, is_patch = utils.create_template(mail, Platform.TELEGRAM)
diff --git a/daemon/telegram/packages/client.py b/daemon/telegram/packages/client.py
index a58aaf8..7e898c5 100644
--- a/daemon/telegram/packages/client.py
+++ b/daemon/telegram/packages/client.py
@@ -3,41 +3,46 @@
# Copyright (C) 2022 Muhammad Rizki <[email protected]>
#
+import logging
from pyrogram import Client
from pyrogram.enums import ParseMode
from pyrogram.types import Message, InlineKeyboardMarkup, InlineKeyboardButton
from typing import Union
from email.message import Message
+
from atom import utils
from enums import Platform
-from logger import BotLogger
from telegram import config
from telegram.database import DB
from .decorator import handle_flood
from exceptions import DaemonException
+log = logging.getLogger("telegram")
+
+
class DaemonTelegram(Client):
def __init__(self, name: str, api_id: int,
- api_hash: str, conn, logger: BotLogger,
- **kwargs
+ api_hash: str, conn, **kwargs
):
super().__init__(name, api_id,
api_hash, **kwargs)
self.db = DB(conn)
- self.logger = logger
async def report_err(self, e: DaemonException):
capt = f"Atom URL: {e.atom_url}\n"
capt += f"Thread URL: {e.thread_url}"
- self.logger.warning(e.original_exception)
+ log.warning(e.original_exception)
await self.send_log_file(capt)
@handle_flood
async def send_log_file(self, caption: str):
- filename = self.logger.handlers[0].baseFilename
+ for handler in log.root.handlers:
+ if isinstance(handler, logging.FileHandler):
+ filename = handler.baseFilename
+
await self.send_document(
config.LOG_CHANNEL_ID,
filename,
@@ -54,7 +59,7 @@ class DaemonTelegram(Client):
url: str = None,
parse_mode: ParseMode = ParseMode.HTML
) -> Message:
- self.logger.debug("[send_text_email]")
+ log.debug("[send_text_email]")
return await self.send_message(
chat_id=chat_id,
text=text,
@@ -79,7 +84,7 @@ class DaemonTelegram(Client):
url: str = None,
parse_mode: ParseMode = ParseMode.HTML
) -> Message:
- self.logger.debug("[send_patch_email]")
+ log.debug("[send_patch_email]")
tmp, doc, caption, url = utils.prepare_patch(
mail, text, url, Platform.TELEGRAM
)
diff --git a/daemon/telegram/packages/decorator.py b/daemon/telegram/packages/decorator.py
index 6bfd8bc..64162f6 100644
--- a/daemon/telegram/packages/decorator.py
+++ b/daemon/telegram/packages/decorator.py
@@ -9,9 +9,12 @@ from typing_extensions import ParamSpec, ParamSpecArgs, ParamSpecKwargs
from functools import wraps
import re
import asyncio
+import logging
+
__all__ = ["handle_flood"]
+log = logging.getLogger("telegram")
T = TypeVar("T")
P = ParamSpec("P")
@@ -23,19 +26,16 @@ def handle_flood(func: Callable[P, Coroutine[Any,Any,T]]) -> Callable[P, Corouti
try:
return await func(*args, **kwargs)
except FloodWait as e:
- # Calling logger attr from the DaemonTelegram() class
- logger = args[0].logger
-
- _flood_exceptions(e, logger)
- logger.info("Woken up from flood wait...")
+ _flood_exceptions(e)
+ log.info("Woken up from flood wait...")
return callback
-async def _flood_exceptions(e, logger):
+async def _flood_exceptions(e):
x = re.search(r"A wait of (\d+) seconds is required", str(e))
if not x:
raise e
n = int(x.group(1))
- logger.info(f"Sleeping for {n} seconds due to Telegram limit")
+ log.info(f"Sleeping for {n} seconds due to Telegram limit")
await asyncio.sleep(n)
diff --git a/daemon/tg.py b/daemon/tg.py
index 26f8374..5233faa 100644
--- a/daemon/tg.py
+++ b/daemon/tg.py
@@ -7,12 +7,10 @@
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from dotenv import load_dotenv
from mysql import connector
-from pyrogram import idle
from atom import Scraper
from telegram.packages import DaemonTelegram
from telegram.mailer import BotMutexes
from telegram.mailer import Bot
-from logger import BotLogger
import os
import logging
import logging.config
@@ -21,9 +19,6 @@ import logging.config
def main():
load_dotenv("telegram.env")
- logger = BotLogger()
- logger.init()
-
logging.config.fileConfig("telegram/telegram.logger.conf")
logging.getLogger("apscheduler").setLevel(logging.WARNING)
logging.getLogger("pyrogram").setLevel(logging.WARNING)
@@ -33,7 +28,6 @@ def main():
api_id=int(os.getenv("API_ID")),
api_hash=os.getenv("API_HASH"),
bot_token=os.getenv("BOT_TOKEN"),
- logger=logger,
conn=connector.connect(
host=os.getenv("DB_HOST"),
user=os.getenv("DB_USER"),
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 14/17] discord: Implement DaemonException and report_err in get_lore_mail.py
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (12 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 13/17] telegram: logger: Refactor all logging method Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 15/17] discord: logger: Refactor all logging method Muhammad Rizki
` (4 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
In the last series, I didn't care so much to look into the Discord's
get_lore_mail.py slash command. I've implemented them same as in the
listener.py.
Signed-off-by: Muhammad Rizki <[email protected]>
---
.../gnuweeb/plugins/slash_commands/get_lore_mail.py | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py b/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py
index 56330f3..84dca83 100644
--- a/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py
+++ b/daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py
@@ -10,6 +10,7 @@ from discord import app_commands
from atom import utils
from atom import Scraper
from enums import Platform
+from exceptions import DaemonException
class GetLoreSC(commands.Cog):
@@ -29,9 +30,11 @@ class GetLoreSC(commands.Cog):
mail = await s.get_email_from_url(url)
text, _, is_patch = utils.create_template(mail, Platform.DISCORD)
except:
- exc_str = utils.catch_err()
- self.bot.logger.warning(exc_str)
- await self.bot.send_log_file(url)
+ e = DaemonException()
+ e.set_thread_url(url)
+ e.set_message(utils.catch_err())
+ await self.bot.report_err(e)
+ return
if is_patch:
m = await self.bot.send_patch_mail_interaction(
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 15/17] discord: logger: Refactor all logging method
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (13 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 14/17] discord: Implement DaemonException and report_err in get_lore_mail.py Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 16/17] refactor: Move requirements.txt Muhammad Rizki
` (3 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Same as the telegram refactors. We use a simple logging method by using
file config and replace all custom logger method to built-in logging
method. This method is more cleaner and less code wasteful.
The reason why we should name the logger like:
`logging.getLogger("dscord")`
Because, if we use "discord" as the logger name, it will conflict the
discord.py's logger, that's why we need to disable the discord.py's
logger not to interrupt our logger.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/dc.py | 8 +-------
daemon/dscord/gnuweeb/client.py | 19 ++++++++++++-------
daemon/dscord/gnuweeb/filters.py | 12 +++++-------
.../dscord/gnuweeb/plugins/events/on_ready.py | 5 ++++-
daemon/dscord/mailer/listener.py | 17 ++++++++++-------
5 files changed, 32 insertions(+), 29 deletions(-)
diff --git a/daemon/dc.py b/daemon/dc.py
index f4eeefb..dade9f1 100644
--- a/daemon/dc.py
+++ b/daemon/dc.py
@@ -13,8 +13,6 @@ from apscheduler.schedulers.asyncio import AsyncIOScheduler
from dscord.gnuweeb import GWClient
from dscord.mailer import Listener
from dscord.mailer import Mutexes
-from enums.platform import Platform
-from logger import BotLogger
from atom import Scraper
@@ -28,9 +26,6 @@ def main():
}
)
- logger = BotLogger(Platform.DISCORD)
- logger.init()
-
logging.config.fileConfig("dscord/discord.logger.conf")
logging.getLogger("apscheduler").setLevel(logging.WARNING)
logging.getLogger("discord").setLevel(logging.WARNING)
@@ -42,8 +37,7 @@ def main():
port=int(os.environ.get("DB_PORT", 3306)),
password=os.getenv("DB_PASS"),
database=os.getenv("DB_NAME")
- ),
- logger=logger
+ )
)
mailer = Listener(
diff --git a/daemon/dscord/gnuweeb/client.py b/daemon/dscord/gnuweeb/client.py
index 7d6433b..a04fd26 100644
--- a/daemon/dscord/gnuweeb/client.py
+++ b/daemon/dscord/gnuweeb/client.py
@@ -3,6 +3,7 @@
# Copyright (C) 2022 Muhammad Rizki <[email protected]>
#
+import logging
import discord
from discord import Interaction, Message
from discord.ext import commands
@@ -14,18 +15,19 @@ from . import models
from atom import utils
from enums import Platform
from exceptions import DaemonException
-from logger.log import BotLogger
from dscord.config import ACTIVITY_NAME, LOG_CHANNEL_ID
from dscord.database import DB
+log = logging.getLogger("dscord")
+
+
class GWClient(commands.Bot):
- def __init__(self, db_conn, logger: BotLogger) -> None:
+ def __init__(self, db_conn) -> None:
self.db = DB(db_conn)
intents = Intents.default()
intents.message_content = True
self.mailer = None
- self.logger = logger
super().__init__(
command_prefix=["$", "."],
description="Just a bot for receiving lore emails.",
@@ -42,7 +44,7 @@ class GWClient(commands.Bot):
async def report_err(self, e: DaemonException):
- self.logger.warning(e.original_exception)
+ log.warning(e.original_exception)
capt = f"Atom URL: {e.atom_url}\n"
capt += f"Thread URL: {e.thread_url}"
await self.send_log_file(capt)
@@ -50,7 +52,10 @@ class GWClient(commands.Bot):
@filters.wait_on_limit
async def send_log_file(self, caption: str):
- filename = self.logger.handlers[0].baseFilename
+ for handler in log.root.handlers:
+ if isinstance(handler, logging.FileHandler):
+ filename = handler.baseFilename
+
channel = self.get_channel(LOG_CHANNEL_ID)
await channel.send(
content=caption,
@@ -61,7 +66,7 @@ class GWClient(commands.Bot):
@filters.wait_on_limit
async def send_text_email(self, guild_id: int, chat_id: int, text: str,
reply_to: Union[int, None] = None, url: str = None) -> Message:
- self.logger.debug("[send_text_email]")
+ log.debug("[send_text_email]")
channel = self.get_channel(chat_id)
return await channel.send(
@@ -78,7 +83,7 @@ class GWClient(commands.Bot):
@filters.wait_on_limit
async def send_patch_email(self, mail, guild_id: int, chat_id: int, text: str,
reply_to: Union[int, None] = None, url: str = None) -> Message:
- self.logger.debug("[send_patch_email]")
+ log.debug("[send_patch_email]")
tmp, doc, caption, url = utils.prepare_patch(
mail, text, url, Platform.DISCORD
)
diff --git a/daemon/dscord/gnuweeb/filters.py b/daemon/dscord/gnuweeb/filters.py
index 28b492c..ccd13e7 100644
--- a/daemon/dscord/gnuweeb/filters.py
+++ b/daemon/dscord/gnuweeb/filters.py
@@ -5,6 +5,7 @@
import asyncio
+import logging
from typing import Any, Callable, TypeVar, Coroutine
from typing_extensions import ParamSpec, ParamSpecArgs, ParamSpecKwargs
from functools import wraps
@@ -13,9 +14,9 @@ import discord
from discord import Interaction
from dscord import config
-from logger import BotLogger
+log = logging.getLogger("dscord")
T = TypeVar("T")
P = ParamSpec("P")
@@ -45,15 +46,12 @@ def wait_on_limit(func: Callable[P, Coroutine[Any,Any,T]]) -> Callable[P, Corout
try:
return await func(*args, **kwargs)
except discord.errors.RateLimited as e:
- # Calling logger attr from the GWClient() class
- logger = args[0].logger
-
_flood_exceptions(e)
- logger.info("Woken up from flood wait...")
+ log.info("Woken up from flood wait...")
return callback
-async def _flood_exceptions(e: "discord.errors.RateLimited", logger: BotLogger):
+async def _flood_exceptions(e: "discord.errors.RateLimited"):
wait = e.retry_after
- logger.info(f"Sleeping for {wait} seconds due to Discord limit")
+ log.info(f"Sleeping for {wait} seconds due to Discord limit")
await asyncio.sleep(wait)
diff --git a/daemon/dscord/gnuweeb/plugins/events/on_ready.py b/daemon/dscord/gnuweeb/plugins/events/on_ready.py
index e7f63cd..4248aad 100644
--- a/daemon/dscord/gnuweeb/plugins/events/on_ready.py
+++ b/daemon/dscord/gnuweeb/plugins/events/on_ready.py
@@ -3,9 +3,12 @@
# Copyright (C) 2022 Muhammad Rizki <[email protected]>
#
+import logging
from discord.ext import commands
+log = logging.getLogger("dscord")
+
class OnReady(commands.Cog):
def __init__(self, bot: "commands.Bot") -> None:
self.bot = bot
@@ -23,4 +26,4 @@ class OnReady(commands.Cog):
t += f"Send `{prefix}sync` message to the Discord channel "
t += "where the bot is running.\n"
- self.bot.logger.info(t)
+ log.info(t)
diff --git a/daemon/dscord/mailer/listener.py b/daemon/dscord/mailer/listener.py
index fc066b7..25e5715 100644
--- a/daemon/dscord/mailer/listener.py
+++ b/daemon/dscord/mailer/listener.py
@@ -6,6 +6,7 @@
import asyncio
import re
+import logging
from mysql.connector.errors import OperationalError, DatabaseError
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from discord import File
@@ -18,6 +19,9 @@ from enums import Platform
from exceptions import DaemonException
+log = logging.getLogger("dscord")
+
+
class Mutexes:
def __init__(self):
self.lock = asyncio.Lock()
@@ -36,7 +40,6 @@ class Listener:
self.scraper = scraper
self.mutexes = mutexes
self.db = client.db
- self.logger = client.logger
self.isRunnerFixed = False
self.runner = None
@@ -46,7 +49,7 @@ class Listener:
# Execute __run() once to avoid high latency at
# initilization.
#
- self.logger.info("Initialize listener...\n")
+ log.info("Initialize listener...\n")
self.sched.start()
self.runner = self.sched.add_job(func=self.__run)
@@ -56,8 +59,8 @@ class Listener:
# TODO(ammarfaizi2):
# Ideally, we also want to log and report this situation.
#
- self.logger.error(f"Database error: {str(e)}")
- self.logger.info("Reconnecting to the database...")
+ log.error(f"Database error: {str(e)}")
+ log.info("Reconnecting to the database...")
#
# Don't do this too often to avoid reconnect burst.
@@ -70,7 +73,7 @@ class Listener:
async def __run(self):
- self.logger.info("Running...")
+ log.info("Running...")
url = None
try:
@@ -128,7 +131,7 @@ class Listener:
email_msg_id = utils.get_email_msg_id(mail)
if not email_msg_id:
md = "email_msg_id not detected, skipping malformed email"
- self.logger.debug(md)
+ log.debug(md)
return False
email_id = self.__get_email_id_sent(
@@ -137,7 +140,7 @@ class Listener:
)
if not email_id:
md = f"Skipping {email_id} because has already been sent to Discord"
- self.logger.debug(md)
+ log.debug(md)
return False
text, files, is_patch = utils.create_template(mail, Platform.DISCORD)
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 16/17] refactor: Move requirements.txt
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (14 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 15/17] discord: logger: Refactor all logging method Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-18 17:53 ` [PATCH v2 17/17] cleanup: Remove the custom logger module Muhammad Rizki
` (2 subsequent siblings)
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Move `requirements.txt` to the outside of the `telegram` directory and
update the required dependencies in it. This requirements file is for
both Telegram and Discord so we don't need to add another requirements
file in their directory.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/{telegram => }/requirements.txt | 2 ++
1 file changed, 2 insertions(+)
rename daemon/{telegram => }/requirements.txt (81%)
diff --git a/daemon/telegram/requirements.txt b/daemon/requirements.txt
similarity index 81%
rename from daemon/telegram/requirements.txt
rename to daemon/requirements.txt
index 193cec9..3a16421 100644
--- a/daemon/telegram/requirements.txt
+++ b/daemon/requirements.txt
@@ -8,3 +8,5 @@ tgcrypto
xmltodict
unicode
python-slugify
+typing-extensions
+discord.py
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 17/17] cleanup: Remove the custom logger module
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (15 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 16/17] refactor: Move requirements.txt Muhammad Rizki
@ 2023-01-18 17:53 ` Muhammad Rizki
2023-01-19 6:54 ` [PATCH v2 00/17] Everything about logger changes and some fixes Ammar Faizi
2023-01-19 6:54 ` Ammar Faizi
18 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-18 17:53 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
We don't need this custom logger anymore since we already use the
built-in `logging` and use .conf file.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/logger/__init__.py | 1 -
daemon/logger/log.py | 52 ---------------------------------------
2 files changed, 53 deletions(-)
delete mode 100644 daemon/logger/__init__.py
delete mode 100644 daemon/logger/log.py
diff --git a/daemon/logger/__init__.py b/daemon/logger/__init__.py
deleted file mode 100644
index bb3145a..0000000
--- a/daemon/logger/__init__.py
+++ /dev/null
@@ -1 +0,0 @@
-from .log import BotLogger
diff --git a/daemon/logger/log.py b/daemon/logger/log.py
deleted file mode 100644
index 8382404..0000000
--- a/daemon/logger/log.py
+++ /dev/null
@@ -1,52 +0,0 @@
-import logging
-import os
-from enums import Platform
-
-
-class BotLogger(logging.Logger):
-
- def __init__(
- self,
- platform: Platform = Platform.TELEGRAM,
- level: int = logging.WARNING
- ) -> None:
-
- self.platform = str(platform.value)
- self.level = level
-
- plat_rep = self.platform.replace("discord", "dscord")
- self.file_name = f"{plat_rep}/" \
- f"{os.getenv('STORAGE_DIR', 'storage')}/" \
- f"{self.platform}.log"
-
- self.file_fmt = "{" \
- "'time': '%(asctime)s'," \
- "'func': '%(name)s: %(funcName)s'," \
- "'path': '%(pathname)s'," \
- "'level': '%(levelname)s'," \
- "'msg': '%(message)s'"\
- "}"
-
- self.stream_fmt = "%(asctime)s | " \
- "%(name)s: %(funcName)s | " \
- "%(levelname)s | " \
- "%(message)s"
-
-
- def init(self):
- super().__init__(self.platform, self.level)
-
- file_handler = logging.FileHandler(self.file_name)
- stream_handler = logging.StreamHandler()
-
- file_fmtr = logging.Formatter(self.file_fmt)
- stream_fmtr = logging.Formatter(self.stream_fmt)
-
- file_handler.setFormatter(file_fmtr)
- stream_handler.setFormatter(stream_fmtr)
-
- file_handler.setLevel(logging.WARNING)
- self.setLevel(logging.INFO)
-
- self.addHandler(file_handler)
- self.addHandler(stream_handler)
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 21+ messages in thread* Re: [PATCH v2 00/17] Everything about logger changes and some fixes
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (16 preceding siblings ...)
2023-01-18 17:53 ` [PATCH v2 17/17] cleanup: Remove the custom logger module Muhammad Rizki
@ 2023-01-19 6:54 ` Ammar Faizi
2023-01-19 12:49 ` Muhammad Rizki
2023-01-19 6:54 ` Ammar Faizi
18 siblings, 1 reply; 21+ messages in thread
From: Ammar Faizi @ 2023-01-19 6:54 UTC (permalink / raw)
To: Muhammad Rizki; +Cc: Alviro Iskandar Setiawan, GNU/Weeb Mailing List
On Thu, Jan 19, 2023 at 12:53:39AM +0700, Muhammad Rizki wrote:
> - Patch 7 is to fix the type annotations for the Telegram's decorator.
You use 'typing_extensions' extension in patch #7.
> - Patch 16 is to move requirements.txt to the daemon and update deps.
But you add 'typing_extensions' in patch #16. This honestly, doesn't
make sense from the test standpoint. You should add 'typing_extensions'
to the requirement list first, then use it later. Each patch, when
applied, should not leave a breakage in the middle.
To save time and internet bandwith, I just reordered it myself because
it's simple enough for me to fix.
--
Ammar Faizi
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v2 00/17] Everything about logger changes and some fixes
2023-01-19 6:54 ` [PATCH v2 00/17] Everything about logger changes and some fixes Ammar Faizi
@ 2023-01-19 12:49 ` Muhammad Rizki
0 siblings, 0 replies; 21+ messages in thread
From: Muhammad Rizki @ 2023-01-19 12:49 UTC (permalink / raw)
To: Ammar Faizi; +Cc: Alviro Iskandar Setiawan, GNU/Weeb Mailing List
On 19/01/2023 13.54, Ammar Faizi wrote:
> On Thu, Jan 19, 2023 at 12:53:39AM +0700, Muhammad Rizki wrote:
>> - Patch 7 is to fix the type annotations for the Telegram's decorator.
>
> You use 'typing_extensions' extension in patch #7.
>
>> - Patch 16 is to move requirements.txt to the daemon and update deps.
>
> But you add 'typing_extensions' in patch #16. This honestly, doesn't
> make sense from the test standpoint. You should add 'typing_extensions'
> to the requirement list first, then use it later. Each patch, when
> applied, should not leave a breakage in the middle.
>
> To save time and internet bandwith, I just reordered it myself because
> it's simple enough for me to fix.
>
Ah yes my bad, thanks anyway!
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 00/17] Everything about logger changes and some fixes
2023-01-18 17:53 [PATCH v2 00/17] Everything about logger changes and some fixes Muhammad Rizki
` (17 preceding siblings ...)
2023-01-19 6:54 ` [PATCH v2 00/17] Everything about logger changes and some fixes Ammar Faizi
@ 2023-01-19 6:54 ` Ammar Faizi
18 siblings, 0 replies; 21+ messages in thread
From: Ammar Faizi @ 2023-01-19 6:54 UTC (permalink / raw)
To: Muhammad Rizki
Cc: Ammar Faizi, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
On Thu, 19 Jan 2023 00:53:39 +0700, Muhammad Rizki wrote:
> This series is about refactor logger method, fix charset issue, and some
> improvements. In this series we use a simple logging method that so
> easily to manage logger config using .conf file.
>
> - What's new?
> v1:
> 1) Add return type annotation for the send_te.xt_mail() and
> send_patch_email() for the Discord bot
>
> [...]
Applied, thanks!
[01/17] telegram: Simplify code to get DB_PORT from env
commit: 1701589a63270d2204d4386630e1cc35d910dd90
[02/17] discord: Simplify code to get DB_PORT from env
commit: d9ab32d7ea83e6d08d705d3d51a5b1d35570a497
[03/17] telegram: logger: Add a telegram.logger.conf
commit: da2ff98f54979669867bca1a69f2ca5d19d10652
[04/17] discord: logger: Add a discord.logger.conf
commit: 5ee8bdab7d3eed174bfd06ab30358027d2d4109e
[05/17] telegram: logger: Initialize the configuration for the Telegram logger.
commit: afc6016ce1ec177b3ae78d65a9fb157292a3d1f6
[06/17] discord: logger: Initialize the configuration for the Discord logger.
commit: 83fa3b5cafaa7c430042404d39480b4382b6dad3
[07/17] telegram: fix: Fix the type annoations for the decorator
commit: 3b420794c629b1adea071ecd6b3d4d421620f43e
[08/17] discord: cleanup: Remove some unnecessary comments
commit: 556c16b605e64746dd174d3a78aefe3ccafd1075
[09/17] discord: fix: Fix the type annotations for the decorator
commit: 57a705431e73b815f209cf340d4a102c5abf13f1
[10/17] discord: typing: Add return type annotations
commit: dd2909aa6ae90b848561d02c8cdf1cb35d3a0420
[11/17] telegram: Implement DaemonException() and report_err() in scrape.py
commit: 16431f083520918a6e29e9fe1e4391a626e7bfe5
[12/17] utils: fix: Fix charset issue for get_decoded_payload()
commit: 858a25d1516d6bef22389d6bbd3577f4c1bc9fc4
[13/17] telegram: logger: Refactor all logging method
commit: 332ffd0c5d197077dd77b4d72d323211eed7b485
[14/17] discord: Implement DaemonException and report_err in get_lore_mail.py
commit: aae8fbb4dcc5a3c1827df70a6ed3763d9dc73619
[15/17] discord: logger: Refactor all logging method
commit: df3476d67c90418710ceaeaff0413e0418d54041
[16/17] refactor: Move requirements.txt
(no commit info)
[17/17] cleanup: Remove the custom logger module
commit: bf2fe963caf19a5002f585cd07541400c5df945b
Best regards,
--
Ammar Faizi <[email protected]>
^ permalink raw reply [flat|nested] 21+ messages in thread