* [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord
@ 2022-09-19 7:26 Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 01/17] telegram: Move the Telegram bot source code Muhammad Rizki
` (16 more replies)
0 siblings, 17 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Hi sir,
This series, Moving the Telegram bot source code to it's specific
directory because there will be a Discord bot so I must to split them.
Refactor the Telegram bot database method to split between insert,
get, and delete to make it more clean and maintainable.
Also, move the atom & utility source code to it's specific directory to
make it reusable functions to reduce code duplication.
Make initial work for Discord bot, it's still undone but it can run.
I just prepared for the event handlers such as on_error handler and
on_ready handler. on_error is to ignore error when any user in the
Discord channel typing a word that doesn't exist in the basic commands
bot like `ComandNotFound: "w"`. on_ready is to display a bot information
when it's successfuly run the bot, also prepared for the database
functions too, the methods pattern just like the Telegram bot,
also combine the tables in db.sql.
There are 17 patches in this series:
- Patch 1 is to split the Telegram bot source code to it's specific dir
- Patch 2 is to refactor Telegram database method to split between funcs
- Patch 3 is to rename some functions to make it more readable
- Patch 4 is to add ignore file for any names that have .env extension
- Patch 5 is to move the Telegram bot runner script outside it's dir
- Patch 6 is to rename mailer directory and listener file
- Patch 7 is to move scraper and utility to it's specific directory
- Patch 8 is to commit initial work for Discord bot
- Patch 9 is to add a print success run in the on_ready event handler
- Patch 10 is to add an error handler in the on_error event handler
- Patch 11 is to move db.sql to the workdir and combine db tables
- Patch 12 is to add database tables for Discord bot in the db.sql file
- Patch 13 is to make an initial Discord bot database instance class
- Patch 14 is to add save_atom() function in database insertion dir
- Patch 15 is to add save_broadcast() function in database insertion dir
- Patch 16 is to add save_discord_mail() func in database insertion dir
- Patch 17 is to add save_email() func in database insertion dir
How to use:
1. Execute db.sql file in the daemon directory
2. Setup .env file, the example is there with suffix .example, this
file name must remove the suffix name .example
3. Setup the config.py in each bot directory such as dscord & telegram,
the example is there with suffix .example & the file name must remove
suffix name .example
4. Run `pip3 install -r requirements.txt` in each bot directory
5. STORAGE_DIR env value must `storage` to make it work fine
6. Run the bot by `python3 dc.py` or `python3 tg.py`
Tested and works fine, please give it a test if there is unexpected
errors, if it does tell me because I'm not in a good mood for coding,
Thanks!
## Changelog
v2 -> v3
- Change commit description for commit 07/17
v1 -> v2
- Remove the ignore file in .gitignore while the file is not added yet.
- Change prefix name each commits from `[]` to `:`
- Change commit description for commit 05/17
Signed-off-by: Muhammad Rizki <[email protected]>
---
Muhammad Rizki (17):
telegram: Move the Telegram bot source code
telegram: Refactor Telegram bot database method
telegram: Renaming some functions in scraper/bot.py
Add ignore file for .env
telegram: Refactor the Telegram bot
telegram: Renames in telegram mailer directory
Move scraper and utility file
discord: Initial work Discord bot
discord: Add success run notice on_ready event
discord: Add error handler on events
Move db.sql to combine database with Discord and Telegram
discord: Add database tables for Discord bot
discord: Add initial Discord bot database instance
discord: Add save_atom() in database insertion
discord: Add save_broadcast() in database insertion
discord: Add save_discord_mail() in database insertion
discord: Add save_email() in database insertion
.gitignore | 4 +-
daemon/{scraper => atom}/__init__.py | 3 +-
daemon/{scraper => atom}/scraper.py | 0
daemon/{scraper => atom}/utils.py | 0
daemon/db.sql | 74 +++++-
daemon/dc.py | 20 ++
daemon/discord.env.example | 11 +
daemon/dscord/config.py.example | 13 ++
daemon/dscord/database/__init__.py | 6 +
daemon/dscord/database/core.py | 17 ++
daemon/dscord/database/methods/__init__.py | 12 +
.../database/methods/insertion/__init__.py | 18 ++
.../database/methods/insertion/insert_atom.py | 27 +++
.../methods/insertion/insert_broadcast.py | 42 ++++
.../methods/insertion/insert_discord.py | 14 ++
.../methods/insertion/insert_email.py | 20 ++
daemon/dscord/gnuweeb/__init__.py | 6 +
daemon/dscord/gnuweeb/client.py | 28 +++
daemon/dscord/gnuweeb/plugins/__init__.py | 17 ++
.../dscord/gnuweeb/plugins/events/__init__.py | 13 ++
.../dscord/gnuweeb/plugins/events/on_error.py | 17 ++
.../dscord/gnuweeb/plugins/events/on_ready.py | 21 ++
daemon/scraper/db.py | 217 ------------------
daemon/{.env.example => telegram.env.example} | 3 +-
daemon/{ => telegram}/config.py.example | 0
daemon/telegram/database/__init__.py | 7 +
daemon/telegram/database/core.py | 20 ++
daemon/telegram/database/methods/__init__.py | 17 ++
.../database/methods/deletion/__init__.py | 14 ++
.../database/methods/deletion/delet_atom.py | 15 ++
.../methods/deletion/delete_broadcast.py | 15 ++
.../database/methods/getter/__init__.py | 18 ++
.../database/methods/getter/get_atom_urls.py | 21 ++
.../methods/getter/get_broadcast_chats.py | 21 ++
.../database/methods/getter/get_email_id.py | 62 +++++
.../methods/getter/get_telegram_reply.py | 33 +++
.../database/methods/insertion/__init__.py | 18 ++
.../database/methods/insertion/insert_atom.py | 27 +++
.../methods/insertion/insert_broadcast.py | 56 +++++
.../methods/insertion/insert_email.py | 27 +++
.../methods/insertion/insert_telegram.py | 21 ++
daemon/telegram/mailer/__init__.py | 8 +
.../bot.py => telegram/mailer/listener.py} | 31 +--
daemon/{ => telegram}/packages/__init__.py | 0
daemon/{ => telegram}/packages/client.py | 6 +-
daemon/{ => telegram}/packages/decorator.py | 0
.../packages/plugins/callbacks/del_atom.py | 6 +-
.../packages/plugins/callbacks/del_chat.py | 6 +-
.../packages/plugins/commands/debugger.py | 2 +-
.../packages/plugins/commands/manage_atom.py | 8 +-
.../plugins/commands/manage_broadcast.py | 8 +-
.../packages/plugins/commands/scrape.py | 8 +-
daemon/{ => telegram}/requirements.txt | 0
daemon/{ => telegram}/storage/.gitignore | 0
daemon/{run.py => tg.py} | 14 +-
55 files changed, 818 insertions(+), 274 deletions(-)
rename daemon/{scraper => atom}/__init__.py (80%)
rename daemon/{scraper => atom}/scraper.py (100%)
rename daemon/{scraper => atom}/utils.py (100%)
create mode 100644 daemon/dc.py
create mode 100644 daemon/discord.env.example
create mode 100644 daemon/dscord/config.py.example
create mode 100644 daemon/dscord/database/__init__.py
create mode 100644 daemon/dscord/database/core.py
create mode 100644 daemon/dscord/database/methods/__init__.py
create mode 100644 daemon/dscord/database/methods/insertion/__init__.py
create mode 100644 daemon/dscord/database/methods/insertion/insert_atom.py
create mode 100644 daemon/dscord/database/methods/insertion/insert_broadcast.py
create mode 100644 daemon/dscord/database/methods/insertion/insert_discord.py
create mode 100644 daemon/dscord/database/methods/insertion/insert_email.py
create mode 100644 daemon/dscord/gnuweeb/__init__.py
create mode 100644 daemon/dscord/gnuweeb/client.py
create mode 100644 daemon/dscord/gnuweeb/plugins/__init__.py
create mode 100644 daemon/dscord/gnuweeb/plugins/events/__init__.py
create mode 100644 daemon/dscord/gnuweeb/plugins/events/on_error.py
create mode 100644 daemon/dscord/gnuweeb/plugins/events/on_ready.py
delete mode 100644 daemon/scraper/db.py
rename daemon/{.env.example => telegram.env.example} (82%)
rename daemon/{ => telegram}/config.py.example (100%)
create mode 100644 daemon/telegram/database/__init__.py
create mode 100644 daemon/telegram/database/core.py
create mode 100644 daemon/telegram/database/methods/__init__.py
create mode 100644 daemon/telegram/database/methods/deletion/__init__.py
create mode 100644 daemon/telegram/database/methods/deletion/delet_atom.py
create mode 100644 daemon/telegram/database/methods/deletion/delete_broadcast.py
create mode 100644 daemon/telegram/database/methods/getter/__init__.py
create mode 100644 daemon/telegram/database/methods/getter/get_atom_urls.py
create mode 100644 daemon/telegram/database/methods/getter/get_broadcast_chats.py
create mode 100644 daemon/telegram/database/methods/getter/get_email_id.py
create mode 100644 daemon/telegram/database/methods/getter/get_telegram_reply.py
create mode 100644 daemon/telegram/database/methods/insertion/__init__.py
create mode 100644 daemon/telegram/database/methods/insertion/insert_atom.py
create mode 100644 daemon/telegram/database/methods/insertion/insert_broadcast.py
create mode 100644 daemon/telegram/database/methods/insertion/insert_email.py
create mode 100644 daemon/telegram/database/methods/insertion/insert_telegram.py
create mode 100644 daemon/telegram/mailer/__init__.py
rename daemon/{scraper/bot.py => telegram/mailer/listener.py} (77%)
rename daemon/{ => telegram}/packages/__init__.py (100%)
rename daemon/{ => telegram}/packages/client.py (95%)
rename daemon/{ => telegram}/packages/decorator.py (100%)
rename daemon/{ => telegram}/packages/plugins/callbacks/del_atom.py (88%)
rename daemon/{ => telegram}/packages/plugins/callbacks/del_chat.py (90%)
rename daemon/{ => telegram}/packages/plugins/commands/debugger.py (97%)
rename daemon/{ => telegram}/packages/plugins/commands/manage_atom.py (91%)
rename daemon/{ => telegram}/packages/plugins/commands/manage_broadcast.py (92%)
rename daemon/{ => telegram}/packages/plugins/commands/scrape.py (91%)
rename daemon/{ => telegram}/requirements.txt (100%)
rename daemon/{ => telegram}/storage/.gitignore (100%)
rename daemon/{run.py => tg.py} (78%)
base-commit: 2582d7e5225d47a01f606808fc71e5e6aa7cb153
--
Muhammad Rizki
^ permalink raw reply [flat|nested] 20+ messages in thread
* [RFC PATCH v3 01/17] telegram: Move the Telegram bot source code
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 02/17] telegram: Refactor Telegram bot database method Muhammad Rizki
` (15 subsequent siblings)
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Move them into their own directory because there is another platform bot
in the future, so it's better to split them.
Acked-by: Ammar Faizi <[email protected]>
Signed-off-by: Muhammad Rizki <[email protected]>
---
.gitignore | 2 +-
daemon/{ => telegram}/.env.example | 0
daemon/{ => telegram}/config.py.example | 0
daemon/{ => telegram}/db.sql | 0
daemon/{ => telegram}/packages/__init__.py | 0
daemon/{ => telegram}/packages/client.py | 0
daemon/{ => telegram}/packages/decorator.py | 0
daemon/{ => telegram}/packages/plugins/callbacks/del_atom.py | 0
daemon/{ => telegram}/packages/plugins/callbacks/del_chat.py | 0
daemon/{ => telegram}/packages/plugins/commands/debugger.py | 0
daemon/{ => telegram}/packages/plugins/commands/manage_atom.py | 0
.../packages/plugins/commands/manage_broadcast.py | 0
daemon/{ => telegram}/packages/plugins/commands/scrape.py | 0
daemon/{ => telegram}/requirements.txt | 0
daemon/{ => telegram}/run.py | 0
daemon/{ => telegram}/scraper/__init__.py | 0
daemon/{ => telegram}/scraper/bot.py | 0
daemon/{ => telegram}/scraper/db.py | 0
daemon/{ => telegram}/scraper/scraper.py | 0
daemon/{ => telegram}/scraper/utils.py | 0
daemon/{ => telegram}/storage/.gitignore | 0
21 files changed, 1 insertion(+), 1 deletion(-)
rename daemon/{ => telegram}/.env.example (100%)
rename daemon/{ => telegram}/config.py.example (100%)
rename daemon/{ => telegram}/db.sql (100%)
rename daemon/{ => telegram}/packages/__init__.py (100%)
rename daemon/{ => telegram}/packages/client.py (100%)
rename daemon/{ => telegram}/packages/decorator.py (100%)
rename daemon/{ => telegram}/packages/plugins/callbacks/del_atom.py (100%)
rename daemon/{ => telegram}/packages/plugins/callbacks/del_chat.py (100%)
rename daemon/{ => telegram}/packages/plugins/commands/debugger.py (100%)
rename daemon/{ => telegram}/packages/plugins/commands/manage_atom.py (100%)
rename daemon/{ => telegram}/packages/plugins/commands/manage_broadcast.py (100%)
rename daemon/{ => telegram}/packages/plugins/commands/scrape.py (100%)
rename daemon/{ => telegram}/requirements.txt (100%)
rename daemon/{ => telegram}/run.py (100%)
rename daemon/{ => telegram}/scraper/__init__.py (100%)
rename daemon/{ => telegram}/scraper/bot.py (100%)
rename daemon/{ => telegram}/scraper/db.py (100%)
rename daemon/{ => telegram}/scraper/scraper.py (100%)
rename daemon/{ => telegram}/scraper/utils.py (100%)
rename daemon/{ => telegram}/storage/.gitignore (100%)
diff --git a/.gitignore b/.gitignore
index 4de282d..3ca3c1b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -140,4 +140,4 @@ data.json
*.patch
# configuration file
-daemon/config.py
+daemon/telegram/config.py
diff --git a/daemon/.env.example b/daemon/telegram/.env.example
similarity index 100%
rename from daemon/.env.example
rename to daemon/telegram/.env.example
diff --git a/daemon/config.py.example b/daemon/telegram/config.py.example
similarity index 100%
rename from daemon/config.py.example
rename to daemon/telegram/config.py.example
diff --git a/daemon/db.sql b/daemon/telegram/db.sql
similarity index 100%
rename from daemon/db.sql
rename to daemon/telegram/db.sql
diff --git a/daemon/packages/__init__.py b/daemon/telegram/packages/__init__.py
similarity index 100%
rename from daemon/packages/__init__.py
rename to daemon/telegram/packages/__init__.py
diff --git a/daemon/packages/client.py b/daemon/telegram/packages/client.py
similarity index 100%
rename from daemon/packages/client.py
rename to daemon/telegram/packages/client.py
diff --git a/daemon/packages/decorator.py b/daemon/telegram/packages/decorator.py
similarity index 100%
rename from daemon/packages/decorator.py
rename to daemon/telegram/packages/decorator.py
diff --git a/daemon/packages/plugins/callbacks/del_atom.py b/daemon/telegram/packages/plugins/callbacks/del_atom.py
similarity index 100%
rename from daemon/packages/plugins/callbacks/del_atom.py
rename to daemon/telegram/packages/plugins/callbacks/del_atom.py
diff --git a/daemon/packages/plugins/callbacks/del_chat.py b/daemon/telegram/packages/plugins/callbacks/del_chat.py
similarity index 100%
rename from daemon/packages/plugins/callbacks/del_chat.py
rename to daemon/telegram/packages/plugins/callbacks/del_chat.py
diff --git a/daemon/packages/plugins/commands/debugger.py b/daemon/telegram/packages/plugins/commands/debugger.py
similarity index 100%
rename from daemon/packages/plugins/commands/debugger.py
rename to daemon/telegram/packages/plugins/commands/debugger.py
diff --git a/daemon/packages/plugins/commands/manage_atom.py b/daemon/telegram/packages/plugins/commands/manage_atom.py
similarity index 100%
rename from daemon/packages/plugins/commands/manage_atom.py
rename to daemon/telegram/packages/plugins/commands/manage_atom.py
diff --git a/daemon/packages/plugins/commands/manage_broadcast.py b/daemon/telegram/packages/plugins/commands/manage_broadcast.py
similarity index 100%
rename from daemon/packages/plugins/commands/manage_broadcast.py
rename to daemon/telegram/packages/plugins/commands/manage_broadcast.py
diff --git a/daemon/packages/plugins/commands/scrape.py b/daemon/telegram/packages/plugins/commands/scrape.py
similarity index 100%
rename from daemon/packages/plugins/commands/scrape.py
rename to daemon/telegram/packages/plugins/commands/scrape.py
diff --git a/daemon/requirements.txt b/daemon/telegram/requirements.txt
similarity index 100%
rename from daemon/requirements.txt
rename to daemon/telegram/requirements.txt
diff --git a/daemon/run.py b/daemon/telegram/run.py
similarity index 100%
rename from daemon/run.py
rename to daemon/telegram/run.py
diff --git a/daemon/scraper/__init__.py b/daemon/telegram/scraper/__init__.py
similarity index 100%
rename from daemon/scraper/__init__.py
rename to daemon/telegram/scraper/__init__.py
diff --git a/daemon/scraper/bot.py b/daemon/telegram/scraper/bot.py
similarity index 100%
rename from daemon/scraper/bot.py
rename to daemon/telegram/scraper/bot.py
diff --git a/daemon/scraper/db.py b/daemon/telegram/scraper/db.py
similarity index 100%
rename from daemon/scraper/db.py
rename to daemon/telegram/scraper/db.py
diff --git a/daemon/scraper/scraper.py b/daemon/telegram/scraper/scraper.py
similarity index 100%
rename from daemon/scraper/scraper.py
rename to daemon/telegram/scraper/scraper.py
diff --git a/daemon/scraper/utils.py b/daemon/telegram/scraper/utils.py
similarity index 100%
rename from daemon/scraper/utils.py
rename to daemon/telegram/scraper/utils.py
diff --git a/daemon/storage/.gitignore b/daemon/telegram/storage/.gitignore
similarity index 100%
rename from daemon/storage/.gitignore
rename to daemon/telegram/storage/.gitignore
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 02/17] telegram: Refactor Telegram bot database method
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 01/17] telegram: Move the Telegram bot source code Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 03/17] telegram: Renaming some functions in scraper/bot.py Muhammad Rizki
` (14 subsequent siblings)
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Refactor the Telegram bot database to make it more clean and
manageable to maintain.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/telegram/database/__init__.py | 7 +
daemon/telegram/database/core.py | 20 ++
daemon/telegram/database/methods/__init__.py | 17 ++
.../database/methods/deletion/__init__.py | 14 ++
.../database/methods/deletion/delet_atom.py | 15 ++
.../methods/deletion/delete_broadcast.py | 15 ++
.../database/methods/getter/__init__.py | 18 ++
.../database/methods/getter/get_atom_urls.py | 21 ++
.../methods/getter/get_broadcast_chats.py | 21 ++
.../database/methods/getter/get_email_id.py | 62 +++++
.../methods/getter/get_telegram_reply.py | 33 +++
.../database/methods/insertion/__init__.py | 18 ++
.../database/methods/insertion/insert_atom.py | 27 +++
.../methods/insertion/insert_broadcast.py | 56 +++++
.../methods/insertion/insert_email.py | 27 +++
.../methods/insertion/insert_telegram.py | 21 ++
daemon/telegram/db.sql | 18 +-
daemon/telegram/packages/client.py | 4 +-
.../packages/plugins/commands/manage_atom.py | 2 +-
.../plugins/commands/manage_broadcast.py | 2 +-
daemon/telegram/scraper/bot.py | 8 +-
daemon/telegram/scraper/db.py | 217 ------------------
22 files changed, 409 insertions(+), 234 deletions(-)
create mode 100644 daemon/telegram/database/__init__.py
create mode 100644 daemon/telegram/database/core.py
create mode 100644 daemon/telegram/database/methods/__init__.py
create mode 100644 daemon/telegram/database/methods/deletion/__init__.py
create mode 100644 daemon/telegram/database/methods/deletion/delet_atom.py
create mode 100644 daemon/telegram/database/methods/deletion/delete_broadcast.py
create mode 100644 daemon/telegram/database/methods/getter/__init__.py
create mode 100644 daemon/telegram/database/methods/getter/get_atom_urls.py
create mode 100644 daemon/telegram/database/methods/getter/get_broadcast_chats.py
create mode 100644 daemon/telegram/database/methods/getter/get_email_id.py
create mode 100644 daemon/telegram/database/methods/getter/get_telegram_reply.py
create mode 100644 daemon/telegram/database/methods/insertion/__init__.py
create mode 100644 daemon/telegram/database/methods/insertion/insert_atom.py
create mode 100644 daemon/telegram/database/methods/insertion/insert_broadcast.py
create mode 100644 daemon/telegram/database/methods/insertion/insert_email.py
create mode 100644 daemon/telegram/database/methods/insertion/insert_telegram.py
delete mode 100644 daemon/telegram/scraper/db.py
diff --git a/daemon/telegram/database/__init__.py b/daemon/telegram/database/__init__.py
new file mode 100644
index 0000000..930e3d9
--- /dev/null
+++ b/daemon/telegram/database/__init__.py
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+
+from .core import DB
diff --git a/daemon/telegram/database/core.py b/daemon/telegram/database/core.py
new file mode 100644
index 0000000..c34d7a8
--- /dev/null
+++ b/daemon/telegram/database/core.py
@@ -0,0 +1,20 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+# Copyright (C) 2022 Ammar Faizi <[email protected]>
+#
+
+
+from .methods import DBMethods
+
+
+class DB(DBMethods):
+ def __init__(self, conn):
+ self.conn = conn
+ self.conn.autocommit = True
+ self.cur = self.conn.cursor(buffered=True)
+
+
+ def __del__(self):
+ self.cur.close()
+ self.conn.close()
diff --git a/daemon/telegram/database/methods/__init__.py b/daemon/telegram/database/methods/__init__.py
new file mode 100644
index 0000000..961b4e0
--- /dev/null
+++ b/daemon/telegram/database/methods/__init__.py
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+# Copyright (C) 2022 Ammar Faizi <[email protected]>
+#
+
+
+from .deletion import Deletion
+from .getter import Getter
+from .insertion import Insertion
+
+
+class DBMethods(
+ Deletion,
+ Getter,
+ Insertion
+): pass
diff --git a/daemon/telegram/database/methods/deletion/__init__.py b/daemon/telegram/database/methods/deletion/__init__.py
new file mode 100644
index 0000000..b206929
--- /dev/null
+++ b/daemon/telegram/database/methods/deletion/__init__.py
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+
+from .delet_atom import DeleteAtom
+from .delete_broadcast import DeleteBroadcast
+
+
+class Deletion(
+ DeleteAtom,
+ DeleteBroadcast
+): pass
diff --git a/daemon/telegram/database/methods/deletion/delet_atom.py b/daemon/telegram/database/methods/deletion/delet_atom.py
new file mode 100644
index 0000000..d8ad4bf
--- /dev/null
+++ b/daemon/telegram/database/methods/deletion/delet_atom.py
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+
+class DeleteAtom:
+
+ def delete_atom(self, atom: str):
+ q = """
+ DELETE FROM tg_atoms
+ WHERE url = %(atom)s
+ """
+ self.cur.execute(q, {"atom": atom})
+ return self.cur.rowcount > 0
diff --git a/daemon/telegram/database/methods/deletion/delete_broadcast.py b/daemon/telegram/database/methods/deletion/delete_broadcast.py
new file mode 100644
index 0000000..d076dec
--- /dev/null
+++ b/daemon/telegram/database/methods/deletion/delete_broadcast.py
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+
+class DeleteBroadcast:
+
+ def delete_broadcast(self, chat_id: int):
+ q = """
+ DELETE FROM tg_broadcasts
+ WHERE chat_id = %(chat_id)s
+ """
+ self.cur.execute(q, {"chat_id": chat_id})
+ return self.cur.rowcount > 0
diff --git a/daemon/telegram/database/methods/getter/__init__.py b/daemon/telegram/database/methods/getter/__init__.py
new file mode 100644
index 0000000..e978c72
--- /dev/null
+++ b/daemon/telegram/database/methods/getter/__init__.py
@@ -0,0 +1,18 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+
+from .get_atom_urls import GetAtomURL
+from .get_broadcast_chats import GetBroadcastChats
+from .get_email_id import GetEmailID
+from .get_telegram_reply import GetTelegramReply
+
+
+class Getter(
+ GetAtomURL,
+ GetBroadcastChats,
+ GetEmailID,
+ GetTelegramReply
+): pass
diff --git a/daemon/telegram/database/methods/getter/get_atom_urls.py b/daemon/telegram/database/methods/getter/get_atom_urls.py
new file mode 100644
index 0000000..04a9315
--- /dev/null
+++ b/daemon/telegram/database/methods/getter/get_atom_urls.py
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+
+class GetAtomURL:
+
+ def get_atom_urls(self):
+ '''
+ Get lore kernel raw email URLs.
+ - Return list of raw email URLs: `List[str]`
+ '''
+ q = """
+ SELECT tg_atoms.url
+ FROM tg_atoms
+ """
+ self.cur.execute(q)
+ urls = self.cur.fetchall()
+
+ return [u[0] for u in urls]
diff --git a/daemon/telegram/database/methods/getter/get_broadcast_chats.py b/daemon/telegram/database/methods/getter/get_broadcast_chats.py
new file mode 100644
index 0000000..d92e879
--- /dev/null
+++ b/daemon/telegram/database/methods/getter/get_broadcast_chats.py
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+
+class GetBroadcastChats:
+
+ def get_broadcast_chats(self):
+ '''
+ Get broadcast chats that are currently
+ listening for new email.
+ - Return list of chat object: `List[Object]`
+ '''
+ q = """
+ SELECT *
+ FROM tg_broadcasts
+ """
+ self.cur.execute(q)
+
+ return self.cur.fetchall()
diff --git a/daemon/telegram/database/methods/getter/get_email_id.py b/daemon/telegram/database/methods/getter/get_email_id.py
new file mode 100644
index 0000000..44a6179
--- /dev/null
+++ b/daemon/telegram/database/methods/getter/get_email_id.py
@@ -0,0 +1,62 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+# Copyright (C) 2022 Ammar Faizi <[email protected]>
+#
+
+
+class GetEmailID:
+
+ def get_email_id(self, email_id, chat_id):
+ '''
+ Determine whether the email needs to be sent to @tg_chat_id.
+ - Return an email id (PK) if it needs to be sent
+ - Return None if it doesn't need to be sent
+ '''
+ if self.__is_sent(email_id, chat_id):
+ return
+
+ res = self.__email_id(email_id)
+ if not bool(res):
+ return
+
+ return int(res[0])
+
+
+ def __is_sent(self, email_id, chat_id):
+ '''
+ Checking if this email has already been sent
+ or not.
+ - Return True if it's already been sent
+ '''
+
+ q = """
+ SELECT tg_emails.id, tg_mail_msg.id FROM tg_emails LEFT JOIN tg_mail_msg
+ ON tg_emails.id = tg_mail_msg.email_id
+ WHERE tg_emails.message_id = %(email_id)s
+ AND tg_mail_msg.chat_id = %(chat_id)s
+ LIMIT 1
+ """
+
+ self.cur.execute(q, {
+ "email_id": email_id,
+ "chat_id": chat_id
+ })
+
+ res = self.cur.fetchone()
+ return bool(res)
+
+
+ def __email_id(self, email_id):
+ '''
+ Get the email id if match with the email message_id.
+ - Return the result if it's match and exists
+ '''
+
+ q = """
+ SELECT id FROM tg_emails WHERE message_id = %(email_id)s
+ """
+
+ self.cur.execute(q, {"email_id": email_id})
+ res = self.cur.fetchone()
+ return res
diff --git a/daemon/telegram/database/methods/getter/get_telegram_reply.py b/daemon/telegram/database/methods/getter/get_telegram_reply.py
new file mode 100644
index 0000000..94e3138
--- /dev/null
+++ b/daemon/telegram/database/methods/getter/get_telegram_reply.py
@@ -0,0 +1,33 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+# Copyright (C) 2022 Ammar Faizi <[email protected]>
+#
+
+
+class GetTelegramReply:
+
+ def get_reply_id(self, email_msg_id, chat_id):
+ '''
+ Get Telegram message ID sent match with
+ email message ID and Telegram chat ID.
+ - Return Telegram message ID if exists: `int`
+ - Return None if not exists`
+ '''
+ q = """
+ SELECT tg_mail_msg.tg_msg_id
+ FROM tg_emails INNER JOIN tg_mail_msg
+ ON tg_emails.id = tg_mail_msg.email_id
+ WHERE tg_emails.message_id = %(email_msg_id)s
+ AND tg_mail_msg.chat_id = %(chat_id)s
+ """
+
+ self.cur.execute(q, {
+ "email_msg_id": email_msg_id,
+ "chat_id": chat_id
+ })
+ res = self.cur.fetchone()
+ if not bool(res):
+ return None
+
+ return res[0]
diff --git a/daemon/telegram/database/methods/insertion/__init__.py b/daemon/telegram/database/methods/insertion/__init__.py
new file mode 100644
index 0000000..3604f82
--- /dev/null
+++ b/daemon/telegram/database/methods/insertion/__init__.py
@@ -0,0 +1,18 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+# Copyright (C) 2022 Ammar Faizi <[email protected]>
+#
+
+from .insert_atom import InsertAtom
+from .insert_broadcast import InsertBroadcast
+from .insert_email import InsertEmail
+from .insert_telegram import InsertTelegram
+
+
+class Insertion(
+ InsertAtom,
+ InsertBroadcast,
+ InsertEmail,
+ InsertTelegram
+): pass
diff --git a/daemon/telegram/database/methods/insertion/insert_atom.py b/daemon/telegram/database/methods/insertion/insert_atom.py
new file mode 100644
index 0000000..ac068ae
--- /dev/null
+++ b/daemon/telegram/database/methods/insertion/insert_atom.py
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+# Copyright (C) 2022 Ammar Faizi <[email protected]>
+#
+
+
+from mysql.connector import errors
+from datetime import datetime
+
+
+class InsertAtom:
+
+ def save_atom(self, atom: str):
+ try:
+ return self.__insert_atom(atom)
+ except errors.IntegrityError:
+ #
+ # Duplicate data, skip!
+ #
+ return None
+
+
+ def __insert_atom(self, atom: str):
+ q = "INSERT INTO tg_atoms (url, created_at) VALUES (%s, %s)"
+ self.cur.execute(q, (atom, datetime.utcnow()))
+ return self.cur.lastrowid
diff --git a/daemon/telegram/database/methods/insertion/insert_broadcast.py b/daemon/telegram/database/methods/insertion/insert_broadcast.py
new file mode 100644
index 0000000..11ebc6e
--- /dev/null
+++ b/daemon/telegram/database/methods/insertion/insert_broadcast.py
@@ -0,0 +1,56 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+# Copyright (C) 2022 Ammar Faizi <[email protected]>
+#
+
+
+from mysql.connector import errors
+from datetime import datetime
+
+
+class InsertBroadcast:
+
+ def save_broadcast(
+ self,
+ chat_id: int,
+ name: str,
+ type: str,
+ created_at: "datetime",
+ username: str = None,
+ link: str = None,
+ ):
+ try:
+ return self.__insert_broadcast(
+ chat_id=chat_id,
+ name=name,
+ type=type,
+ created_at=created_at,
+ username=username,
+ link=link
+ )
+ except errors.IntegrityError:
+ #
+ # Duplicate data, skip!
+ #
+ return None
+
+
+ def __insert_broadcast(
+ self,
+ chat_id: int,
+ name: str,
+ type: str,
+ created_at: "datetime",
+ username: str = None,
+ link: str = None,
+ ):
+ q = """
+ INSERT INTO tg_broadcasts
+ (chat_id, username, name, type, link, created_at)
+ VALUES
+ (%s, %s, %s, %s, %s, %s)
+ """
+ values = (chat_id, username, name, type, link, created_at)
+ self.cur.execute(q, values)
+ return self.cur.lastrowid
diff --git a/daemon/telegram/database/methods/insertion/insert_email.py b/daemon/telegram/database/methods/insertion/insert_email.py
new file mode 100644
index 0000000..adbe86b
--- /dev/null
+++ b/daemon/telegram/database/methods/insertion/insert_email.py
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+# Copyright (C) 2022 Ammar Faizi <[email protected]>
+#
+
+
+from mysql.connector import errors
+from datetime import datetime
+
+
+class InsertEmail:
+
+ def save_email(self, email_msg_id):
+ try:
+ return self.__insert_email(email_msg_id)
+ except errors.IntegrityError:
+ #
+ # Duplicate data, skip!
+ #
+ return None
+
+
+ def __insert_email(self, email_msg_id):
+ q = "INSERT INTO tg_emails (message_id, created_at) VALUES (%s, %s)"
+ self.cur.execute(q, (email_msg_id, datetime.utcnow()))
+ return self.cur.lastrowid
diff --git a/daemon/telegram/database/methods/insertion/insert_telegram.py b/daemon/telegram/database/methods/insertion/insert_telegram.py
new file mode 100644
index 0000000..8e0615c
--- /dev/null
+++ b/daemon/telegram/database/methods/insertion/insert_telegram.py
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+# Copyright (C) 2022 Ammar Faizi <[email protected]>
+#
+
+
+from datetime import datetime
+
+
+class InsertTelegram:
+
+ def save_telegram_mail(self, email_id, tg_chat_id, tg_msg_id):
+ q = """
+ INSERT INTO tg_mail_msg
+ (email_id, chat_id, tg_msg_id, created_at)
+ VALUES (%s, %s, %s, %s);
+ """
+ self.cur.execute(q, (email_id, tg_chat_id, tg_msg_id,
+ datetime.utcnow()))
+ return self.cur.lastrowid
diff --git a/daemon/telegram/db.sql b/daemon/telegram/db.sql
index ac6ed0d..c55c464 100644
--- a/daemon/telegram/db.sql
+++ b/daemon/telegram/db.sql
@@ -7,8 +7,8 @@ SET sql_mode = 'NO_AUTO_VALUE_ON_ZERO';
SET NAMES utf8mb4;
-DROP TABLE IF EXISTS `emails`;
-CREATE TABLE `emails` (
+DROP TABLE IF EXISTS `tg_emails`;
+CREATE TABLE `tg_emails` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`message_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci NOT NULL,
`created_at` datetime NOT NULL,
@@ -18,8 +18,8 @@ CREATE TABLE `emails` (
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci;
-DROP TABLE IF EXISTS `tg_emails`;
-CREATE TABLE `tg_emails` (
+DROP TABLE IF EXISTS `tg_mail_msg`;
+CREATE TABLE `tg_mail_msg` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`email_id` bigint unsigned NOT NULL,
`chat_id` bigint NOT NULL,
@@ -30,12 +30,12 @@ CREATE TABLE `tg_emails` (
KEY `chat_id` (`chat_id`),
KEY `tg_msg_id` (`tg_msg_id`),
KEY `created_at` (`created_at`),
- CONSTRAINT `tg_emails_ibfk_2` FOREIGN KEY (`email_id`) REFERENCES `emails` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
+ CONSTRAINT `tg_mail_msg_ibfk_2` FOREIGN KEY (`email_id`) REFERENCES `tg_emails` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci;
-DROP TABLE IF EXISTS `atom_urls`;
-CREATE TABLE `atom_urls` (
+DROP TABLE IF EXISTS `tg_atoms`;
+CREATE TABLE `tg_atoms` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`url` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci NOT NULL,
`created_at` datetime NOT NULL,
@@ -45,8 +45,8 @@ CREATE TABLE `atom_urls` (
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci;
-DROP TABLE IF EXISTS `broadcast_chats`;
-CREATE TABLE `broadcast_chats` (
+DROP TABLE IF EXISTS `tg_broadcasts`;
+CREATE TABLE `tg_broadcasts` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`chat_id` bigint NOT NULL,
`username` varchar(32),
diff --git a/daemon/telegram/packages/client.py b/daemon/telegram/packages/client.py
index 282daf6..820c3e2 100644
--- a/daemon/telegram/packages/client.py
+++ b/daemon/telegram/packages/client.py
@@ -9,7 +9,7 @@ from pyrogram.types import Message, InlineKeyboardMarkup, InlineKeyboardButton
from typing import Union
from email.message import Message
from scraper import utils
-from scraper.db import Db
+from database import DB
from .decorator import handle_flood
@@ -18,7 +18,7 @@ class DaemonClient(Client):
api_hash: str, conn, **kwargs):
super().__init__(name, api_id,
api_hash, **kwargs)
- self.db = Db(conn)
+ self.db = DB(conn)
@handle_flood
diff --git a/daemon/telegram/packages/plugins/commands/manage_atom.py b/daemon/telegram/packages/plugins/commands/manage_atom.py
index bcb2f35..4ba422a 100644
--- a/daemon/telegram/packages/plugins/commands/manage_atom.py
+++ b/daemon/telegram/packages/plugins/commands/manage_atom.py
@@ -25,7 +25,7 @@ async def add_atom_url(c: DaemonClient, m: Message):
if not is_atom:
return await m.reply("Invalid Atom URL")
- inserted = c.db.insert_atom(text)
+ inserted = c.db.save_atom(text)
if inserted is None:
return await m.reply(f"This URL already listened for new email.")
diff --git a/daemon/telegram/packages/plugins/commands/manage_broadcast.py b/daemon/telegram/packages/plugins/commands/manage_broadcast.py
index ffb5a6b..6d75c36 100644
--- a/daemon/telegram/packages/plugins/commands/manage_broadcast.py
+++ b/daemon/telegram/packages/plugins/commands/manage_broadcast.py
@@ -20,7 +20,7 @@ async def add_broadcast(c: DaemonClient, m: Message):
else:
chat_name = m.chat.title
- inserted = c.db.insert_broadcast(
+ inserted = c.db.save_broadcast(
chat_id=m.chat.id,
name=chat_name,
type=str(m.chat.type),
diff --git a/daemon/telegram/scraper/bot.py b/daemon/telegram/scraper/bot.py
index 7adfb12..9ef5508 100644
--- a/daemon/telegram/scraper/bot.py
+++ b/daemon/telegram/scraper/bot.py
@@ -112,7 +112,7 @@ class Bot():
tg_chat_id, text,reply_to, url
)
- self.db.insert_telegram(email_id, m.chat.id, m.id)
+ self.db.save_telegram_mail(email_id, m.chat.id, m.id)
for d, f in files:
await m.reply_document(f"{d}/{f}", file_name=f)
await asyncio.sleep(1)
@@ -124,11 +124,11 @@ class Bot():
def __need_to_send_to_telegram(self, email_msg_id, tg_chat_id):
- email_id = self.db.save_email_msg_id(email_msg_id)
+ email_id = self.db.save_email(email_msg_id)
if email_id:
return email_id
- email_id = self.db.need_to_send_to_tg(email_msg_id, tg_chat_id)
+ email_id = self.db.get_email_id(email_msg_id, tg_chat_id)
return email_id
@@ -141,4 +141,4 @@ class Bot():
if not reply_to:
return None
- return self.db.get_tg_reply_to(reply_to, tg_chat_id)
+ return self.db.get_reply_id(reply_to, tg_chat_id)
diff --git a/daemon/telegram/scraper/db.py b/daemon/telegram/scraper/db.py
deleted file mode 100644
index 58601d1..0000000
--- a/daemon/telegram/scraper/db.py
+++ /dev/null
@@ -1,217 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-#
-# Copyright (C) 2022 Muhammad Rizki <[email protected]>
-# Copyright (C) 2022 Ammar Faizi <[email protected]>
-#
-
-from datetime import datetime
-import mysql
-
-
-class Db():
- def __init__(self, conn):
- self.conn = conn
- self.conn.autocommit = True
- self.cur = self.conn.cursor(buffered=True)
-
-
- def __del__(self):
- self.cur.close()
- self.conn.close()
-
-
- def save_email_msg_id(self, email_msg_id):
- try:
- return self.__save_email_msg_id(email_msg_id)
- except mysql.connector.errors.IntegrityError:
- #
- # Duplicate data, skip!
- #
- return None
-
-
- def __save_email_msg_id(self, email_msg_id):
- q = "INSERT INTO emails (message_id, created_at) VALUES (%s, %s)"
- self.cur.execute(q, (email_msg_id, datetime.utcnow()))
- return self.cur.lastrowid
-
-
- def insert_telegram(self, email_id, tg_chat_id, tg_msg_id):
- q = """
- INSERT INTO tg_emails
- (email_id, chat_id, tg_msg_id, created_at)
- VALUES (%s, %s, %s, %s);
- """
- self.cur.execute(q, (email_id, tg_chat_id, tg_msg_id,
- datetime.utcnow()))
- return self.cur.lastrowid
-
-
- #
- # Determine whether the email needs to be sent to @tg_chat_id.
- #
- # - Return an email id (PK) if it needs to be sent.
- # - Return None if it doesn't need to be sent.
- #
- def need_to_send_to_tg(self, email_msg_id, tg_chat_id):
- q = """
- SELECT emails.id, tg_emails.id FROM emails LEFT JOIN tg_emails
- ON emails.id = tg_emails.email_id
- WHERE emails.message_id = %(email_msg_id)s
- AND tg_emails.chat_id = %(tg_chat_id)s
- LIMIT 1
- """
-
- self.cur.execute(
- q,
- {
- "email_msg_id": email_msg_id,
- "tg_chat_id": tg_chat_id
- }
- )
- res = self.cur.fetchone()
- if bool(res):
- #
- # This email has already been sent to
- # @tg_chat_id.
- #
- return None
-
- q = """
- SELECT id FROM emails WHERE message_id = %(email_msg_id)s
- """
- self.cur.execute(q, {"email_msg_id": email_msg_id})
- res = self.cur.fetchone()
- if not bool(res):
- #
- # Something goes wrong, skip!
- #
- return None
-
- return int(res[0])
-
-
- def get_tg_reply_to(self, email_msg_id, tg_chat_id):
- q = """
- SELECT tg_emails.tg_msg_id
- FROM emails INNER JOIN tg_emails
- ON emails.id = tg_emails.email_id
- WHERE emails.message_id = %(email_msg_id)s
- AND tg_emails.chat_id = %(chat_id)s
- """
-
- self.cur.execute(
- q,
- {
- "email_msg_id": email_msg_id,
- "chat_id": tg_chat_id
- }
- )
- res = self.cur.fetchone()
- if not bool(res):
- return None
-
- return res[0]
-
-
- def insert_atom(self, atom: str):
- try:
- return self.__save_atom(atom)
- except mysql.connector.errors.IntegrityError:
- #
- # Duplicate data, skip!
- #
- return None
-
-
- def __save_atom(self, atom: str):
- q = "INSERT INTO atom_urls (url, created_at) VALUES (%s, %s)"
- self.cur.execute(q, (atom, datetime.utcnow()))
- return self.cur.lastrowid
-
-
- def delete_atom(self, atom: str):
- q = """
- DELETE FROM atom_urls
- WHERE url = %(atom)s
- """
- try:
- self.cur.execute(q, {"atom": atom})
- return True
- except:
- return False
-
-
- def get_atom_urls(self):
- q = """
- SELECT atom_urls.url
- FROM atom_urls
- """
- self.cur.execute(q)
- urls = self.cur.fetchall()
-
- return [u[0] for u in urls]
-
-
- def insert_broadcast(
- self,
- chat_id: int,
- name: str,
- type: str,
- created_at: "datetime",
- username: str = None,
- link: str = None,
- ):
- try:
- return self.__save_broadcast(
- chat_id=chat_id,
- name=name,
- type=type,
- created_at=created_at,
- username=username,
- link=link
- )
- except mysql.connector.errors.IntegrityError:
- #
- # Duplicate data, skip!
- #
- return None
-
-
- def __save_broadcast(
- self,
- chat_id: int,
- name: str,
- type: str,
- created_at: "datetime",
- username: str = None,
- link: str = None,
- ):
- q = """
- INSERT INTO broadcast_chats
- (chat_id, username, name, type, link, created_at)
- VALUES
- (%s, %s, %s, %s, %s, %s)
- """
- values = (chat_id, username, name, type, link, created_at)
- self.cur.execute(q, values)
- return self.cur.lastrowid
-
-
- def delete_broadcast(self, chat_id: int):
- q = """
- DELETE FROM broadcast_chats
- WHERE chat_id = %(chat_id)s
- """
- self.cur.execute(q, {"chat_id": chat_id})
- return self.cur.rowcount > 0
-
-
- def get_broadcast_chats(self):
- q = """
- SELECT *
- FROM broadcast_chats
- """
- self.cur.execute(q)
-
- return self.cur.fetchall()
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 03/17] telegram: Renaming some functions in scraper/bot.py
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 01/17] telegram: Move the Telegram bot source code Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 02/17] telegram: Refactor Telegram bot database method Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 04/17] Add ignore file for .env Muhammad Rizki
` (13 subsequent siblings)
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Just renaming some functions...
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/telegram/scraper/bot.py | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/daemon/telegram/scraper/bot.py b/daemon/telegram/scraper/bot.py
index 9ef5508..a7087ad 100644
--- a/daemon/telegram/scraper/bot.py
+++ b/daemon/telegram/scraper/bot.py
@@ -4,6 +4,7 @@
# Copyright (C) 2022 Ammar Faizi <[email protected]>
#
+from pyrogram.types import Message
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from packages import DaemonClient
from scraper import Scraper
@@ -72,7 +73,7 @@ class Bot():
chats = self.db.get_broadcast_chats()
for chat in chats:
async with self.mutexes.send_to_tg:
- should_wait = await self.__send_to_tg(url, mail,
+ should_wait = await self.__send_mail(url, mail,
chat[1])
if should_wait:
@@ -80,7 +81,7 @@ class Bot():
# @__must_hold(self.mutexes.send_to_tg)
- async def __send_to_tg(self, url, mail, tg_chat_id):
+ async def __send_mail(self, url, mail, tg_chat_id):
email_msg_id = utils.get_email_msg_id(mail)
if not email_msg_id:
#
@@ -89,7 +90,7 @@ class Bot():
#
return False
- email_id = self.__need_to_send_to_telegram(email_msg_id,
+ email_id = self.__mail_id_from_db(email_msg_id,
tg_chat_id)
if not email_id:
#
@@ -99,16 +100,16 @@ class Bot():
return False
text, files, is_patch = utils.create_template(mail)
- reply_to = self.get_tg_reply_to(mail, tg_chat_id)
+ reply_to = self.get_reply(mail, tg_chat_id)
url = str(re.sub(r"/raw$", "", url))
if is_patch:
- m = await self.client.send_patch_email(
+ m: "Message" = await self.client.send_patch_email(
mail, tg_chat_id, text, reply_to, url
)
else:
text = "#ml\n" + text
- m = await self.client.send_text_email(
+ m: "Message" = await self.client.send_text_email(
tg_chat_id, text,reply_to, url
)
@@ -123,7 +124,7 @@ class Bot():
return True
- def __need_to_send_to_telegram(self, email_msg_id, tg_chat_id):
+ def __mail_id_from_db(self, email_msg_id, tg_chat_id):
email_id = self.db.save_email(email_msg_id)
if email_id:
return email_id
@@ -132,7 +133,7 @@ class Bot():
return email_id
- def get_tg_reply_to(self, mail, tg_chat_id):
+ def get_reply(self, mail, tg_chat_id):
reply_to = mail.get("in-reply-to")
if not reply_to:
return None
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 04/17] Add ignore file for .env
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
` (2 preceding siblings ...)
2022-09-19 7:26 ` [RFC PATCH v3 03/17] telegram: Renaming some functions in scraper/bot.py Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-20 8:45 ` Ammar Faizi
2022-09-19 7:26 ` [RFC PATCH v3 05/17] telegram: Refactor the Telegram bot Muhammad Rizki
` (12 subsequent siblings)
16 siblings, 1 reply; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Adding *.env value in the .gitignore to ignore any file which has .env
extension inside the daemon directory.
Signed-off-by: Muhammad Rizki <[email protected]>
---
.gitignore | 1 +
1 file changed, 1 insertion(+)
diff --git a/.gitignore b/.gitignore
index 3ca3c1b..e77815f 100644
--- a/.gitignore
+++ b/.gitignore
@@ -140,4 +140,5 @@ data.json
*.patch
# configuration file
+daemon/*.env
daemon/telegram/config.py
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 05/17] telegram: Refactor the Telegram bot
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
` (3 preceding siblings ...)
2022-09-19 7:26 ` [RFC PATCH v3 04/17] Add ignore file for .env Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 06/17] telegram: Renames in telegram mailer directory Muhammad Rizki
` (11 subsequent siblings)
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Moving the Telegram bot runner script outside the telegram directory and
rename it to `tg.py`. In this state, adapt the import statement for
Telegram bot file that are affected to avoid breakage after the main
file movement.
Reason:
There will be a Discord bot with `dc.py` runner script in the same
directory with `tg.py`. With this, both can share the same modules
that reduce the code duplication.
Signed-off-by: Muhammad Rizki <[email protected]>
---
.../.env.example => telegram.env.example} | 3 ++-
daemon/telegram/packages/client.py | 4 ++--
.../packages/plugins/callbacks/del_atom.py | 6 +++---
.../packages/plugins/callbacks/del_chat.py | 6 +++---
.../telegram/packages/plugins/commands/debugger.py | 2 +-
.../packages/plugins/commands/manage_atom.py | 6 +++---
.../packages/plugins/commands/manage_broadcast.py | 6 +++---
.../telegram/packages/plugins/commands/scrape.py | 8 ++++----
daemon/telegram/scraper/bot.py | 4 ++--
daemon/{telegram/run.py => tg.py} | 14 +++++++-------
10 files changed, 30 insertions(+), 29 deletions(-)
rename daemon/{telegram/.env.example => telegram.env.example} (82%)
rename daemon/{telegram/run.py => tg.py} (78%)
diff --git a/daemon/telegram/.env.example b/daemon/telegram.env.example
similarity index 82%
rename from daemon/telegram/.env.example
rename to daemon/telegram.env.example
index 99aa46a..a070dd0 100644
--- a/daemon/telegram/.env.example
+++ b/daemon/telegram.env.example
@@ -15,4 +15,5 @@ DB_NAME=
# Telegram Chat ID that want to send to when receive new email.
TG_SEND_TO=
-STORAGE_DIR=storage/
+# Storage directory to save patch files
+STORAGE_DIR=
diff --git a/daemon/telegram/packages/client.py b/daemon/telegram/packages/client.py
index 820c3e2..e419d40 100644
--- a/daemon/telegram/packages/client.py
+++ b/daemon/telegram/packages/client.py
@@ -8,8 +8,8 @@ from pyrogram.enums import ParseMode
from pyrogram.types import Message, InlineKeyboardMarkup, InlineKeyboardButton
from typing import Union
from email.message import Message
-from scraper import utils
-from database import DB
+from telegram.scraper import utils
+from telegram.database import DB
from .decorator import handle_flood
diff --git a/daemon/telegram/packages/plugins/callbacks/del_atom.py b/daemon/telegram/packages/plugins/callbacks/del_atom.py
index 1510d60..94b2489 100644
--- a/daemon/telegram/packages/plugins/callbacks/del_atom.py
+++ b/daemon/telegram/packages/plugins/callbacks/del_atom.py
@@ -3,10 +3,10 @@
# Copyright (C) 2022 Muhammad Rizki <[email protected]>
#
-from packages import DaemonClient
-from scraper import utils
+from telegram.packages import DaemonClient
+from telegram.scraper import utils
from pyrogram.types import CallbackQuery
-import config
+from telegram import config
@DaemonClient.on_callback_query(config.admin_only, group=1)
diff --git a/daemon/telegram/packages/plugins/callbacks/del_chat.py b/daemon/telegram/packages/plugins/callbacks/del_chat.py
index 26c6dd8..ae21179 100644
--- a/daemon/telegram/packages/plugins/callbacks/del_chat.py
+++ b/daemon/telegram/packages/plugins/callbacks/del_chat.py
@@ -3,10 +3,10 @@
# Copyright (C) 2022 Muhammad Rizki <[email protected]>
#
-from packages import DaemonClient
-from scraper import utils
+from telegram.packages import DaemonClient
+from telegram.scraper import utils
from pyrogram.types import CallbackQuery
-import config
+from telegram import config
@DaemonClient.on_callback_query(config.admin_only, group=2)
diff --git a/daemon/telegram/packages/plugins/commands/debugger.py b/daemon/telegram/packages/plugins/commands/debugger.py
index ae2d31d..7f6f367 100644
--- a/daemon/telegram/packages/plugins/commands/debugger.py
+++ b/daemon/telegram/packages/plugins/commands/debugger.py
@@ -7,7 +7,7 @@ from pyrogram import Client, filters, enums
from pyrogram.types import Message
from textwrap import indent
import io, import_expression, contextlib, traceback
-import config
+from telegram import config
@Client.on_message(
diff --git a/daemon/telegram/packages/plugins/commands/manage_atom.py b/daemon/telegram/packages/plugins/commands/manage_atom.py
index 4ba422a..a4ccc46 100644
--- a/daemon/telegram/packages/plugins/commands/manage_atom.py
+++ b/daemon/telegram/packages/plugins/commands/manage_atom.py
@@ -5,9 +5,9 @@
from pyrogram.types import Message
from pyrogram import filters
-from packages import DaemonClient
-from scraper import utils
-import config
+from telegram.packages import DaemonClient
+from telegram.scraper import utils
+from telegram import config
@DaemonClient.on_message(
diff --git a/daemon/telegram/packages/plugins/commands/manage_broadcast.py b/daemon/telegram/packages/plugins/commands/manage_broadcast.py
index 6d75c36..16a494a 100644
--- a/daemon/telegram/packages/plugins/commands/manage_broadcast.py
+++ b/daemon/telegram/packages/plugins/commands/manage_broadcast.py
@@ -5,9 +5,9 @@
from pyrogram.types import Message
from pyrogram import filters, enums
-from packages import DaemonClient
-from scraper import utils
-import config
+from telegram.packages import DaemonClient
+from telegram.scraper import utils
+from telegram import config
@DaemonClient.on_message(
diff --git a/daemon/telegram/packages/plugins/commands/scrape.py b/daemon/telegram/packages/plugins/commands/scrape.py
index 45b1581..a83420c 100644
--- a/daemon/telegram/packages/plugins/commands/scrape.py
+++ b/daemon/telegram/packages/plugins/commands/scrape.py
@@ -6,10 +6,10 @@
from pyrogram.types import Message
from pyrogram import filters
-from packages import DaemonClient
-from scraper import Scraper
-from scraper import utils
-import config
+from telegram.packages import DaemonClient
+from telegram.scraper import Scraper
+from telegram.scraper import utils
+from telegram import config
import shutil
import re
import asyncio
diff --git a/daemon/telegram/scraper/bot.py b/daemon/telegram/scraper/bot.py
index a7087ad..ac183be 100644
--- a/daemon/telegram/scraper/bot.py
+++ b/daemon/telegram/scraper/bot.py
@@ -6,8 +6,8 @@
from pyrogram.types import Message
from apscheduler.schedulers.asyncio import AsyncIOScheduler
-from packages import DaemonClient
-from scraper import Scraper
+from telegram.packages import DaemonClient
+from telegram.scraper import Scraper
from . import utils
import asyncio
import shutil
diff --git a/daemon/telegram/run.py b/daemon/tg.py
similarity index 78%
rename from daemon/telegram/run.py
rename to daemon/tg.py
index 5360395..86b0226 100644
--- a/daemon/telegram/run.py
+++ b/daemon/tg.py
@@ -5,20 +5,20 @@
#
from apscheduler.schedulers.asyncio import AsyncIOScheduler
-from scraper import BotMutexes
+from telegram.scraper import BotMutexes
from dotenv import load_dotenv
from mysql import connector
-from packages import DaemonClient
-from scraper import Scraper
-from scraper import Bot
+from telegram.packages import DaemonClient
+from telegram.scraper import Scraper
+from telegram.scraper import Bot
import os
def main():
- load_dotenv()
+ load_dotenv("telegram.env")
client = DaemonClient(
- "storage/EmailScraper",
+ "telegram/storage/EmailScraper",
api_id=int(os.getenv("API_ID")),
api_hash=os.getenv("API_HASH"),
bot_token=os.getenv("BOT_TOKEN"),
@@ -29,7 +29,7 @@ def main():
database=os.getenv("DB_NAME")
),
plugins=dict(
- root="packages.plugins"
+ root="telegram.packages.plugins"
),
)
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 06/17] telegram: Renames in telegram mailer directory
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
` (4 preceding siblings ...)
2022-09-19 7:26 ` [RFC PATCH v3 05/17] telegram: Refactor the Telegram bot Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 07/17] Move scraper and utility file Muhammad Rizki
` (10 subsequent siblings)
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Rename a mailer directory and listener file.
Directory:
- telegram/scraper => telegram/mailer
File:
- telegram/scraper/bot.py => telegram/mailer/listener.py
Affected codes are changed too for imports.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/telegram/{scraper => mailer}/__init__.py | 4 ++--
daemon/telegram/{scraper/bot.py => mailer/listener.py} | 2 +-
daemon/telegram/{scraper => mailer}/scraper.py | 0
daemon/telegram/{scraper => mailer}/utils.py | 0
daemon/telegram/packages/client.py | 2 +-
daemon/telegram/packages/plugins/callbacks/del_atom.py | 2 +-
daemon/telegram/packages/plugins/callbacks/del_chat.py | 2 +-
daemon/telegram/packages/plugins/commands/manage_atom.py | 2 +-
daemon/telegram/packages/plugins/commands/manage_broadcast.py | 2 +-
daemon/telegram/packages/plugins/commands/scrape.py | 4 ++--
10 files changed, 10 insertions(+), 10 deletions(-)
rename daemon/telegram/{scraper => mailer}/__init__.py (77%)
rename daemon/telegram/{scraper/bot.py => mailer/listener.py} (98%)
rename daemon/telegram/{scraper => mailer}/scraper.py (100%)
rename daemon/telegram/{scraper => mailer}/utils.py (100%)
diff --git a/daemon/telegram/scraper/__init__.py b/daemon/telegram/mailer/__init__.py
similarity index 77%
rename from daemon/telegram/scraper/__init__.py
rename to daemon/telegram/mailer/__init__.py
index 4294302..75f27df 100644
--- a/daemon/telegram/scraper/__init__.py
+++ b/daemon/telegram/mailer/__init__.py
@@ -5,5 +5,5 @@
#
from .scraper import Scraper
-from .bot import BotMutexes
-from .bot import Bot
+from .listener import BotMutexes
+from .listener import Bot
diff --git a/daemon/telegram/scraper/bot.py b/daemon/telegram/mailer/listener.py
similarity index 98%
rename from daemon/telegram/scraper/bot.py
rename to daemon/telegram/mailer/listener.py
index ac183be..f92ae1d 100644
--- a/daemon/telegram/scraper/bot.py
+++ b/daemon/telegram/mailer/listener.py
@@ -7,7 +7,7 @@
from pyrogram.types import Message
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from telegram.packages import DaemonClient
-from telegram.scraper import Scraper
+from . import Scraper
from . import utils
import asyncio
import shutil
diff --git a/daemon/telegram/scraper/scraper.py b/daemon/telegram/mailer/scraper.py
similarity index 100%
rename from daemon/telegram/scraper/scraper.py
rename to daemon/telegram/mailer/scraper.py
diff --git a/daemon/telegram/scraper/utils.py b/daemon/telegram/mailer/utils.py
similarity index 100%
rename from daemon/telegram/scraper/utils.py
rename to daemon/telegram/mailer/utils.py
diff --git a/daemon/telegram/packages/client.py b/daemon/telegram/packages/client.py
index e419d40..950665b 100644
--- a/daemon/telegram/packages/client.py
+++ b/daemon/telegram/packages/client.py
@@ -8,7 +8,7 @@ from pyrogram.enums import ParseMode
from pyrogram.types import Message, InlineKeyboardMarkup, InlineKeyboardButton
from typing import Union
from email.message import Message
-from telegram.scraper import utils
+from telegram.mailer import utils
from telegram.database import DB
from .decorator import handle_flood
diff --git a/daemon/telegram/packages/plugins/callbacks/del_atom.py b/daemon/telegram/packages/plugins/callbacks/del_atom.py
index 94b2489..57be8c6 100644
--- a/daemon/telegram/packages/plugins/callbacks/del_atom.py
+++ b/daemon/telegram/packages/plugins/callbacks/del_atom.py
@@ -4,7 +4,7 @@
#
from telegram.packages import DaemonClient
-from telegram.scraper import utils
+from telegram.mailer import utils
from pyrogram.types import CallbackQuery
from telegram import config
diff --git a/daemon/telegram/packages/plugins/callbacks/del_chat.py b/daemon/telegram/packages/plugins/callbacks/del_chat.py
index ae21179..854b172 100644
--- a/daemon/telegram/packages/plugins/callbacks/del_chat.py
+++ b/daemon/telegram/packages/plugins/callbacks/del_chat.py
@@ -4,7 +4,7 @@
#
from telegram.packages import DaemonClient
-from telegram.scraper import utils
+from telegram.mailer import utils
from pyrogram.types import CallbackQuery
from telegram import config
diff --git a/daemon/telegram/packages/plugins/commands/manage_atom.py b/daemon/telegram/packages/plugins/commands/manage_atom.py
index a4ccc46..37b06cb 100644
--- a/daemon/telegram/packages/plugins/commands/manage_atom.py
+++ b/daemon/telegram/packages/plugins/commands/manage_atom.py
@@ -6,7 +6,7 @@
from pyrogram.types import Message
from pyrogram import filters
from telegram.packages import DaemonClient
-from telegram.scraper import utils
+from telegram.mailer import utils
from telegram import config
diff --git a/daemon/telegram/packages/plugins/commands/manage_broadcast.py b/daemon/telegram/packages/plugins/commands/manage_broadcast.py
index 16a494a..b99b712 100644
--- a/daemon/telegram/packages/plugins/commands/manage_broadcast.py
+++ b/daemon/telegram/packages/plugins/commands/manage_broadcast.py
@@ -6,7 +6,7 @@
from pyrogram.types import Message
from pyrogram import filters, enums
from telegram.packages import DaemonClient
-from telegram.scraper import utils
+from telegram.mailer import utils
from telegram import config
diff --git a/daemon/telegram/packages/plugins/commands/scrape.py b/daemon/telegram/packages/plugins/commands/scrape.py
index a83420c..253993f 100644
--- a/daemon/telegram/packages/plugins/commands/scrape.py
+++ b/daemon/telegram/packages/plugins/commands/scrape.py
@@ -7,8 +7,8 @@
from pyrogram.types import Message
from pyrogram import filters
from telegram.packages import DaemonClient
-from telegram.scraper import Scraper
-from telegram.scraper import utils
+from telegram.mailer import Scraper
+from telegram.mailer import utils
from telegram import config
import shutil
import re
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 07/17] Move scraper and utility file
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
` (5 preceding siblings ...)
2022-09-19 7:26 ` [RFC PATCH v3 06/17] telegram: Renames in telegram mailer directory Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 08/17] discord: Initial work Discord bot Muhammad Rizki
` (9 subsequent siblings)
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Move these files to the daemon/atom to make it reusable in
future use.
Reason:
Because there will be a Discord bot. With this, both can share same
modules to reduce code duplication.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/atom/__init__.py | 8 ++++++++
daemon/{telegram/mailer => atom}/scraper.py | 0
daemon/{telegram/mailer => atom}/utils.py | 0
daemon/telegram/mailer/__init__.py | 1 -
daemon/telegram/mailer/listener.py | 4 ++--
daemon/telegram/packages/client.py | 2 +-
daemon/telegram/packages/plugins/callbacks/del_atom.py | 2 +-
daemon/telegram/packages/plugins/callbacks/del_chat.py | 2 +-
daemon/telegram/packages/plugins/commands/manage_atom.py | 2 +-
.../packages/plugins/commands/manage_broadcast.py | 2 +-
daemon/telegram/packages/plugins/commands/scrape.py | 4 ++--
daemon/tg.py | 6 +++---
12 files changed, 20 insertions(+), 13 deletions(-)
create mode 100644 daemon/atom/__init__.py
rename daemon/{telegram/mailer => atom}/scraper.py (100%)
rename daemon/{telegram/mailer => atom}/utils.py (100%)
diff --git a/daemon/atom/__init__.py b/daemon/atom/__init__.py
new file mode 100644
index 0000000..423d775
--- /dev/null
+++ b/daemon/atom/__init__.py
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+# Copyright (C) 2022 Ammar Faizi <[email protected]>
+#
+
+from .scraper import Scraper
+from . import utils
diff --git a/daemon/telegram/mailer/scraper.py b/daemon/atom/scraper.py
similarity index 100%
rename from daemon/telegram/mailer/scraper.py
rename to daemon/atom/scraper.py
diff --git a/daemon/telegram/mailer/utils.py b/daemon/atom/utils.py
similarity index 100%
rename from daemon/telegram/mailer/utils.py
rename to daemon/atom/utils.py
diff --git a/daemon/telegram/mailer/__init__.py b/daemon/telegram/mailer/__init__.py
index 75f27df..95cbe96 100644
--- a/daemon/telegram/mailer/__init__.py
+++ b/daemon/telegram/mailer/__init__.py
@@ -4,6 +4,5 @@
# Copyright (C) 2022 Ammar Faizi <[email protected]>
#
-from .scraper import Scraper
from .listener import BotMutexes
from .listener import Bot
diff --git a/daemon/telegram/mailer/listener.py b/daemon/telegram/mailer/listener.py
index f92ae1d..decf85f 100644
--- a/daemon/telegram/mailer/listener.py
+++ b/daemon/telegram/mailer/listener.py
@@ -7,8 +7,8 @@
from pyrogram.types import Message
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from telegram.packages import DaemonClient
-from . import Scraper
-from . import utils
+from atom import Scraper
+from atom import utils
import asyncio
import shutil
import re
diff --git a/daemon/telegram/packages/client.py b/daemon/telegram/packages/client.py
index 950665b..4f9c596 100644
--- a/daemon/telegram/packages/client.py
+++ b/daemon/telegram/packages/client.py
@@ -8,7 +8,7 @@ from pyrogram.enums import ParseMode
from pyrogram.types import Message, InlineKeyboardMarkup, InlineKeyboardButton
from typing import Union
from email.message import Message
-from telegram.mailer import utils
+from atom import utils
from telegram.database import DB
from .decorator import handle_flood
diff --git a/daemon/telegram/packages/plugins/callbacks/del_atom.py b/daemon/telegram/packages/plugins/callbacks/del_atom.py
index 57be8c6..b750e1c 100644
--- a/daemon/telegram/packages/plugins/callbacks/del_atom.py
+++ b/daemon/telegram/packages/plugins/callbacks/del_atom.py
@@ -4,7 +4,7 @@
#
from telegram.packages import DaemonClient
-from telegram.mailer import utils
+from atom import utils
from pyrogram.types import CallbackQuery
from telegram import config
diff --git a/daemon/telegram/packages/plugins/callbacks/del_chat.py b/daemon/telegram/packages/plugins/callbacks/del_chat.py
index 854b172..90b557e 100644
--- a/daemon/telegram/packages/plugins/callbacks/del_chat.py
+++ b/daemon/telegram/packages/plugins/callbacks/del_chat.py
@@ -4,7 +4,7 @@
#
from telegram.packages import DaemonClient
-from telegram.mailer import utils
+from atom import utils
from pyrogram.types import CallbackQuery
from telegram import config
diff --git a/daemon/telegram/packages/plugins/commands/manage_atom.py b/daemon/telegram/packages/plugins/commands/manage_atom.py
index 37b06cb..99df7f7 100644
--- a/daemon/telegram/packages/plugins/commands/manage_atom.py
+++ b/daemon/telegram/packages/plugins/commands/manage_atom.py
@@ -6,7 +6,7 @@
from pyrogram.types import Message
from pyrogram import filters
from telegram.packages import DaemonClient
-from telegram.mailer import utils
+from atom import utils
from telegram import config
diff --git a/daemon/telegram/packages/plugins/commands/manage_broadcast.py b/daemon/telegram/packages/plugins/commands/manage_broadcast.py
index b99b712..0aa70de 100644
--- a/daemon/telegram/packages/plugins/commands/manage_broadcast.py
+++ b/daemon/telegram/packages/plugins/commands/manage_broadcast.py
@@ -6,7 +6,7 @@
from pyrogram.types import Message
from pyrogram import filters, enums
from telegram.packages import DaemonClient
-from telegram.mailer import utils
+from atom import utils
from telegram import config
diff --git a/daemon/telegram/packages/plugins/commands/scrape.py b/daemon/telegram/packages/plugins/commands/scrape.py
index 253993f..d4d10a9 100644
--- a/daemon/telegram/packages/plugins/commands/scrape.py
+++ b/daemon/telegram/packages/plugins/commands/scrape.py
@@ -7,8 +7,8 @@
from pyrogram.types import Message
from pyrogram import filters
from telegram.packages import DaemonClient
-from telegram.mailer import Scraper
-from telegram.mailer import utils
+from atom import Scraper
+from atom import utils
from telegram import config
import shutil
import re
diff --git a/daemon/tg.py b/daemon/tg.py
index 86b0226..5f8c21e 100644
--- a/daemon/tg.py
+++ b/daemon/tg.py
@@ -5,12 +5,12 @@
#
from apscheduler.schedulers.asyncio import AsyncIOScheduler
-from telegram.scraper import BotMutexes
from dotenv import load_dotenv
from mysql import connector
+from atom import Scraper
from telegram.packages import DaemonClient
-from telegram.scraper import Scraper
-from telegram.scraper import Bot
+from telegram.mailer import BotMutexes
+from telegram.mailer import Bot
import os
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 08/17] discord: Initial work Discord bot
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
` (6 preceding siblings ...)
2022-09-19 7:26 ` [RFC PATCH v3 07/17] Move scraper and utility file Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 09/17] discord: Add success run notice on_ready event Muhammad Rizki
` (8 subsequent siblings)
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Just initialize Discord bot, the feature is not available yet, only
running the bot and has game activity.
Signed-off-by: Muhammad Rizki <[email protected]>
---
.gitignore | 1 +
daemon/dc.py | 20 ++++++++++++++++
daemon/discord.env.example | 11 +++++++++
daemon/dscord/config.py.example | 13 +++++++++++
daemon/dscord/gnuweeb/__init__.py | 6 +++++
daemon/dscord/gnuweeb/client.py | 28 +++++++++++++++++++++++
daemon/dscord/gnuweeb/plugins/__init__.py | 11 +++++++++
7 files changed, 90 insertions(+)
create mode 100644 daemon/dc.py
create mode 100644 daemon/discord.env.example
create mode 100644 daemon/dscord/config.py.example
create mode 100644 daemon/dscord/gnuweeb/__init__.py
create mode 100644 daemon/dscord/gnuweeb/client.py
create mode 100644 daemon/dscord/gnuweeb/plugins/__init__.py
diff --git a/.gitignore b/.gitignore
index e77815f..ad5ade0 100644
--- a/.gitignore
+++ b/.gitignore
@@ -142,3 +142,4 @@ data.json
# configuration file
daemon/*.env
daemon/telegram/config.py
+daemon/dscord/config.py
diff --git a/daemon/dc.py b/daemon/dc.py
new file mode 100644
index 0000000..ff29abe
--- /dev/null
+++ b/daemon/dc.py
@@ -0,0 +1,20 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+import os
+from dotenv import load_dotenv
+
+from dscord.gnuweeb import GWClient
+
+
+def main():
+ load_dotenv("discord.env")
+
+ client = GWClient()
+ client.run(os.getenv("DISCORD_TOKEN"), log_handler=None)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/daemon/discord.env.example b/daemon/discord.env.example
new file mode 100644
index 0000000..60fcfac
--- /dev/null
+++ b/daemon/discord.env.example
@@ -0,0 +1,11 @@
+# Input your Discord bot token below
+DISCORD_TOKEN=
+
+# Input your MySQL connection below
+DB_HOST=
+DB_USER=
+DB_PASS=
+DB_NAME=
+
+# Storage directory to save patch files
+STORAGE_DIR=
diff --git a/daemon/dscord/config.py.example b/daemon/dscord/config.py.example
new file mode 100644
index 0000000..87eca75
--- /dev/null
+++ b/daemon/dscord/config.py.example
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+# Paste the admin role ID below
+# to filter only admin who can
+# access add/delete lore commands.
+ADMIN_ROLE_ID =
+
+# The activity name like "Playing Genshin Impact"
+# you set the value as "Genshin Impact"
+ACTIVITY_NAME =
diff --git a/daemon/dscord/gnuweeb/__init__.py b/daemon/dscord/gnuweeb/__init__.py
new file mode 100644
index 0000000..f3d814a
--- /dev/null
+++ b/daemon/dscord/gnuweeb/__init__.py
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+from .client import GWClient
diff --git a/daemon/dscord/gnuweeb/client.py b/daemon/dscord/gnuweeb/client.py
new file mode 100644
index 0000000..fa07fb1
--- /dev/null
+++ b/daemon/dscord/gnuweeb/client.py
@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+import discord
+from discord.ext import commands
+from discord import Intents
+from dscord.config import ACTIVITY_NAME
+
+
+class GWClient(commands.Bot):
+ def __init__(self) -> None:
+ intents = Intents.default()
+ intents.message_content = True
+ super().__init__(
+ command_prefix=["$", "."],
+ description="Just a bot for receiving lore emails.",
+ intents=intents,
+ activity=discord.Game(ACTIVITY_NAME)
+ )
+
+
+ async def setup_hook(self):
+ await self.load_extension(
+ name=".gnuweeb.plugins",
+ package="dscord"
+ )
diff --git a/daemon/dscord/gnuweeb/plugins/__init__.py b/daemon/dscord/gnuweeb/plugins/__init__.py
new file mode 100644
index 0000000..b9a7e37
--- /dev/null
+++ b/daemon/dscord/gnuweeb/plugins/__init__.py
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+
+from discord.ext import commands
+
+
+async def setup(bot: "commands.Bot"):
+ pass
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 09/17] discord: Add success run notice on_ready event
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
` (7 preceding siblings ...)
2022-09-19 7:26 ` [RFC PATCH v3 08/17] discord: Initial work Discord bot Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 10/17] discord: Add error handler on events Muhammad Rizki
` (7 subsequent siblings)
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Adding a bot info and success message on the on_ready event.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/dscord/gnuweeb/plugins/__init__.py | 8 ++++++-
.../dscord/gnuweeb/plugins/events/__init__.py | 11 ++++++++++
.../dscord/gnuweeb/plugins/events/on_ready.py | 21 +++++++++++++++++++
3 files changed, 39 insertions(+), 1 deletion(-)
create mode 100644 daemon/dscord/gnuweeb/plugins/events/__init__.py
create mode 100644 daemon/dscord/gnuweeb/plugins/events/on_ready.py
diff --git a/daemon/dscord/gnuweeb/plugins/__init__.py b/daemon/dscord/gnuweeb/plugins/__init__.py
index b9a7e37..ab4cb7f 100644
--- a/daemon/dscord/gnuweeb/plugins/__init__.py
+++ b/daemon/dscord/gnuweeb/plugins/__init__.py
@@ -5,7 +5,13 @@
from discord.ext import commands
+from .events import Events
+
+
+class Plugins(
+ Events
+): pass
async def setup(bot: "commands.Bot"):
- pass
+ await bot.add_cog(Plugins(bot))
diff --git a/daemon/dscord/gnuweeb/plugins/events/__init__.py b/daemon/dscord/gnuweeb/plugins/events/__init__.py
new file mode 100644
index 0000000..d9a891f
--- /dev/null
+++ b/daemon/dscord/gnuweeb/plugins/events/__init__.py
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+from .on_ready import OnReady
+
+
+class Events(
+ OnReady,
+): pass
diff --git a/daemon/dscord/gnuweeb/plugins/events/on_ready.py b/daemon/dscord/gnuweeb/plugins/events/on_ready.py
new file mode 100644
index 0000000..cb7de29
--- /dev/null
+++ b/daemon/dscord/gnuweeb/plugins/events/on_ready.py
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+from discord.ext import commands
+
+
+class OnReady(commands.Cog):
+ def __init__(self, bot: "commands.Bot") -> None:
+ self.bot = bot
+
+
+ @commands.Cog.listener()
+ async def on_ready(self):
+ t = "[ GNU/Weeb Bot is connected ]\n\n"
+ t += f"ID : {self.bot.user.id}\n"
+ t += f"Name : {self.bot.user.display_name}\n"
+ t += f"Tags : {self.bot.user}\n\n"
+ t += "Ready to get the latest of lore kernel emails."
+ print(t)
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 10/17] discord: Add error handler on events
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
` (8 preceding siblings ...)
2022-09-19 7:26 ` [RFC PATCH v3 09/17] discord: Add success run notice on_ready event Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-20 8:37 ` Ammar Faizi
2022-09-19 7:26 ` [RFC PATCH v3 11/17] Move db.sql to combine database with Discord and Telegram Muhammad Rizki
` (6 subsequent siblings)
16 siblings, 1 reply; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Adding error handler to ignore `CommandNotFound` error while someone
sending a message that is not exists for the bot basic commands.
See image: https://i.ibb.co/qRcCz6Z/image.png
Signed-off-by: Muhammad Rizki <[email protected]>
---
.../dscord/gnuweeb/plugins/events/__init__.py | 2 ++
.../dscord/gnuweeb/plugins/events/on_error.py | 17 +++++++++++++++++
2 files changed, 19 insertions(+)
create mode 100644 daemon/dscord/gnuweeb/plugins/events/on_error.py
diff --git a/daemon/dscord/gnuweeb/plugins/events/__init__.py b/daemon/dscord/gnuweeb/plugins/events/__init__.py
index d9a891f..7074e4d 100644
--- a/daemon/dscord/gnuweeb/plugins/events/__init__.py
+++ b/daemon/dscord/gnuweeb/plugins/events/__init__.py
@@ -4,8 +4,10 @@
#
from .on_ready import OnReady
+from .on_error import OnError
class Events(
OnReady,
+ OnError
): pass
diff --git a/daemon/dscord/gnuweeb/plugins/events/on_error.py b/daemon/dscord/gnuweeb/plugins/events/on_error.py
new file mode 100644
index 0000000..e1e4e28
--- /dev/null
+++ b/daemon/dscord/gnuweeb/plugins/events/on_error.py
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+from discord.ext import commands
+
+
+class OnError(commands.Cog):
+ def __init__(self, bot: "commands.Bot") -> None:
+ self.bot = bot
+
+
+ @commands.Cog.listener()
+ async def on_command_error(self, _, err):
+ if isinstance(err, commands.CommandNotFound):
+ pass
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 11/17] Move db.sql to combine database with Discord and Telegram
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
` (9 preceding siblings ...)
2022-09-19 7:26 ` [RFC PATCH v3 10/17] discord: Add error handler on events Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 12/17] discord: Add database tables for Discord bot Muhammad Rizki
` (5 subsequent siblings)
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Moving db.sql to daemon directory because this database will be combine
together with Discord and Telegram, just split them between tables.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/{telegram => }/db.sql | 0
1 file changed, 0 insertions(+), 0 deletions(-)
rename daemon/{telegram => }/db.sql (100%)
diff --git a/daemon/telegram/db.sql b/daemon/db.sql
similarity index 100%
rename from daemon/telegram/db.sql
rename to daemon/db.sql
--
Muhammad Rizki
^ permalink raw reply [flat|nested] 20+ messages in thread
* [RFC PATCH v3 12/17] discord: Add database tables for Discord bot
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
` (10 preceding siblings ...)
2022-09-19 7:26 ` [RFC PATCH v3 11/17] Move db.sql to combine database with Discord and Telegram Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 13/17] discord: Add initial Discord bot database instance Muhammad Rizki
` (4 subsequent siblings)
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Add some tables for Discord bot in the same database with Telegram bot
in db.sql
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/db.sql | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 56 insertions(+)
diff --git a/daemon/db.sql b/daemon/db.sql
index c55c464..d7665db 100644
--- a/daemon/db.sql
+++ b/daemon/db.sql
@@ -59,4 +59,60 @@ CREATE TABLE `tg_broadcasts` (
KEY `created_at` (`created_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci;
+-- [ SPLITTER ] ------------------------------------------------------------------------------
+
+DROP TABLE IF EXISTS `dc_emails`;
+CREATE TABLE `dc_emails` (
+ `id` bigint unsigned NOT NULL AUTO_INCREMENT,
+ `message_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci NOT NULL,
+ `created_at` datetime NOT NULL,
+ PRIMARY KEY (`id`),
+ UNIQUE KEY `message_id` (`message_id`),
+ KEY `created_at` (`created_at`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci;
+
+
+DROP TABLE IF EXISTS `dc_mail_msg`;
+CREATE TABLE `dc_mail_msg` (
+ `id` bigint unsigned NOT NULL AUTO_INCREMENT,
+ `email_id` bigint unsigned NOT NULL,
+ `channel_id` bigint NOT NULL,
+ `dc_msg_id` bigint unsigned NOT NULL,
+ `created_at` datetime NOT NULL,
+ PRIMARY KEY (`id`),
+ KEY `email_id` (`email_id`),
+ KEY `channel_id` (`channel_id`),
+ KEY `dc_msg_id` (`dc_msg_id`),
+ KEY `created_at` (`created_at`),
+ CONSTRAINT `dc_mail_msg_ibfk_2` FOREIGN KEY (`email_id`) REFERENCES `dc_emails` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
+) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci;
+
+
+DROP TABLE IF EXISTS `dc_atoms`;
+CREATE TABLE `dc_atoms` (
+ `id` bigint unsigned NOT NULL AUTO_INCREMENT,
+ `url` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci NOT NULL,
+ `created_at` datetime NOT NULL,
+ PRIMARY KEY (`id`),
+ UNIQUE KEY `url` (`url`),
+ KEY `created_at` (`created_at`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci;
+
+
+DROP TABLE IF EXISTS `dc_broadcasts`;
+CREATE TABLE `dc_broadcasts` (
+ `id` bigint unsigned NOT NULL AUTO_INCREMENT,
+ `guild_id` BIGINT UNSIGNED NOT NULL,
+ `channel_id` BIGINT UNSIGNED NOT NULL,
+ `channel_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci NOT NULL,
+ `channel_link` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci NOT NULL,
+ `created_at` datetime NOT NULL,
+ PRIMARY KEY (`id`),
+ KEY `guild_id` (`guild_id`),
+ UNIQUE KEY `channel_id` (`channel_id`),
+ KEY `channel_name` (`channel_name`),
+ KEY `channel_link` (`channel_link`),
+ KEY `created_at` (`created_at`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci;
+
-- 2022-07-07 14:25:28
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 13/17] discord: Add initial Discord bot database instance
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
` (11 preceding siblings ...)
2022-09-19 7:26 ` [RFC PATCH v3 12/17] discord: Add database tables for Discord bot Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 14/17] discord: Add save_atom() in database insertion Muhammad Rizki
` (3 subsequent siblings)
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Adding a Discord bot database class instance.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/dscord/database/__init__.py | 6 ++++++
daemon/dscord/database/core.py | 17 +++++++++++++++++
2 files changed, 23 insertions(+)
create mode 100644 daemon/dscord/database/__init__.py
create mode 100644 daemon/dscord/database/core.py
diff --git a/daemon/dscord/database/__init__.py b/daemon/dscord/database/__init__.py
new file mode 100644
index 0000000..c720e19
--- /dev/null
+++ b/daemon/dscord/database/__init__.py
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+from .core import DB
diff --git a/daemon/dscord/database/core.py b/daemon/dscord/database/core.py
new file mode 100644
index 0000000..1a25388
--- /dev/null
+++ b/daemon/dscord/database/core.py
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+# Copyright (C) 2022 Ammar Faizi <[email protected]>
+#
+
+
+class DB:
+ def __init__(self, conn):
+ self.conn = conn
+ self.conn.autocommit = True
+ self.cur = self.conn.cursor(buffered=True)
+
+
+ def __del__(self):
+ self.cur.close()
+ self.conn.close()
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 14/17] discord: Add save_atom() in database insertion
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
` (12 preceding siblings ...)
2022-09-19 7:26 ` [RFC PATCH v3 13/17] discord: Add initial Discord bot database instance Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 15/17] discord: Add save_broadcast() " Muhammad Rizki
` (2 subsequent siblings)
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Add save_atom() for saving atom url into the database.
Signed-off-by: Muhammad Rizki <[email protected]>
---
daemon/dscord/database/methods/__init__.py | 12 +++++++++
.../database/methods/insertion/__init__.py | 12 +++++++++
.../database/methods/insertion/insert_atom.py | 27 +++++++++++++++++++
3 files changed, 51 insertions(+)
create mode 100644 daemon/dscord/database/methods/__init__.py
create mode 100644 daemon/dscord/database/methods/insertion/__init__.py
create mode 100644 daemon/dscord/database/methods/insertion/insert_atom.py
diff --git a/daemon/dscord/database/methods/__init__.py b/daemon/dscord/database/methods/__init__.py
new file mode 100644
index 0000000..189ec67
--- /dev/null
+++ b/daemon/dscord/database/methods/__init__.py
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+
+from .insertion import Insertion
+
+
+class DBMethods(
+ Insertion
+): pass
diff --git a/daemon/dscord/database/methods/insertion/__init__.py b/daemon/dscord/database/methods/insertion/__init__.py
new file mode 100644
index 0000000..632aa55
--- /dev/null
+++ b/daemon/dscord/database/methods/insertion/__init__.py
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+#
+
+
+from .insert_atom import InsertAtom
+
+
+class Insertion(
+ InsertAtom
+): pass
diff --git a/daemon/dscord/database/methods/insertion/insert_atom.py b/daemon/dscord/database/methods/insertion/insert_atom.py
new file mode 100644
index 0000000..9468eca
--- /dev/null
+++ b/daemon/dscord/database/methods/insertion/insert_atom.py
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Muhammad Rizki <[email protected]>
+# Copyright (C) 2022 Ammar Faizi <[email protected]>
+#
+
+
+from mysql.connector import errors
+from datetime import datetime
+
+
+class InsertAtom:
+
+ def save_atom(self, atom: str):
+ try:
+ return self.__insert_atom(atom)
+ except errors.IntegrityError:
+ #
+ # Duplicate data, skip!
+ #
+ return None
+
+
+ def __insert_atom(self, atom: str):
+ q = "INSERT INTO dc_atoms (url, created_at) VALUES (%s, %s)"
+ self.cur.execute(q, (atom, datetime.utcnow()))
+ return self.cur.lastrowid
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 15/17] discord: Add save_broadcast() in database insertion
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
` (13 preceding siblings ...)
2022-09-19 7:26 ` [RFC PATCH v3 14/17] discord: Add save_atom() in database insertion Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 16/17] discord: Add save_discord_mail() " Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 17/17] discord: Add save_email() " Muhammad Rizki
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Add save_broadcast() to save discord guild channel to the database for
send new lore emails to the channels.
Signed-off-by: Muhammad Rizki <[email protected]>
---
.../database/methods/insertion/__init__.py | 4 +-
.../methods/insertion/insert_broadcast.py | 42 +++++++++++++++++++
2 files changed, 45 insertions(+), 1 deletion(-)
create mode 100644 daemon/dscord/database/methods/insertion/insert_broadcast.py
diff --git a/daemon/dscord/database/methods/insertion/__init__.py b/daemon/dscord/database/methods/insertion/__init__.py
index 632aa55..4192735 100644
--- a/daemon/dscord/database/methods/insertion/__init__.py
+++ b/daemon/dscord/database/methods/insertion/__init__.py
@@ -5,8 +5,10 @@
from .insert_atom import InsertAtom
+from .insert_broadcast import InsertBroadcast
class Insertion(
- InsertAtom
+ InsertAtom,
+ InsertBroadcast
): pass
diff --git a/daemon/dscord/database/methods/insertion/insert_broadcast.py b/daemon/dscord/database/methods/insertion/insert_broadcast.py
new file mode 100644
index 0000000..e68f9b4
--- /dev/null
+++ b/daemon/dscord/database/methods/insertion/insert_broadcast.py
@@ -0,0 +1,42 @@
+from mysql.connector import errors
+from datetime import datetime
+
+
+class InsertBroadcast:
+
+ def save_broadcast(
+ self,
+ guild_id: int,
+ channel_id: int,
+ channel_name: str,
+ channel_link: str = None,
+ ):
+ try:
+ return self.__insert_broadcast(
+ guild_id=guild_id,
+ channel_id=channel_id,
+ channel_name=channel_name,
+ channel_link=channel_link
+ )
+ except errors.IntegrityError:
+ #
+ # Duplicate data, skip!
+ #
+ return None
+
+
+ def __insert_broadcast(
+ self,
+ guild_id: int,
+ channel_id: int,
+ channel_name: str,
+ channel_link: str = None,
+ ):
+ q = """
+ INSERT INTO dc_broadcasts
+ (guild_id, channel_id, channel_name, channel_link, created_at)
+ VALUES (%s, %s, %s, %s, %s)
+ """
+ values = (guild_id, channel_id, channel_name, channel_link, datetime.utcnow())
+ self.cur.execute(q, values)
+ return self.cur.lastrowid
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 16/17] discord: Add save_discord_mail() in database insertion
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
` (14 preceding siblings ...)
2022-09-19 7:26 ` [RFC PATCH v3 15/17] discord: Add save_broadcast() " Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 17/17] discord: Add save_email() " Muhammad Rizki
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Add save_discord_mail() to save lore email that has been sent in the
Discord guild channel for checking if this email already been sent to
the channel and has reply to header then the bot reply to that message
if the reply id is exist in the database.
Signed-off-by: Muhammad Rizki <[email protected]>
---
.../dscord/database/methods/insertion/__init__.py | 4 +++-
.../database/methods/insertion/insert_discord.py | 14 ++++++++++++++
2 files changed, 17 insertions(+), 1 deletion(-)
create mode 100644 daemon/dscord/database/methods/insertion/insert_discord.py
diff --git a/daemon/dscord/database/methods/insertion/__init__.py b/daemon/dscord/database/methods/insertion/__init__.py
index 4192735..dd3ecf4 100644
--- a/daemon/dscord/database/methods/insertion/__init__.py
+++ b/daemon/dscord/database/methods/insertion/__init__.py
@@ -6,9 +6,11 @@
from .insert_atom import InsertAtom
from .insert_broadcast import InsertBroadcast
+from .insert_discord import InsertDiscord
class Insertion(
InsertAtom,
- InsertBroadcast
+ InsertBroadcast,
+ InsertDiscord
): pass
diff --git a/daemon/dscord/database/methods/insertion/insert_discord.py b/daemon/dscord/database/methods/insertion/insert_discord.py
new file mode 100644
index 0000000..bb423fd
--- /dev/null
+++ b/daemon/dscord/database/methods/insertion/insert_discord.py
@@ -0,0 +1,14 @@
+from datetime import datetime
+
+
+class InsertDiscord:
+
+ def save_discord_mail(self, email_id, channel_id, dc_msg_id):
+ q = """
+ INSERT INTO dc_mail_msg
+ (email_id, channel_id, dc_msg_id, created_at)
+ VALUES (%s, %s, %s, %s);
+ """
+ self.cur.execute(q, (email_id, channel_id, dc_msg_id,
+ datetime.utcnow()))
+ return self.cur.lastrowid
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC PATCH v3 17/17] discord: Add save_email() in database insertion
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
` (15 preceding siblings ...)
2022-09-19 7:26 ` [RFC PATCH v3 16/17] discord: Add save_discord_mail() " Muhammad Rizki
@ 2022-09-19 7:26 ` Muhammad Rizki
16 siblings, 0 replies; 20+ messages in thread
From: Muhammad Rizki @ 2022-09-19 7:26 UTC (permalink / raw)
To: Ammar Faizi
Cc: Muhammad Rizki, Alviro Iskandar Setiawan, GNU/Weeb Mailing List
Add save_email() to save raw lore emails to the database.
Explanation:
The functionality is like save_discord_mail() but it's still differ.
Save lore emails if it's not exist in database yet, if it does then
ignore.
Signed-off-by: Muhammad Rizki <[email protected]>
---
.../database/methods/insertion/__init__.py | 4 +++-
.../methods/insertion/insert_email.py | 20 +++++++++++++++++++
2 files changed, 23 insertions(+), 1 deletion(-)
create mode 100644 daemon/dscord/database/methods/insertion/insert_email.py
diff --git a/daemon/dscord/database/methods/insertion/__init__.py b/daemon/dscord/database/methods/insertion/__init__.py
index dd3ecf4..5137b84 100644
--- a/daemon/dscord/database/methods/insertion/__init__.py
+++ b/daemon/dscord/database/methods/insertion/__init__.py
@@ -7,10 +7,12 @@
from .insert_atom import InsertAtom
from .insert_broadcast import InsertBroadcast
from .insert_discord import InsertDiscord
+from .insert_email import InsertEmail
class Insertion(
InsertAtom,
InsertBroadcast,
- InsertDiscord
+ InsertDiscord,
+ InsertEmail
): pass
diff --git a/daemon/dscord/database/methods/insertion/insert_email.py b/daemon/dscord/database/methods/insertion/insert_email.py
new file mode 100644
index 0000000..0fab378
--- /dev/null
+++ b/daemon/dscord/database/methods/insertion/insert_email.py
@@ -0,0 +1,20 @@
+from mysql.connector import errors
+from datetime import datetime
+
+
+class InsertEmail:
+
+ def save_email(self, email_msg_id):
+ try:
+ return self.__insert_email(email_msg_id)
+ except errors.IntegrityError:
+ #
+ # Duplicate data, skip!
+ #
+ return None
+
+
+ def __insert_email(self, email_msg_id):
+ q = "INSERT INTO dc_emails (message_id, created_at) VALUES (%s, %s)"
+ self.cur.execute(q, (email_msg_id, datetime.utcnow()))
+ return self.cur.lastrowid
--
Muhammad Rizki
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [RFC PATCH v3 10/17] discord: Add error handler on events
2022-09-19 7:26 ` [RFC PATCH v3 10/17] discord: Add error handler on events Muhammad Rizki
@ 2022-09-20 8:37 ` Ammar Faizi
0 siblings, 0 replies; 20+ messages in thread
From: Ammar Faizi @ 2022-09-20 8:37 UTC (permalink / raw)
To: Muhammad Rizki; +Cc: Alviro Iskandar Setiawan, GNU/Weeb Mailing List
On 9/19/22 2:26 PM, Muhammad Rizki wrote:
> Adding error handler to ignore `CommandNotFound` error while someone
> sending a message that is not exists for the bot basic commands.
> See image:https://i.ibb.co/qRcCz6Z/image.png
Please don't use a screenshot for something like this, just dump the log
in the commit message. See this patch for example:
https://lore.kernel.org/lkml/[email protected]/
--
Ammar Faizi
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC PATCH v3 04/17] Add ignore file for .env
2022-09-19 7:26 ` [RFC PATCH v3 04/17] Add ignore file for .env Muhammad Rizki
@ 2022-09-20 8:45 ` Ammar Faizi
0 siblings, 0 replies; 20+ messages in thread
From: Ammar Faizi @ 2022-09-20 8:45 UTC (permalink / raw)
To: Muhammad Rizki; +Cc: Alviro Iskandar Setiawan, GNU/Weeb Mailing List
On 9/19/22 2:26 PM, Muhammad Rizki wrote:
> Adding *.env value in the .gitignore to ignore any file which has .env
> extension inside the daemon directory.
Please use v1 for writing persuasive verbs in the commit message.
s/Adding/Add/
Apply this rule to all pathces.
Thanks!
--
Ammar Faizi
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2022-09-20 8:45 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-09-19 7:26 [RFC PATCH v3 00/17] Refactor Telegram & initial work Discord Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 01/17] telegram: Move the Telegram bot source code Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 02/17] telegram: Refactor Telegram bot database method Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 03/17] telegram: Renaming some functions in scraper/bot.py Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 04/17] Add ignore file for .env Muhammad Rizki
2022-09-20 8:45 ` Ammar Faizi
2022-09-19 7:26 ` [RFC PATCH v3 05/17] telegram: Refactor the Telegram bot Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 06/17] telegram: Renames in telegram mailer directory Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 07/17] Move scraper and utility file Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 08/17] discord: Initial work Discord bot Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 09/17] discord: Add success run notice on_ready event Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 10/17] discord: Add error handler on events Muhammad Rizki
2022-09-20 8:37 ` Ammar Faizi
2022-09-19 7:26 ` [RFC PATCH v3 11/17] Move db.sql to combine database with Discord and Telegram Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 12/17] discord: Add database tables for Discord bot Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 13/17] discord: Add initial Discord bot database instance Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 14/17] discord: Add save_atom() in database insertion Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 15/17] discord: Add save_broadcast() " Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 16/17] discord: Add save_discord_mail() " Muhammad Rizki
2022-09-19 7:26 ` [RFC PATCH v3 17/17] discord: Add save_email() " Muhammad Rizki
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox