[PATCH v2 0/8] Fix some bugs and add some features

[Muhammad Rizki <kiizuha@gnuweeb.org>]

Hi sir,

This is v2 revision of fix some bugs and add some features.
This series contain add if statement to ignore text/html and remove
a re.sub().

This series is to fix some bugs, improve some codes and add some new
features. These bugs should have been fixed now, and the email file
attachments should have been removed after all attachments have been
sent.

Known bugs:
1. Email payload extraction result become unicode if the email payload
   contain non-UTF8 characters like chinese, japanese, and similar
   like that.
2. remove_patch() doesn't remove all file attachments properly.

Improvements:
1. Improve fix_utf8_char() to make it stable for both Discord and
   Telegram bot.
2. Improve remove_patch() to make it all file attachments removed
   after sending them.
3. Fix `/lore {raw atom url}` to add a "telegram" onto the
   create_template() platform parameter.
4. Improvement on extract_body() to ignore whenever the email contain
   text/html content-type.

New features:
1. Add send_text_mail_interaction() for the `/lore {raw atom url}` slash
   command.
2. Add send_patch_mail_interaction() for the `/lore {raw atom url}`
   slash command.
3. Add `/lore {raw atom url}` slash command.

There are 8 patches in this series:
- Patch 1 is to add send_text_mail_interaction()
- Patch 2 is to add send_patch_mail_interaction()
- Patch 3 is to add `/lore` slash command
- Patch 4 is to improve fix_utf8_char() code
- Patch 5 is to improve remove_patch() code to make it more stable
- Patch 6 is to add manage_payload() for manage email payload extraction
- Patch 7 is to fix the Telegram `/lore` command
- Patch 8 is to improve extract_body() to ignore text/html content-type

How to use:
1. Execute the db.sql file in the daemon directory,
2. Setup .env file, the example is there with suffix .example, this
   file name must remove the suffix name .example,
3. Set up the config.py in each bot directory, such as dscord and
   telegram. The example is there with suffix .example & the file name
   must remove suffix name .example,
4. Run `pip3 install -r requirements.txt` in each bot directory,
5. STORAGE_DIR env value must `storage` to make it work fine,
6. Run the bot by `python3 dc.py` or `python3 tg.py`.

Both tested. But, I want to make sure if it's already fixed and stable.
So, don't forget to test it too, thanks.

## Changelog

v1 -> v2:
- Remove re.sub() in the fix_utf8_char() when set unescape to True
- Add if statement logic to ignore text/html in the extract_body()

Signed-off-by: Muhammad Rizki <kiizuha@gnuweeb.org>
---

Muhammad Rizki (8):
  discord: Add send_text_mail_interaction()
  discord: Add send_patch_mail_interaction()
  discord: Add get lore mail slash command
  atom: Small change for fix_utf8_char()
  atom: Improve remove_patch()
  atom: add manage_payload()
  telegram: Fix get lore command
  atom: Improve extract_body()

 daemon/atom/utils.py                          | 32 +++++++++++----
 daemon/dscord/gnuweeb/client.py               | 23 +++++++++++
 .../plugins/slash_commands/__init__.py        |  2 +
 .../plugins/slash_commands/get_lore_mail.py   | 39 +++++++++++++++++++
 daemon/dscord/mailer/listener.py              |  4 +-
 daemon/telegram/mailer/listener.py            |  3 +-
 .../packages/plugins/commands/scrape.py       |  5 +--
 7 files changed, 93 insertions(+), 15 deletions(-)
 create mode 100644 daemon/dscord/gnuweeb/plugins/slash_commands/get_lore_mail.py


base-commit: d9b20dab81202b93f48d5365ad680796e5839d80
--
Muhammad Rizki