Re: [PATCH v1 4/7] atom: Improve fix_utf8_char()

[Muhammad Rizki <kiizuha@gnuweeb.org>]

On 19/10/2022 23.59, Ammar Faizi wrote:
> On 10/18/22 3:16 PM, Muhammad Rizki wrote:
>> -def fix_utf8_char(text: str, html_escape: bool = True):
>> +def fix_utf8_char(text: str, unescape: bool = True):
>>       t = text.rstrip().replace("�"," ")
>> -    if html_escape:
>> -        t = html.escape(html.escape(text))
>> +    if unescape:
>> +        t = html.unescape(html.unescape(text))
>> +        reg = 
>> re.compile('<.*?>|&([a-z0-9]+|#[0-9]{1,6}|#x[0-9a-f]{1,6});')
>> +        t = reg.sub('', t)
>>       return t
> 
> You do html.unescape() twice, then remove all HTML special chars and
> tags. I don't understand why we should do that. Can you explain a bit
> on what is going on here?
> 

You said an HTML tag in the email payload should be empty or removed, so 
I created the re.sub() to remove the HTML tag. I forgot where you said that.