From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server-vie001.gnuweeb.org X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,URIBL_ZEN_BLOCKED_OPENDNS autolearn=ham autolearn_force=no version=3.4.6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gnuweeb.org; s=new2025; t=1757218985; bh=iX4ILAbFN7eLAZ8unGhxHe8wRCgZVqmnc1Ykc289BC8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To:Message-ID:Date:From:Reply-To:Subject:To: Cc:In-Reply-To:References:Resent-Date:Resent-From:Resent-To: Resent-Cc:User-Agent:Content-Type:Content-Transfer-Encoding; b=ojI3j19aidyrs/WCkT9Fg9/0zFsqk3XM7sz6OrJujV2hJ/wHwG0+dXFqPxZgoJLOz XgpMTG3a4hwBBnCqh1Tec+LuhNWiXqv8bYfc+VzbCxIDyUmqMp/64eT1PoQZ/mvpIT vz3Ae6+9DqaavEunf4dkcC8Kp2arFYGgbc2K2GyCEL/OjB4wjulBRy7dhcE6fx182/ +FOzpT6CEsNhAHtQlcmTpYMCaXvX7RZJbX7vyNj7ZEl84s/vKqQg9E8RTW20ube5SD 6W1JJ7CkOeP9dczK8Pzrrfz+vlEp1Ti7Uf70P3he0ADOLPqgOxssG93KB8ai9KRkpn 0f2KAb++z3H+g== Received: from biznet-home.integral.gnuweeb.org (unknown [114.10.153.232]) by server-vie001.gnuweeb.org (Postfix) with ESMTPSA id 80715312798D; Sun, 7 Sep 2025 04:23:04 +0000 (UTC) Date: Sun, 7 Sep 2025 13:22:49 +0900 From: Ammar Faizi To: Alviro Iskandar Setiawan Cc: Ahmad Gani , GNU/Weeb Mailing List Subject: Re: [PATCH gwproxy v8 2/2] gwproxy: refactor code base to add experimental raw DNS backend Message-ID: References: <20250829075557.598176-1-reyuki@gnuweeb.org> <20250829075557.598176-3-reyuki@gnuweeb.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: List-Id: On Sat, Sep 06, 2025 at 08:30:54PM +0700, Alviro Iskandar Setiawan wrote: > Let me know your design if you have one too. I have a small proposal. Writing this email from my Android phone right now. I am still traveling. Sitting and waiting are boring, so I decided to exercise my brain designing this solution. It keeps my brain active. Have these two structs: struct stack_u16 { uint16_t sp; uint16_t bp; uint16_t *arr; }; struct dns_resolver { struct stack_u16 stack; struct gwp_conn_pair **sess_map; uint16_t sess_map_cap; }; [ The struct dns_resolver MAY also cover socket, addr, etc. Now my primary point here is about the session mapping data structure. ] 1) @stack is used to keep track of unused indexes in @sess_map. - Push all unused indexes into the @stack, the index will be used as the DNS query txid. - When creating a DNS query, pop the stack, use the popped number as the txid. The txid is also used to store the corresponding gwp_conn_pair session (sess_map[txid] = ptr to conn session). - When the sess_map[txid] is no longer used, set (sess_map[txid] = NULL) and push the txid back into the @stack. 2) @sess_map_cap is used to keep track of the allocated size of @sess_map. Don't allocate 65536 at once to keep the memory footprint low when the proxy server is not fully utilized. - Start allocating @sess_map from size 16. Push 15, 14, 13, ..., 0 into the @stack. - Double the size when the slot is exhausted. When double-ing the size, the new allocated indexes are all pushed into the @stack. The slot exhaustion happens when the @stack is empty, it can be identified at the pop operation. - @sess_map array can be expanded up to 65536. - @sess_map MAY only be shrunk back to 16 when all members in @sess_map are NULL. When to shrink the array can be identified at the push operation. If the number of elements in the @stack is equal to @sess_map_cap, that means @sess_map is 100% free and can be reset to the initial state (back to size 16). The content of the @stack MUST also be reset at the shrink operation to keep the next pop and push operations in sync. 3) Handle more than 65535 clients in a single thread? Doable! Just have more than one struct dns_resolver per thread, create a new DNS resolver in the same thread once the 65535 limit is hit. Client 0 to 65535, use DNS resolver A. Client 65536 to 131071, use DNS resolver B (new UDP socket!). and so on... I doubt we will reach that limit. I think it is better to add more threads to utilize more CPU cores than keep adding more clients in a single thread. But we should still support it. -- Ammar Faizi