From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server-vie001.gnuweeb.org X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,URIBL_ZEN_BLOCKED_OPENDNS autolearn=ham autolearn_force=no version=3.4.6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gnuweeb.org; s=new2025; t=1757227183; bh=KMJPA3SxPaz/lZlwqtnr58iQAR0iXKblbZCCSVWG4Zg=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type:Content-Transfer-Encoding:Message-ID:Date:From: Reply-To:Subject:To:Cc:In-Reply-To:References:Resent-Date: Resent-From:Resent-To:Resent-Cc:User-Agent:Content-Type: Content-Transfer-Encoding; b=FdJTsxC4qGrr+3cBledTAMZ8PJz4py0SZO0VdCLqF9kGup8FoHBjv5XnsP4/kI3Jo jMzPDNx8Gf3EhPZrq9s3kyKlNQiur7YM4FXYm5RVQ8/XjVvQ2XfSXwXLz29tVZQgBW 96bb6mp2/WxE6BExXT0PCYRlfnlDm5xzooZEaNAgzH3yNCJLdfkXVDd7YaHL0QdEfY Vp0TM8inqRdyEa9zLeMoIriQRlU0zhu5LptNwjbUrH4rO/eILSkTLiPXhi0T/8qY/X XxSyk6VB9HcKO/0KEfDB0QCMdoMowSE9B6Ni84UWvRUUdcJb0G7OhCAeSHcDGy/ZWD 8t94kqfrzkBmg== Received: from mail-lf1-f47.google.com (mail-lf1-f47.google.com [209.85.167.47]) by server-vie001.gnuweeb.org (Postfix) with ESMTPSA id 86B5131279D5 for ; Sun, 7 Sep 2025 06:39:43 +0000 (UTC) Received: by mail-lf1-f47.google.com with SMTP id 2adb3069b0e04-55f76454f69so3179602e87.0 for ; Sat, 06 Sep 2025 23:39:43 -0700 (PDT) X-Forwarded-Encrypted: i=1; AJvYcCUiQrTjxgxrn3unTQDR+eyeRiCbfNr101No8wYS1NFftFIdEUgejHb4HKYsyjeiOvxsiAfR@vger.gnuweeb.org X-Gm-Message-State: AOJu0YxOueA1v7ZmsgpqjjgshhZLlKQJyJBthFZa0f7Jzrx+tc9zWuPZ gYZjUiTYqKVLuSMjoGQoeLTZrBlOJxuqx2Hy/8665EkrWaEysqqfuu3ng1iVuu1uJ4e8Guw1EHg SoM3Arf/sRfQc8P69in3rZ/1jGeYdVV4= X-Google-Smtp-Source: AGHT+IE0uxYxRHXA+e/br7vNy4DEYNXqk0fXDR+5LL/ADieIbKqlQsUB5GEhzj5olWUyook0FSY4NmBZS8p6AkQL6Vs= X-Received: by 2002:a05:651c:3543:20b0:333:fff0:7d21 with SMTP id 38308e7fff4ca-33b4c479492mr8317961fa.10.1757227182929; Sat, 06 Sep 2025 23:39:42 -0700 (PDT) MIME-Version: 1.0 References: <20250829075557.598176-1-reyuki@gnuweeb.org> <20250829075557.598176-3-reyuki@gnuweeb.org> In-Reply-To: From: Ahmad Gani Date: Sun, 7 Sep 2025 13:39:05 +0700 X-Gmail-Original-Message-ID: X-Gm-Features: Ac12FXzCz9vm5rPLEsHLOQrTDh4iOzlwtvT-UuwqJH0HK-t7_tcnTXTxnq8Bvmw Message-ID: Subject: Re: [PATCH gwproxy v8 2/2] gwproxy: refactor code base to add experimental raw DNS backend To: Ammar Faizi Cc: Alviro Iskandar Setiawan , "GNU/Weeb Mailing List" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-Id: On Sun, Sep 7, 2025 at 11:23=E2=80=AFAM Ammar Faizi wrote: > Have these two structs: > > struct stack_u16 { > uint16_t sp; > uint16_t bp; > uint16_t *arr; > }; > > struct dns_resolver { > struct stack_u16 stack; > struct gwp_conn_pair **sess_map; > uint16_t sess_map_cap; > }; > > [ The struct dns_resolver MAY also cover socket, addr, etc. Now my > primary point here is about the session mapping data structure. ] > > 1) @stack is used to keep track of unused indexes in @sess_map. > > - Push all unused indexes into the @stack, the index will be used as > the DNS query txid. > > - When creating a DNS query, pop the stack, use the popped number as > the txid. The txid is also used to store the corresponding > gwp_conn_pair session (sess_map[txid] =3D ptr to conn session). > > - When the sess_map[txid] is no longer used, set > (sess_map[txid] =3D NULL) and push the txid back into the @stack. Is using a stack more advantageous than incrementing txid until it recycles itself? we can still keep track of unused indexes by checking if the session_map[txid] is NULL or not. > 2) @sess_map_cap is used to keep track of the allocated size of > @sess_map. Don't allocate 65536 at once to keep the memory footprint > low when the proxy server is not fully utilized. I think 65536 is a pretty low memory footprint in modern systems; as Sir Alviro noted, it's around 0.25 or 0.5 MB. Maybe it's more beneficial to allocate at once in the stack rather than add instructions for growing/shrinking the allocated memory? > 3) Handle more than 65535 clients in a single thread? Doable! > > Just have more than one struct dns_resolver per thread, create a new > DNS resolver in the same thread once the 65535 limit is hit. > > Client 0 to 65535, use DNS resolver A. > Client 65536 to 131071, use DNS resolver B (new UDP socket!). > and so on... > > I doubt we will reach that limit. I think it is better to add more > threads to utilize more CPU cores than keep adding more clients in > a single thread. But we should still support it. Yeah, I prefer to close the connection and add more workers if the concurrent clients in that worker are beyond 65535. But closing the connection because of this makes gwproxy not a robust application; I will consider allowing it to scale beyond 65535 in a single worker. -- Ahmad Gani