Re: [PATCH gwproxy v8 2/2] gwproxy: refactor code base to add experimental raw DNS backend

[Ahmad Gani <reyuki@gnuweeb.org>]

On Sun, Sep 7, 2025 at 11:23 AM Ammar Faizi wrote:
> Have these two structs:
>
> struct stack_u16 {
>         uint16_t sp;
>         uint16_t bp;
>         uint16_t *arr;
> };
>
> struct dns_resolver {
>         struct stack_u16 stack;
>         struct gwp_conn_pair **sess_map;
>         uint16_t sess_map_cap;
> };
>
> [ The struct dns_resolver MAY also cover socket, addr, etc. Now my
>   primary point here is about the session mapping data structure. ]
>
> 1) @stack is used to keep track of unused indexes in @sess_map.
>
>    - Push all unused indexes into the @stack, the index will be used as
>      the DNS query txid.
>
>    - When creating a DNS query, pop the stack, use the popped number as
>      the txid. The txid is also used to store the corresponding
>      gwp_conn_pair session (sess_map[txid] = ptr to conn session).
>
>    - When the sess_map[txid] is no longer used, set
>      (sess_map[txid] = NULL) and push the txid back into the @stack.

Is using a stack more advantageous than incrementing txid until it
recycles itself? we can still keep track of unused indexes by checking
if the session_map[txid] is NULL or not.

> 2) @sess_map_cap is used to keep track of the allocated size of
>    @sess_map. Don't allocate 65536 at once to keep the memory footprint
>    low when the proxy server is not fully utilized.

I think 65536 is a pretty low memory footprint in modern systems; as Sir
Alviro noted, it's around 0.25 or 0.5 MB.

Maybe it's more beneficial to allocate at once in the stack rather than
add instructions for growing/shrinking the allocated memory?

> 3) Handle more than 65535 clients in a single thread? Doable!
>
>    Just have more than one struct dns_resolver per thread, create a new
>    DNS resolver in the same thread once the 65535 limit is hit.
>
>       Client     0 to 65535,  use DNS resolver A.
>       Client 65536 to 131071, use DNS resolver B (new UDP socket!).
>       and so on...
>
>    I doubt we will reach that limit. I think it is better to add more
>    threads to utilize more CPU cores than keep adding more clients in
>    a single thread. But we should still support it.

Yeah, I prefer to close the connection and add more workers if the
concurrent clients in that worker are beyond 65535.

But closing the connection because of this makes gwproxy not a robust
application; I will consider allowing it to scale beyond 65535 in
a single worker.

--
Ahmad Gani