Configuring Internal Socket Management

Jump to: navigation, search

Table of Contents >> Appendix A: Configuration Notes

Socket Lookup

A socket is not inserted in the socket lookup table until it is either connected or bound to a non zero port for TCP, bind without restriction for UDP, sendto(), or listen() occurs. If a TCP socket is bound to port zero, then getsockname() will not return the selected random port until connect(), or listen() has been called. So an application should call getsockname() after listen(), if it binds to port zero.

The socket lookup table has been split into two separate tables, one for non TCP sockets, and one for TCP sockets. The TCP socket lookup table now uses the 4-tuple as a search criteria (i.e. the local address is included in the search). Prior to this the local address search was done in a separate linked list. This is to speed up the TCP sockets lookup.

We have conditionally added a hash table for each socket lookup table. The two hash tables can be used either with a doubly linked list per bucket, or a Red Black tree per bucket. Using a doubly linked list or using a Red Black tree per bucket is a compile time decision.

  • By default the Red Black tree code is not enabled.
  • By default the hash tables are enabled (if TM_OPTIMIZE_SIZE is not defined), and the hash tables' sizes are TM_SOC_INDEX_MAX.
  • The hash table sizes are configurable at compile time or at run time, but prior to starting the stack.
  • You can use
  1. two hash tables with a doubly-linked list per bucket (default)
  2. two hash tables with a red black tree per bucket
  3. no hash table with a single doubly-linked list per socket lookup table
  4. no hash table with a single red black tree per socket lookup table.

Configuring the Hash Table sizes

By default, (if TM_OPTIMIZE_SIZE is not defined) two hash tables are dynamically allocated with TM_SOC_INDEX_MAX buckets. Each bucket's size is either the size of a pointer (if TM_USE_SOCKET_RB_TREE is defined), or the size of 2 pointers and an integer otherwise. The number of buckets is configurable at compile time, or run time.

Compile Time Configuration

Note Note: If TM_OPTIMIZE_SIZE is defined, then the hash tables are not enabled.

Run Time Configuration

  • To change the number of buckets in the non TCP sockets hash table at run time, call tfInitTreckOptions() with the TM_OPTION_SOTB_NON_CON_HASH_SIZE option name.

Enabling the Red Black Trees

By default the hash tables use a doubly-linked list for sockets that end up in the same bucket. This can be changed to a red black tree at compile time. To do so, define TM_USE_SOCKET_RB_TREE in <trsystem.h>.

Note Note: If TM_USE_SOCKET_RB_TREE, and TM_DISABLE_SOCKET_HASH_LIST are both defined, then a single red black tree per socket look up table will be used.

Enable the Red Black tree, if you have limited data space, and you need to either disable the hash tables, or reduce their sizes dramatically and you will use a lot of sockets, and you have enough ROM for the Red Black tree code.

UDP Performance Enhancement

For applications that are using hash table socket lookup (i.e. TM_DISABLE_SOCKET_HASH_LIST is not defined), the standard UDP socket lookup algorithm narrows down the search by hashing on the destination port of the incoming UDP datagram. This is perfect for most applications, as it executes fast and usually gets to a list of only one or two sockets.

For applications that have thousands of UDP sockets with many sockets bound to the same local port, the standard search can degrade to a linear search. By uncommenting the following compile time macro in your trsystem.h, you enable code that hashes on both source and destination ports and IP addresses as well as address family (IPv4 or IPv6) and device binding (only if TM_USE_STRONG_ESL is defined). In some applications, this detailed hash lookup can arrive at the destination UDP socket faster than the standard algorithm.

#define TM_USE_UDP_LOOKUP_PERF

Because of the diverse binding combinations, this feature may require multiple passes through the hash lookup algorithm before it finds a relevant socket. As a result, most applications will see a slight decrease in performance when TM_USE_UDP_LOOKUP_PERF is defined. The following list gives some examples of the iterative search process:

  • sockets may be bound to a particular local IP address or any local IP address,
  • sockets may be bound to a particular remote IP address and port if connect() is used,
  • sockets may be bound to a particular interface if Strong ESL is used,
  • a socket may be bound to TM_WILD_PORT to receive on any local port,
  • IPv4 datagrams can be received by IPv6 sockets if there is no IPv4 socket bound to the incoming port and the IPv6 socket has not been restricted by the setsockopt() IPV6_V6ONLY option.

Random Port Insertion

When a socket is inserted in a socket lookup table, and the user did not bind the socket, a random port is selected. Selecting a random port that does not conflict when thousands of sockets are used could take a lot of CPU time. If the number of concurrent sockets reaches a pre-defined configurable threshold, and the stack detects a port collision when picking a random port, the stack will switch to selecting increasing port numbers (instead of random port numbers) so that the stack does not spend too much time finding a random port with no conflict. This threshold is configurable at both compile time and run time. By default the threshold is 304.

TCP sockets threshold

Threshold of concurrent TCP sockets before we switch from a random local port selection to a sequential local port selection is given by the TM_SOTB_TCP_CON_PORT_THRS macro. The user can redefine the TM_SOTB_TCP_CON_PORT_THRS in <trsystem.h>, to change the threshold of concurrent TCP sockets that will cause the stack to switch to a sequential port selection when a port conflict occurs when inserting a connecting socket in the socket table lookup. The user can also change this threshold at run time at any time using the tfSetTreckOptions() API, with the TM_OPTION_SOTB_TCP_CON_PORT_THRS option.

Non TCP sockets

Threshold of non TCP sockets before we switch from a random local port selection to a sequential local port selection is given by the TM_SOTB_NON_CON_PORT_THRS macro. The user can redefine the TM_SOTB_NON_CON_PORT_THRS in <trsystem.h>, to change the threshold of concurrent non TCP sockets that will cause the stack to switch to a sequential port selection when a port conflict occurs when inserting a non TCP socket in the socket table lookup. The user can also change this threshold at run time at any time using the tfSetTreckOptions() API, with the TM_OPTION_SOTB_NON_CON_PORT_THRS option.


Table of Contents >> Appendix A: Configuration Notes