Académique Documents
Professionnel Documents
Culture Documents
Page 1 of 7
Please Note: When mentioned Temporary, it means that after reboot configuration will be erased while when mentioned Permanently, it means that after reboot the configurations will remain intact.
A value of 0 associates a new TCP/IP connection with the CPU that creates the connection. A value of 1 associates the connection with multiple squeues that belong to different CPUs. The number of squeues that are used to fanout the connection is based upon ip_soft_rings_cnt. Consider setting this parameter to 1 to spread the load across all CPUs in certain situations. For example, when the number of CPUs exceed the number of NICs, and one CPU is not capable of handling the network load of a single NIC, change this parameter to 1. Temporary - ndd -set /dev/tcp ip_squeue_fanout 1 Permanently insert into /etc/system - set ip:ip_squeue_fanout = 1
Remarks In Solaris, the available range of TCP/IP ports is 0 to 65535. However, there are some restrictions that apply: Ports in the range 0 to 1023 are reserved for privileged (root) services, such as telnetd, ftpd, and so on. Ports in the range 1024 to tcp_smallest_anon_port-1 are used for user
http://www.eilfa.com/solaris/solaris-10-tcp-tuning.html
12/02/13
Page 2 of 7
services such as NFS server daemon, FONT server, and so on. This leaves the range 32768 to 65535 available for general TCP/IP connections. To limit the range of the port numbers allocated for the general use, the following two ndd(1M) parameters can be used: tcp_smallest_anon_port: This determines the smallest TCP port number that may be used for an anonymous connection. Solaris allocates anonymous ports above 32768. The default value is 32768. tcp_largest_anon_port: This is the largest TCP port number that may be used for anonymous connections. The default value of this is 65535.
kernel sockets
The kernel keeps a list of sockets in the TIME_WAIT state. When the list is full, failures start to occur. If your server is getting new client connections faster than it can bleed off sockets in the TIME_WAIT state, the list will ultimately get full. Decreasing the timeout increases the bleed-off rate. Default - 60000
Temporary - ndd -set /dev/tcp tcp_time_wait_interval 3000 Setting this didnt give a better performance for a lighty web server.
Check UNDER /etc/system: Controls the hash table size in the TCP module for all TCP connections (default 512). Temporary - ndd -set tcp:tcp_conn_hash_size=8192 Controls the hash table size in an IP module for all active (in ESTABLISHED state) TCP connections(default 512). Permanently insert into /etc/system - set ipc_tcp_conn_hash_size=8192
congestion window
The maximum (Default 4)initial congestion window (cwnd) size in MSS of a TCP connection tcp_rexmit_interval_initial Temporary - ndd -set /dev/tcp tcp_slow_start_initial 1 ndd -set /dev/tcp tcp_slow_start_after_idle 1 When to Change? For more information, see tcp_slow_start_initial.
http://www.eilfa.com/solaris/solaris-10-tcp-tuning.html
12/02/13
Page 3 of 7
ndd -set /dev/tcp tcp_slow_start_initial 2 ndd -set /dev/tcp tcp_slow_start_initial 1 When to Change? Do not change the value. If the initial cwnd size causes network congestion under special circumstances, decrease the value.
TIME_WAIT ports
These ensure that TIME_WAIT ports either get reused or closed fast. insert into /etc/system Linux net.ipv4.tcp_fin_timeout = 1 in Solaris tcp_time_wait_interval Linux net.ipv4.tcp_tw_recycle = 1 TCP memory Linux net.core.rmem_max = 16777216 Linux net.core.rmem_default = 16777216 Linux net.core.netdev_max_backlog = 262144 in Solaris to tcp_conn_req_max_q Linux tcp_slow_start_after_idle = 262144
Linux net.ipv4.tcp_synack_retries & net.ipv4.tcp_syn_retries = 2 in Solaris tcp_rexmit_interval_min 400 tcp_rexmit_interval_max 60000 tcp_ip_abort_interval 480000 tcp_rexmit_interval_initial 3000
conntrack
You shouldn't be using conntrack on a heavily loaded server anyway, but these are suitably high for our uses, insuring that if conntrack gets turned on, the box doesn't die net.ipv4.netfilter.ip_conntrack_max = 1048576 net.nf_conntrack_max = 1048576
http://www.eilfa.com/solaris/solaris-10-tcp-tuning.html
12/02/13
Page 4 of 7
In Solaris use Dtrace script to track connections Dtrace script to track connections
TCP keepalive
TCP keepalive is a feature provided by many TCP implementations, including Solaris, as a way to clean up idle connections in situations like the ones mentioned above. Applications must enable this feature with the SO_KEEPALIVE socket option via the setsockopt(3SOCKET) socket call. Once enabled, a keepalive probe packet is sent to the other end of the socket provided the connection has remained in the ESTABLISHED state and has been idle for the specified time frame. This time frame is the value specified by the TCP tunable tcp_keepalive_interval.
http://www.eilfa.com/solaris/solaris-10-tcp-tuning.html
12/02/13
Page 5 of 7
A keepalive probe packet is handled just like any other TCP packet which requires an acknowledgment (ACK) from the other end of the socket connection. It will be retransmitted per the standard retransmission backoff algorithm. If no response is received by the time specified for the other TCP tunable, tcp_ip_abort_interval, the connection is terminated, as would be the case for any other unacknowledged packet. Hence the actual maximum idle time of a connection utilizing TCP keepalive, which has no responding peer will therefore be: tcp_keepalive_interval + tcp_ip_abort_interval Default valuses respectively 7200000 480000 The above parameters are global and will affect the entire system. Keep in mind that TCP keepalive probes have no effect on inactive connections as long as the remote host is still responding to probes. However care should be taken to ensure the above parameters remain at a high enough value to avoid unnecessary traffic and other issues such as prematurely closing active connections in situations where a few packets have gone missing. Temporary - ndd -set /dev/tcp tcp_keepalive_interval 300000
Backlog Queue
The backlog queue is a large memory structure used to handle incoming packets with the SYN flag set until the moment the three-way handshake process is completed. An operating system allocates part of the system memory for every incoming connection. We know that every TCP port can handle a defined number of incoming requests. The backlog queue controls how many half-open connections can be handled by the operating system at the same time. When a maximum number of incoming connections is reached, subsequent requests are silently dropped by the operating system. As mentioned before, when we detect a lot of connections in the SYN RECEIVED state, host is probably under a SYN flooding attack. Moreover, the source IP addresses of these incoming packets can be spoofed. To limit the effects of SYN attacks we should enable some built-in protection mechanisms. Additionally, we can sometimes use techniques such as increasing the backlog queue size and minimizing the total time where a pending connection in kept in allocated memory (in the backlog queue). Run this command to count how many half-open connections are in the backlog queue at the moment netstat -s -P tcp | grep tcpHalfOpenDrop In Sun Solaris there are two parameters which control the maximum number of connections. The first parameter tcp_conn_req_max_q controls the total number of full connections. The second tcp_conn_req_max_q0 parameter defines how many halfopen connections are allowed without the dropping of incoming requests. In Sun Solaris 8, the default value is set to 1024. Using the ndd command we can modify this value.
http://www.eilfa.com/solaris/solaris-10-tcp-tuning.html
12/02/13
Page 6 of 7
It is pretty simple really: never change these parameters unless connections are refused because the values are too low. The only way to determine this empirically is to use netstat s | fgrep i listendrop. If tcpListenDrop is non-zero, increase tcp_conn_req_max_q. If tcpListenDropQ0 is non-zero, increase tcp_conn_req_max_q0. Temporary - ndd -set /dev/tcp tcp_conn_req_max_q 128 OR 262144 Temporary - ndd -set /dev/tcp tcp_conn_req_max_q0 1024 OR 30000
TCP/IP statistics
These set of commands will present some of the TCP/IP statistics you will need in order to follow every change you make in your TCP/IP stack.
netstat -nP tcp | grep WAIT | wc -l;netstat -nP tcp |wc -l;netstat -s -P tcp | grep -E "tcpL" netstat -I bnx0 10 iostat -xn 10
TCP/IP script
This script will assist you in configuring the TCP/IP parameters on your system.
http://www.eilfa.com/solaris/solaris-10-tcp-tuning.html
12/02/13
Page 7 of 7
#!/sbin/sh ndd -set /dev/ip ip_forward_src_routed 0 #(Defalut value alreay set) ndd -set /dev/tcp tcp_rev_src_routes 0 #(Defalut value alreay set) ndd -set /dev/ip ip_forward_directed_broadcasts 0 #(Defalut value alreay set) *ndd -set /dev/tcp tcp_conn_req_max_q0 4096 #(Defalut value 1024) *ndd -set /dev/tcp tcp_conn_req_max_q 1024 #(Defalut value 128) ###Prevent System responding to ICMP timestamp requests ndd -set /dev/ip ip_respond_to_timestamp 0 #(Defalut value alreay set) ###Prevent System responding to ICMP timestamp Broadcast ndd -set /dev/ip ip_respond_to_timestamp_broadcast 0 #(Defalut value alreay set) ndd -set /dev/ip ip_respond_to_address_mask_broadcast 0 #(Defalut value alreay set) ndd -set /dev/ip ip_respond_to_echo_broadcast 0 ndd -set /dev/arp arp_cleanup_interval 60000 ndd -set /dev/ip ip_ire_arp_interval 60000 ndd -set /dev/ip ip_ignore_redirect 1 ndd -set /dev/ip ip_strict_dst_multihoming 1 ndd -set /dev/ip ip_send_redirects 0 ########END of TCP/IP script###############
http://www.eilfa.com/solaris/solaris-10-tcp-tuning.html
12/02/13