[AIX] TCP streaming workload tuning

< TCP streaming workload tuning >

Streaming workloads는 많은 양의 data를 하나의 endpoint에서 다른 endpoint로 옮기는 것이다. 예를 들어 file transfer, backup 또는 restore workload 또는 bulk data transfer등이다.

이 동작은 bandwidth와 관련이 있으나 또한 end-to-end latency를 봐야 한다.

The primary tunables that affect TCP performance for streaming applications are the following:

- tcp_recvspace

- tcp_sendspace

- rfc1323

- MTU path discovery

- tcp_nodelayack

- sb_max

- Adpater options, such as checksum offload and TCP Large Send

The following table shows suggested sizes for the tunable values to obtain optimal performance, based on the type of adapter and the MTU size:

Device	Speed	MTU size	tcp_sendspace	tcp_recvspace	sb_max1	rfc1323
Token Ring	4 or 16 Mbit	1492	16384	16384	32768	0
Ethernet	10 Mbit	1500	16384	16384	32768	0
Ethernet	100 Mbit	1500	16384	16384	65536	0
Ethernet	Gigabit	1500	131072	65536	131072	0
Ethernet	Gigabit	9000	131072	65535	262144	0
Ethernet	Gigabit	9000	262144	1310722	524288	1
Ethernet	10 Gigabit	1500	131072	65536	131072	0
Ethernet	10 Gigabit	9000	262144	131072	262144	1
ATM	155 Mbit	1500	16384	16384	131072	0
ATM	155 Mbit	9180	65535	655353	131072	0
ATM	155 Mbit	65527	655360	6553604	1310720	1
FDDI	100 Mbit	4352	45056	45056	90012	0
Fiber Channel	2 Gigabit	65280	655360	655360	1310720	1

(1) sb_max의 디폴트 값을 1048576으로 사용할 것을 제안함.

(2) Performance is slightly better when using these options, with rfc1323 enabled, on jumbo frames on Gigabit Ethernet

(3) Certain combinations of TCP send and receive space will result in very low throughput, (1Mbit or less). To avoid this problem, set the tcp_sendspace tunable to a minimum of three times the MTU size or greater or equal to the receiver's

tcp_recvspace value.

(4) TCP has only a 16-bit value to use for its window size. This translates to a maximum window size of 65536 bytes. For adapters that have large MTU sizes(for example 32KB or 64KB), TCP streaming performance might be very poor. For example, on a device with a 64KB MTU size, and with a tcp_recvspace set to 64KB, TCP can only send one packet and then its window closes. It must wait for an ACK back from the receiver before it can send again. This problem can be solved in one of the following ways:

- Enable rfc1323, which enhances TCP and allows it to overcome the 16-bit limit so that it can use a window size larger than

64KB. You can then set the tcp_recvspace tunable to a large value, such as 10 times the MTU size, which allows TCP to stream data and thus provides good performance.

- Reduce the MTU size of the adapter. For example, use the ifconfig at0 mtu 16384 command to set the ATM MTU size to 16KB. This causes TCP to compute a smaller MSS value.