菜鸟学Linux 第111篇笔记 Memory

菜鸟学Linux 第111篇笔记 Memory

建议查看原文（因为复制版的格式可能有问题)
原文出自 Winthcloud　链接：

内容总览
内存子系统组件
Memory提升
Viewing system calls
Strategies for using memory
Tunning page allocation
Tuning overcommit
Slab cache
ARP cache
Page cache
调优策略(redhat 6 performance tuning guide官方文档)
进程间通信相关的调优

内存子系统组件
slab allocator
buddy system
kswapd
pdflush
mmu

虚拟化环境
PA --> HA --> MA
虚拟机转换：PA-->HA

GuestOS, OS
Shadow PT

Memory提升
lbs

Hugetlbfs
查看是否启用Hugetlbfs
cat /proc/meminfo | grep Huge

启用大页面(永久有效)
/etc/sysctl.conf
添加 vm.nr_hugepages = n

即时启用
sysctl -w vm.nr_hugepages=n

Configure hugetlbfs if needed by application
创建hugepage并挂载
mkdir /hugepages
mount -t hugetlbfs none /hugepages

Viewing system calls
Trace every system call made by a program
strace -o /tmp/strace.out -p PID
grep mmap /tmp/strace.out

Summarize system calls
strace -c -p PID or
strace -c COMMAND

Strategies for using memory
Reduce overhead for tiny memory objects
Slab cache
cat /proc/slabinfo

Reduce or defer service time for slower subsystems
Filesystem metadata: buffer cache(slab cache)
Disk IO: page cache
Interprocess communications: shared memory
Network IO: buffer cache, arp cache, connection tracking

使用buffer cache 缓存文件元数据
使用page cache缓存Disk IO
使用shm完成进程间通信
使用buffer cache, arp cache和connection tracking提升网络IO性能

Considerations when tunning memory
How should pages be reclaimed to avoid pressure?
Larger writes are usually more efficient due to re-sorting

Tunning page allocation
Set using

vm.min_free_kbytes
Tuning vm.min_free_kbytes only be necessary when an application regularly
needs to allocate a large block of memory, then frees that same memory

It may well be the case that the system has too little disk bandwith, too
little CPU power, or too little memory to handle its load.

Consequences
Reduces service time for demand paging
Memory is not available for other useage
Can cause pressure on ZONE_NORMAL

内存耗尽会使系统崩溃

Tuning overcommit
Set using

cat /proc/sys/vm/overcommit_memory
vm.overcommit_memory
0 = heuristic overcommit
1 = always overcommit
2 = commit all RAM plus a percentage of swap (may be > 100)

vm.overcommit_ratio
specified the percentage of physical memory allowed to be
overcommited when the vm.overcommit_memory set to 2

View Committed_AS in /proc/meminfo
An estimate of how much RAM is required to avoid an out of memory (OOM)
condition for the current workload on a system

OOM
Overcommit Of Memory

Slab cache
Tiny kernel objects are stored in slab
Extra overhead of tracking is better than using 1 page/object
Example: filesystem metadata(dentry and inode caches)

Monitoring
/proc/slabinfo
slabtop
vmstat -m

Tuning a particular slab cache
echo "cache_name limit batchcount shared" > /proc/slabinfo
limit the maximum number of objects that will be cached for each CPU

batchcount the maximum number of global cache objects that will be
trasferred to the per-CPU cache when it becomes empty

shared the sharing behavior for Symmetric MultiProcessing(SMP) systems

ARP cache
ARP entries map hardware address to protocol address
cached in /proc/net/arp
By default, the cache is limited to 512 entries as a soft limit
and 1024 entries as a hard limit

Garbage collection removes stale or older entries

Insufficient ARP cache leads to
Intermittent timeouts between hosts
ARP thrashing

Too much ARP cache puts pressure on ZONE_NORMAL
List entries
ip neighbor list
Flush cache
ip neighbor flush dev ethX

Tuning ARP cache
Adjust where the gc will leave arp table alone
net.ipv4.neigh.default.gc_thresh1
default 128

Soft upper limit
net.ipv4.neigh.default.gc_thresh2
default 512
Becomes hard limit after 5 seconds

Hard upper limit
net.ipv4.neigh.default.gc_thresh3

Garbage collection frequency in seconds
net.ipv4.neigh.default.gc_interval

Page cache
A large percentage of paging activity is due to I/O requests
File reads: each page of file read from disk into memory
These pages form the page cache

Page cache is always checked for IO requests
Drectory reads
Reading and writing regular files
Reading and writing via block device files, DISK IO
Accessing memory mapped files, mmap
Accessing swapped out pages

Page in the page cache are associated with file data

Tuning page cache
View page cache allocation in /proc/meminfo
Tune length/size of memory
vm.lowmen_reserve_ratio
vm.vfs_cache_pressure

Tune arrival/completion rate
vm.page-cluster
vm.zone_reclaim_mode

vm.lowmen_reserve_ratio
For some specialised workloads on highmem machines it is dangerous for the
kernel to allow process memory to be allocated from the "lowmem" zone

Linux page allocator has a mechanism which prevents allocations which could
use highmem from using too much lowmem

The 'lowmem_reserve_ratio' tunable determines how aggressive the kernel is
in defending these lower zones

If you have a machine which uses highmem or ISA DMA and Your applications
are using mlock(), or if you are running with no swap then you probably
should change the lowmem_reserve_ratio setting

vfs_cache_pressure
Controls the tendency of the kernel to reclaim the memory which is used for
caching of directory and inode objects

At the default value of vfs_cache_pressure=100 the kernel will attempt to
reclaim dentries and inodes at a "fair" rate with respect to pagecache and
swapcache reclaim

Decreasing vfs_cache_pressure causes the kernel to prefer to retain dentry
and inode caches

When vfs_cache_pressure=0, the kernel will never reclaim dentries and
inodes due to memory pressure and this can easily lead to out-of-memory
conditions

Increasing vfs_cache_pressure beyond 100 causes the kernel to prefer to
reclaim dentries and inodes.

page-cluster
page-cluster controls the number of pages which are written to swap in
single attempt

It is a logarithmic value-setting it to zero means "1 page", setting it to
1 means "2 pages", setting it to 2 means "4 pages", etc

The default value is three (eight pages at a time)
There may be some small benefits in tuning this to a different value if
your workload is swap-intensive

zone_reclaim_mode
Zone_reclaim_mode allows someone to set more or less aggressive approaches
to reclaim memory when a zone runs out of memory

If it is set to zero then no zone reclaim occurs

Allocations will be satisfied from other zones/nodes in the system

This is value ORed together of
1 = Zone reclaim on
2 = Zone reclaim writes dirty pages out
4 = Zone reclaim swaps pages

Anonymous pages
Anonymous pages can be another large consumer of data

Are not associated with a file, but instead contain:
Program data - arrays, head allocations, etc
Anonymous memory regions
Dirty memory mapped process private pages
IPC shared memory regions pages

View summary usage
grep Anon /proc/meminfo
cat /proc/PID/statm
Anonymous pages = RSS - Shared

Anonymous pages are eligible for swap

调优策略
硬件调优： 硬件选型
软件调优： 内核调优 /proc, /sys
应用调优

内核调优
1. 进程管理，CPU
2. 内存调优
3. I/O 调优
4. 文件系统
5. 网络子系统

调优思路
1. 查看各项性能指标，定位瓶颈
2. 调优

红帽官方提供一份文档 redhat 6 performance tuning guide 可以搜索到

进程间通信相关的调优
ipcs (interprocess communication facilities)

进程间通信管理命令
ipcs
ipcrm

shared memory
kernel.shmmni
Specifies the maximum number of shared memory segments
system-wide, default = 4096

kernel.shmall
Specifies the total amount of shared memory, in pages, that
can be used at one time on the system, default=2097152

This should be at least kernel.shammax/PAGE_SIZE

kernel.shmmax
Specifies the maximum size of a shared memory segment that
can be created

messages
kernel.msgmnb
Specifies the maximum number of bytes in single message
queue, default = 16384

kernel.msgmni
Specifies the maximum number of message queue identifiers,
default=16

kernel.msgmax
Specifies the maximum size of a message that can be passed
between processes

This memory cannot be swapped, default=8192