Month: January 2011

zfs set mountpoint=/mount/point/dir mypool/myfs
74/433

Soalris doesn’t have md5sum, so there’s a workaround
digest -a md5 -v /path/file
Thx to Justin Lim
19/433

If for any reason your vcs node falls into STALE_ADMIN_WAIT, issue
# hasys -force <nodename>
193/433

To determine the maximum size of a shared memory segment, run:
# cat /proc/sys/kernel/shmmax
2147483648
The default shared memory limit for SHMMAX can be changed in the proc file system without reboot:
# echo 2147483648 > /proc/sys/kernel/shmmax
Alternatively, you can use sysctl(8) to change it:
# sysctl -w kernel.shmmax=2147483648
To make a change permanent, add the following line to the file /etc/sysctl.conf (your setting may vary).
This file is used during the boot process.
# echo “kernel.shmmax=2147483648” >> /etc/sysctl.conf
We also got this:
Checking kernel
222/433
parameters
Checking for
semmsl=250; found semmsl=250. Passed
Checking for
semmns=32000; found semmns=32000. Passed
Checking for
semopm=100; found semopm=32. Failed
<<<<
Checking for
semmni=128; found semmni=128. Passed
Checking for
shmmax=536870912; found shmmax=33554432. Failed
<<<<
Checking for
shmmni=4096; found shmmni=4096. Passed
Checking for
shmall=2097152; found shmall=2097152. Passed
Checking for
file-max=65536; found file-max=751357. Passed
Checking for
VERSION=2.6.9; found VERSION=2.6.9-55.0.0.0.2.ELlargesmp.
Passed
Checking for
ip_local_port_range=1024 – 65000; found ip_local_port_range=32768
– 61000. Failed
<<<<
Checking for
rmem_default=262144; found rmem_default=135168. Failed
<<<<
Checking for
rmem_max=262144; found rmem_max=135168. Failed
<<<<
Checking for
wmem_default=262144; found wmem_default=135168. Failed
<<<<
Checking for
wmem_max=262144; found wmem_max=135168. Failed
<<<<
Check complete. The
overall result of this check is: Failed
<<<<
Problem: The kernel
parameters do not meet the minimum requirements (see
above).
Recommendation: Perform
operating system specific instructions to update the kernel
parameters.
223/433
so for semopm we do a:
[root@irwdbd1 ~]# ipcs -ls
—— Semaphore Limits ——–
max number of arrays = 128
max semaphores per array = 250
max semaphores system wide = 32000
max ops per semop call = 32
semaphore max value = 32767
and set it to be changed on reboot as:
[root@server ~]# echo “kernel.sem=250 32000 100 128” >> /etc/sysctl.conf
set it also live as:
[root@server ~]# echo “250 32000 100 128” > /proc/sys/kernel/sem
For the rest, i just added these lines to /etc/sysctl.conf:
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default=262144
net.core.wmem_default=262144
net.core.rmem_max=262144
net.core.wmem_max=262144
and ran a
[root@server ~]# /sbin/sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
kernel.sem = 250 32000 100 128
224/433
kernel.shmmax = 536870912
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 262144
net.core.wmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_max = 262144
You can also run these commands to get the variables oracle recommends to tune:
sysctl -a | grep -i shmmni
sysctl -a | grep -i ‘kernel.sem’
sysctl -a | grep -i ip_local_port_range
sysctl -a | grep -i rmem_default
sysctl -a | grep -i rmem_max
sysctl -a | grep -i wmem_default
sysctl -a | grep -i wmem_max
sysctl -a | grep -i shmmax
You will get something like:
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_default = 262144
net.core.wmem_max = 262144
kernel.shmmax = 2147483648
225/433

if you’re bad at transforming hex to dec or viceversa
http://www.softwareeffect.com/tools/netmask.html
– rdircio
246/433

If you don’t have direct access from your PC to the cluster server, but another unix jumphost has, you can do the same
as for the dracs and the ilos.
The connectivity is
PC-jumphost-clusternode
in your PC you can do a:
ssh -g jumphost -L 14141:clusternode:14141 -L 14150:clusternode:14150
Then, open the “Veritas Cluster Manager” software in your PC, and point it to “localhost”, and you’ll be
managing the remote cluster 🙂
For Veritas Enterprise Administrator (vea), we only need to forward one port:
ssh -g jumphost -L 2148:veanode: 2148

This one replicates all from /200gb/DOCS to the server called “sol” and removes files in “sol” that we deleted locally. the
“c” is for checksum, the “t” is for updating
rsync -crtva –delete /200gb/DOCS/ root@sol:/200gb/DOCS/
if we want to copy locally from /a to /b, but never change /a
rsync -crtva /a/ /b/
32/433

If you want to open, for instance, port 32 to any peer:
# iptables -I INPUT -p tcp –dport 32 -j ACCEPT
85/433

Still on the performance+sysstat topic, I found useful to use “sar -ruq” to obtain memory, cpu and runqueue stats in one
command
root@kraftek:~# sar -ruq 3 10000
Linux 2.6.21.5-smp (kraftek) 02/15/2008
06:51:28 PM CPU %user %nice %system %iowait %steal %idle
06:51:31 PM all 97.01 0.00 2.99 0.00 0.00 0.00
06:51:28 PM kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree kbswpused %swpused
kbswpcad
06:51:31 PM 32528 866560 96.38 42564 178940 0 0 0.00 0
06:51:28 PM runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15
06:51:31 PM 1 256 1.85 2.01 1.37
06:51:31 PM CPU %user %nice %system %iowait %steal %idle
06:51:34 PM all 96.67 0.00 3.33 0.00 0.00 0.00
06:51:31 PM kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree kbswpused %swpused
kbswpcad
06:51:34 PM 32156 866932 96.42 42624 178940 0 0 0.00 0
06:51:31 PM runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15
06:51:34 PM 1 256 1.85 2.01 1.37
Nmon
To add to our previous performance post, we can equip our linux server with nmon, from IBM.
I’m not an IBM fan, this is free, but closed source!!!, despite this, it looks good, get it at:
http://www-941.haw.ibm.com/collaboration/wiki/display/WikiPtype/nmon
131/433

If your HP rilo,rib,ilo card (i don’t get these names) is not accessible, and the server is booted up to linux, you can
configure its address using “hponcfg”
theserver:~ # hponcfg -f rdpm.xml
Firmware Revision = 1.91 Device type = iLO Driver name = cpqci
Script succeeded
Where rdpm.xml has contents:
<RIBCL VERSION=”2.0″>
<LOGIN USER_LOGIN=”currentuser” PASSWORD=”currentpassword”>
<USER_INFO MODE=”write”>
<MOD_USER USER_LOGIN=”Administrator”>
<PASSWORD value=”apassword” />
</MOD_USER>
</USER_INFO>
<RIB_INFO MODE=”write”>
<MOD_NETWORK_SETTINGS>
<ENABLE_NIC value=”Yes” />
<REG_DDNS_SERVER value=”No” />
<PING_GATEWAY value=”No” />
<DHCP_DOMAIN_NAME value=”Yes” />
<SPEED_AUTOSELECT value=”No” />
<NIC_SPEED value=”100″ />
<FULL_DUPLEX value=”Yes” />
<DHCP_ENABLE value=”No” />
<IP_ADDRESS value=”172.168.120.106″ />
<SUBNET_MASK value=”255.255.255.0″ />
<DNS_NAME value=”theserver-ilo” />
<DOMAIN_NAME value=”rilos.com” />
<REG_WINS_SERVER value=”No” />
</MOD_NETWORK_SETTINGS>
</RIB_INFO>
</LOGIN>
</RIBCL>
124/433