高手帮忙请进,linux系统oracle数据库死机？？？
请教高手帮忙分析一下问题，最近linux服务器经常出现死机，
只装了oracle10g，其他没有装应用程序，
服务器配置。8G内存
CPU 4G
oracle数据库参数：
SGA 1.5G
PGA 750M
shared_pools 900M
Java 16M
large 16M
以下监控服务器参数如下:
[root@localhost ~]# vmstat 2 10;
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
2 0 4521344 316976 6968 3870564 7 27 40 180 93 89 2 4 94 0 0
2 0 4521344 305204 6992 3881092 912 0 912 13210 421 280 11 36 51 2 0
2 0 4521344 292184 7008 3891632 960 0 960 28 422 282 11 36 48 4 0
2 0 4521340 274436 7016 3902276 978 0 980 30 632 376 11 38 46 6 0
3 0 4521336 262408 7036 3912456 880 0 880 13224 557 328 13 33 48 6 0
2 0 4521328 253116 7048 3923136 960 0 964 12 600 373 11 37 47 5 0
2 0 4521324 241344 7072 3933064 864 0 866 13232 624 379 10 36 48 5 0
2 0 4521324 229192 7084 3943656 928 0 928 24 503 327 12 35 48 4 0
3 0 4521320 215048 7092 3954336 974 0 974 18 476 332 10 38 49 2 0
1 1 4521320 203524 7116 3965040 944 0 944 13198 513 339 11 36 48 4 0
[root@localhost ~]# vmstat 2 10;
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
2 0 4508404 15080 7524 4103340 8 27 41 182 93 89 2 4 94 0 0
2 0 4508392 14972 7532 4103448 894 34 900 13188 572 430 13 34 50 4 0
2 0 4508380 14344 7516 4100676 946 1586 948 1618 536 427 11 38 49 3 0
2 0 4508368 14944 7524 4099664 976 812 980 940 472 349 10 37 47 6 0
3 0 4508372 16532 7512 4097752 896 1776 896 14860 436 309 12 34 48 6 0
2 0 4504904 14840 7516 4104960 1304 0 1304 26 500 401 12 37 50 1 0
2 0 4504900 14556 7532 4108264 922 0 922 13186 398 286 11 35 51 2 0
2 1 4504896 14724 7544 4106492 1032 382 1032 412 636 347 12 35 50 2 0
2 0 4504900 14660 7532 4105736 1050 1058 1062 1110 675 435 11 37 49 3 0
2 0 4504904 14284 7532 4103372 912 1210 914 14276 458 310 11 35 51 4 0
TOP命令:
Tasks: 271 total, 3 running, 266 sleeping, 0 stopped, 2 zombie
Cpu(s): 15.9%us, 36.2%sy, 0.0%ni, 43.2%id, 4.5%wa, 0.0%hi, 0.1%si, 0.1%st
Mem: 7793664k total, 7777292k used, 16372k free, 7620k buffers
Swap: 16386260k total, 4503856k used, 11882404k free, 4086828k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
8348 oracle 25 0 5790m 1.9g 38m R 100.1 26.0 14:07.99 oracle
7824 oracle 18 0 2846m 591m 74m R 86.8 7.8 18:20.20 oracle
9319 oracle 15 0 1680m 75m 71m S 2.3 1.0 0:00.61 oracle
8849 oracle 15 0 1680m 83m 79m S 1.3 1.1 0:01.47 oracle
239 root 10 -5 0 0 0 S 0.3 0.0 0:28.09 kswapd0
1 root 15 0 10348 572 544 S 0.0 0.0 0:00.45 init
2 root RT -5 0 0 0 S 0.0 0.0 0:00.10 migration/0
3 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
5 root RT -5 0 0 0 S 0.0 0.0 0:00.03 migration/1
6 root 34 19 0 0 0 S 0.0 0.0 0:00.04 ksoftirqd/1
7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/1
8 root RT -5 0 0 0 S 0.0 0.0 0:00.02 migration/2
9 root 34 19 0 0 0 S 0.0 0.0 0:00.02 ksoftirqd/2
10 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/2
11 root RT -5 0 0 0 S 0.0 0.0 0:00.02 migration/3
12 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/3
13 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/3
14 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/0
15 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 events/1
16 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/2
17 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/3
18 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khelper
19 root 11 -5 0 0 0 S 0.0 0.0 0:00.00 kthread
21 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 xenwatch
22 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 xenbus
27 root 10 -5 0 0 0 S 0.0 0.0 0:00.10 kblockd/0
28 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/1
29 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 kblockd/2
30 root 10 -5 0 0 0 S 0.0 0.0 0:00.03 kblockd/3
31 root 16 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid
145 root 16 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/0
146 root 17 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/1
147 root 17 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/2
148 root 17 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/3
152 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khubd
154 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kseriod
237 root 15 0 0 0 0 S 0.0 0.0 0:02.06 pdflush
238 root 15 0 0 0 0 S 0.0 0.0 0:01.67 pdflush
240 root 16 -5 0 0 0 S 0.0 0.0 0:00.00 aio/0
[root@localhost ~]# iostat 2,3;
Linux 2.6.18-164.el5xen (localhost.localdomain) 2011?1′11?02?￥
avg-cpu: %user %nice %system %iowait %steal %idle
2.16 0.00 4.05 0.32 0.00 93.46
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 16.63 338.25 1487.59 18092563 79568436
sda1 0.00 0.03 0.00 1802 4
sda2 12.56 241.65 1242.80 12925594 66474688
sda3 1.20 21.36 24.75 1142410 1323872
sda4 0.00 0.00 0.00 8 0
sda5 2.87 75.20 220.05 4022325 11769872
avg-cpu: %user %nice %system %iowait %steal %idle
7.24 0.00 18.48 26.59 0.00 47.69
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 978.50 29896.00 3904.00 59792 7808
sda1 0.00 0.00 0.00 0 0
sda2 1.00 4.00 8.00 8 16
sda3 2.00 16.00 4.00 32 8
sda4 0.00 0.00 0.00 0 0
sda5 975.50 29876.00 3892.00 59752 7784
[root@localhost ~]# free -m
total used free shared buffers cached
Mem: 7611 7595 15 0 8 3
581
-/+ buffers/cache: 4005 3605
Swap: 16002 3394 12607
WORKLOAD REPOSITORY report for
DB Name DB Id Instance Inst num Release RAC Host
MES 1822183937 mes 1 10.2.0.4.0 NO localhost.localdomain
Snap Id Snap Time Sessions Cursors/Session
Begin Snap: 7214 02-Nov-11 00:00:33 34 5.1
End Snap: 7216 02-Nov-11 02:00:38 44 5.8
Elapsed: 120.07 (mins)
DB Time: 373.66 (mins)
[解决办法]
alert_sid.log内容
[解决办法]
- SQL code
--通过ID,找出SQLTEXT/* 脚本 getsql.sql 日期 2008-10-23*/SELECT sql_text FROM v$sqltext a WHERE (a.hash_value,a.address) IN ( SELECT DECODE(sql_hash_value,0,prev_hash_value,sql_hash_value), DECODE(sql_hash_value,0,prev_sql_addr,sql_address) FROM v$session b WHERE b.paddr=(SELECT addr FROM v$process c WHERE c.spid='&pid'))ORDER BY piece ASC
[解决办法]
内存的使用量也非常高
[解决办法]
为什么使用那么多交换分区？
什么东西耗费那么多内存？
AWR看看，内存和CPU消耗在哪里
[解决办法]
up
------解决方案--------------------
看现在的内存，死机应该不至于吧。曾经管理过双节点的 9i RAC，高峰时 top 出来的 free mem 通产在 3-4M 也没出现过死机。
1.5G SGA，900M 给了shared pool，似乎有点儿高。
查一下 alert.log 吧