Ubuntu 16.04 Kernel Crash Dump


Ubuntu 16.04 Kernel Crash Dump

1. apt install linux-crashdump

2. /etc/default/kdump-tools by including the following line:

USE_KDUMP=1

3.  kdump-config show

DUMP_MODE:        kdump

USE_KDUMP:        1

KDUMP_SYSCTL:     kernel.panic_on_oops=1

KDUMP_COREDIR:    /var/crash

crashkernel addr: 0x2b000000

   /var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinuz-4.4.0-89-generic

kdump initrd:

   /var/lib/kdump/initrd.img: symbolic link to /var/lib/kdump/initrd.img-4.4.0-89-generic

current state:    ready to kdump

kexec command:

  /sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-4.4.0-89-generic root=UUID=505002b4-e453-4e7c-9eb5-a9af5ccf1391 ro irqpoll nr_cpus=1 nousb systemd.unit=kdump-tools.service" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz

4. cat /proc/cmdline

BOOT_IMAGE=/boot/vmlinuz-4.4.0-89-generic root=UUID=505002b4-e453-4e7c-9eb5-a9af5ccf1391 ro crashkernel=384M-:128M

crashkernel=384M-2G:64M,2G-:128M

The above value means:

1).if the RAM is smaller than 384M, then don't reserve anything (this is the "rescue" case)

2).if the RAM size is between 386M and 2G (exclusive), then reserve 64M

3).if the RAM size is larger than 2G, then reserve 128M

5. cat /proc/sys/kernel/sysrq

If a value of 0 is returned the feature is disabled. Enable it with the following command :

sudo sysctl -w kernel.sysrq=1

6. echo c > /proc/sysrq-trigger

[   31.659002] SysRq : Trigger a crash

[   31.659749] BUG: unable to handle kernel NULL pointer dereference at           (null)

[   31.662668] IP: [<ffffffff8139f166>] sysrq_handle_crash+0x16/0x20

[   31.662668] PGD 3bfb9067 PUD 368a7067 PMD 0

[   31.662668] Oops: 0002 [#1] SMP

[   31.662668] CPU 1

....

7.  ls /var/crash/

201708070427  kexec_cmd  linux-image-4.4.0-89-generic-201708070427.crash

8. apport-unpack /var/crash/linux-image-4.4.0-89-generic-201708070427.crash ~/tmp/

9. crash needs d gsym to work. Check whether debug directory exist in /usr/lib or not.If not, download it from
http://ddebs.ubuntu.com/pool/main/l/linux/

The dbgsym should be the same with the kernel version.

10. crash /usr/lib/debug/boot/vmlinux-4.4.0-89-generic  /var/crash/201708070427/dump.201708070427