Add a system call to the linux kernel in Ubuntu

1) Install the tools to build a kernel:

sudo apt-get install fakeroot build-essential crash kexec-tools makedumpfile kernel-wedge

sudo apt-get build-dep linux

sudo apt-get install git-core libncurses5 libncurses5-dev libelf-dev asciidoc binutils-dev

sudo apt-get build-dep --no-install-recommends linux-image-$(uname -r)

apt-get source linux-image-$(uname -r)

2) Get the kernel to edit:

sudo apt-get install linux-source

mkdir ~/src

cd ~/src

tar xjvf /usr/src/linux-source-<version-number-here>.tar.bz2

cd linux-source-<version-number-here>

*or go to kernel.org and download version you wish, uncompress it and store in /usr/src(see notes below)

3) Edit the kernel:

These are files that will have to be edited or created for the system call to work.

a) change syscall table

folder: arch/x86/syscalls/

edit syscall_32.tbl for 32-bit processors:

at end of file add: 350 i386 myservice sys_myservice

(might not be 350 for your linux kernel)



*or edit syscall_64.tbl for 64-bit processors:

at end of non-specific system call numbers add: 313 64 myservice sys_myservice



b) add header file to include folder:

folder: include/linux/syscalls.h

at very end of file(after #endif) add: asmlinkage int sys_myservice



c) create the system call:

folder: /kernel

create the system call and save as myservice.c:

#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/syscalls.h>
#include <linux/linkage.h>

asmlinkage int sys_myservice(void) {
printk(KERN_EMERG "my service is running");
return 0;
}

d) create the Kconfig file:

folder: /kernel

create the Kconfig and save as Kconfig.myservice:

config MYSERVICE
bool "prints my service is running"
default y
help
This will print my service is running

e) edit the Makefile of Kernel directory:

folder: /kernel

edit Makefile: add myservice.o to obj-y list



f) edit Makefile of kernel

folder: linux-<version-number-here>

edit Makefile at .extraversion line: EXTRAVERSION = .syscall(or whatever you want)

4) Create the config file:

cp -vi /boot/config-`uname -r` .config(this should be different config name, so look up /boot/configsomething instead of uname -r)

make menuconfig

5) Build the kernel:(this will take a while so maybe make a cup of tea or coffee. also this is safer in fakeroot)

export CONCURRENCY_LEVEL=3 #this is for dual processors quadcore would be...

make-kpkg clean # only needed if you want to do a "clean" build

fakeroot make-kpkg --initrd --append-to-version=-some-string-here kernel-image kernel-headers

echo vesafb | sudo tee -a /etc/initramfs-tools/modules

echo fbcon | sudo tee -a /etc/initramfs-tools/modules

6) Install the kernel:

sudo dpkg -i linux-image-2.6.20-16-2be-k7_2.6.20-16_i386.deb(your version not 2.6.20...)

sudo dpkg -i linux-headers-2.6.20-16-2be-k7_2.6.20-16_i386.deb(same here)

sudo cp /usr/share/doc/kernel-package/examples/etc/kernel/postinst.d/initramfs /etc/kernel/postinst.d/initramfs

sudo mkdir -p /etc/kernel/postrm.d/

sudo cp /usr/share/doc/kernel-package/examples/etc/kernel/postrm.d/initramfs /etc/kernel/postrm.d/initramfs

7) Change grub to show your new kernel at boot:

cd /boot

sudo mkinitramfs -k -o initrd.img-2.6.32.15+drm33.5-mylucid 2.6.32.15+drm33.5-mylucid(your initrd.img-3.5.something....)

sudo update-grub2

8) Boot into new kernel:

a) restart computer and hold down shift to get to grub menu

b) select Advaced options for Ubuntu

c) select your version of the kernel

9) Create the file to test system call:

a) create the source file to save system call

create file named test_myservice.c:

#include <stdlib.h>

int myservice() {
int ret;
__asm__("movl $350,%eax");
__asm__("int $0x80");
__asm__("movl %eax, -4(%ebp)");
return ret;
}

int main()  {
int ret;
printf("invoking system call myservice...\n");
ret = myservice();
if(ret<0) {
printf("not working");
exit(1);
}
printf("system call executed.\n");
return 0;
}

b) compile the source and

gcc -o test_myservice.c myservice

c) run the service

./myservice

10) Check the call was logged in the kernel error log:

folder: /var/log/

open: kern.log

search for: "my service"

Helpful links:

http://blog.techveda.org/index.php/adding-system-calls-linux-kernel-3-5-x/

https://help.ubuntu.com/community/Kernel/Compile

http://kernel.org/

http://kernelnewbies.org/KernelBuild

http://www.wiley.com/college/sc/silberschatz/projectch2.pdf

Notes:

*the method presented is a mix and match of the web pages listed above and a few others

you might need to mess with it to get it to work correctly for your system

*make sure computer is connected to internet

*to enter directory as root:

alt+F2

gksudo nautilus /directory

or

alt+F2 gksu nautilus

*to compile something with multicore processors and speed up build:

example for dual core:

export CONCURRENCY_LEVEL=3

*or -j 3

number should be 1 + number of cores your processor has

*to list kernel version:

uname -r

*edit source file:

install VIM