среда, 11 декабря 2013 г.

KVM storage caching modes perfomance comparison

In my everyday work i need perfomance rich KVM VMs on my workstation. I already use virtio drivers for network and storage. But what about tune it more?

First, I've moved my VMs from qcow images to raw LVM volumes.

Next, I've switced disk cache mode to 'unsafe' as most fastest. I don't care about data integrity if smth goes wrong.

But decided to compare different modes. Detailed description about cache modes could be found here.

Cache modes differ mostly by used or not "page cache" of host operating sytem and used or not "write cache" of host hardware disk (See table 1).

Table 1

Disk Write Cache (Host)
Page Cache (Host)
writethrough
×
none
×
writeback
unsafe
Used, but ignores transfer operations
Used, but ignores transfer operations

I've done disk perfomance testing with different cache modes on raw LVM volume. Used bonnie++ benchmark. 5 tests for every cache mode. Full results here.

Table 2

Write (sequental block), MBs
Read (sequental block), MBs
writethrough
35
2337 (page cache)
none
49
103
writeback
45
2381 (page cache)
unsafe
54
2257 (page cache)

Most fastest mode is "unsafe". But do not use "unsafe" mode if you need data integrity.
Optimal mode is "writeback", it has good write perfomance and used page cache, that increases read perfomance a lot.
Safest mode is writethrough(by default in libvirt), but slowest.

Resume:
unsafe cache mode gave me +57% disk perfomance.

Specs:
Disk: Seagate Barracuda 500G (ST500DM002-1BC142)
MB: ASUS P8H67-M EVO
CPU: Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
OS: ARCH linux 3.11.6-1-ARCH x86_64 (libvirt 1.1.4-1, qemu 1.6.1-2)

вторник, 29 октября 2013 г.

Deploy Openstack by FUEL on libvirt: pre-install network configuration

Mirantis has released FUEL 3.2 deployment automation tool for OpenStack. It has a lot of documentation and QuickStart scripts for VirtualBox, but if you prefer libvirt or may be pure KVM, you probably plunge into networking issues or need to read and understand fuel networking. I'll try to explain quick way  how to prepare networking on linux box for FUEL 3.2.

Fuel 3.2 uses 5 networks by default:
  • FUEL- pxe and puppet - 10.20.0.0/24
  • PUBLIC - public APIs and floating range - 172.16.0.0/24 (VLAN 100) - internet access
  • MANAGEMENT - internal OpenStack communications -  192.168.0.0/24 (VLAN 101)
  • STORAGE - iscsi, swift - 192.168.1.0/24 (VLAN 102)
  • FIXED - for openstack projects - 10.0.0.0/16 (VLAN 103 or VLAN range)
You could cultivate all networks in single linux bridge. But it need some magic like routing, NAT and DNS.

1) create bridge
brctl addbr br0
ip link set up dev br0

2) add address and DNS forwarding server. 10.20.0.1 is used by default as dns server.
ip addr add 10.20.0.1/24 dev br0
dnsmasq -a 10.20.0.1

3) add VLAN interface, add gateway and configure NAT for PUBLIC network
ip link add link br0 name br0.100 type vlan id 100
ip link set up br0.100
ip addr add 172.16.0.1/24 dev br0.100
iptables -t nat -A POSTROUTING -s 172.16.0.0/24 ! -d 172.16.0.0/24 -j MASQUERADE

4) You should create VMs w/ single interface attached to br0.

After that install Fuel master node from iso. And follow steps in documentation.

среда, 26 июня 2013 г.

Arch Linux with Awesome wm

Weeks ago i've switched from Ubuntu to Arch Linux. Why? I don't want to depend from some commercialized way of Ubuntu, also close to me Arch principles "... elegance, code correctness, minimalism, and simplicity ..." (see https://en.wikipedia.org/wiki/Arch_Linux and https://en.wikipedia.org/wiki/KISS_principle). Why not gentoo? ha,ha, it is to hardcore for me. Arch has a very useful wiki. You could start from this guide.

Let's try to install Arch with Awesome window manager.

WARN: Please try it on Virtual machine first.

Installing base system

1. Create bootable USB (https://wiki.archlinux.org/index.php/USB_Installation_Media)



wget http://mirror.yandex.ru/archlinux/iso/2013.06.01/archlinux-2013.06.01-dual.iso
dd bs=4M if=archlinux-2013.06.01-dual.iso of=/dev/sdx

Where "/dev/sdx" NOT mounted USB stick.

2. Boot from this stick. It prompts you into root console.

3. Partitioning with LVM.

Use "lsblk" to find target block device (i.e. /dev/sda). I prefer fdisk for partitioning. Command below will create two partitions (200M and all remain space).

echo -e "o\nn\n\n\n\n+200M\nn\n\n\n\n\nw\n"|fdisk /dev/sda

mkfs.ext2 /dev/sda1
pvcreate /dev/sda2
vgcreate vgroot /dev/sda2
lvcreate -n root -L 10G vgroot
lvcreate -n swap -L 1G vgroot
lvcreate -n home -L 5G vgroot
mkfs.ext4 /dev/vgroot/root
mkfs.ext4 /dev/vgroot/home
mkswap /dev/vgroot/swap
mount /dev/vgroot/root /mnt
mkdir /mnt/{boot,home}
mount /dev/vgroot/home /mnt/home
mount /dev/sda1 /mnt/boot
swapon /dev/vgroot/swap

4. Connect to the Internet. 

By default DHCP service is enabled, but if you need some other network settings (i.e. static or wireless) read this. To check Internet:

ping -c 3 www.google.com


5. Install base system.

Enable preferred mirror in /etc/pacman.d/mirrorlist and

pacstrap /mnt base base-devel


6. Configure system.

genfstab -p /mnt >> /mnt/etc/fstab
arch-chroot /mnt
echo "myhostname" > /etc/hostname
ln -s /usr/share/zoneinfo/Europe/Moscow /etc/localtime
sed -i -e 's/#ru_RU.UTF-8/ru_RU.UTF-8/' -e 's/#en_US.UTF-8/en_US.UTF-8/' /etc/locale.gen
locale-gen
echo 'LANG="en_US.UTF-8"' > /etc/locale.conf

Edit /etc/mkinitcpio.conf. Add "lvm2" hook between "block" and "filesystems".

...
HOOKS="base udev autodetect modconf block lvm2 filesystems keyboard fsck"
...

mkinitcpio -p linux

Set root password with "passwd" and create regular user:

useradd -m -s /bin/bash user1
passwd user1

Enable DHCP service:

systemctl enable dhcpcd.service


7. Installing GRUB.

See also https://wiki.archlinux.org/index.php/GRUB

grub-install --recheck /dev/sda
grub-mkconfig -o /boot/grub/grub.cfg


8. Umount and reboot

exit
umount /mnt/{boot,home,}

Installing Awesome wm

This is cool tiling window manager. Official page http://awesome.naquadah.org/.

Execute commands below from root.

1. Install video driver.

Detailed instruction here.

pacman -S xf86-video-vesa

For nvidia "xf86-video-nv", ATI "xf86-video-ati"

2. Install X server, awesome wm and slim dm.

pacman -S xorg-server xorg-server-utils xorg-xinit xterm awesome slim
systemctl enable slim.service

3. Configure xinitrc.

su - user1
echo "exec awesome" > .xinitrc
mkdir -p .config/awesome
cp /etc/xdg/awesome/rc.lua .config/awesome/
reboot

4. Using awesome.

For advanced usage should read official wiki

Keys:

win+{1, 2, 3,.., left, right} - switch between tags (like screens in gnome)
win+r - execute command (i.e. "firefox")
win+enter - open terminal
win+{j, k} - switch between windows
win+tab - switch to prev. window
win+w - open awesome menu

rc.lua:

In internet a lot of info about customizing rc.lua. Here is only simple example. This will set capslock key to switch keyboard layout.

echo 'awful.util.spawn_with_shell("setxkbmap -layout 'us,ru' -option 'grp:caps_toggle,grp_led:scroll'")' >> .config/awesome/rc.lua

Check syntax:

awesome -k .config/awesome/rc.lua

Then type "ctrl+win+r" to restart awesome.

Now you are free to install necessary software with "pacman", customize awesome with "rc.lua", etc. ,etc. ,etc.

понедельник, 10 июня 2013 г.

Rebuild CirrOS tiny cloud image for VMware

We've faced problems when tried to boot CirrOS tiny cloud image as guest OS on VMware ESXi managed by OpenStack.

Problem description: It boots up without any storage disk or NIC. By default OpenStack (with VMwareESXDriver) uses LSILogic SCSI controler (Fusion MPT) for VMware guests. Seems like there is no drivers(modules) for this type of SCSI controler. Let's try to add it.

To add this additional drivers we need to rebuild CirrOS as described in official README . I've done all stuff on Ubuntu 12.04 LTS.

* Get build dependencies, clone source code repo.

sudo apt-get -y install bison flex texinfo build-essential gettext ncurses-dev unzip bzr qemu-kvm cvs quilt
bzr branch lp:cirros
cd cirros

* Download buildroot and setup environment and apply local cirros patches to it.

br_ver="2012.05"
mkdir -p ../download
ln -snf ../download download
( cd download && wget http://buildroot.uclibc.org/downloads/buildroot-${br_ver}.tar.gz )
tar -xvf download/buildroot-${br_ver}.tar.gz
ln -snf buildroot-${br_ver} buildroot
./bin/mkcabundle > src/etc/ssl/certs/ca-certificates.crt
( cd buildroot && QUILT_PATCHES=$PWD/../patches-buildroot quilt push -a )
make ARCH=i386 br-source

 * Build buildroot for a given arch. ARCH should be set to 'i386', 'x86_64' or 'arm'.
make ARCH=i386 OUT_D=$PWD/output/i386

This will do a full buildroot build, which will take a while. The output that CirrOS is interested in is output/i386/rootfs.tar. That file is the full buildroot filesystem, and is used as input for subsequent steps here.

 * Download a kernel to use. I've used ubuntu virtual i386 kernel.

wget https://launchpad.net/ubuntu/+archive/primary/+files/linux-image-3.2.0-41-virtual_3.2.0-41.66_i386.deb -O download/linux-image-3.2.0-41-virtual_3.2.0-41.66_i386.deb

* Patch cirros bundle script. In simple, this script used to rebuild rootfs.tar with custom kernel. It creates two initramfs files: "standart" and "smaller" w/o kernel modules. By default it uses second one. To prevent it and use standart initramfs do:
sed -i 's/cp "${initramfs}.smaller"/cp "${initramfs}"/' bin/bundle

* Add necessary kernel modules. Bundle script gets modules to include in image from "src/etc/modules" . For Fusion MPT devices (i.e. lsiLogic scsi controller) support add "mptbase" and "mptscsih" modules:

echo -e "mptbase\nmptscsih" >> src/etc/modules

*  build disk images using special script bin/bundle

sudo ./bin/bundle -v output/$ARCH/rootfs.tar download/linux-image-3.2.0-41-virtual_3.2.0-41.66_i386.deb output/$ARCH/images
Resulting images located in output/i386/images . You could use bootable qcow2 image "output/i386/images/disk.img" or ami (blank.img), ari (initramfs), aki (kernel). OpenStack supports now only flat vmdk images for vmware and we need to convert qcow2 to raw:

qemu-img convert -O raw output/i386/images/disk.img output/i386/images/disk-raw.img

How to add image to glance:
glance add name="cirros-vmware" disk_format=raw container_format=bare is_public=true vmware_adaptertype="lsiLogic" vmware_disktype="preallocated" vmware_ostype="otherGuest" < disk-raw.img
 

понедельник, 3 июня 2013 г.

How to upload files thru rdp from linux

If you want to copy files from Linux to remote Windows machine you could do it thru RDP (rdesktop utility).

rdesktop -g 90% -r disk:mydir=/home/user/mydir 10.0.0.2

It will open graphical RDP session and you could see your shared folder from "My Computer" page.

вторник, 5 февраля 2013 г.

Ubuntu splits all after 127.0.0.1 in dns-nameservers flag

Ubuntu manages local DNS settings by "resolvconf" package and settings could be set in /etc/network/interfaces file.

For example
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
  iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
  address 192.168.122.20
  netmask 255.255.255.0
  gateway 192.168.122.1
  dns-nameservers 127.0.0.1 8.8.8.8
  dns-search local.lan cloud.lan
In detail "man interfaces" or "man resolvconf".


I have local dns server on 127.0.0.1. And i've plunged in strange behavior with "dns-nameservers" flag.

If i set
dns-nameservers 8.8.8.8 127.0.0.1
It works like a charm.
cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 8.8.8.8
nameserver 127.0.0.1
search local.lan cloud.lan

But if I put 127.0.0.1 as first nameserver
dns-nameservers 127.0.0.1 8.8.8.8
There is no nameservers after 127.0.0.1 ...
cat /etc/resolv.conf 
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.0.1
search local.lan cloud.lan
But I want to be able to use local dns as preferable plus other public nameserver.

What i've found here /etc/resolvconf/update.d/libc (this file related to resolvconf package)
...
# Set TRUNCATE_NAMESERVER_LIST_AFTER_LOOPBACK_ADDRESS=no
# to allow additional nameserver addresses to be listed in
# resolv.conf after an initial loopback address 127.* or ::1.

...

SOLUTION:
echo "TRUNCATE_NAMESERVER_LIST_AFTER_LOOPBACK_ADDRESS=no" >> /etc/default/resolvconf

Used ubuntu 12.04 LTS