clint0922的部落格

第一次嘗試在linux上架設vm，不透過圖形介面，使用CLI來進行架設，在此紀錄一些注意事項，其中也遇到不少問題，日後有新資訊會繼續補上

(一)硬體需求

不需要硬體特別強大的伺服器，使用2個4-core的processor、16GB的Memory，以及160GB的HDD，就可游刃有餘地跑8~10台VM，每台VM使用1虛擬CPU，1GB的Memory、1個VCPU和大約15GB左右的硬碟空間。

(二)虛擬技術

必須使用有支援Virtualization Technology(VT)的CPU，並且在BIOS裡面啟動(此功能為非必須，可以增進VM的效能，還有待查證)

可以藉由下列指令查詢CPU是否有支援VT的指令集

# grep -E 'svm|vmx' /proc/cpuinfo
- vmx is for Intel processors
- svm is for AMD processors

(三)所需RPM

# yum install kvm python-virtinst libvirt libvirt-python virt-manager virt-viewer libguestfs-tools

(四)建立VM的前置作業(使用虛擬的bridge，必須讓一張實體網卡當作虛擬bridge，此張實體卡將不會有實體IP)

(i)先關閉NetworkManager 
chkconfig NetworkManager off
chkconfig network on
service NetworkManager stop
service network start

(ii)先把網卡停用
ifdown eth0

(iii)修改網卡內容,設定檔位置 /etc/sysconfig/network-scripts/ifcfg-eth0(不需要設定IP)
DEVICE="eth0"
BOOTPROTO="none"
NM_CONTROLLED="yes"
ONBOOT="yes"
BRIDGE="br0" 

(iv)新增一個Bridge
/etc/sysconfig/network-scripts/ifcfg-br0
DEVICE="br0"
TYPE="Bridge"
ONBOOT="yes"
DELAY=0
BOOTPROTO="dhcp"


(v)service network restart

(vi)使用brctl檢查birdge是否成功

brctl show

(五)建立VM

自己使用的參數

virt-install --network bridge:br0 --name vm1 --ram=1024 --vcpus=1 --disk path=/vm1.img,size=10 --graphics none --location=ftp://192.168.151.1/CENTOS64x64 --extra-args="ks=ftp://192.168.151.1/final_13.cfg console=tty0 console=ttyS0,115200"

管理VM 

使用virsh

開起vm1

virsh start vm1

關閉

virsh shutdown vm1

List all VMs

virsh list

connect VM

virsh console vm1

切換Host & VM

在vm下 按下鍵盤 " ] + Ctrl "

In order to assign devices in KVM, you’ll need a system which supports VT-d. This has nothing to do with the VT-x support of your CPU, VT-d needs to be supported by both your chipset on your motherboard and by your CPU.

If you are in doubt whether your motherboard or CPU supports VT-d or not, the Xen VT-d wikipage has some pointers of VT-d enabled chipsets, motherboards and CPUs:

http://wiki.xensource.com/xenwiki/VTdHowTo 

If your hardware doesn’t have an IOMMU (“Intel VT-d” support in case of Intel – “AMD I/O Virtualization Technology” support in case of AMD), you’ll not be able to assign devices in KVM. Some work towards allowing this were done, but the code never made it into KVM, due to various issues with the code. At the moment it doesn’t seem like device assignment without hardware support, will ever be integrated into KVM.

Assignment of graphics cards are not officially supported at the moment, but there has been some success passing through a secondary Radeon HD 5850 as a VM’s secondary display.

资料:http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM

在host选择PCI pass through设备

reboot and verify that your system has IOMMU support

AMD Machine

Intel Machine

If you get no output you’ll need to fix this before moving on. Check if your hardware supports VT-d and check that it has been enabled in BIOS.

NOTE: If you still get an error “No IOMMU found.” Check dmesg for errors suggesting your BIOS is broken. Another possible reason: CONFIG_DMAR_DEFAULT_ON is not set. In that case, pass “intel_iommu=on” as kernel parameter to enable it. AMD uses different kernel parameter than Intel, on AMD you need to pass “iommu=pt iommu=1″.

请看附录:No IOMMU found 解决

选择要使用的透传设备

友情提示:使用透传设备时,拿USB控制器作实验,可能鼠标键盘不能使用.请谨慎.

将设备从宿主机上解除绑定

使用echo命令,将设备从host机器上解除绑定,将来用于guest机器. For example:

关闭虚拟机

关闭虚拟机,修改配置文件.

使用libvirt进行pci pass through

修改虚拟机配置文件

<devices> …
    <hostdev mode=’subsystem’ type=’pci’ managed=’yes’>
      <source>
         <address domain=’0x000′ bus=’0x00′ slot=’0x1b’ function=’0x0’/>
      </source>
   </hostdev>
</devices>

在修改完虚拟机配置文件后,运行虚拟机.

使用qemu进行pci pass through

使用qemu进行pci pass through也是一样的简单的.我们需要一个已经创建好操作系统的虚拟机.

在qemu命令行运行以下命令以启动虚拟机

这样就将设备挂载到虚拟机上了.

• 参数-device pci-assign,host=00:1b.0说的是使用一个pci设备,并提供一个设备的地址.
• 参数-monitor stdio是使用一个标准的控制台输出.在命令行中进行输入命令,等等迁移的时候也在这里输入命令.

附录1:No IOMMU found 解决

启动虚拟机的时候出现了iommu的问题.以供大家参考

1.查看错误日志说明

在配置好XML文件后,启动虚拟机,遇到一个问题.

error: Failed to start domain vdisk
error: Unable to read from monitor: Connection reset by peer

查看虚拟机日志( cat /var/log/libvirt/qemu/vdisk.log )信息.

char device redirected to /dev/pts/3
No IOMMU found. Unable to assign device "hostdev0"
qemu-system-x86_64: -device pci-assign,configfd=20,host=00:1b.0,id=hostdev0,bus=pci.0,addr=0x4: Device 'pci-assign' could not be initialized
2013-07-08 06:41:23.256+0000: shutting down

上网查阅资料,说是要在BIOS上设置虚拟化,然后在引导程序里也要设置iommu.可以查看一下自己的电脑信息是否开启了.

2.查看信息gurb的引导信息

可以发现,我的还未开启intel_iommu=on选项.所以接下来我们来激活它.

3.激活intel_iommu=on

Activate Intel VT-d in the kernel

Activate Intel VT-d in the kernel by appending the intel_iommu=on parameter to the kernel line of the kernel line in the/boot/grub/grub.conf file. The example below is a modified grub.conf file with Intel VT-d activated.

对于intel的cpu和amd的cpu,在grub配置上是不同的,具体的配置请参考文章:http://pve.proxmox.com/wiki/Pci_passthrough

4.更新grub

在编辑完grub文件后,需要更新

5.重启电脑,使其生效

发现开机已经启动了inte _iommu=on了.再次启动虚拟机已经就不会出现这个bug了.

附录2 PCI pass through 失败要关闭SELinux

我用的是Fedora 18 ,将SELinux给disalbed了,但是发现PCI pass through出先问题.上网看到文章PCI passthrough fails in qemu-kvm unless selinux is disabled 说,要将selinux设置成permissive模式,于是这个问题解决了.

參考 http://www.chenyudong.com/archives/add-pci-pass-through-device-to-guest-vm-with-libvirt-and-qemu.html

參考資料

http://linux.dell.com/files/whitepapers/KVM_Virtualization_in_RHEL_6_made_easy.pdf

http://benjr.tw/8189

http://34e.cc/355

http://www.chenyudong.com/archives/add-pci-pass-through-device-to-guest-vm-with-libvirt-and-qemu.html