{{Header}}
{{#seo:
|description=Development Notes for adding KVM support to {{project_name_long}}.
}}
{{intro|
Development Notes for adding KVM support to {{project_name_long}}.
}}
= Links =
* [[KVM|{{project_name_short}} KVM documentation]]
* [https://phabricator.whonix.org/project/board/16/ KVM workboard]
* [https://phabricator.whonix.org/maniphest/query/qmC_a5t_IZ13/#R KVM {{project_name_short}} Bug Tracker]
* [https://github.com/{{project_name_short}}/whonix-libvirt whonix-libvirt github]
= Continuous Integration =
* [https://travis-ci.org/github/{{project_name_short}}/{{project_name_short}}-libvirt Travis CI]
* https://www.whonix.org/old-forum/index.php/topic,829.0.html forum thread
== Converting QEMU command line args to libvirt domain xml==
https://stackoverflow.com/questions/26267173/converting-qemu-command-line-args-to-libvirt-domain-xml#28588704
{{CodeSelect|code=
virsh domxml-from-native qemu-argv
}}
= Blockers =
None.
= Non-Blockers =
== Bullseye and Beyond ==
=== Test headless admin on Bullseye ===
https://etbe.coker.com.au/2016/12/24/video-mode-kvm/
QEMU changes will likely need kernel commandline adjustment to adjust and re-enable console video mode.
== Document new features and open relevant upstream tickets ==
== Canokey ==
https://www.qemu.org/docs/master/system/devices/canokey.html#id9
https://github.com/canokeys/canokey-qemu/issues/6
https://qemu.readthedocs.io/en/v8.2.1/system/devices/canokey.html
https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg03913.html
https://forums.whonix.org/t/canokey-kvms-emulated-gpg-smartcard/19981
Should supercede the emulated TPM + swtpm-tools suggested here.
* Opened a Debian RFP: https://lists.debian.org/debian-wnpp/2024/06/msg00422.html
* Asked Canokey devs to clarify the security properties of the virtual implementation: https://github.com/canokeys/canokey-qemu/issues/8
== VirtIO Bluetooth ==
Unsure about its usecases but cool to have anyhow.
https://www.phoronix.com/news/VirtIO-Bluetooth-Next
==QEMU LLVM Support==
https://www.phoronix.com/news/QEMU-6.0-Released
Interesting is the LLVM CFI/LTO support that can be leveraged for more security. open a Debian feature request asking if they plan to build it with these flags.
== aarch64 ==
Packages needed on x86 host:
Debian built Openstack images tailored for KVM environments can be fetched from here:
https://cdimage.debian.org/cdimage/openstack/
These images don't ship with a DE but were essential in coming up with a VM config that works. Here's the XML for a machine that successfully booted and reached the systemc console with minimal modifcations from defaults. Note that on an actual aach64 host, the domain type would need to be set to 'kvm' instead to run without CPU emulation. More modifications may be needed to this section, but it woun;t be possible to know without hands-on experience:
== VirtIO Sound ==
Now supported in Linux 5.13. Move on from the current emulated device whenever it's available in stable.
https://www.phoronix.com/news/VirtIO-Sound-Driver-Linux-5.13
== Virtio-Video ==
Paravirtual webcam access and screen sharing capability soon to land in KVM. Worth documenting when ready.
https://www.phoronix.com/news/VirtIO-Video-2020
== Emulated TPM ==
KVM has recently added virtualized TPM support. They can be very valuable from a security standpoint (and more trustworthy than hardware TPMs). A big thing is they can be used like virtual smartcards providing security like Qubes' split-gpg but for more types of software keys like OpenSSH, OpenSSL and more.
To boot a VM after adding the emulated TPM virtual hardware, on the host install:
sudo apt install swtpm-tools
'''Update:'''
* GPG 2.3 now supports TPM2 https://www.gnupg.org/blog/20210315-using-tpm-with-gnupg-2.3.html
* OpenSSH has added TPM2 support and can import existing keys as of 1.7.0-rc0https://github.com/tpm2-software/tpm2-pkcs11/releases/tag/1.7.0-rc0
https://blog.ledger.com/ssh-with-tpm/
The upstream TPM2 project is looking at consolidating the multiple code projects out there into an upstream implementation superseding the projects below.
https://github.com/tpm2-software/tpm2-tools/issues/518
https://github.com/tpm2-software/tpm2-pkcs11
To summarize in someone else's words:
TPMs are useful as an
alternative to key cards: they provide the same security against key
theft and the same cryptographic protections. The main difference is
that TPMs are universally present in every laptop, so they provide a
simple and ubiquitous solution to key security. The only real down
side is that unlike key cards, TPM protected keys cannot be transferred
between laptops, you must instead keep an offline backup copy of the
key can then be transferred to the TPM of any new laptop.
The way TPM protection works is slightly different from key cards.
Instead of moving the key inside the card, the TPM converts any given
key to a TPM specific representation (meaning it's encrypted by a
special key that only the TPM possesses). The TPM represenation must
be stored offline somewhere and if it is lost, so is the protected key.
The way I implemented this is to use the TPM to convert the key to
protected representation and then store it in the shadow_info of a
shadowed-private-key using a shadow type of tpm2-v1. The TPM can
handle an arbitrary number of keys, but the price is the shadow_info
stores the keys and must be preserved.
https://lwn.net/Articles/768419/
https://blog.hansenpartnership.com/using-your-tpm-as-a-secure-key-store/
Patches for GPG support for TPMs https://lists.gnupg.org/pipermail/gnupg-devel/2018-January/033350.html has been submitted but I have yet to see what happened. Needs following up for sure so it can make it before the next freeze hopefully.
To be done:
* How to secure OpenSSH keys
* Suggesting Tor supports this for Onion service keys and even session key material if TPM detected.
Contents of TPM can be encrypted in the libvirt version in Bullseye.
This nifty code supports the major applications on TPM 2
https://github.com/irtimmer/tpm2-pk11
https://github.com/irtimmer/tpm2-pk11/wiki RFP: https://bugs.debian.org/941951
* OpenSSH Client (SSH key in TPM)
* Firefox (Private key of Client certificate in TPM)
* GnuPG using gnupg-pkcs11-scd (PGP key in TPM)
https://phabricator.whonix.org/T931
https://packages.debian.org/{{Stable project version based on Debian codename}}/simple-tpm-pk11
simple-tpm-pk11 provides tools to create a key in your TPM (Trusted Platform Module) chip which can then be used with SSH. The package comes with a library that you can use as “PKCS11Provider” in your SSH configuration file.
swtpm binary missing in Debian Buster and hasn't been packaged yet.
https://listman.redhat.com/archives/libvirt-users/2019-October/msg00008.html
Opened a RFP.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=941939
GPG 2.3 has added support for binding keys to TPM2 chips. Needs to be tested if it works with the vTPM.
https://www.phoronix.com/news/GnuPG-2.3-Released
== VirtIO-FS ==
As of Linux 5.4 a newer and faster implementation of shared folder filesystems has been merged upstream. This replaces the 9p fs requirement with a shared memory design that has better performance. Will look into changing shared folder service in the VM to use it by Bullseye.
https://www.phoronix.com/news/VirtIO-FS-Linux-5.4-File-System
https://fosdem.org/2020/schedule/event/vai_virtio_fs/attachments/slides/3666/export/events/attachments/vai_virtio_fs/slides/3666/virtio_fs_A_Shared_File_System_for_Virtual_Machines_FOSDEM.pdf
There could be inferior security of this design compared to the 9p architecture. This uses the fuse protocol and allows direct access to host memory pages for increased performance. A Rust Implementation of virtiofsd is planned.
https://blog.vmsplice.net/2020/04/virtio-fs-has-landed-in-qemu-50.html
== Image Rebasing ==
https://documentation.suse.com/sles/11-SP4/#redirectmsg
https://web.archive.org/web/20170523161045/http://nairobi-embedded.org/manipulating_disk_images_with_qemu-img.html
Play with image rebasing. Advantages: disk space savings, one time update for all derived images.
This feature can be used to create what Qubes calls "templates".
== NAT Port Forwarding ==
Instructions for port forwarding to GW. Useful if configuring GW as Tor relay. Relevant if
/when this is supported in {{project_name_short}}
https://wiki.libvirt.org/page/Networking#Forwarding_Incoming_Connections
==Virgl3D==
Still cannot enable on Buster because of an QEMU process restrictions which were fixed upstream, but unfortunately after the freeze.
https://bugzilla.redhat.com/show_bug.cgi?id=1659484
https://www.kraxel.org/blog/2016/09/using-virtio-gpu-with-libvirt-and-spice/
[https://www.kraxel.org/blog/2019/09/display-devices-in-qemu/ Quote] Gerd Hoffman (libvirt dev) ''vhost-user virtio gpu'':
This is good from the security perspective, especially if you want use virgl 3D acceleration, and it also helps with opengl performance.
QXL has been superseded by virtio-vga where most development happens (and security attention probably). 2D acceleration as a distinct mechanism is outdated and 3D mode is where all types of graphical acceleration will be handled:
This device has support for 2D acceleration. This becomes more and more useless though as modern display devices don't have dedicated 2D acceleration support any more and use the 3D engine for everything. The same happens on the software side, modern desktops are rendering with opengl or vulkan instead of using 2D acceleration.
VMs with the same or related OS have the same canvas fingerprint. So Firefox in all Debian, Ubuntu etc VMs have the same fingerprint. Tor browser blocks fingerprinting, but Iceweasel doesn't. But if some bug allowed canvas fingerprinting in {{project_name_short}} Tor Browser, it would be the same as Firefox in another Debian-related VM.
Asked upstream for help testing this:
[https://listman.redhat.com/archives/libvir-list/2016-December/msg01157.html libvirt: Do different Virgil3D users running have the same CANVAS fingerprint? bancfc at openmailbox.org]
Given that the 3D enabled virtio-gpu is also a virtual GPU it should show the same model for all guests and therefore I predict it will be the same as with results of virtualized graphics without this feature.
https://listman.redhat.com/archives/libvir-list/2017-January/msg00296.html
Vulkan support is being added by Google to the virtio-gpu
https://www.phoronix.com/news/VirtIO-GPU-Venus-Vulkan
Demi Marie Obenour (Qubes core developer) [https://github.com/QubesOS/qubes-issues/issues/4234#issuecomment-1664860957 quote]:
VirGL and Venus run the userspace driver (OpenGL and Vulkan respectively) on the host. This means that they provide a hardware-independent API to the guest, but also means that the entire userspace stack becomes guest-accessable attack surface. This attack surface is very large, and Linux graphics developers have stated on IRC that it is not a security boundary. Therefore, @marmarek has decided that Qubes OS will never use VirGL or Venus and I agree with his decision.
virtGPU native contexts, on the other hand, expose the kernel ioctl API to the guest. This API is accessible to unprivileged userspace processes, which means it is a supported security boundary. It is also much smaller than the OpenGL or Vulkan APIs, which means that its attack surface is vastly smaller. As a bonus, native contexts offer near-native performance, which should be enough even for games and other demanding tasks.
The kernel ioctl API (also known as the userspace API or uAPI) is hardware-dependent, so virtGPU native contexts are only supported on a subset of hardware. Currently, Intel, AMD, and Adreno GPUs are supported using the upstream i915, amdgpu, and freedreno drivers.
Xen supports grant-based virtio, so virtio-GPU should not be incompatible with running QEMU in a stubdomain. The virtio-GPU emulator will need to run in dom0, but its job is much simpler than that of QEMU (it is only emulating a single device) and so the attack surface should (hopefully!) be acceptable.
todo: read https://www.kraxel.org/blog/2021/05/virtio-gpu-qemu-graphics-update/
== Audit Output of virsh domxml-to-native ==
Does it look sense? Do we understand all of it? Is there anything we should change?
HulaHoop: Output makes sense. Needs to be updated because of recent config changes. Some upcoming.
See [https://travis-ci.org/github/{{project_name_short}}/{{project_name_short}}-libvirt CI log] for output of running against other xml files.
- Tested the same command on Debian+KVM+GW,WS,Kicksecure and the output is:
=== Whonix-Gateway ===
== Kernel Direct Boot ==
Minor importance.
Document how to do a kernel direct boot. For passing special options for debugging options such as disabling apparmor in kernel. Interesting, because even if grub was broken for some reason, VM would still be bootable.
HulaHoop: There is an example here: https://libvirt.org/formatdomain.html#direct-kernel-boot feel free to change it to suit {{project_name_short}}.
== Hardware side-channel research ==
Ongoing. Interesting developments worth watching out for at https://gruss.cc
== Host to VM Communication ==
Name of chapter "Host to VM Communication" might be wrong. Any better idea?
VirtualBox supports VBoxManage setextradata which can be used on the host to define a pair of setting and content. The host can set arbitrary information which can be used inside the VM.
https://www.virtualbox.org/manual/ch09.html#changeacpicust
Does KVM support something similar?
This would be useful for Whonix-Host. Some setting could be set on Whonix-Host which then could be read inside {{project_name_short}} KVM VMs. That would be useful to spare users from going again through setup-dist (disclaimer) and perhaps anon-connection-wizard (ACW).
This also talks about KVM: https://serverfault.com/questions/727347/solidworks-activation-license-mode-is-not-supported-in-this-virtual-environment
Maybe the same mechanism could be used.
What is type=0 vs type=1?
Other alternatives... Or does KVM nowadays support SLIC tables?
Something else.
Not good: network based communications. To keep this simple/reliable some of the above mechanisms would be better.
Forum discussion: https://forums.whonix.org/t/host-to-kvm-vm-communication-using-slic-setextradata/10619
= In Progress =
== Shared Folder ==
Run as non-root!
{{CodeSelect|code=
groups
}}
Check if being a member of groups:
* libvirt
* kvm
Create folder /mnt/shared.
{{CodeSelect|code=
sudo mkdir -p /mnt/shared
}}
Optional. Should already be the case. Make sure folder /mnt is owned by user and group root.
{{CodeSelect|code=
sudo chown root:root /mnt
}}
Optional. Should already be the case. Make sure folder /mnt is readable by any linux user account and linux user group ("public").
{{CodeSelect|code=
sudo chmod o+r /mnt
}}
https://askubuntu.com/questions/862289/difference-between-executable-directory-vs-executable-files
{{CodeSelect|code=
sudo chmod o+x /mnt
}}
Set owner of folder /mnt/shared to user libvirt and group kvm.
{{CodeSelect|code=
sudo chown --recursive libvirt:kvm /mnt/shared
}}
Permission hardening. Prevent "public" from reading the shared folder.
{{CodeSelect|code=
sudo chmod --recursive o-rwx /mnt/shared
}}
Allow folder /mnt/shared to be readable and writeable by user libvirt and group kvm.
{{CodeSelect|code=
sudo chmod --recursive ug+rw /mnt/shared
}}
https://askubuntu.com/questions/862289/difference-between-executable-directory-vs-executable-files
{{CodeSelect|code=
sudo chmod ug+x /mnt/shared
}}
= Done =
== QCOW2 Preallocation Settings ==
Preallocating metadata speeds up disk use in guest at expense of time during disk creation.
https://web.archive.org/web/20220929043923/https://www.jamescoyle.net/how-to/2060-qcow2-physical-size-with-different-preallocation-settings
Other options like falloc and full cause the disk to loose sparse properties.
== QCOW2 L2 Subcluster Allocation ==
https://blogs.igalia.com/berto/2020/12/03/subcluster-allocation-for-qcow2-images/
Problem:
Because of that qcow2 images with larger cluster sizes tend to:
grow faster, wasting more disk space and duplicating data. increase the amount of necessary I/O during cluster allocation,reducing the allocation performance.
Why reducing cluster size is not a solution:
Unfortunately, reducing the cluster size is in general not an option because it also has an impact on the amount of metadata used internally by qcow2 (reference counts, guest-to-host cluster mapping). Decreasing the cluster size increases the number of clusters and the amount of necessary metadata. This has direct negative impact on I/O performance, which can be mitigated by caching it in RAM, therefore increasing the memory requirements (the aforementioned post covers this in more detail).
The new '''extended L2 entries''' feature in QEMU 5.2+ gives us the best of all worlds:
*Sixteen times less metadata per unit of allocation, greatly reducing the amount of necessary L2 cache.
*Much faster I/O during allocating when the image has a backing file, up to 10-15 times more I/O operations per second for the same cluster size in my tests (see chart below).
*Smaller images and less duplication of data.
How to use:
In order to test this you simply need to create an image with extended_l2=on, and you also probably want to use a larger cluster size (the default is 64 KB, remember that every cluster has 32 subclusters). Here is an example:
$ qemu-img create -f qcow2 -o extended_l2=on,cluster_size=128k img.qcow2 1T
And that’s all you need to do. Once the image is created all allocations will happen at the subcluster level.
Change commit:
https://github.com/adrelanos/Whonix/pull/1/commits/16f5404eb929b6748178948ceb509211c95a016c
== Biometric Fingerprinting ==
This problem is related to all hypervisors and noted here to keep track. New info added not in mail list message:
Upstream tickets:
https://lists.nongnu.org/archive/html/qemu-discuss/2016-03/msg00023.html
https://web.archive.org/web/20161002204349/https://secure-os.org/pipermail/desktops/2016-March/000108.html
https://bugzilla.redhat.com/show_bug.cgi?id=1318813
https://marc.info/?l=linux-kernel&m=145877344732456&w=2
https://lists.freedesktop.org/archives/wayland-devel/2016-March/027607.html
https://lists.x.org/archives/xorg-devel/2016-March/049159.html
https://www.phoronix.com/news/Anti-Keystroke-FP-Wayland-Kern
https://mailman.stanford.edu/pipermail/liberationtech/2016-March/015947.html
https://web.archive.org/web/20170222094419/https://mailman.boum.org/pipermail/tails-dev/2016-March/010485.html
https://lists.torproject.org/pipermail/tor-talk/2015-July/038624.html
Testing sites listed here:
https://forums.whonix.org/t/help-wanted-kvm-users-biometrics-fingerprinting-testing/2233
Related code:
https://cgit.freedesktop.org/spice/spice-gtk/tree/src/channel-inputs.h
https://cgit.freedesktop.org/spice/spice-gtk/tree/src/channel-inputs.c
https://git.qemu.org/?p=qemu.git;a=blob;f=hw/input/ps2.c
Virtualization is useful in presenting an identical environment and set of "hardware" for each user which goes a long way in creating an anonymity set of systems. That way a system attacker, advertisers and online trackers would not be able to fingeprint a user or their hardware.
The problem: Tracking techniques have become more sophisticated with time. They advanced from simple cookies to browser/device fingerprinting (which Tor Browser focuses on defeating) to user behavior fingerprinting. The latter is about profiling how a user types on a keyboard or uses a mouse [2].
Keystroke dynamics is a super creepy way to track users based on how long they press keys (dwell time) and the time between key presses (gap time). This is extremely accurate at identifying individuals because of how unique these measurements are. Advertising networks (Google, Facebook...) that fingeprprint users on both the clearnet and Tor can deanonymize users. This technique is already actively used in the wild [6][7].
Potential Solutions:
Since input devices are all emulated its a great opportunity to stop this profiling technique.
* A security researcher designed a proof of concept plugin for Chrome browser that mitigates this. Implementing something like the PoC addon in [1] known as KeyBoardPrivacy. Some random delay in milliseconds in a 50 millisecond range for dwell and gap times for the emulated keyboards is enough to skew the values to render this attack useless while not affecting performance.
* The changes made to Tor Borwser to make JS timers more coarse grained but constant (250ms for keyboard events) were not enough to stop keystroke dynamics fingerprinting because a malicious script can evict the cache and allow extrapolation of true timing events within 1-5ms accuracy .[3][5] Their goal is to instead add jitter to the timers [4]. A similar solution proposed in [4] can be implemented in all QEMU-KVM timers to mitigate both attacks.
EDIT: Second solution not the way to go. High precision timers in virtualizers have bad security implications. They enable sidechannel attacks on crypto operations running on other VMs as well as aid in biometric fingerprinting. A multi hypervisor study on timer accuracy shows most have high accuracy[8] by necessity. Increasing coarseness in the hypervisor's code can lead to guest slow downs or instability. This varies by hypervisor. CPU load does NOT affect timer accuracy [in KVM] however IO does. A stress/stress-ng daemon installed on GW that runs some varying workload may introduce a skew to timer data.
EDIT 2:
Even if the user's destination does not itself surreptitiously sniff their biometrics, any one observing the traffic of any interactive JS application can generate a model for your biometric statistics. [16]
[1] https://paul.reviews/behavioral-profiling-the-password-you-cant-change/
[2] https://web.archive.org/web/20221114004700/http://jcarlosnorte.com/security/2016/03/06/advanced-tor-browser-fingerprinting.html
[3] https://www.lightbluetouchpaper.org/2015/07/30/double-bill-password-hashing-competition-keyboardprivacy/#comment-1288166
[4] https://gitlab.torproject.org/legacy/trac/-/issues/16110
[5] https://gitlab.torproject.org/legacy/trac/-/issues/1517
[6] https://web.archive.org/web/20200430120856/http://scraping.pro/no-captcha-recaptcha-challenge/
[7] https://nakedsecurity.sophos.com/2013/11/01/facebook-to-silent-track-users-cursor-movements-to-see-which-ads-we-like-best/
[8] Achieving High Resolution Timer Events in Virtualized Environment https://web.archive.org/web/20220716012044/http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4503740/pdf/pone.0130887.pdf
[9] https://archive.ph/20160528163257/http://dataprotectioncenter.com/featured/keystroke-profiling/
[10] https://arstechnica.com/information-technology/2015/07/how-the-way-you-type-can-shatter-anonymity-even-on-tor/
[11] Spoofing key-press latencies with a generative keystroke dynamics model [DOI: 10.1109/BTAS.2015.7358795] https://ieeexplore.ieee.org/document/7358795
[12] "I Am Robot: (Deep) Learning to Break Semantic Image CAPTCHAs" https://web.archive.org/web/20220224013800/http://www.cs.columbia.edu/~polakis/papers/sivakorn_eurosp16.pdf
[13] I can be You: Questioning the use of Keystroke Dynamics as Bio
metrics https://web.archive.org/web/20220331095121/http://flyer.sis.smu.edu.sg/ndss13-tey.pdf
[14] How I reverse-engineered Google Docs to play back any document's keystrokes https://news.ycombinator.com/item?id=8562483
[15] An Efficient User Verification System via Mouse Movements [DOI=10.1.1.220.9410] https://web.archive.org/web/20190303210911/http://www.cs.wm.edu/~hnw/paper/ccs11.pdf
[16] Knowing the User’s Every Move – User Activity Tracking
for Website Usability Evaluation and Implicit Interaction https://www.medien.ifi.lmu.de/pubdb/publications/pub/atterer2006www/atterer2006www.pdf
[17] Eliminating Fine Grained Timers in Xen https://cseweb.ucsd.edu/~bvattikonda/docs/xentimers-ccsw11.pdf
[18] https://web.archive.org/web/20220526220057/https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3835878/#B13
[19] https://www.cs.cmu.edu/~maxion/pubs/KillourhyMaxion08a.pdf (The Effect of Clock Resolution
on Keystroke Dynamics) millisecond resolution is vital
[20] https://web.archive.org/web/20210412030235/https://www.comp.nus.edu.sg/~tsim/documents/keystroke-dynamics-published.pdf (Keystroke Dynamics in a General Setting) Compared fingerprinting users in a general setting in contrast to typing a password. Measured dynamics of typing common words. Another surprising finding is non-English users were even more susceptible to keystroke fingerprinting than English ones.
higher level behavior fingerprinting:
https://www.researchgate.net/profile/Roman_Yampolskiy/publication/256169398_Game_Playing_Tactic_as_a_Behavioral_Biometric_for_Human_Identification/links/569c3c1d08aea1476954742c.pdf
higher level "cognitive" behavioral biometrics are application specific. That paper uses descriptive statistics for actions that are very specific to the game and don't really apply anywhere else.
=== Mouse Tracking ===
User re-authentication via mouse movements
https://web.archive.org/web/20230519163324/https://csis.pace.edu/~ctappert/it691-projects/mouse-pusara.pdf
Relatively successful tracking active readers however mouse movements are not enough to fingerprint users. That was back in 2004.
ON USING MOUSE MOVEMENTS AS A BIOMETRIC
https://www.cs.sjsu.edu/faculty/pollett/papers/shivanipaper.pdf
https://www.cs.sjsu.edu/faculty/pollett/masters/Semesters/Spring04/Shivani/CS298Presentation.pdf
Needs more work to achieve high accuracy. 2004.
An Efficient User Verification System via Mouse Movements
https://web.archive.org/web/20190303210911/http://www.cs.wm.edu/~hnw/paper/ccs11.pdf
high accuracy achieved in limited situations - actice authentication during log-on. Does not clear EU false positive requirements however so they recommend it for combining with keystroke dynamics as extra confirmation:
In the scenario of static verification, a user is required to perform a series of mouse movements and its mouse data is verified within a certain amount of time (e.g., login time). A good example of this scenario is a click-based graphical password for user login, where five clicks are estimated to be made in no more than 25 seconds.
By contrast, in the scenario of continuous verification, a user’s mouse data is continuously collected and verified throughout the entire session. This is non-intrusive to users and meets the goal of passive monitoring. However, the frequency of user mouse actions varies significantly in different sessions. In general, the average frequency of user mouse actions will be much lower than that of the static scenario.
Even for the same user at different times, the number of mouse events per unit time varies a lot. However, to the best of our knowledge, our work is the first to achieve high accuracy with a reasonably small number of mouse events.
[1] http://www.ijicic.org/ijicic-ksi-03.pdf - User Authentication using Rhythm Click Characteristics for Non-Keyboard Devices
[2] https://web.archive.org/web/20160617233804/http://www.ibm.com/developerworks/library/os-userauth-mouse/index.html - IBM perl guide for fingerprinting mouse click-hold times
search terms:
Mouse movement authentication
=== Solutions ===
Researcher Vinnie Monaco has kindly volunteered to write tools that mitigate these attacks:
anti-keystroke defense:
https://github.com/vmonaco/kloak
mouse movement defense:
Upcoming
== clocksource ==
https://forums.whonix.org/t/available-clocksource-current-clocksource
== Upcoming XML Changes - Stable Next ==
When Stretch hits stable disable:
smm which is not even supported by qemu yet.
Check if switch to virtio-input devices means virtio=serial can now be removed.
== Apply Clean-filter traffic? ==
Advantages are its easy to set a new IP for a new WS. Has many rules to prevent MAC, IP, ARP spoofing. Can also protect the GW from DoS by using connection tracking and rate-limiting.
Sounds good but I still think a completely separate internal network is the best option to ensure separartion between multi WSs. The GW should be able to withstand DoS on its own or there will be big problems. I think its safer if libvirt knows nothing about the traffic on the internal network and does not parse anything on there.
To have a multi-WS implementation with different internal networks likely requires a big effort on the {{project_name_short}} development side. Not worth it when setting a new GW is easy and costs little resources - also provides more assurance against theoretical attacks on a shared GW.
== RNG ==
===Rate limit===
This is necessary to prevent malicious entropy starvation of other VMs and the host.
Test code for /dev/random throughput
https://unix.stackexchange.com/questions/13771/dd-if-dev-random-is-randomly-bottlenecked-with-big-time-lags-but-i-have-no-i
dd if=/dev/random bs=1k count=1
Re-reading the article it seems the choice made is OK. The limit sets the max a guest can use to be 25% of that of the host.
/dev/random provides high quality numbers but stops when entropy is poor: the Linux kernel keeps a entropy pool of 4 KB (see /proc/sys/kernel/random/poolsize); when the pool is empty, the kernel blocks causing a delay until it can provide the requested random numbers.
Note: Just after , you can add a line like to prevent a guest from consuming more than 1 KB per second of pseudo random numbers. Otherwise, one single guest could consume all the available pseudo random numbers at the expense of all the others.
Also applied for GW since some crypto requiring operations under the control of WS processes can abuse the entropy available to it if unchecked.
===Haveged===
Cross-posted
haveged relies on the RDTSC instruction, that apparently is useless in some virtualized environments. Also, the quality of random numbers output by HAVEGE is unclear, and the topic of many discussions.
https://tails.boum.org/contribute/design/random/#rngd
haveged is not useful on KVM
rdtsc or any tsc counter provides very accurate clock tick information so better to block.
== blkiotune ==
Its possible to limit IO throughput per VM however with disk speeds varying a lot it will be impossible to find an option that makes sense for everybody. Nonetheless still interesting to have. Documented below is a IO test and how to set limits.
https://fedoraproject.org/wiki/QA:Testcase_Virtualization_IO_Throttling
== Secure Clipboard ==
https://bugzilla.redhat.com/show_bug.cgi?id=1320263
https://gitlab.freedesktop.org/spice/spice-gtk/-/issues/97
== CPU Masking ==
KVM cpus support a baseline of features by default. You can mask out the problematic ones and don't have to worry about the extra ones it doesn't support because it will be masked out anyhow (because it was never supported in the first place).
The only bad instructions we should filter out are a subset of whatever instructions are listed under the virtual cpu from the output of the cpu_map.xml list
cat /usr/share/libvirt/cpu_map.xml
I figured out safe defaults and will do a pull request. NB clflush was abused to carry out the rowhammer attack so its blacklisted.aes will be passed through for crypto performance - it doesn't mess with random number generation.
qemu64
https://libvirt.org/formatdomain.html#cpu-model-and-topology
Informative link about cpu flag functionality:
https://unix.stackexchange.com/questions/43539/what-do-the-flags-in-proc-cpuinfo-mean
https://www.berrange.com/posts/2010/02/15/guest-cpu-model-configuration-in-libvirt-with-qemukvm/
"Every hypervisor has its own policies for what a guest will see for its CPUs by default, Xen just passes through the host CPU, with QEMU/KVM the guest sees a generic model called “qemu32” or “qemu64”. "
cat output:
== Passed to Command Line ==
Minor importance.
Libvirt "only" runs the qemu-kvm command line utility with the correct command line parameters?
Is there a way to make libvirt tell with what parameters libvirt is calling qemu-kvm?
HulaHoop:
Yes its possible. Conversion of commands is supported in bot directions. See documentation here: https://libvirt.org/drvqemu.html#converting-from-domain-xml-to-qemu-args
Nice and useful. Added to unit_test:
https://github.com/{{project_name_short}}/whonix-libvirt/commit/4bcac51b264b9d78042899be005e4c06549224ab
== Haveged test for KVM + virtio-rng Result ==
NB Patrick has conducted the tests without virtio-rng and they worked too.
Reults with virtio-rng:
0 failed individual tests with THRESHOLD 0.001000 on 514 individual tests
PASS: nist/test.sh
==================
All 2 tests passed
==================
make[2]: Leaving directory `/home/user/haveged-1.4'
make[1]: Leaving directory `/home/user/haveged-1.4'
== Two Separate {{project_name_gateway_long}} ==
https://www.whonix.org/old-forum/index.php/topic,397.msg2973.html#new
Possible and documented.
== Preserve Comments in libvirt / virsh XML Files ==
Asked on stackexchange:
https://unix.stackexchange.com/questions/148163/is-it-possible-to-use-comments-in-virsh-libvirt-xml-files
Asked on libvirt mailinglist and here is their response:
https://listman.redhat.com/archives/libvir-list/2014-August/msg00681.html
A description attribute can be used to carry human readable metadata.
== Activate Watchdog? ==
libvirt can be told to ignore a guest's request for restart. The actual action would be a silent shutdown. Same for when a crash happens, libvirt would just leave the machine in its shutdown state instead of booting it again.
This all hinges on what you think about the attack i described earlier. Should we just advise people not to do things on their host that would leak timestamps? Like browsing for instance. From what I got from the wiki, this is the only thing done on a Linux host that could leak this information.
Later I learned that TCP Sequence numbers could still leak time information, but its more difficult to exploit than other methods.
Patrick wrote: "the attack i described earlier" in unclear for other readers and I am also not so sure about it anymore.
Patrick wrote: Not sure exactly what you want to defend here, but I guess this info might invalid its premises. The ACPI reset command is not required. An adversary could simply use kexec. It allows to reboot a machine without ACPI reset command. kexec's purpose is to save time by not going through BIOS initialization routines. Therefore KVM ignoring ACPI reset command probably won't be a big help.
HulaHoop: Agreed. I went ahead and re-added it.
== Add pvspinlock when supported in more recent version of libvirt ==
Was not accepting it because its a fresh feature with not support in the version of libvirt I run. Its been added as of libvirt 1.2 as per the reply.
https://listman.redhat.com/archives/libvir-list/2014-August/msg01010.html
HulaHoop: Done.
== Preconfigured Shared Folders ==
Patrick asks: For better usability, should we add the necessary settings for shared folders by default to whonix_gateway.xml and whonix_workstation.xml?
Patrick comments: I could then, if kvm is detected (easy using virt-what), automatically mount the shared folder in {{project_name_short}} VMs. Could be implemented into the https://github.com/{{project_name_short}}/shared-folder-help shared-folder-help package.
Patrick asks: If we decide to do so, what would be good default paths for the host shared folder? /mnt/shared_gw and /mnt/shared_ws?
Patrick said: HulaHoop reported, that "virsh define" will fail if the shared folder in the xml does not exist.
See also: https://github.com/Whonix/Whonix/issues/223
== Kernel Samepage Merging (KSM) ==
Any changes to the images required? Probably adding this init script:
https://github.com/dnaeon/ksm-init.d-debian (looks good and simple - trivial to package)
Any other changes required?
Any changes to documentation or xml required to make use of it?
While not needed to enable this functionality for {{project_name_short}} directly,its useful for Debian host users. Its inclusion in {{project_name_short}} will definitely help when running Nested Virtualization though so please add this.
To make use of it, it needs to be enabled host-side following this guide
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/virtualization_administration_guide/chap-ksm
. It is used automatically by libvirt without further intervention, but certain parameters such as huge page sharing can be, optionally, used through the XML.
https://libvirt.org/formatdomain.html#memory-backing
Using it on the host is as simple as starting the two services ''ksm'' and ''ksmtuned'' following the guide above.
No need to adjust the number of pages it uses anymore as newer versions automatically handle it with bigger RAM amounts as to give a bigger performance improvement compared to earlier versions that used just 2000 pages of RAM.
How to verify that KSM is working and looking at its stats.
https://www.suse.com/documentation/sles11/book_kvm/data/kvm_qemu_ksm.html
Notes: What appeared to be effects of KSM was actually just the Linux Guests not using all allocated RAM by default. Non Linux guests usually take over the entire assigned RAM allocated space while Linux does not.
By default KSM kicks in when there is very little free free RAM on the host. Settings must be tweaked in ksmtuned.conf to change this behavior. Experimenting in progress...
KSM security implications?
3.
ATTACK ON MEMORY
DEDUPLICATION
Memory deduplication may be subject to a memory disclosure
attack from the attacker’s VM to the victim’s VM. The attack
guesses a process running on a victim’s VM or a file downloaded
by a browser.
Memory deduplication shares 4KB pages which have the
same contents. When data is written to a deduplicated page, the
page is re-created with a copy of its contents (Copy On Write).
This causes the write access time to be slower than normal,
because it includes the overhead to re-create the same page. An
attacker can measure access time to determine whether a page was
deduplicated by another VM.
https://web.archive.org/web/20220616034225/https://staff.aist.go.jp/k.suzaki/EuroSec2011-suzaki.pdf
Enable Hugepages for guests?
Yes. I am seeing benchmarks of 40% improvement of performance for all workloads in a guest according to the literature. However using it will prevent use of memory ballooning and swapping
https://fedoraproject.org/wiki/Features/KVM_Huge_Page_Backed_Memory#Detailed_Description
and is subject to lack of adequate sVirt protections.
https://bugzilla.redhat.com/show_bug.cgi?id=515741
Just confirmed from Dan Walsh, SELinux policy developer that this is no longer the case.
The sacrificing of one performance feature for another may not be a problem afterall. Since 2012 there has been an effort to teach KSM how to play nice with THP transparent Hugetables.
https://web.archive.org/web/20140429193804/http://www.linux-kvm.org/wiki/images/1/19/2012-forum-memory-mgmt.pdf
More info
https://fedoraproject.org/wiki/Features/KVM_Huge_Page_Backed_Memory#Benefit_to_Fedora
Documentation:
A kind soul posted how to interpret what ksmuned.conf's parameters mean in plain English in the first post. Its worth mirroring in our documentation:
https://forum.proxmox.com/threads/ksm-with-2-6-35.5041/
No need to play around with the defaults. ksmtuned should handle memory overcommitment adjustments dynamically when the host is under pressure.
More accurate instructions on starting ksm and ksmtuned services and keeping them that way at boot. They will be summarized in the documentation too:
https://archive.ph/jwgVS
https://archive.ph/TSWww
https://archive.ph/6HBJc
https://archive.ph/5EWIv
TO DO: packaging https://github.com/Whonix/Whonix/issues/39#issuecomment-53991872
TO DO: Document on KVM wikipage how users can enable this permanently.
Documentation and risk assessment completed.
Done.
== Pass qemu commands via libvirt? ==
Not a good idea, has potential security and stability risks.
https://listman.redhat.com/archives/libvir-list/2014-September/msg00288.html
Useful in some cases such as guest debugging via qemu's gdb while retainging libvirt's security framework. Relevant documentation added.
== memory balloon ==
https://rwmj.wordpress.com/2010/07/17/virtio-balloon/
Any changes to documentation, xml or images required to make use of it?
HulaHoop: Just having the device added is enough to take advantage of it. At the moment this feature needs manual intervention to make use of it which limits its utility in the real world. However, towards the end of 2013 work is being done to have this feature automated and mainlined
https://www.linux-kvm.org/page/Projects/auto-ballooning
This space will be watched for developments; as soon as its incorporated upstream and an option added in libvirt I will enable it in the configuration.
Memory ballooning is indeed another vector for side channel atacks on memory. Confirmed in a follow-up by side-channel attacks researcher Daniel Gruss:
"Not Rowhammer, but there has already been a side-channel attack on memory ballooning earlier this year: https://ieeexplore.ieee.org/document/7562068/"
Status: feature disabled for {{project_name_short}} 14
Warned upstream and advised them against enabling it by default:
https://listman.redhat.com/archives/libvir-list/2016-December/msg01156.html
== SPICE security implications ==
https://lists.freedesktop.org/archives/spice-devel/2013-December/015810.html
https://lists.freedesktop.org/archives/spice-devel/2014-January/015815.html
Protections applying to QEMU apply here too.
== How to check if virtio storage access is in use or an eventually existing fallback driver? ==
=== Question ===
How to check if virtio storage access '''virtio_blk''' really is in use or an eventually existing fallback driver?
https://www.linux-kvm.org/page/Boot_from_virtio_block_device
https://wiki.libvirt.org/page/Virtio
=== Asked on unix.stackexchange ===
https://unix.stackexchange.com/questions/148055/kvm-how-to-check-if-virtio-storage-access-is-in-use
=== lsmod won't help ===
The following.
lsmod | grep virtio
Only shows, that the virtio kernel module is loaded. It is not hard to load arbitrary kernel modules for hardware that you don't have installed ("sudo modprobe virtio-blk"). The question remains, is this kernel module actually in use or an eventually existing fallback driver?
=== modinfo won't help ===
modinfo virtio-blk
Won't help, because it only shows general information it would also show on VBox systems (after running "sudo modprobe virtio-blk").
=== ls Test ===
/dev/sda and /dev/sda1 should not exist.
ls -la /dev/sda
ls -la /dev/sda1
/dev/vda and /dev/vda1 should exist.
ls -la /dev/vda
ls -la /dev/vda1
=== hwinfo Test ===
See which driver is loaded for each particular device.
hwinfo --short
== How to check if virtio network is really in use or an eventually existing fallback driver? ==
How to check, that '''virtio_pci''' is really being used and not some fallback driver?
/sys/devices/virtio-pci/0/net/eth0/statistics/rx_bytes seems not to exist on Debian Wheezy.
== How to enable AppArmor / SELinux for KVM on Debian? ==
Details on how to install, configure and enable SELinux enforcing mode:
https://wiki.debian.org/SELinux/Setup#Steps_to_setup_SELinux <-- This is a start, but incomplete. When following these instructions it gets difficult at the "audit2why -al" step, because it shows a ton of policy violations which are non-trivial to fix without learning a lot about SELinux.
Why Bother?
SELinux is more robust than Apparmor because its label based vs file-path based. But his comes at the expense of being difficult to write policies. However the plans I have for it in mind, it won't matter since we won't be writing SELinux profiles, but leveraging the one already present in sVirt as a protection layer around application sandboxes.
HulaHoop: This is probably a result of not having done a relabel of your filesystem after enabling SELinux. This aspect can be handled by a script from the link below. I'll need to test this at some point. Any files created before SELinux was prsent creates conflicts with policy when it is enforced. A relabel must also run if you want to isolate software that has a policy but was introduced after SELinux was enabled. Example KVM if we decide to ship it for nested purposes in the future.
https://debian-handbook.info/browse/stable/sect.selinux.html
Another guiede to test with:
https://web.archive.org/web/20111211210506/http://blog.danielcosta.pt/?p=440
Note: Both MAC systems cannot be run simultaneously run. This is not supported by LSM and may also be a source of conflicts.
For Debian hosts, AppArmor makes more sense.
https://libvirt.org/drvqemu.html#apparmor-svirt-confinement
AppArmor profiles for Ubuntu could be ported to Debian:
https://bazaar.launchpad.net/~ubuntu-branches/ubuntu/utopic/libvirt/utopic/files/head:/debian/apparmor/
DONE. DOCUMENTED.
== Set cluster_size 512 - 2MB for better performance? ==
HulaHoop wrote: The qcow2 defaults seem fine to me, however the one parameter of interest we should play with is image's : 'cluster_size' 512 - 2MB the larger the size the better performance but the size is bigger. I think we should try the largest and see if the image size stays reasonable.
Patrick wrote:
When using ({{project_name_short}} 7.7.8.0):
qemu-img \
convert \
-p \
-O qcow2 \
-o preallocation=metadata \
"$WHONIX_BINARY/$VMNAME.img" \
"$WHONIX_BINARY/$VMNAME-$derivative-maker_whonix_version_new.qcow2"
du -h --apparent-size file.img
100 GB
du -h file.img
2,5 GB
When using ({{project_name_short}} 7.7.8.0):
qemu-img \
convert \
-p \
-O qcow2 \
-o cluster_size=2M \
-o preallocation=metadata \
"$WHONIX_BINARY/$VMNAME.img" \
"$WHONIX_BINARY/$VMNAME-$derivative-maker_whonix_version_new.qcow2"
du -h --apparent-size file.img
2,9 GB
du -h file.img
2,9 GB
File size grew a bit. Not too much. We could use this.
It remains to be tested, if the images work well using these settings. (The apparent size should be shown as 100 GB, but is only 2,9 GB. Needs testing if 100 GB can be used after the image is in use.)
"qemu-img info file.img" however, shows 2,9 GB actual size and 100 GB virtual size.
git tag 7.8.0.1 and above will be using the "-o cluster_size=2M" option.
HulaHoop said: I am yet to document this in the KVM page lets do this at the end. Is there any tangible speed gains?
== Exporting/Cloning of virtual machines ==
Important links:
* http://libvirt.org/sources/virshcmdref/html-single/
* https://unix.stackexchange.com/questions/8351/how-to-create-a-dupe-of-a-kvm-libvirt-virt-manager-vm
* https://linux.die.net/man/1/virt-clone
Archived thread SE
I'm a bit lost with virt-manager / libvirt / KVM.
I've got a working KVM VM (Windows XP) which works nicely.
The VM is backed by a 4GB file or so (a .img).
Now I want to do something very simple: I want to duplicate my VM.
I thought "OK, no problem, let's copy the 4GB file and copy the XML" file.
But then the libvirt FAQ states in all uppercase: "you SHOULD NOT CARE WHERE THE XML IS STORED"
https://wiki.libvirt.org/page/FAQ
OK fine, I shouldn't care. But then how do I duplicate my VM?
I want to create a new VM that is a copy of that VM.
Which will make a copy of this-vm, named that-vm, and takes care of duplicating storage devices. Nothing new here except detail.
More to the point, What the FAQ is saying is that the XML domain descriptions are not directly editable, you need to go through libvirt. To complete the steps taken by the virt-clone command, you could:
You cannot "clone" a running vm, stop it. suspend and destroy are also valid options for less graceful cloning
virsh shutdown this.vm
copy the storage.
cp /var/lib/libvirt/images/{this-vm,that-vm}.img
dump the xml for the original
virsh dumpxml this-vm > /tmp/that-vm.xml
hardware addresses need to be removed, libvirt will assign new addresses automatically
sed -i /uuid/d /tmp/that-vm.xml
sed -i '/mac address/d' /tmp/that-vm.xml
and actually rename the vm: (this also updates the storage path)
# dump the xml for the virtual isolated network
virsh net-dumpxml {{project_name_short}} > /tmp/whonix.xml
# hardware addresses need to be removed, libvirt will assign
# new addresses automatically
sed -i /uuid/d /tmp/whonix.xml
sed -i '/mac address/d' /whonix.xml
== management tools ==
There are a lot [https://www.linux-kvm.org/page/Management_Tools management tools]. Are any of these useful for {{project_name_short}} use case?
== Nested Virtualization ==
As a minor point, documenting how to use KVM with nested virtualization would be nice.
Forum thread:
https://forums.whonix.org/t/nested-virtulization-needs-64-bit-whonx-guest
This is only possible using a 64 bit kernel. Install a 64 bit kernel. Don't get confused by the package name. Works with '''AMD''' and '''Intel'''.
== qemu i386 support ==
Why are our qemu xml files currently called ${VMNAME}_qemu-x86-x64.xml. Are they dependent on x86-x64? Should they work on i386 systems as well? Maybe just must be removed? Can you test please?
Any QEMU architecture works on any host supported by QEMU.i386 tested and working but pointless when x86-64 works on i386 hosts too.
==Disable Microphone Input==
Figure out how to disable Microphone input to Guests. While a nice feature for VoIP, is a very dangerous setting to have by default as its unexpected.
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/section-libvirt-dom-xml-sound-devices.html
https://libvirt.org/formatdomain.html
'''Somewhat Solved:'''
After speaking with Cole Robinson, libvirt developer, its clear that this is a feature provided by SPICE rather than something having to do with the sound card. I found that all sound cards have this after testing yesterday. When I changed Spice to VNC this no longer happens. Before you say that we shouldn't use spice, its important to note that audio does not function without it at all.
I've asked on the mailinglists if there is some xml attribute to disable the feature. From what I've seen in the documentation this is not possible. However it can be accomplished through the Host's audio popup menu in the taskbar. The devices show up with the name "virt-manager: record" and can be set to mute as with any other device on the host.
https://libvirt.org/formatdomain.html#graphical-framebuffers
Other spice channel functionality provided:
Valid channel names include main, display, inputs, cursor, playback, record (all since 0.8.6); smartcard (since 0.8.8); and usbredir (since 0.9.12).
Mailinglist reply:
https://listman.redhat.com/archives/virt-tools-list/2014-August/msg00002.html
Currently not supported through libvirt but planned. Not a problem, we can just document how to disable its easy and gui based. For those running from commandlines, they should just revert to a non SPICE adapter and they'll be fine. They are not missing much because they are not using the workstation as a desktop.
Bugzilla Ticket:
https://bugzilla.redhat.com/show_bug.cgi?id=1126641
== Document: how to leave KVM when no X is running? ==
Situation... User is in terminal in a VM... No X is running ("sudo service kdm stop"). User wants to switch back to the host... How to do this?
The emulated tablet device handles this by not allowing the mouse to be captured by the guest.
Its still possible though:
serverfault.com/questions/457603/any-way-to-release-focus-on-a-kvm-guest-in-virt-manager-without-havin-g-to-click
press Ctrl_L & Alt_L
==Audio Retasking==
SPEAKE(a)R: Turn Speakers to Microphones for Fun
and Profit
https://arxiv.org/ftp/arxiv/papers/1611/1611.07350.pdf
conclusion:
While virtual sound devices might be retaskable with software too the malware still cannot access the soundcard settings of the host to retask it too given that the hypervisor is not broken.
Replies from the QEMU devs. They confirm my conclusion that no risk posed to host. Also virtual soundcards optionally have half-duplex modes that make audio input impossible.
https://lists.nongnu.org/archive/html/qemu-devel/2016-11/msg04754.html2
Replies:
https://lists.nongnu.org/archive/html/qemu-devel/2016-11/msg04899.html2https://lists.nongnu.org/archive/html/qemu-devel/2016-11/msg04906.html2https://lists.nongnu.org/archive/html/qemu-devel/2016-11/msg04904.html2
== libguestfs import/export feature request ==
Let's together write a feature request for an import/export feature for libguestfs, similar to VirtualBox's import/export feature. Then post in [https://bugzilla.redhat.com/buglist.cgi?component=libguestfs&product=Virtualization%20Tools libguestfs's bug tracker].
Irrelevant see virt-manager feature instead.
== virt-manager import/export feature request ==
Let's together write a feature request for an import/export feature for virt-manager, similar to VirtualBox's import/export feature. Then post in [https://bugzilla.redhat.com/buglist.cgi?product=Virtualization%20Tools&classification=Community&component=virt-manager&bug_status=NEW&bug_status=ASSIGNED virt-manager's bug tracker].
Update on import/export Support:
Gnome Boxes, another simplified GUI for libvirt will have an OVF import feature added. Note that OVF is not the vmware vmdk specfic format OVA. In this sense making it a true standard.
Its in the works and the Gnome contributor behind it has posted that he plans to make it into a stand-alone library so it could be used by other projects. Meaning it could be easily included in virt-manager too.
His post:
https://zee-nix.blogspot.com/2014/05/berlin-dx-hackfest-boxes-rain-and.html
Initial commit:
https://web.archive.org/web/20140701164007/https://gitorious.org/govf
Update 2016:
Past efforts have come to a stop. Opened new discussion:
https://listman.redhat.com/archives/libvir-list/2016-March/msg00165.html
== virt-v2v ==
Importing KVM to KVM - Not what its for: https://rwmj.wordpress.com/2015/09/18/importing-kvm-guests-to-ovirt-or-rhev/#content
=== How to install virt-v2v on Debian? ===
virt-v2v is not yet in Debian. [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=743261 wishlist request]
=== Can we copy and paste virt-v2v in meanwhile? ===
Since virt2v2 is not yet in Debian, maybe it is a single script that can replace the [https://github.com/{{project_name_short}}/derivative-maker/blob/master/libvirt/whonix_libvirt_import whonix_libvirt_import] script?
HulaHoop: It is my opinion that we shelve the automated script idea completely and use virt-v2v when its possible/available. If virt-v2v does not work out, we should instead do the following: 1) ship multiple templates according to containment systems we want to support 2) A readme file with instructions on
[a] how to copy the sparse file properly to any location a user wants if they wish
[b] how to introduce the sparse file into priviledged dirs like /var/lib..
[c] how to import xml for machines
[d] how to autostart the isolated network
[e] how to edit xml files in the case that they want to use non-default paths
The build process would be scripted to update the version number of the name and disk img xml fields and thats it.
The absence of an easy way to import a machine is too big of an issue. We cannot compensate for a shortcoming as big as this coming from upstream. It would take tremendous effort as we have seen and some people might not be happy that it doesn't do X. So I say we leave it as described above.
=== How to export a VM using virt-v2v? ===
This is what probably must be done in the build script. We got qcow2 files and xml. What command must be used to export a VM?
=== How to export a VM using virt-v2v? ===
This is probably what users have to do to import. What command must be used to import a VM?
=== Closed ===
Looks like we've settled on not using virt-v2v for either importing {{project_name_short}} VMs or for cloning VMs. Probably mainly because v2v is not in Debian and hence doesn't make things simpler.
== Shared Folder ==
We should add instructions on how to use shared folders.
https://www.linux-kvm.org/page/9p_virtio
Done: [[KVM#Shared_Folders]]
== Script/automate creation and upload of raw images? ==
Because raw images have better performance than raw images? https://www.linux-kvm.org/page/Qcow2
True that performance is better, but I don't think it supports snapshots AFAIK.
For something that supports snapshots and good performance lets stick with qcow2.
== write generic import script ==
Maybe write generic import script as per: https://www.whonix.org/old-forum/index.php/topic,159.msg2412.html#msg2412
A hellish solution and way too much work. Your time is spent better elsewhere. I'll use my time to get xml import validation working.
== Script/automate creation of VM description files? ==
So not following manual instructions is required anymore, but copy/paste some commands end up with the correct VM configuration.
Once the necessary commands are documented, Patrick will be happy to automate the steps with a script.
HulaHoop created [https://github.com/{{project_name_short}}/derivative-maker/tree/master/libvirt xml files], because kvm xml files can not be created using libvirt as VirtualBox VM setting files can be created using vboxmanage. This is not a problem.
== How to add a random time offset? ==
Having a small random offset (similar to [[Advanced_Security_Guide#Network_Time_Synchronization]]) would be a good thing.
Patrick said: Thanks to [https://github.com/{{project_name_short}}/bootclockrandomization bootclockrandomization] this is no longer necessary.
== xml does not validate ==
virsh --version
0.9.12.3
virt-xml-validate whonix_gateway.xml
whonix_gateway.xml:24: element pm: Relax-NG validity error : Element domain has extra content: pm
whonix_gateway.xml fails to validate
virt-xml-validate whonix_workstation.xml
whonix_workstation.xml:24: element pm: Relax-NG validity error : Element domain has extra content: pm
whonix_workstation.xml fails to validate
virt-xml-validate whonix_network.xml
whonix_network.xml:1: element network: Relax-NG validity error : Invalid attribute connections for element network
whonix_network.xml fails to validate
kvmclock had to be under rtc for it to work. The Network xml validates on my machine without changes however. Please check if it works and post back.
The virt-xml-validate I tested with is a newer version. Try updating to a more recent one and try again, I think it may be buggy or not updated enough to interpret new schema they may have added since .9x was released.
Patrick said: HulaHoop said this is only due to different KVM versions and won't cause issues.
Patrick said: Leaving automatic validation out for now. Can still be done manually after editing the xml files.
== Run virt-xml-validate during build process ==
We should use.
virt-xml-validate libvirt.xml
Patrick will be happy to add this command to the build process when using --kvm (or --qemu?) switch. Scripting this should be simple and the build process could fail early and closed (break).
Patrick said: Since validation is too version specific (see above), we're not doing this.
== Weaker warning against KVM in whonixcheck ==
As soon as all blockers have been sorted out. We could recommend using [[KVM]] instructions.
Done.
== Leak Tests? ==
HulaHoop sorted that out in [[Special:AWCforum/st/id306/]].
== Hardware serials? ==
HulaHoop sorted that out in [[Special:AWCforum/st/id306/]].
==How to Edit in nano with virsh?==
su
# EDITOR=nano virsh edit ''GuestMachineName''
== whonixcheck KVMClock ==
Whonixcheck should check if /sys/devices/system/clocksource/clocksource0/current_clocksource is kvm-clock and warn if it does.
Implemented: https://github.com/{{project_name_short}}/derivative-maker/commit/2584c3bdcd9a98181c7535f963a615d257e0fefd
== Is it possible to add 64, 128 or even more MB to the virtual graphics card using the command line to boost graphics performance? ==
Yes. Can be realised by virsh command as per the steps in here:
https://serverfault.com/questions/346301/how-to-increase-video-memory-in-libvirt-kvm-gui
== KVM Instructions recommend to use a no cache policy? ==
As per [https://web.archive.org/web/20210410122008/http://www.ilsistemista.net/index.php/virtualization/11-kvm-io-slowness-on-rhel-6.html?start=7 blog post] it is recommended to use a no cache policy. Do our KVM instructions need changes to incorporate this?
Yes if its recommended then lets do this, Its easily done through the GUI by going to: disk → Cache policy: set to none
== Script/automate creation and upload of qcow2 images? ==
Done.
== Verifiable Builds for qcow2 images==
The https://github.com/{{project_name_short}}/derivative-maker/blob/master/help-steps/analyze_image requires supports raw, vdi, vmdk, qcow and qcow2 images as well as .ova appliances.
== metadata preallocated images ==
For better performance. (As per [https://web.archive.org/web/20210120040738/http://www.ilsistemista.net/index.php/virtualization/11-kvm-io-slowness-on-rhel-6.html blog post].) Done in 7.7.8.0.
== qcow2 compression ==
When uncompressed, upload would take ~250 hours per image. Now compressing using tar. [Not using qcow2's compression feature.]
== How to disable KVMClock? ==
In xml file add:
What does KVMClock do? "kvmclock or KVM pvclock lets guests read the host’s wall clock time." This is bad. [https://rwmj.wordpress.com/2010/10/15/kvm-pvclock/ source]
How to adjust KVM to have clock=vm on host.
See https://www.linux-kvm.org/page/KVMClock
https://libvirt.org/formatdomain.html#time-keeping
https://listman.redhat.com/archives/libvir-list/2010-March/msg01293.html
https://web.archive.org/web/20141021123003/https://doc.opensuse.org/products/draft/SLES/SLES-kvm_sd_draft/cha.qemu.running.html#cha.qemu.running.gen_opts.rtc
ls /sys/devices/system/clocksource/clocksource0/current_clocksource
To ensure that you have successfully isolated the vm clock form the host's you must run this in the vm:
sudo chmod -x /etc/init.d/sdwdate
You should verify that kvmclock is indeed disabled by running this command as root.
Make sure kvmclock is not listed as a response any longer.
Documented for reference but is irrelevant as of now since this problem is solved.
https://bugzilla.redhat.com/show_bug.cgi?id=783921#c6 < This link provides the solution to disabling kvmclock and verifying that it is so.
whonixcheck (since {{project_name_short}} 8) would tell you if kvmclock is enabled. whonixcheck --verbose will tell you, that it is not enabled.
== Fix subnet range in KVM instructions ==
As found out by zweeble in https://www.whonix.org/old-forum/index.php/topic,328.0.html, subnet range must be adjusted to make KVM work.
HulaHoop: I think until we can be sure that this is a problem it shouldn't be considered a blocker
== Readme Commands and Warnings ==
Until we compile and test a list of virsh commands needed for operating on vm templates and associated disk storage, we should shot release KVM version to the public.
# virsh net-define [path]/whonix_network.xml
Internal network 'Whonix' defined from network xml file residing in the following directory path.
# virsh net-autostart {{project_name_short}} Network 'Whonix' marked to be autostarted upon boot.
# virsh net-start {{project_name_short}} Network 'Whonix' has been started
== Leak ==
* [https://www.whonix.org/old-forum/index.php/topic,243.msg1636.html New FIN ACK / RST ACK Leak Test]
* [https://www.whonix.org/old-forum/index.php/topic,243.msg1636.html#msg1636 Reported by HulaHoop]
* Was a [https://www.whonix.org/old-forum/index.php/topic,331.msg2269.html#msg2269 false positive] due to a bug in leak testing instructions (now fixed).
== Instructions include how to set clock to UTC? ==
As per https://libvirt.org/formatdomain.html#time-keeping
The '''clock''' setting.
Quote:
The offset attribute takes four possible values, allowing fine grained control over how the guest clock is synchronized to the host. NB, not all hypervisors support all modes.
How to check, that it is set to '''utc'''?
It is set to utc by default. A quick look at the xml file will show:
== How to use -rtc clock=vm ? ==
KVM's '''-rtc''' option as per https://web.archive.org/web/20141021123003/https://doc.opensuse.org/products/draft/SLES/SLES-kvm_sd_draft/cha.qemu.running.html#cha.qemu.running.gen_opts.rtc
This is the original qemu patchset that implements clock=vm in the timer back-ends: https://listman.redhat.com/archives/libvir-list/2010-March/msg01293.html
If you need to isolate the VM Guest clock from the host one, specify clock=vm instead of the default clock=host.
How to use '''-rtc clock=vm''' ?
How to check, that KVM '''-rtc''' is set to '''clock=vm''' ?
Does not seem to be a valid or accepted parameter by libvirt as its always sanitized from the xml file upon saving. But it is accepted by QEMU directly. However I noticed that even with kvmclock enabled, I no longer see any change in a guest's time in response to a host skew of the time. In the recent past this was not true, as any change was immediately reflected in the Guest clock immediately. kvmclock on its own can be disabled and confirmed to be so as outlined below.
== What is timer pit? ==
What is timer pit?
HulaHoop:
Whatever it was, its now gone. I overlooked this by mistake.
We are using.
Start {{project_name_gateway_short}} in virt-manager.
=== xml creation ===
How can we create an xml file, that can be imported using virt-image?
Possible help:
https://fedoraproject.org/wiki/Features/ApplianceTools#Release_Engineering
Maybe we need to learn from their fedora-aos.xml to structure {{project_name_short}} xml.
Since virt-image has been deprecated, this is no option. Source:
https://blog.wikichoon.com/2014/04/deprecating-little-used-tool-virt-image1.html
== How to get virtio running for storage? ==
=== Initial Instructions ===
Virtio-blk now works
Here are the steps, which I'll throw altogether here before the server downtime. Will move into the wiki later:
https://www.linux-kvm.org/page/Boot_from_virtio_block_device < Original steps that needed to be changed so it can apply to Debian.
Quote
in guest os, change /boot/grub/device.map from "(hd0) /dev/sda" to "(hd0) /dev/vda"
in guest os, change /boot/grub/grub.cfg from "root=/dev/sda1" to "root=/dev/vda1", if you are using UUID, then no need to do this step.
I changed all mention of sda1 to vda1 in the grub file. I changed the disk controller type from virtmanager to virtio which takes care of this at the libvirt xml level. Now the vm boots normally with no hangups and I feel a little faster.
A bit of advice, that I'll add so you can decide:
Quote
bancfc: the better option would be using LABELs and UUIDs and such in the source image
unless this is a one time conversion kind of thing
=== Using LABELs ===
Patrick asks: How to use LABELs?
Patrick says: Nevermind. We're now using UUIDs.
=== Using UUIDs ===
Patrick writes: In {{project_name_short}} 9, fstab will look like this.
Maybe using UUID and even a fixed one will aid this feature.
HulaHoop: Yes it will. I found very encouraging answers to your inquiries on the forum that I'll post here:
1. activate virtio-blk in a booted image
2. check virtio-blk is really in use (https://www.whonix.org/wiki/Dev/KVM#How_to_check_if_virtio_storage_access_is_in_use_or_an_eventually_existing_fallback_driver.3F)
3. see how we can automate this using {{project_name_short}} build script / {{project_name_short}} packages
1. From my IRC chats with KVM devs, the virtio driverof concern has been included in all distros for a while now and is used by simply having libvirt select it so.
2. lsmod : lists all currently running modules
modinfo : information about associated devices
hwinfo --short : see which driver is loaded for each particular device
3. UUID is the answer from what I recall from different sources inluding KVM documentation. From a post towards the bottom of a thread, someone describes how to configure the kernel to use UUID to boot:
forums.gentoo.org/viewtopic-p-6668919.html
=== /boot/grub/grub.cfg issue ===
Patrick comments: I think /boot/grub/grub.cfg is a unstable solution. That file says on top.
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
So when there is a kernel upgrade, /boot/grub/grub.cfg will be regenerated and the changes will be lost.
Patrick asks: How do we apply these settings in a way, so they do survive an "sudo update-grub" (which is automatically run on every kernel upgrade).
Patrick says: this is no longer required, since we're now using UUID's in grub.cfg.
=== System gets unbootable? ===
Patrick comments: After editing /boot/grub/grub.cfg, in worst case, maybe, the system will be no longer bootable then. You can test/simulate this case by running "sudo update-grub". Please try.
HulaHoop comments: I found a solution, one that survives kernel and grub upgrades: https://grumpymole.blogspot.com/2007/05/ubuntu-how-to-edit-grub-boot-parameters.html
2.2 Making a change survive upgrades
Depending on how frequently you upgrade your system, you might find that when a new kernel is installed, or grub is upgraded (there may be other scenarios), the /boot/grub/menu.lst file may be replaced by a newer version, or a new line might be added for the new kernel. In this case, your "permanent" changes might be as permanent as you thought.
The best way to address this is to change the "#kopt..." line in /boot/grub/menu.lst. Note that the line is not commented out, even though it starts with a "#". See the excerpt below.
### BEGIN AUTOMAGIC KERNELS LIST
## lines between the AUTOMAGIC KERNELS LIST markers will be modified
## by the debian update-grub script except for the default options below
## DO NOT UNCOMMENT THEM, Just edit them to your needs
## ## Start Default Options ##
## default kernel options
## default kernel options for automagic boot options
## If you want special options for specific kernels use kopt_x_y_z
## where x.y.z is kernel version. Minor versions can be omitted.
## e.g. kopt=root=/dev/hda1 ro
## kopt_2_6_8=root=/dev/hdc1 ro
## kopt_2_6_8_2_686=root=/dev/hdc2 ro
# kopt=root=UUID=fccafcc7-d7cc-4594-9459-a8f0db7b9f7f ro
Add the parameter changes to the end of the line "#kopt...". These parameters will then be automatically added to any new kernel entries and should also survive an upgrade of grub
Patrick comments: Have you tested this?
Patrick comments: I am not sure editing /boot/grub/grub.cfg is a robust solution. We cannot directly ship a /boot/grub/grub.cfg in a deb, because it is an auto generated file by grub-mkconfig. (When that deb was updated, /boot/grub/grub.cfg would be outdated, system unbootable, you know?) Maybe a package could be created, that appends /boot/grub/grub.cfg, but still, any solution not touching /boot/grub/grub.cfg should be preferred. If we could edit/extend /etc/default/grub, /etc/default/grub.d or /etc/grub.d, that would be much more robust. What is the official suggestion by kvm devs to solve this as a distro?
Patrick says: this is no longer required, since we're now using UUID's in grub.cfg.
=== Virtio Block Latency ===
Not really important but is interesting:
https://www.linux-kvm.org/page/Virtio/Block/Latency
HulaHoop: Yes it could have been the situation that there is latency as the data shows, but I'm sure this was improved by now.
=== Well what'ya know It is a bug! ===
HulaHoop wrote:
The reason I struggled so much to get this working is because of an ancient bug that has a workaround in the build process by naming disks to vdX instead of sdX.
Please generate a test image with these changes.
https://bugs.launchpad.net/ubuntu/+source/vm-builder/+bug/517067
Please read this comment for the workarounds:
https://bugs.launchpad.net/ubuntu/+source/vm-builder/+bug/517067/comments/4
> b.) linux/udev : you could suggest that the vdX devices should be symlinked or other to sdX so that your fstab finds the devices independent of how they're connected. → A suggestion for linux/udev. Not {{project_name_short}}.
Patrick says: this is no longer required, since we're now using UUID's in grub.cfg.
=== Solved ===
https://www.whonix.org/old-forum/index.php/topic,401.msg3099.html#msg3099
== Enable virtio by default? ==
Probably yes?
https://wiki.debian.org/DebianKVMGuests
== Boot from virtio block device? ==
What is https://www.linux-kvm.org/page/Boot_from_virtio_block_device good for? Required?
== Disable Microphone Feature Request ==
Let's together write a feature request for an option to disable sound. Maybe like this.
Adds only speaker, no microphone.
Then post it... Where?
Done:
https://bugzilla.redhat.com/show_bug.cgi?id=1126641
This is finally it. Added in Libvirt 4.4.0. Need to add it when Buster rolls around.
''qemu: Add support for 'output' audio codec
Support QEMU's 'hda-output' codec advertising only a line-out for ich6 and ich9 sound devices.''
== Debian Network NAT Issue ==
error: Requested operation is not valid: network 'default' is not active
* Now talking on #kvm
* Topic for #kvm is: KVM wiki website is https://www.linux-kvm.org/page/Main_Page - try status and faq pages, no really, read them! || you can also check the qemu manual at https://www.qemu.org/docs/master/ || don't paste on chan, use https://privatebin.net/ instead || libvirt/virt-manager support is #virt on irc.oftc.net
* Topic for #kvm set by aliguori at Wed Mar 30 13:38:24 2011
Hi How do I check if virtio storage is in use or an existing fallback driver for an emulated device?
there's no fallback in kvm like in xen
so a simple grep should verify if a virtio device is used then - not merely attached I mean?
well... if you really want to check, it is going to be /dev/vdX
and lspci will show virtio block device
Thanks iggy I have a few more questions that I'd really appreciate your help with
ask away
I am trying to isolate the vm clock so no changes on the host can affect it. This is currently possible with -rtc clock=vm afaik,
but it would be much more helpful for my project's purpose if this can be done through something like libvirt
is this possible with libvirt?
that I don't know
the appealing part of using libvirt is the xml settings file it creates, but I think something similar can be done with qemu-kvm cfgfiles option
* iggy hates xml
But that means all settings would have to be imitated but directly for the qemu-kvm utility which is fine if I can get it to do the same things
namely the functionality of isoalted internal networking between two vms
is that possible with qemu-kvm directly?
libvirt has an option to pass options directly to qemu-kvm
please tell me how this is done :-)
or where I can know this.
ok nevermind, I just want to know if this means that a setting in the xml file is passed directly to qemu-kvm then?
... You might try asking in the libvirt channel
I did but no one seems to reply...
I know it is possible, I don't remember it off the top of my head
Thank you so far for your replies. I am actually working on a project that uses virtualization to route all of a vm's traffic through a TOR instance running on another network facing gateway vm.
Its known as {{project_name_short}} Just letting you know that I'd appreciate any help on getting the main issues with porting it to KVM solved
so we can prepare this for a greater good.
This is the bugtracker page I am working on resolving: https://www.whonix.org/wiki/Dev/KVM#How_to_get_virtio_running_for_storage.3F
iggy do you mind if I post the contents of this thread on our forum?
no...
Is there any security drawbacks from using a raw image file?
bancfc: security drawbacks? like what?
iggy: are its contents directly interpreted by the host or is that when lvm is used only?
I'm not sure what you mean by "contents directly interpreted", but raw devices/files just means there's a 1:1 correlation between guest request and host request
like a virus or advanced persistent threat running in the vm
the host doesn't "interpret" what it is reading/writing... it just becomes a read/write in the host
ok good just checking. iAnother question is raw snapshotting possible albeit not directly?
I saw something about a differencing image applied in qcow format, yes?
only if there's something underlying the files that supports it (btrfs, lvm, zfs, etc)
so not at the "block" level I guess?
iggy: just a question or two about storage if thats ok, becuase we are trying to see which can give maximum performance
lvm > raw files > qcow2 > everything else
Is qcow2 the standard? Is qed recommended instead? When is the qcow"3" changes included into the present format?
didn't mean to send this as you had already responded
but anyhow qed seems to have been designed as a speedier replacement for qcow2. But qcow2 were adding changes to become more competitive again
I wanted to know what you think about this and thats all. thanks
bancfc: qed is deprecated (as all of it is performance improvements that actually ended up helping were backported to qcow2)
the features are handled by feature flags in the qcow2 format, so there's technically no need for a "qcow3" format
as long as you don't use the features that qed didn't support (i.e. snapshots, etc.), qcow2 should be as fast
iggy: thanks alot for your help
np
* Now talking on #tor
* Topic for #tor is: Welcome to #tor, the discussion channel for Tor users and operators | https://www.torproject.org/ | https://www.torproject.org/docs/faq | https://tor.stackexchange.com/ | Developer discussion on #tor-dev | It is 'Tor', not 'TOR' | non-tor talk => #nottor
* Topic for #tor set by ChanServ!services@services.oftc.net at Sun Feb 2 13:56:09 2014
* Riastradh (~riastradh@82JAAD1W2.tor-irc.dnsbl.oftc.net) has joined #tor
I am working on the {{project_name_short}} KVM port and ran the leaktest for this bug reported here: https://lists.torproject.org/pipermail/tor-talk/2014-March/032503.html
It turns out that this issue applies to us too and its an upstream bug
i believe you
upstream meaning what -- the kernel? iptables?
I am thinking iptables
You guys can probably get to them easier than we can
is there a patch?
* drtor (~drtor@9YYAAK2T4.tor-irc.dnsbl.oftc.net) has joined #tor
We are looking at some workaround in iptables but are not sure how well this will work
but one thing for sure is it needs to be fixed upstream because it affects any Linux platform that uses Tor as a transproxy
AFAIK Patrick -Whoix leader- doesn't write kernel level code
https://www.whonix.org/old-forum/index.php/topic,243.0.html
* realitygaps has quit (Remote host closed the connection)
According to my findings, the leak can be seen in tcpdump running on the host, as direct communication to Google without going through any Tor node. Its a catastrophe by all means
a kernel patch is not necessarily appropriate, and upstream might well not consider it a bug
the iptables workaround i gave is in a followup post from mike in that thread
it would appear that "it needs to be fixed upstream" is just false
i only skimmed that forum page quickly, and i really don't want to wade through every bit of it
it contains my workaround based on --state INVALID
That's fine we will apply your advice, but are you sure that there is nothing fundamentally wrong with the kernel or iptables though?
but it doesn't clearly seem to have been used
apply it and test, a lot
there is not yet any solid case made that there is something "fundamentally wrong"
using iptables to redirect for transproxying relies on connection tracking
i think the conclusion is "it is hard to use these things safely"
which is not really a bug in upstream so much as a challenge with the approach
you have to test
people don't test thoroughly
they just read a man page or get some cookbook approach from some post, and run with it
that is why something like this goes unnoticed for so long
(gee, sounds like some other notable problems ...)
Sorry to throw another link your way, but so far this is what we do when testing:
https://www.whonix.org/wiki/Dev/Leak_Tests#FIN_ACK_.2F_RST_ACK_-_Leak_Test
Any advice if this is enough or anything we should add to ensure its safe after a workaround?
* quiliro has quit (Quit: Leaving.)
do violent tests with your networking connection
You are very sharp btw. That bug was a great find
interrupt it and restore it at various moments, for various periods
* blumenkraft (~eristisk@a1.networktest.34sp.com) has joined #tor
well actually i never announced or reported the problem, so even though i fixed the problem for myself a couple years ago, mike appropriately gets credit for finding it. i just get credit for a better workaround.
* amiller has quit (Remote host closed the connection)
y'all need to understand that connection tracking was never intended to be more than a loose fit around categories of traffic, not some leakproof box
do you know if this applies to nftables too?
Its gradually becoming the successor to iptables and was added in 3.13
when the kernel's info on a connection is lost or dropped (timeout), it is gone forever, so subsequent traffic won't be redirected
i haven't used it and don't know. unless there's been some massively radical re-architecting of connection tracking, it is unlikely to be totally different, though there could be minor differences in behavior.
the tails approach is starting to look a bit smarter
* eristisk has quit (Ping timeout: 480 seconds)
tails is robus but it has to secure an entire rich os stack to protect the user
approach to torifying the traffic i mean
ok
i don't argue that wrapping stuff in vm's can make it harder for an attacker to break out
in that tails doesn't rely on transproxy redirection, yes it is better
what specifics are you talking about for torifying maybe we can replicate that?
but torifying outgoing streams is harder to make safe than explicit proxy settings
* darkclown (~darkclown@p2011-ipbf7306marunouchi.tokyo.ocn.ne.jp) has joined #tor
* anong0 has quit (Ping timeout: 480 seconds)
right, we've always warned that transproxying is not as great as it seems
"set your proxy settings to use tor, only allow tor to use the network"
* Gentlecat has quit (Ping timeout: 480 seconds)
so, implicitly, "don't allow anything that is not configured to use tor to use the network"
and most people just ignore the warnings about transproxying, because "just stuffing everything through tor" seems just what they want, and so simple. but not really.
* mdik is now known as Guest6988
* mdik (~mdik@brln-4d0c41d4.pool.mediaWays.net) has joined #tor
the problem is when a box is rooted the software that is configured to use Tor can do things it shouldn't do
but I get your point
* Guest6988 has quit (Ping timeout: 480 seconds)
its those people who think stuffing flash through a transproxy pipe think their safe
that problem only exists for you because you insist on running tor in isolation
* huseby has quit (Ping timeout: 480 seconds)
"be sure to send unexpected traffic over tor so it can't hurt us"
vs "be sure to drop unexpected traffic so it can't hurt us"
option b sure seems wiser
then a bad guy who breaks in can, at worst, generate traffic that looks expected so it goes through tor
Useability wise, just having unique circuit per PID might be reasonable compromise.
* TheCthulhu1 has quit (Remote host closed the connection)
i think that's a totally separate topic
arma: an important goal we are trying to achieve is for operators of onion services to be able to have a fighting chance in case they are targeted.
Trying to educate users "don't throw * at tor, but selectively firewall everything" seems like a losing battle though.
it does? screw those users then.
better to write software that works well
and the ones who want to use the safe software will use it
right, it can only be won or lost if you consider it to be a battle. users screw themselves, every day
and we can't do much for the ones who choose to do it wrong
a better analogy might be leading a horse to water
* Rym has quit (Ping timeout: 480 seconds)
That sounds like somewhat elitist stance. "You dont deserve privacy because you're dumb" vs "Safe defaults built in."
obviously better defaults are better, if they actually exist and are actually safer
(I can see how it can go downhill from there if users are babysitted too much, though)
but defaults never cover everything, because software is flexible, and people want it to be
The less decisions that need to be made in security, the better everyone will be
and hence why transproxy is an idea worth working towards
so give them only the applications that you know behave well, with those applications configured to use tor correctly
that principle is correct but the conclusion about transproxying is wrong
letting them pick their own apps and hope they are safe is a recipe for surprises again and again
we do, but we can't guarantee that nothing dangerous happens when these apps are compromised unless it *all* goes through Tor
Ultimately the problem with transproxy is just that it is difficult to identify the source app.
* sd uses assigned source port ranges, but it is far from ideal
Kgpg, TorBrowser and xchat come by default
even if it all goes through tor, you can't guarantee that nothing dangerous happens, you only have considered the first layer of potential problems
updated system and AppArmor are other layers we have
that's all very nice and good
but against a well funded adversary you must throuw everything in the toolbox and the toolbox itself to protect users
actually it is just more and more desperate band-aiding
the effort would be much better directed towards thoroughly auditing apps
Auditing firefox is out of our league
but we have verifiable builds for {{project_name_short}} Velope, that goal is far from realistic when we're talking huge blobs of code such as web browser or libpurple.
it is the true longterm challenge that people get scared away from all the time
* mttp_ (~mttp@9YYAAK2WG.tor-irc.dnsbl.oftc.net) has joined #tor
and the "community" pays for it dearly with things like heartbleed