The information on this page is written for a host running Proxmox but should be easy to adapt to any machine running LXC/LXD.
Since a device is being shared between two systems there are almost certainly some security implications and I haven't been able to determine what degree of security you're giving up to share a GPU.
#### Determine Device Major/Minor Numbers To allow a container access to the device you'll have to know the devices major/minor numbers. This can be found easily enough by running `ls -l` in `/dev/`. As an example to pass through the integated UHD 630 GPU from an Core i7 8700k you would first list the devices where are created under `/dev/dri`. ```bash root@blackbox:~# ls -l /dev/dri total 0 drwxr-xr-x 2 root root 80 May 12 21:54 by-path crw-rw---- 1 root video 226, 0 May 12 21:54 card0 crw-rw---- 1 root render 226, 128 May 12 21:54 renderD128 ``` From that you can see the major device number is `226` and the minors are `0` and `128`. #### Provide LXC Access In the configuration file you'd then add lines to allow the LXC guest access to that device and then also bind mount the devices from the host into the guest. In the example above since both devices share the same major number it is possible to use a shorthand notation of `226:*` to represent all minor numbers with major number `226`. ```diff # /etc/pve/lxc/*.conf + lxc.cgroup.devices.allow: c 226:* rwm + lxc.mount.entry: /dev/dri/card0 dev/dri/card0 none bind,optional,create=file,mode=0666 + lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file ``` #### Allow `unprivileged` Containers Access In the example above we saw that `card0` and `renderD128` are both owned by `root` and have their groups set to `video` and `render`. Because the "unprivilged" part of LXC unprivileged container works by mapping the UIDs (user IDs) and GIDs (group IDs) in the LXC guest namespace to an unused range of IDs on host, it is necessary to create a custom mapping for that namespace that maps those groups in the LXC guest namespace to the host groups while leaving the rest unchanged so you don't lose the added security of running an unprivilged container. First you need to give root permission to map the group IDs. You can look in `/etc/group` to find the GIDs of those groups, but in this example `video` = `44` and `render` = `108` on our host system. You should add the following lines that allow `root` to map those groups to a new GID. ```diff # /etc/subgid + root:44:1 + root:108:1 ``` Then you'll need to create the ID mappings. Since you're just dealing with group mappings the UID mapping can be performed in a single line as shown on the first line addition below. It can be read as "remap `65,536` of the LXC guest namespace UIDs from `0` through `65,536` to a range in the host starting at `100,000`." You can tell this relates to UIDs because of the `u` denoting users. It wasn't necessary to edit `/etc/subuid` because that file already gives root permission to perform this mapping. You have to do the same thing for groups which is the same concept but slightly more verbose. In this example when looking at `/etc/group` in the LXC guest it shows that `video` and `render` have GIDs of `44` and `106`. Although you'll use `g` to denote GIDs everything else is the same except it is necessary to ensure the custom mappings cover the whole range of GIDs so it requires more lines. The only tricky part is the second to last line that shows mapping the LXC guest namespace GID for `render` (`106`) to the host GID for `render` (`108`) because the groups have different GIDs. ```diff # /etc/pve/lxc/*.conf lxc.cgroup.devices.allow: c 226:* rwm lxc.mount.entry: /dev/dri/card0 dev/dri/card0 none bind,optional,create=file,mode=0666 lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file + lxc.idmap: u 0 100000 65536 + lxc.idmap: g 0 100000 44 + lxc.idmap: g 44 44 1 + lxc.idmap: g 45 100045 61 + lxc.idmap: g 106 108 1 + lxc.idmap: g 107 100107 65429 ``` Beaues it can get confusing to read I just wanted show each line with some comments... ```diff + lxc.idmap: u 0 100000 65536 // map UIDs 0-65536 (LXC namespace) to 100000-165535 (host namespace) + lxc.idmap: g 0 100000 44 // map GIDs 0-43 (LXC namspace) to 100000-100043 (host namespace) + lxc.idmap: g 44 44 1 // map GID 44 to be the same in both namespaces + lxc.idmap: g 45 100045 61 // map GIDs 45-105 (LXC namspace) to 100045-100105 (host namespace) + lxc.idmap: g 106 108 1 // map GID 106 (LXC namspace) to 108 (host namespace) + lxc.idmap: g 107 100107 65429 // map GIDs 107-65536 (LXC namspace) to 100107-165536 (host namespace) ``` #### Add `root` to Groups Because `root`'s UID and GID in the LXC guest's namespace isn't mapped to `root` on the host you'll have to add any users in the LXC guest to the groups `video` and `render` to have access the devices. As an example to give `root` in our LXC guest's namespace access to the devices you would simply add `root` to the `video` and `render` group. ```bash usermod --append --groups video,render root ``` #### Potential Alernative [lxc.mount.entry - static uid/gid in LXC guest](https://www.reddit.com/r/Proxmox/comments/q5wbl0/lxcmountentry_static_uidgid_in_lxc_guest/) #### Resources [Proxmox: Unprivileged LXC containers](https://pve.proxmox.com/wiki/Unprivileged_LXC_containers) # LXC NIC Passthrough On the rare occation you have a good reason to forgo the small overhead of an _veth_ [(Virtual Ethernet)](https://man7.org/linux/man-pages/man4/veth.4.html) device connected to an [ethernet bridge](https://wiki.archlinux.org/index.php/Network_bridge) it is possible to pass a physical network interface directly to a LXC host. To pass a physical device you just need to provide `lxc.net.[index].type` and `lxc.net.[index].link` parameters in the LXC config. You may optionally provide a name for the link as well with `lxc.net.[index].name`. Just be sure your index value is unique among all network interfaces fot the LXC container including those Proxmox may add if you running your LXC hosts on Proxmox. ``` lxc.net.0.type: phys lxc.net.0.link: enp1s0 # optional lxc.net.0.name: eth0 ``` # netfilter/iptable logging > Logging from network namespaces other than init has been disabled since kernel 3.10 in order to prevent host kernel log flooding from inside a container. Source: [lxc-users.linuxcontainers.narkive.com](https://lxc-users.linuxcontainers.narkive.com/fuIM2bZg/lxc-and-netfilter-log) There are two ways to get logging working on guests running in Namespaces. The first is to simply enable it on even though it is off by default due to the security concerns mentioned above. The second _and better_ way is to use User space logging which doesn't carry the same restrictions because it doesn't interact with Kernel space in the same way. Besides the User space logging method being the best security practice, anytime it is possible to modify the host machine less it is better in my opinion. ## Method 1: Userspace Logging (on guest) Install `ulogd2` ```bash apt install ulogd2 ``` Replace `LOG` in any `iptable/netfilter` rules with `NFLOG` ```diff - -A INPUT -j LOG + -A INPUT -j NFLOG ``` Source: [lxadm.com](https://lxadm.com/Iptables:_LOG_target_not_working_in_LXD_containers) ## Method 2: Enable Logging In Namespaces (on host) > Logging from network namespaces other than init has been disabled since kernel 3.10 in order to prevent host kernel log flooding from inside a container.The information on this page is written for a host running Proxmox but should be easy to adapt to any machine running LXC/LXD.
#### Locate Bus/Device ```bash root@vault:~# lsusb Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 003: ID 13d3:3273 IMC Networks 802.11 n/g/b Wireless LAN USB Mini-Card Bus 001 Device 004: ID 10c4:8a2a Silicon Labs HubZ Smart Home Controller Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub ``` #### Determine Device Major/Minor Numbers ```bash root@vault:~# ls -l /dev/bus/usb/001/004 crw-rw-r-- 1 root root 189, 3 Oct 3 17:17 /dev/bus/usb/001/004 ``` From that you can see the major device number is `189` and the minor is `3`. #### Provide LXC Access In the configuration file you'd then add lines to allow the LXC guest access to that device and then also bind mount the devices from the host into the guest. In the example above since both devices share the same major number it is possible to use a shorthand notation of `189:*` to represent all minor numbers with major number `189`. ```diff # /etc/pve/lxc/*.conf + lxc.cgroup.devices.allow: c 189:* rwm + lxc.mount.entry: /dev/bus/usb/001/020 dev/bus/usb/001/020 none bind,optional,create=file,mode=664 ``` #### Allow `unprivileged` Containers Accessincomplete
#### Resources [USB Passthrough to an LXC (Proxmox)](https://medium.com/@konpat/usb-passthrough-to-an-lxc-proxmox-15482674f11d)