# mini # Physical Hardware ## Basic Components ### Compute ![](https://www.amd.com/system/files/82446-raven-am4-ryzen-3-pib-left-facing-1260x709.png)#### Processor [AMD Ryzen 3 3200G](https://en.wikichip.org/wiki/amd/ryzen_3/3200g) - Economical CPU that has 4 cores and integrated graphics so a PCIe slot can be saved since a GPU isn't necessary.
Cores / Threads4 / 4
Base Frequency3.6 GHz
Burst Frequency4.0 GHz
Cache4MB L3 Cache
TDP65W
![](https://www.vortez.net/articles_file/40128_asrock_b450m_pro4_front_box.jpg)### Motherboard [Asrock B450M Steel Legend](https://www.asrock.com/mb/amd/b450%20steel%20legend/index.asp) - Small form factor mATX board that is being reused from a prototyping project. Being an mATX board it is very limited in expansion.
**Manufacturer**Asrock
**Model**B450M Steel Legend
**CPU**AMD AM4 Socket
**Chipset**AMD Promontory B450
**Memory**- 4x DDR4 DIMM supporting up to 64GB - Dual channel memory architecture
**Display**- 1x HDMI 1.4 - 1x DisplayPort 1.2
**Networking**- 1x Realtek RTL8111H 1GbE LAN
**Expansion**- 1x PCIe 3.0 x16 - 1x PCIe 2.0 x16 - 1x PCIe 2.0 x1
**Storage**- 1x M.2 PCIe x4/x2 - 1x M.2 SATA x4/x2 - 4x SATA3
**USB**- 1x USB 3.1 Gen 2 (Type-C) - 1x USB 3.1 Gen 2 (Type-A) - 4x USB 3.1 Gen 1 (Type-A) - 2x USB 2.0
### Memory
Slot 1 ![](https://alfawazit.com/wp-content/uploads/2020/06/CMK16GX4M1A2400C16-Gallery-VENG-LPX-BLK-03.png)Corsair Vengeance LPX 8GB DDR4 2666MHz (1x8GB) - 2Rx8 Dual Rank - CAS Latency 16 - timing 16-18-18-35 - 1.2V
Slot 2 ![](https://alfawazit.com/wp-content/uploads/2020/06/CMK16GX4M1A2400C16-Gallery-VENG-LPX-BLK-03.png)Corsair Vengeance LPX 8GB DDR4 2666MHz (1x8GB) - 2Rx8 Dual Rank - CAS Latency 16 - timing 16-18-18-35 - 1.2V
Slot 3 ![](https://alfawazit.com/wp-content/uploads/2020/06/CMK16GX4M1A2400C16-Gallery-VENG-LPX-BLK-03.png)Corsair Vengeance LPX 8GB DDR4 2666MHz (1x8GB) - 2Rx8 Dual Rank - CAS Latency 16 - timing 16-18-18-35 - 1.2V
Slot 4 ![](https://alfawazit.com/wp-content/uploads/2020/06/CMK16GX4M1A2400C16-Gallery-VENG-LPX-BLK-03.png)Corsair Vengeance LPX 8GB DDR4 2666MHz (1x8GB) - 2Rx8 Dual Rank - CAS Latency 16 - timing 16-18-18-35 - 1.2V
![](https://www.fractal-design.com/wp-content/uploads/2019/06/Define-Mini-C_1.jpg)### Case [Fractal Design - Define Mini C (Blackout)](https://www.fractal-design.com/products/cases/define/define-mini-c/black/) - A fantastic case with an attractive minimalistic design that in a mATX form factor.
**Manufacturer**Fractal Design
**Model**Define Mini C (Blackout)
**Features**- Sound dampening panels - Excellent build quality - 2x 3.5" Drive Bays - 2x 2.5" Drive Bays
### Storage
**\#****Capacity****Interface****Type****Manufacturer & Model****Speed**
1x ![](https://shop.westerndigital.com/content/dam/store/en-us/assets/products/internal-storage/wd-black-sn750-nvme-ssd/gallery/without-heatsink/wd-black-sn750-nvme-ssd-Noheatsink2.png.thumb.1280.1280.png)256GBNVMeSSDWestern Digital Black WDS256G1X0CPCIe 3.0 x2
2x ![](https://ssdcomparison.com/wp-content/uploads/2015/04/Crucial-MX100-Vs-M500.png)512GBSATASDDCrucial MX100SATA3 6.0Gb/s
1x ![](https://cdn.cnetcontent.com/syndication/feeds/wd/inline-content/92/0519E12CA7ABBD8CF2D927E60BC1553957DDFB1C_WDBLUEPCDESKTOPHARDDRIVE1TB_gallery.png)1TBSATAHDDWestern Digital WD10EZEXSATA3 6.0Gb/s
1x ![](https://5.imimg.com/data5/SELLER/Default/2021/2/TS/GU/XU/58564276/cctv-hard-disk-500x500.png)1TBSATAHDDSeagate ST1000DM003SATA3 6.0Gb/s
### Cooling
CPU ![](https://avadirect-freedomusainc1.netdna-ssl.com/Pictures/500/11927135_3.png)AMD Wraith Spire
Case (Front) ![](https://cdn-reichelt.de/bilder/web/xxl_ws/E201/NF-A14-PWM_04.png)[Noctua NF-A14 PWM 140mm](https://noctua.at/en/products/fan/nf-a14-pwm)
Case (front) ![](https://img.gigatron.rs/img/products/large/image5a54b5d0d3252.png)[Noctua NF-F12 PWM 120mm](https://noctua.at/en/products/fan/nf-f12-pwm)
Case (rear) ![](https://img.gigatron.rs/img/products/large/image5a54b5d0d3252.png)[Noctua NF-F12 PWM 120mm](https://noctua.at/en/products/fan/nf-f12-pwm)
![](https://images.evga.com/products/gallery//png/220-G2-0550-Y2_XL_4.png)### Power Supply
ManufacturerEVGA
Model[SuperNOVA 550 G2](https://www.evga.com/products/product.aspx?pn=120-GP-0850-X1)
Features- 550W - fully module
### UPS n/a ## Add-On Cards ---
PCI 3.0 x16 ![](https://bookstack.swigg.net/uploads/images/gallery/2021-04/scaled-1680-/gigabyte-radeon560.png)[Radeon RX 560 Gaming OC 4G (rev. 1.0)](https://www.gigabyte.com/us/Graphics-Card/GV-RX560GAMING-OC-4GD-rev-10)- PCIe Gen3 x8
PCI 3.0 x1 ![](https://bookstack.swigg.net/uploads/images/gallery/2021-04/scaled-1680-/inatek-quad-usb.png)[Inateck 4 Ports PCIe to USB 3.0](https://www.inateck.com/inateck-kt4004-4-port-usb-3-0-pci-e-express-card-for-mac-pro.html)- PCIe Gen3 x1
PCI 2.0 x16 ![](https://bookstack.swigg.net/uploads/images/gallery/2021-04/scaled-1680-/10gtek-single-sfp.png)[10Gtek Intel 82599ES SFP+ PCIe x8](https://www.amazon.com/gp/product/B01LZRSQM9)- PCIe Gen3 x8 - SFP+ 10GbE port - SR-IOV
# Host Configuration # Base Install ## Operating System ---------------------- [Proxmox Virtual Environment](https://www.proxmox.com/en/) 6.x ## Configuration

Proxmox configuration has been transitioned to being automated by an Ansible Role

# Networking (out-of-date) ### Configuration Because I don't want my main management interface to ever change names, I explicitly give it a name based on its MAC address. ```diff # /etc/systemd/network/10-management-net.link + [Match] + MACAddress=70:85:c2:fe:4c:b7 + + [Link] + Name=man0 ``` ### Bridges | Master | Bridge | IP Address | Gateway | Description | |--------|--------|-------------|----------|-------------| | man0 | vmbr0 | 10.0.2.5/21 | 10.0.2.1 | Main Interface (slower Realtek NIC) | | enp6s0 | -- | -- | -- | Intel 10GbE SFP+ (used for PCI passthrough) # Common Software ### Install fail2ban This blocks connections that make repeated failed attempts to authenticate. SSH is covered by default which is what I am interested in, and I'll add additional config to similarly block too many repeated auth failures against the Proxmox web interface. ```bash apt install fail2ban ``` ```diff # /etc/fail2ban/jail.local + [proxmox] + enabled = true + port = https,http,8006 + filter = proxmox + logpath = /var/log/daemon.log + maxretry = 3 + # 1 hour + bantime = 3600 ``` ### Install sysfsutils Sysfs is a virtual file system in Linux kernel 2.5+ that provides a tree of system devices. This package provides the program 'systool' to query it: it can list devices by bus, class, and topology. In addition this package ships a configuration file /etc/sysfs.conf which allows one to conveniently set sysfs attributes at system bootup (in the init script etc/init.d/sysfsutils). ```bash apt install sysfsutils ``` ### Install Netdata Monitoring Install [Netadata](https://www.netdata.cloud/) so that I can get a detailed view of system metrics. It will also be used as a datasource for [LXC / Conception / Prometheus](https://bookstack.swigg.net/books/blackboxhermz/page/lxc-conception-prometheus) so I can look at metrics over a larger timeframe. ```bash apt update apt install curl bash <(curl -Ss https://my-netdata.io/kickstart.sh) ``` # Setup PCI Passthrough See [PCI Passthrough](https://bookstack.swigg.net/books/how-to-guides/page/pci-passthrough) for more detail as to why I am doing these things. Proxmox doesn't need a GPU, so blacklist the GPU and prepare it to be passed for a guest machine. ### Enable Kernel Modules ```diff # /etc/modules # /etc/modules: kernel modules to load at boot time. # # This file contains the names of kernel modules that should be loaded # at boot time, one per line. Lines beginning with "#" are ignored. + vfio_pci + vfio + vfio_iommu_type1 + vfio_virqfd ``` ### Bind `vfio-pci` Driver to Devices ```diff # /etc/modprobe.d/vfio.conf + # AMD Radeon RX 560 [1002:67ff,1002:aae0] + alias pci:v00001002d000067FFsv00001458sd000022FFbc03sc00i00 vfio-pci + alias pci:v00001002d0000AAE0sv00001458sd0000AAE0bc04sc03i00 vfio-pci + + options vfio-pci ids=1002:aae0,1002:67ff disable_vga=1 ``` ### Rebuild `initramfs` The `initramfs` needs to be rebuilt to reflect the changes I just did. ```bash update-initramfs -u ``` ### Update Bootloader Proxmox uses `systemd-boot` as the bootloader so I have to make sure to update the boot entries #### Update Kernel Parameters ```diff # /etc/kernel/cmdline - root=ZFS=rpool/ROOT/pve-1 boot=zfs + root=ZFS=rpool/ROOT/pve-1 boot=zfs amd_iommu=on iommu=on video=efifb:off pcie_acs_override=multifunction ``` #### Rebuild Bootloader Options ```bash pve-efiboot-tool refresh ``` # Storage & Backups (out-of-date) ### Setup ZFS Scrub (Data Integrity) Automate [ZFS scrubbing](https://wiki.archlinux.org/index.php/ZFS#Scrubbing) so the data integrity on disks is actively monitored, repaired if necessary, and I'm alerted if there is a problem with my disks. #### Create systemd Service/Timer ([source](https://github.com/lnicola/systemd-zpool-scrub)) Create a simple systemd servcie template for scrubbing ZFS pools. ```diff # /etc/systemd/system/zpool-scrub@.service + [Unit] + Description=Scrub ZFS Pool + Requires=zfs.target + After=zfs.target + + [Service] + Type=oneshot + ExecStartPre=-/usr/sbin/zpool scrub -s %I + ExecStart=/usr/sbin/zpool scrub %I ``` Then create a systemd timer template for periodically running that service. I am running the scrub weekly, but semi-monthly or monthly would almost certainly be ok too. ```diff # /etc/systemd/system/zpool-scrub@.timer + [Unit] + Description=Scrub ZFS pool weekly + + [Timer] + OnCalendar=weekly + Persistent=true + + [Install] + WantedBy=timers.target ``` #### Enable ZFS Scrub ```bash systemctl daemon-reload systemctl enable --now zpool-scrub@rpool.timer ``` ### Setup Sanoid/Syncoid (Data Backup) Run [Sanoid](https://github.com/jimsalterjrs/sanoid) for automating snapshots and Syncoid for remote backups. Unfortunately this isn't available in repositories so you have to build it yourself. However the author makes it fairly simple. #### Install ([source](https://github.com/jimsalterjrs/sanoid)) ```bash apt-get install build-essential debhelper dpkg-buildpackage libcapture-tiny-perl libconfig-inifiles-perl pv lzop mbuffer sudo git clone https://github.com/jimsalterjrs/sanoid.git cd sanoid ln -s packages/debian . dpkg-buildpackage -uc -us apt install ../sanoid_*_all.deb ``` #### Configure Sanoid I want to take hourly snapshots of both of my ZFS pools because sometimes I am not as careful or thoughtful as I should be about what I am doing at any given moment. ```diff # /etc/sanoid/sanoid.conf + [template_proxmox] + frequently = 0 + hourly = 24 + daily = 7 + weekly = 4 + monthly = 1 + yearly = 0 + autosnap = yes + autoprune = yes + + [rpool] + use_template = template_proxmox + process_children_only = yes + recursive = yes + + [rpool/ROOT] + use_template = rpool + process_children_only = yes + recursive = yes + + [rpool/data] + use_template = template_proxmox + weekly = 1 + monthly = 1 + process_children_only = yes + recursive = yes ``` Maybe this is a sin, but I'd like my snapshots to be in local time so I don't have to do the (admittedly simple) conversion in my head. ```diff # /usr/lib/systemd/system/sanoid.service [Service] - Environment=TZ=UTC + Environment=TZ=EST ``` #### Configure Syncoid I haven't decided where I want to replicate to yet so I haven't configured syncoid yet. # VM / macOSAMD ## Description This VM is for running macOS via dedicated hardware so I have something faster than my laptop. ## Configuration ### Resources
HostnameCPUMemory
MiMac12 vCPU16GB
### Storage
DiskControllerSizePurpose
local-zfs:vm-100-disk-1ide01MNVRAM
local-zfs:vm-100-disk-0virtio200MBEFI boot loader
local-zfs:vm-100-disk-2virtio1TBboot disk
### PCI Passthrough
NameBDFSettings
Fresco USB 3.0 Controller04:00n/a
AMD RX 560 GPU08:00pcie=1,x-vga=1
Intel 10GbE SFP+ NIC06:00n/a
### Networking #### Interfaces
IDNameBridgeIP AddressComments
eno0n/avmbr010.0.2.5/21 (DHCP)1GbE
eno1n/a--10.0.10.5/24 (manual)10GbE
#### Configuration Because we want macOS to route all traffic destined for [blackbox.hermz](https://bookstack.swigg.net/books/blackboxhermz) over our 10GbE network we need to adjust our routing table to redirect all traffic to [LXC / Routeman](https://bookstack.swigg.net/books/blackboxhermz/page/lxc-routeman) first. Knowing that blackbox.hermz has ip addresses `10.0.2.2`, and `10.0.2.3` and all of its services run with `10.0.4.x` we can easily setup some updated routes. ```bash # 10.0.2.2 (main interface) and 10.0.2.3 (admin interface) ip route add 10.0.2.2/31 dev en1 via 10.0.10.6 # 10.0.4.x (services running on blackbox) ip route add 10.0.4.0/24 dev en1 via 10.0.10.6 ``` This works perfectly except it isn't persistent.