diff --git a/esxi/readme.md b/esxi/readme.md
index 6959a56..62d14fd 100644
--- a/esxi/readme.md
+++ b/esxi/readme.md
@@ -1,22 +1,101 @@
# Esxi
###### guide-by-example
+
+
# Purpose
-Type 1 hypervisor hosting virtual machines, running straight on metal.
+Type 1 hypervisor hosting virtual machines, running straight on metal,
+managed through web GUI.
+
+ESXi is made by vmware. It is a commercial product part of vSphere,
+but offers free version when one registers on their site.
+The free version has limit of max 2 physical cpus, max 8 vCPU per VM,
+and no access to API which prevents ease of making backups of the VMs.
+But for backups there is solution of using ghettovcb script.
+
+ESXi is also considered bit picky when it comes to hardware.
+Having natively supported network card and disk controller is not given.
+
+
# Basic settings
-https://www.dbappweb.com/2020/08/20/how-to-change-the-default-gateway-for-vmware-vsphere-esxi/
+### Password complexity
-* esxcfg-route
-* esxcfg-route 10.65.26.25
+* Host > Manage > System > Advanced settings > Password quality control
+ `retry=10 min=1,1,1,1,1`
-https://blog.techygeekshome.info/2021/04/vmware-esxi-esxcli-commands-to-update-host-dns-servers/
+### New user
-* esxcli network ip dns server add --server=8.8.8.8
-* esxcli network ip dns server remove --server=1.1.1.1
-* esxcli network ip dns server list
+* Add new user
+ Host > Manage > Security & users > Users > Add user
+ Set name and password.
+* Set permissions for the new user
+ Host > Actions > Permissions
+ Set the same user as before, pick role
+
+### Interface
+
+* Right top corner > user name
+ * Auto-refresh=60
+ * Settings, turn off everything - statistics,
+ recent only, welcome message, visual effects
+
+### NTP time sync
+
+* Host > Manage > System > Time & date > Edit NTP Settings
+ * Use Network Time Protocol (enable NTP client)
+ * Start and stop with host
+ * `pool.ntp.org`
+ * Host > Manage > Services > search for ntpd > Start
+
+### Hostname and domain
+
+* ssh in
+* `esxcli system hostname set --host esxi-2023`
+* if domain on network
+ `esxcli system hostname set --domain example.local`
+
+### Network
+
+Should just work, but if there is more complex setup,
+like if a VM serves as a firewall...
+Be sure you ssh in and try to `ping google.com` to see if network and DNS work.
+
+To check and set the default gateway
+
+* `esxcfg-route`
+* `esxcfg-route 10.65.26.25`
+
+To [change DNS server](https://blog.techygeekshome.info/2021/04/vmware-esxi-esxcli-commands-to-update-host-dns-servers/)
+
+* `esxcli network ip dns server list`
+* `esxcli network ip dns server add --server=8.8.8.8`
+* `esxcli network ip dns server remove --server=1.1.1.1`
+* `esxcli network ip dns server list`
+
+To disable ipv6
+
+* `esxcli network ip set --ipv6-enabled=false`
+
+# Logs
+
+[Documentation](https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.monitoring.doc/GUID-832A2618-6B11-4A28-9672-93296DA931D0.html)
+
+Host > Monitor > Logs
+
+The one worth knowing about
+
+* shell.log - History of shell commands when SSH in
+* syslog.log - General info of what's happening on the system.
+* vmkernel.log - Activities of esxi and VMs
+
+Will update with some actual use, when I use logs.
+
+Logs from systems in VMs are in >Virtual Machines > Name-of-VM > Monitor > Logs
+
+
# Backups using ghettoVCB
@@ -65,7 +144,8 @@ Looks like this `/vmfs/volumes/6187f7e1-c584077c-d7f6-3c4937073090`
Gotta know basics how to edit files with ancient `vi`
* Config file template is in `/opt/ghettovcb/ghettoVCB.conf`
- Make copy of it `cp /opt/ghettovcb/ghettoVCB.conf /opt/ghettovcb/ghetto_1.conf`
+ Make copy of it
+ `cp /opt/ghettovcb/ghettoVCB.conf /opt/ghettovcb/ghetto_1.conf`
* Only edit this file, for starter setting where to copy backups
`vi /opt/ghettovcb/ghetto_1.conf`
`VM_BACKUP_VOLUME=/vmfs/volumes/6187f7e1-c584077c-d7f6-3c4937073090/Backups`
@@ -134,24 +214,76 @@ To execute it periodicly cron is used.
`kill $(cat /var/run/crond.pid)`
`crond`
+Logs about backups are in `/tmp`
+
### Restore from backup
[Documentation](https://communities.vmware.com/t5/VI-VMware-ESX-3-5-Documents/Ghetto-Tech-Preview-ghettoVCB-restore-sh-Restoring-VM-s-backed/ta-p/2792996)
-*
-
-
-Logs about backups are in `/tmp`
-
-# disk reclamation
-
+* In webgui create a full path where to restore the VM
+* The restore-config-template-file is in `/opt/ghettovcb/`
+ named `ghettoVCB-restore_vm_restore_configuration_template`
+ Make copy of it
+ `cp /opt/ghettovcb/ghettoVCB-restore_vm_restore_configuration_template /opt/ghettovcb/vms_to_restore_list`
+* Edit this file, adding new line, in which separated by `;` are:
+ * path to the backup, the directory with date in name
+ * path where to restore this backup
+ * disk type - 1=thick | 2=2gbsparse | 3=thin | 4=eagerzeroedthick
+ * optional - new name of the VM
+ `vi /opt/ghettovcb/vms_to_restore_list`
+ ```
+ "/vmfs/volumes/6187f7e1-c584077c-d7f6-3c4937073090/Backups/OPNsense/OPNsense-2023-04-16_04-00-00;/vmfs/volumes/6378107d-b71bee00-873d-b42e99f40944/OPNsense_restored;3;OPNsense-restored"
+ ```
+* Execute the restore script with the config given as a parameter.
+ `opt/ghettovcb/bin/ghettoVCB-restore.sh -c /opt/ghettovcb/vms_to_restore_list`
+* Register the restored VM.
+ If it's in the same location as the original was, it should just go through.
+ If the location is different then esxi asks if it was moved or copied.
+ * Copied - You are planning to use both VMs at the same time,
+ selecting this option generates new UUID for the VM, new MAC address,
+ maybe some other hardware identifiers as well.
+ * Moved - All old settings are kept, for restoring backups this is usually
+ the correct choice.
# Switching from Thick to Thin disks
+Kinda issue is that vmdk are actually two files.
+Small plain `.vmdk` that holds some info, and the `flat.vmdk` with actual
+gigabytes of data of the disk. In webgui this fact is hidden.
-vmkfstools --punchzero "./TrueNAS_scale-thin.vmdk"
+* have backups
+* down the VM
+* unregister the VM
+* ssh in
+* navigate to where its vmdk files are in datastore
+ `cd /vmfs/volumes/6187f7e1-c584077c-d7f6-3c4937073090/linux/`
+* execute command that converts the vmdk
+ `vmkfstools -i "./linux.vmdk" -d thin "./linux-thin.vmdk"`
+* zeropunch the image file, so that unused blocks are properly zeroed.
+ `vmkfstools --punchzero "./linux-thin.vmdk"`
+* remove or move both original files
+ `rm linux.vmdk`
+ `rm linux-flat.vmdk`
+* In webgui navigate to the datastore.
+ Use `move` command to rename thin version to the original name.
+ This changes the values in `linux.vmdk` to point to correct `flat.vmdk`
+* register the VM back to esxi gui.
-unmap, windows, `fsck_ufs -Ey /dev/da0p3`
+# Disk space reclamation
+
+If you run VMs with thin disks, the idea is that it uses only as much space
+as is needed. But if you copy 50GB file to a VM, then deletes it, it's not always
+seamless that the VMDK shrinks by 50GB too.
+
+Correctly functioning reclamation can save time and space for backups.
+
+* Modern windows should just work, did just one test with win10.
+* linux machines need fstrim run that marks blocks as empty.
+* Unix machine, like opnsense based on FreeBSD needed to be started from ISO,
+ so that partition is not mounted and executed
+ `fsck_ufs -Ey /dev/da0p3`
+ afterwards it needed one more run of vmkfstools --punchzero "./OPNsense.vmdk"
+ And it still uses roughly twice as much space as it should.
# links
@@ -160,5 +292,3 @@ unmap, windows, `fsck_ufs -Ey /dev/da0p3`
* https://forums.unraid.net/topic/30507-guide-scheduled-backup-your-esxi-vms-to-unraid-with-ghettovcb/
* https://blog.kingj.net/2016/07/03/how-to/backing-up-vmware-esxi-vms-with-ghettovcb/
* https://sudonull.com/post/95754-Backing-up-ESXi-virtual-machines-with-ghettoVCB-scripts#esxi-3
-
-#### email