Skip to content

Best Practices#

How to Use Root External IPs in Tenants in VergeOS

In VergeOS Virtual Data Centers leveraging Tenants, a tenant may need a public IP address (External IP) from the root External network for use by VMs inside the tenant space.

Using Network Blocks

Network Blocks can be used to assign a group of IPs as a single unit, and often represent the most straightforward method of using an External IP inside a tenant space for VM NICs. They are created on the root External network, or the external interface upon which the IPs you intend to use are routable.

Pro/Cons

Pro - Allows direct assignment of a public IP to a tenant VM's network interface.
Pro - Leverages built-in Layer 3 functions, maintaining full visibility of the configuration for diagnostic and troubleshooting.
Con - Requires at minimum 4 total IP addresses to deliver 1 usable IP to a device; a /30 network block.

Creating a Network Block

In the VergeOS system root:

  1. Navigate to the network that represents the 'edge' of your VergeOS system; this is very often the root External network.
  2. Click Network Blocks from the left menu.
  3. Click New to create a new block.
  4. Enter your Network Block IP address in CIDR format. (a.b.c.d/n)
  5. Optional : Add a helpful description to your block.
  6. Since we're using this block with a tenant, set "Owner Type" to Tenant and set "Owner" to the tenant you'd like to assign this block to.
  7. Click Submit to save and assign the Block.

Block Addressing

When creating Network Blocks, VergeOS will validate the Block when applying the firewall rules in further steps. Failure to set a proper starting IP for a given range will result in an error. e.g. 10.1.2.108/30 would be a valid block. 10.1.2.110/30, while representing in many cases the same block, will fail to validate in the following steps and not function. Validate with a subnet calculator if you are unsure.

  1. Return to the main page for the network you just added the Block to.
  2. At the top, note the "Rule changes need to be applied" message. You may click "Apply Rules" here, or click "Rules" in the left menu to validate before applying. You may also leave rule application for later, but you must return and apply before your Block will be routed and functional.
  3. Navigate to the Tenant Networks view and, using the top filters, select "Needs FW Apply" "Yes"; your tenant network should be listed.
  4. Select the network with a click, then Apply Rules in the left menu to complete delivery of the Network Block to the tenant.

In the Tenant interface:

  1. Navigate to your External network. You will find your root-assigned Network Block listed with the description "External Network Address from service provider".
  2. Select the Block with a click, then click New Network on the left menu. This will create a new Internal network in the tenant using the address block.
  3. Note the Address Type is automatically set to Static with the Network Block address already set. Dynamic DHCP will also be enabled with the available IP range remaining usable already filled.
  4. Give the new network a name.
  5. When finished with any other customizations you require, click Submit to create the network.
  6. You will exit to the new network. Click Power On from the left menu to bring the network online. You will be presented a confirmation window to prevent accidental power-on.

Attach a VM NIC to the network with DHCP enabled to have the IP assigned to the NIC automatically. Alternatively, set the IP address(es) within your guest OS manually.

Creating a Virtual Wire

Virtual Wires are another common method to consume root-level IP space by tenant workloads. These are roughly analogous to physical wires in that they allow Layer 2 network traffic to "skip" routed network segments, in this case allowing a Tenant Internal network to communicate directly with a network outside of VergeOS. This may be a "WAN" network directly, or other network configured outside of VergeOS that has address space usable and is routable out to the internet.

Pro/Cons

Pro - Simple configuration within VergeOS, bypassing internal Layer 3 routing configuration.
Pro - Allows direct usage of External IPs on edge devices by consumers.
Pro - Minimal address space overhead; only the IP addresses used by clients.
Con - Virtual Wires only function when both networks they connect are running on the same node.
-This requires the External network and Tenant Node1 use a High Availability (HA) Grouping to maintain their grouping, which may impact HA event expectations.
Con - May make troubleshooting and diagnostic more difficult by removing VergeOS WebUI and native routing visibility.

For instructions on creating a Virtual Wire, see Creating a Virtual Wire.

Once your Virtual Wire is in place, virtual machines and other workloads with NICs connected to the Internal network the vwire is attached to will have a Layer 2 connection out of VergeOS and will function similarly to a VLAN in a traditional switch with regards to addressing and routing.

Address Translation

If a workload must have a consistent IP address, but does not need the address assigned to it directly, Address Translation may be the best method. This allows standard Layer 3 routing from your Public/External IP pool to any given workload in VergeOS via the built-in Rules and Networking system.

Pro/Cons

Pro - Follows standard and well understood routing conventions.
Pro - Allows full route visibility and control within VergeOS WebUI panels, which can aid troubleshooting and future changes.
Con - Does not allow the end device to be assigned the Public IP natively.
Con - Not all network traffic survives Address Translation, particularly if source/destination validation is required.

Due to the very extensive and flexible nature of VergeOS's network possibilities, we will provide 2 example configurations, with the Address Translation at differing points in the routing journey.

DNAT and SNAT Rules on Tenant Internal Network

  1. Navigate to the network that represents the 'edge' of your VergeOS system; this is very often the root External network.
  2. Click IP Addresses from the left menu.
  3. Click New to create a new IP.
  4. Set "Type" to Virtual IP.
  5. Fill in the IP Address.
  6. Optional : Add a helpful description.
  7. Set "Owner Type" to "Tenant"
  8. Set "Owner" to the tenant you'd like to assign this IP to.
  9. Click Submit to save.
  10. You will be returned to the "IP Addresses" view.
  11. Return to the main page for the network you just added the Block to.
  12. At the top, note the "Rule changes need to be applied" message. You may click "Apply Rules" here, or click "Rules" in the left menu to validate before applying. You may also leave rule application for later, but you must return and apply before your Address will be routed and functional.
  13. Navigate to the Tenant Networks view and, using the top filters, select "Needs FW Apply" "Yes"; your tenant network should be listed.
  14. Select the network with a click, then Apply Rules in the left menu to complete delivery of the IP Address to the tenant.

In the Tenant interface:

  1. Navigate to your External network. You will find your root-assigned IP Address listed with the description "External IP from service provider".
  2. Select the IP Address with a click, then click Edit on the left menu.
  3. Set "Owner Type" to "Network".
  4. Set "Owner" to the network your VM's NIC is attached to.
  5. Click Submit to save.
  6. Return to the Tenant External network view.
  7. Click Apply Rules to activate the automatic rule created to route your IP.
  8. Navigate to the Tenant network you set in step 4.

DNAT Option: (If your workload is compatible)

  1. Click Rules in the left panel.
  2. Click New in the left panel to create a new Rule.
  3. Give your rule a Name.
  4. Optional : Write a helpful Description..
  5. Set "Action" to Translate.
  6. Set "Destination Type" to My IP Addresses.
  7. Select the IP Address you've passed along from the list.
  8. Set "Target" to either:
  9. "Type" My IP Addresses and select the Static IP Address you have already configured for this VM NIC in VergeOS.
  10. OR "Type" IP/Custom and manually enter the static IP you have set the Local IP you have set on the VM NIC already.
  11. Click Submit to save.

SNAT Configuration: (Required for outgoing translation)

  1. Click Rules in the left panel.
  2. Click New in the left panel to create a new Rule.
  3. Give your rule a Name.
  4. Optional Write a helpful Description.
  5. Set "Action" to Translate.
  6. Set "Source" to either: (Using the same IP as the "Target" from the previous steps)
  7. "Type" My IP Addresses and select the Static IP Address you have already configured for this VM NIC in VergeOS.
  8. OR "Type" IP/Custom and manually enter the static IP you have set the Local IP you have set on the VM NIC already.
  9. Set "Destination Type" to My IP Addresses.
  10. Select the IP Address you've passed along from the list.
  11. Set "Pin" to Top to set this Rule above others, ensuring it is applied early.
  12. Click Submit to save.

Routing Option: (May be useful if your workload is not DNAT compatible)

If your workload does not support DNAT, clients must access it with the native IP on the device, AND you have only 1 IP available, there is an alternative to the "DNAT Option" above. Follow the "DNAT Option" instructions above, and at Step 5 set your action to Route rather than Translate. This will send traffic for the public IP to the VM via the native private IP. Then, on the NIC in your Guest OS, set a secondary IP with the public IP and a /32 (255.255.255.255) subnet. Follow the "SNAT Configuration" as written to translate the outbound traffic. This option is entirely GuestOS dependent and may not work in all situations.

To use this method, follow the "DNAT Option" instructions above with 1 change, Step 5. changes to: 5. Set "Action" to Route.

Set the Public/External IP address as a secondary address on the VM NIC, inside the guest OS. This will allow traffic bound for the IP to be routed to the native internal IP, then allow the guest OS to handle it on the /32 single IP. The "SNAT Configuration" steps will likely still need to be followed; the outbound traffic from the VM NIC will still originate from the local IP, not the public assigned as a secondary address, and thus Source NAT will need to change it on the way out.

Document Information

  • Last Updated: 2025-02-17
  • VergeOS Version: 4.13.3

How to Achieve Network Micro-Segmentation on VergeOS

Network micro-segmentation is a security approach that divides a network into isolated segments, each with its own security controls. This article explains how to implement micro-segmentation using VergeOS's powerful networking features.

Key Features for Micro-Segmentation

VergeOS provides several features that enable effective network micro-segmentation:

  1. Internal Networks: Create multiple isolated virtual networks.
  2. Network Rules: Implement granular firewall rules.
  3. IPSec and WireGuard VPNs: Establish encrypted tunnels between networks.
  4. Tenant Isolation: Separate virtual data centers for strong multi-tenancy.
  5. Network Aliases: Group IP addresses/networks for easier policy management.
  6. Port Mirroring: Monitor traffic for security analysis.

Implementing Micro-Segmentation

Follow these steps to achieve network micro-segmentation on VergeOS:

1. Design Your Network Topology

  • Create separate internal networks for different applications or workloads.
  • Each internal network provides a default-secure environment.

Example: 

- Web Application Network
- Database Network
- Management Network
- Development Network

2. Configure Network Rules

Use network rules to control traffic between internal networks and VMs:

  1. Navigate to the network dashboard.
  2. Select "Rules" from the left menu.
  3. Click "New" to create a rule.
  4. Set the action (e.g., Accept, Drop), protocol, and direction.
  5. Define source and destination networks/IPs.
  6. Apply the rule.

Example rule: Allow web servers to access the database on a specific port: 

Action: Accept
Protocol: TCP
Direction: Outgoing
Source: Web Application Network
Destination: Database Network
Destination Port: 3306

3. Utilize Network Aliases

Group IP addresses or networks for easier policy management:

  1. Go to the network dashboard.
  2. Select "Aliases" from the left menu.
  3. Click "New" to create an alias.
  4. Name the alias and add IP addresses or networks.
  5. Use the alias in network rules.

Example: 

Alias Name: Web Servers
IP Addresses: 192.168.1.10, 192.168.1.11, 192.168.1.12

4. Implement VPN Tunnels

For sensitive traffic between networks, use IPSec or WireGuard VPNs:

  1. Navigate to the VPN configuration section.
  2. Choose IPSec or WireGuard.
  3. Configure the VPN settings (e.g., encryption, authentication).
  4. Apply the VPN to the desired networks.

5. Leverage Tenant Isolation

For multi-tenant environments:

  • Create separate tenants for different departments or customers.
  • Each tenant has its own set of isolated internal networks.

6. Monitor and Adjust

Use port mirroring to monitor traffic:

  1. Go to the network dashboard.
  2. Enable port mirroring for the networks you want to monitor.
  3. Analyze the traffic and adjust network rules as needed.

Best Practices

  • Follow the principle of least privilege: only allow necessary traffic.
  • Regularly review and update network rules.
  • Use descriptive names for networks, rules, and aliases.
  • Document your network topology and segmentation strategy.

By utilizing these features and following these steps, you can create a highly segmented network architecture on VergeOS, implementing zero trust principles and reducing the potential attack surface.

Document Information

  • Last Updated: 2024-08-29
  • vergeOS Version: 4.12.6

VM Disk Discard

The Discard option on a VM Disk in VergeOS is responsible for managing unused storage blocks by issuing TRIM or DISCARD commands. When Discard is enabled, the system automatically frees up unused blocks, helping to maintain efficient storage usage on the vSAN.

Enabling or Disabling Disk Discard

When creating or editing a VM disk, you have the option to enable or disable Discard. By default, Discard is enabled, and it is highly recommended to leave it enabled for optimal storage efficiency. When Discard is disabled, deleted files on the virtual disk do not immediately free up the corresponding storage, leading to potential overuse of storage resources.

Here’s what happens when Discard is enabled:

  • The system periodically identifies and frees unused disk blocks.
  • vSAN storage remains optimized, as unused blocks are reclaimed.
  • Disk space usage more accurately reflects the actual data stored on the VM.

Only disable Discard for performance reasons

Disabling Discard can lead to storage inefficiencies and should only be done for specific performance-related reasons. Always consult with VergeOS Support before disabling this feature.

Why Use Disk Discard?

  • Efficient Storage Management: When a file is deleted from a VM, the unused blocks are immediately flagged as free, allowing the vSAN to reuse that space for other data.
  • Improved Disk Performance: Discard operations help maintain a clean and optimized storage system, reducing overhead from managing fragmented or unused blocks.
  • Space Reclamation: Particularly in environments with high storage churn (i.e., frequent file creation and deletion), Discard ensures that space is consistently reclaimed, avoiding storage bloat.

When to Disable Disk Discard

In rare circumstances, you may need to disable Discard to improve performance, particularly on certain workloads where the overhead of issuing TRIM/DISCARD commands may cause delays or slowdowns. Before making this change, it's critical to understand the trade-offs in terms of storage efficiency and consult with VergeOS Support for further guidance.

By keeping Discard enabled, you ensure that VergeOS optimizes storage for virtual machines, maintaining high efficiency and minimizing wasted space.

Document Information

  • Last Updated: 2024-08-29
  • vergeOS Version: 4.12.6

Database Performance Best Practices on VergeOS

Running databases on VergeOS can deliver excellent performance when properly configured. Below are the recommended best practices for optimizing database performance, specifically for databases running on Microsoft Windows within the VergeOS environment.

Key Adjustments for Optimizing Performance

  • Backup Location: Always store database backups directly within VergeOS. This ensures faster read/write access and better overall performance for backup operations.

  • Decrypt Databases Before Migration: If your database is encrypted, it is recommended to decrypt the database before migrating it onto VergeOS. This prevents potential performance degradation during the initial transfer and ensures optimal operation within the environment.

  • Disable Automatic Defragmentation: The built-in Windows automatic defrag job should be disabled. Defragging on virtual disks can lead to performance issues, especially on SSDs or highly fragmented virtual environments like VergeOS.

  • Disable Windows Defender: Windows Defender, while useful for security, can significantly affect database performance due to real-time scanning. Disable Defender or, at the very least, exclude the following from being scanned:

    • The path or drive where the database files are stored
    • Log files and transaction log drives

  • Turn Off Non-Essential Windows Services: Many Windows services are unnecessary for database performance and can consume valuable resources. Disable any non-essential services to free up CPU, RAM, and disk I/O for database operations.

  • Core Overcommitment: Avoid overcommitting CPU cores, especially if multiple database VMs are sharing the same physical CPU cores. Overcommitting can lead to resource contention, reducing performance for each VM.

  • Disable Fsync on Log Drives: For virtual disks used to store logs, consider disabling Fsync. This can improve I/O performance by reducing the frequency of forced data synchronization on log writes. You can do this by editing the specific virtual disk settings in VergeOS.

    Important

    Disabling Fsync can lead to data loss in case of a crash. Ensure this aligns with your database's tolerance for I/O performance versus reliability trade-offs.

  • Core Allocation Limits: Limit the number of cores allocated to a database VM so that it does not exceed the physical cores available on a single CPU socket. Do not count hyperthreaded cores in this calculation.

    Example: On a dual-socket system with 8 physical cores per socket (16 cores, 32 threads), limit the VM to use 8 cores (1 physical socket), avoiding overcommitment across multiple sockets.

  • Memory Allocation: Ensure that the memory allocated to the database VM does not exceed the physical memory available in a single CPU socket. This ensures memory access is localized and reduces memory access latency.

Network and Storage Optimizations

  • Use VirtIO Drivers: For network performance, it is recommended to use VirtIO network drivers whenever possible. These drivers are optimized for performance in virtual environments and provide significantly better throughput compared to emulated drivers (e.g., e1000).

    You can download the latest stable VirtIO drivers from the following link: Download VirtIO Drivers

  • Monitor Disk I/O: Regularly monitor the disk I/O performance, especially on high-write activity volumes like log files or tempdb files, and consider allocating high-performance SSD or NVMe drives if required.

By following these best practices, you can optimize the performance and reliability of your database systems running on VergeOS. If further fine-tuning is required, VergeOS supports additional features that can help monitor and improve VM performance in real-time.

Document Information

  • Last Updated: 2024-08-29
  • vergeOS Version: 4.12.6

VMware-Backup-DR-Guide

Overview

The VergeOS VMware feature provides a direct interface with vSphere (storage independent) to run a backup agent for VMware virtual machines. The VergeOS agent initiates snapshots, with the ability to access both full and incremental backups for either a one-time import or ongoing backup and DR for vSphere environments.

Setting up VMware Backups - High-level Steps

  • Create a VMware Service (This creates a direct connection to the vSphere environment.)
  • Create Schedule(s).
  • Assign Schedules to VMs (different schedules can be assigned to different VMs.)

Creating a VMware Service

The first step to creating a backup/import of VMware VMs to VergeOS is to create a VMware Service. The VMware service establishes a direct agent connection with vSphere; network access and admin login credentials to the vSphere environment is required.

To Create a New VMware Service:

  1. From the Main Dashboard, click Backup/DR.
  2. Click VMware on the left menu. (Any existing VMware Services will appear in the list view.)
  3. Click New on the left menu.

VMware Service Settings:

  1. Enter a Name for the VMware service (required).
  2. Typically, it is recommended to keep the 2 cores and 2GB RAM default settings in place; this will be suitable for almost all situations.
  3. Optionally, a specific Cluster can be selected (or leave as --Default--).
  4. Optionally, a Failover Cluster can be selected (or leave as --Default--).
  5. Optionally, a Preferred Node can be selected on which to run this service (or leave as --None-- for the system to select Node.)
  6. Optionally, enter a Description to record additional information, if desired.
  7. Select desired On Power Loss setting:
    • Last State - Service  will only be powered on if it was on at the time of power loss.
    • Leave Off - Service will not be powered on when power is restored (regardless of its state at the time of power loss).
    • Power On - Service will be powered on when power is restored (regardless of its state at the time of power loss).
  8. Select a Network on which to run the VMware service. 

DHCP is required on the selected network.

vSphere Settings:

dr1-1.png

  1. Enter the vSphere DNS or IP (required).  The address should be reachable from the network selected for the service. 

It is recommended to connect to the vSphere cluster rather than an individual ESX(i) node.

  1. The default vSphere Port is 443; this is the typical listening port for VMware client connections.  Change to alternate port if needed. 
  2. Enter the vSphere User name.
  3. Enter the vSphere Password for the above user.
  4. The Allow Insecure Certificates option can be selected if the vSphere address is not using a certificate signed by a certificate authority (e.g. self-signed certificates). 
  5. Click Submit to save the new service. 
  6. You are returned to the VMware Services listing where the new service will appear.  Click the Service to select it.  
  7. Click Power On on the left menu to start the service and attempt connection to vSphere.

After the service is started, double-click to bring up the VMWare Service Dashboard. 

Modify Advanced vSphere Settings (Optional):

Once the service is  powered on, advanced vSphere settings can be changed, if desired. 

Default settings will be appropriate for most installations.

Click Edit on the left menu.

  • Max Concurrent VM backups - the number of simultaneous VMware backups.  The default setting (4) is typically appropriate; however, this number can be increased to speed up backup processes from Vsphere systems with high CPU resources and adequate available bandwidth.  Conversely, this setting can be reduced for systems with lower CPU resources/lower available bandwidth.  
  • Name for the auto-created snapshot during backup - the name given to the temporary, VMware-created snapshot used during the backup operation.
  • Default VM backup schedule - defines the backup schedule to be assigned automatically to all new VMware VMs discovered by the service.  Initially, this is set to  --None-, which will set new VMs to use no Schedule (no backups) by default.  After Schedules are created, the default can be changed to assign a specific backup schedule to any newly detected VMs.
  • Automatically enable change tracking per VM - this setting will automatically turn on the VMware CBT (changed block tracking) feature for each VM included in differential and thin-provisioned backups. By default, this setting is enabled (Enabled is recommended).  If this setting is disabled, and CBT is not otherwise enabled on VMware,  a differential backup will default back to a full backup (backup logs will indicate this change.)
  • Backup storage tier - the VergeOS storage tier in which to store backup data.   By default, this is set to tier 4.  Note: Changing this setting affects new Full Backups only.  (In other words: if a backup has already taken place to a different tier, differential backups will continue to be stored in that tier; the new setting will take effect as soon as another Full backup is performed. 

When vSphere settings have been changed as needed, click Submit.  

Advanced VSphere Settings:

  1. Click Refresh VMs on the left menu to discover VMware Virtual Machines. This will initiate a connection to the vSphere system and detected VMs will appear in the VMware Virtual Machines section of the page.  

On the Dashboard, check the Status (top left). If the service successfully connected to the vSphere system, the status will show as  Online and Running.

An Error Status indicates the connection was not made due to: incorrect login credentials, insecure SSL (without enabling the option for insecure SSL), invalid address, or a network issue reaching the VSphere system. 

See Appendix A: Troubleshooting Connection Errors for more information.

Creating a VMware Backup Schedule

A schedule is a grouping of backup Tasks.  A single schedule might include various backups, such as hourly, daily, weekly and monthly backups, and allow for taking backups at different intervals, each with different retention rules.  Additionally, different types of backups can be included within the same schedule: Full (thick provisioned), Full (thin provisioned), and Differential. 

Different schedules can be created to be applied to different VMs, for example a general schedule could be used for production VMs, while a less rigorous schedule is applied to development and testing VMs; yet another schedule that includes frequent backups with shorter retention might be applied to SQL VMs, etc. 

Default Schedule

When a new VMWare Service is created, a Schedule named “Default” is created automatically. This Schedule can be modified to fit your organization’s needs.  You can also create any number of new Schedules.  

To Create a New Schedule:

  1. From the VMware Service Dashboard (Main Dashboard -> Backup/DR → VMware -> double click VMware service in the list.)  
  2. Click Schedules on the left menu.

  3. Click New on the left menu.

  4. Enter a Name for the new Schedule.

  5. Optionally, a Description can be entered to record more information.
  6. Click Submit.
  7. The Schedules list will appear.  Double-click the new Schedule.

At this point the Schedule is just an empty container; one or more tasks need to be added to the Schedule.  

  1. Click New Task on the left menu or click the + Add Task option on the Schedule Dashboard. 

  2. Enter a descriptive Name for the Task (for ex: Midnight_7days; weekly_1monthretention; yearly_perpetual, etc.)

  3. Select the desired Scheduling for the backup Task.  (Granular options allow for great flexibility in task scheduling.)

Example Task Scheduling:

Ex: Every weekday  at 5:15 PM

dr3.png

Ex: Every 2 hours, from 7 AM - 5 PM, except for Sunday:

dr4.png

Ex: Monthly, on the last day of the month:

dr6.png

Ex: One time only, on 2019-04-01 at Noon:

dr7.png

  1. By default, a recurring Task is set to run perpetually.  Optionally,  a Task Expiration can be defined which will cause the Task to cease on the selected date and time.  To set an expiration for the Task: De-select the Never checkbox and enter desired expiration date and time. 

  2. By default, the Backup Job Name will default to: ScheduleName - TaskName-YYYY-MM-DD HH:MM (ex: prodschedule -  hourly - 2019-01-29 11:00 for a backup created from the “prodschedule” schedule, “hourly” task, at 11 AM).  Optionally, a Backup Job Name can be defined and can include any combination of these formatted date variables: 

  • %Y 4-digit year
  • %m 2-digit month (01 to 12)
  • %d 2-digit day of the month (01 to 31)
  • %H 2-digit hour (24-hour clock)
  • %M 2-digit minute (00 to 59)

Example: The entry: “%m-%d-%Y:%H%M-sqlbackup”, run on Jan 26, 2019 at 11AM produces a backup named “01-26-2019:11:00-sqlbackup”  

  1. Select the desired Backup Job Retention; this is the amount of time to keep the backup.  (Units that can be selected: Minutes,  Days (default), Hours, Years, Forever). 

After a backup is run, the expiration of individual Backup Job instances can be modified manually; backup job instances can also be manually deleted before the expiration date/time.

  1. The Quiesce Snapshots option can be selected to invoke the VMWare quiesce feature (Note: VMware Guest Tools required.)  When this option is enabled, VMWare pauses running processes on the guest operating system so that the file system contents are in a known consistent state when the snapshot is taken; this process might include such operations as flushing dirty buffers from the Operating System’s in-memory cache to disk, or other application-specific tasks. Consult VMware documentation for more information about the quiesce feature.

  2. Optionally,  Minimum Backup Jobs to Keep can be selected.   This setting overrides Individual backup expirations to keep the specified minimum number of backups (most recent) in place.  This can provide a safety-net, intended to prevent all backups from expiring before new backups are created: for situations such as a system being powered off for a period of time or an interval of backup errors. 

  3. Select a Backup Mode.  

  • Differential - only transfers changes since the last Full VMware  backup.  Because of the  way that differential backups are stored in the vSAN, a differential backup can be used directly and does not rely on a full backup or other differentials for a restore operation. 

This requires Changed Block Tracking (CBT) enabled on vSphere VMs.

  • Full Backup (Thick provisioned) - Full Backup, requesting all blocks from VMware.
  • Full Backup (Thin provisioned) - Full Backup, requesting only allocated blocks from VMware.

This requires Changed Block Tracking (CBT) enabled on vSphere VMs.

* Differential and Thin Provisioned Full backups  utilize the CBT vSphere feature.  Please see Appendix B for information and considerations regarding this feature.

Using Differential and Full Backups

A Full backup is needed initially and should also be done on a regular basis.  Differential backups are quicker and use fewer resources/bandwidth as only changes since the last full backup are requested.   A prudent strategy will include performing full backups regularly (ex: daily, weekly, bi-weekly), with differential backups at intervals in between.

  1. When the Task is configured as desired, click Submit.  
  2. You are returned to the Schedule page and the new task will appear in the Tasks section.  Click the + Add Task button and repeat the above steps to append additional tasks to the schedule. 

Assigning Schedules

Once the VMware service is created and successfully connects to the VSphere system, the list of discovered VMware Virtual Machines will appear on the VMware Service Dashboard.    dr8.png By default, all VMs have their schedule set to --None--.

To apply a Schedule to VM(s):

  1. From the VMware Service Dashboard, click Virtual Machines on the left menu. 
  2. Select the desired VM(s) from the list.  (Selected VMs show a checked box on the left.) Hint: If you’d like to select all VMs in the list, click the checkbox in the upper left corner.  

  3. Click Edit Backup Schedules on the left menu.

  4. Select the Schedule from the dropdown list and click Submit.

  5. The Backup Schedule assigned to each VM is displayed in the VMware VMs listing.  

Setting the Default Backup Schedule

The default VM Backup Schedule can be defined to automatically assign a backup schedule to all new VMware VMs discovered by the service.   

  1. From the VMware Service Dashboard, click Edit on the left menu. 
  2. In Default VM Backup Schedule, select the desired Schedule from the dropdown list.  
  3. Click Submit to save the change. 

The Default Backup Schedule is displayed on the VMware Service Dashboard.

Manual Backups

Manual backups can also be performed on VMs using the VMware service; this can be helpful in creating a backup immediately before maintenance work,  such as a guest OS upgrade, application update, or other configuration changes. 

To Perform a Manual VM Backup:

  1. From the VMware Service Dashboard, click Virtual Machines on the left menu. 
  2. Select one or more VM(s) in the list.  (Hint: to select all VMs click the checkbox in the upper left corner.)
  3. Click Backup on the left menu. 
  4. A Confirmation dialog will appear; click Yes to proceed with the backup.  
  5. Return to the VMware Service Dashboard (Hint: you can use the breadcrumb at the top or the browser back button to return to the Service Dashboard.)
  6. Click Backup Jobs on the left menu. 

The Manual Backup should appear at the top of the listing and will display a status of “Running” until it is finished, at which point the status will show as “Complete”.

For manual backups, the Name displayed will be the name of the first VM selected for backup, and the Schedule Task column will indicate  a Manual backup.  Additional columns display the Number of VMs backed up (VM Count),  Started and Finished time and, the Expires setting for the backup. 

To Change the Name and/or Expiration of a Backup Job:

  1. Double-click the Backup Job in the listing.

  2. The Backup Job Dashboard displays.  Click Edit on the left menu. 

  3. Make changes to Name/Expires fields as desired. 
  4. Click Submit to save the changes.

To Delete a Backup Job:

  1. Double-click the Backup Job in the listing.
  2. The Backup Job Dashboard displays.  Click Delete on the left menu. 
  3. Click Yes to confirm the delete operation.

Restores

File-level

The VM is imported to the VergeOS environment (From the Backup Job Dashboard, double click the individual VM -> click Import VM.)

VM is powered on in the VergeOS environment where files can be extracted to the VergeOS NAS and accessed via CIFS or NFS.

Restore systems to a VMware environment

Individual VMs or entire VMware system backups can be pushed back to the VMware environment. 

DR/Business Continuity

VMware VMS are powered up in VergeOS from the backup.  Built-in Site-Sync provides the mechanism to synchronize VMware backups offsite to be prepared for quick recovery in the event of a disaster or primary facility outage. 

Appendix A

Troubleshooting VMware Connection Errors

Note: Check Logs (at the bottom of the Dashboard page) for possible additional information.

  • Verify input of correct Address (IP or domain name)

  • Verify network connection 

  • If using a domain name, verify DNS resolution of the name. 
  • The network on which the VMware service is running must have access to the Vsphere address/port provided on port 443 (or port selected). 
  • The network on which the VMware service is running must be DHCP.  

The built-in Diagnostics engine can assist in testing the network connection.  (VMware Services Dashboard -> View Service -> Diagnostics)

  • Verify Login Credentials to vsphere

    • Must be the correct username/password for a VSphere administrator account

  • SSL Certificate

    • If using a self-signed certificate, the option to allow insecure certificates must be enabled.  To modify an existing VMware service: From the VMware Service Dashboard -> Edit -> check the box for Disable SSL host certificate verification

Appendix B

VMware’s Changed Block Tracking (CBT)

Differential and Full(Thin Provisioned) backups utilize VMware’s CBT feature, to request only blocks that have changed since the last full backup, or blocks in use.  This can provide for quicker operations that utilize less bandwidth.  (There is a VergeOS option to automatically turn on CBT for all VMs.)  The following VMware KB article provides more information, including VMware requirements for using CBT: https://kb.vmware.com/s/article/1020128_

CBT Considerations/Cautions

Utilizing CBT to provide faster and more efficient backups is generally fine.  However, it is important to consider that defects in CBT can compromise backups that have utilized the feature. This is a source issue that is not controlled by any third-party agent accessing VMware for backup.  

The following strategies are recommended to mitigate potential risks posed by using the CBT feature: 

  1. As a VMware customer/user, stay abreast of known issues and apply available updates and patches as they become available. In the past, there have been bugs involving the CBT feature, for which VMware has provided patches to fix known CBT defects.
  2. Although VergeOS stores all backups in the vSAN such that they are stand-alone (any backup, including differential, can be used directly and does not rely on another backup for restore operations), a prudent backup strategy will include a schedule of both Full backups and Differential backups in between.  For example, a common schedule used by many organizations is to run a Full backup weekly or twice weekly and differentials on days in between.  
  3. When possible, use Full-Thick Provisioned backup for those that are intended for long-term retention.

Document Information

  • Last Updated: 2024-09-03
  • VergeOS Version: 4.12.6

Best Practices - Running a pfSense Virtual Firewall

pfSense is a widely-used open-source firewall and router software that can be run as a virtual machine (VM) within VergeOS. Leveraging pfSense inside VergeOS allows for highly customizable and flexible firewall configurations. Below are the best practices for creating and maintaining a pfSense virtual firewall in VergeOS.

1. Configuring Disk and Network Interfaces

When deploying pfSense as a virtual machine, ensure that the disk and network interfaces are configured for optimal performance.

  • Disk and Network Interface Type: Set both the disk and network interfaces to VirtIO. This configuration provides better performance compared to the default options.
    • VirtIO for NIC: By default, VergeOS may configure the network interface to use E1000. While this can work, it's recommended to switch to VirtIO for better throughput and reduced CPU overhead. Failure to use VirtIO can lead to intermittent traffic issues or slowness, especially under high network loads.
    • VirtIO for Storage: Using VirtIO for storage ensures faster disk I/O, reducing bottlenecks in the system when dealing with large firewall logs or managing stateful connections.

2. Disabling Hardware Checksum Offloading

In certain environments, pfSense may experience network performance issues like packet loss, slowness, or connection timeouts. This is commonly due to hardware checksum offloading on virtualized NICs.

  • Disable Hardware Checksum Offloading:
    • Within the pfSense UI, navigate to System > Advanced > Networking and disable Hardware Checksum Offloading.
    • When enabled, pfSense offloads the processing of checksums to the virtual NIC. However, this feature is better suited for physical NICs, and in virtualized environments, it can cause performance degradation by generating unnecessary processing overhead on the virtual machine.

pfSense NIC Offloading Settings

3. Assigning Adequate Resources

  • CPU and RAM Allocation:

    • Depending on the size of your network and the complexity of your firewall rules, ensure you assign adequate CPU cores and RAM to your pfSense VM.
    • For small to medium environments, 2 CPU cores and 2GB of RAM is usually sufficient. For more complex configurations or higher network traffic, consider increasing these resources to ensure optimal performance.

  • Disk Space

    • Allocate enough disk space for system logs, caching, and configuration backups. Start with at least 10GB of disk space and increase based on the features in use, such as VPNs or IPS/IDS logging.

4. Snapshots and Rollbacks

  • Use VergeOS Snapshots:

    • Before making significant changes to your pfSense configuration or performing major upgrades, create a VergeOS snapshot of the pfSense VM. This allows for a quick rollback in case of misconfiguration or failure.

  • Automate Snapshots:

    • Automate your snapshots for pfSense to ensure regular backups of your firewall’s state. These snapshots can be scheduled in VergeOS and easily restored when needed.

Following these best practices ensures that pfSense operates efficiently and securely within VergeOS, providing a reliable and high-performance firewall solution for your virtualized network environment.

Document Information

  • Last Updated: 2024-08-29
  • vergeOS Version: 4.12.6

Proper Power Sequence

Proper Shutdown Sequence for a VergeOS Environment

To power off a cluster (a collection of two or more nodes) follow these steps:

  1. Check any running workloads on each node of the cluster. Navigate to the node dashboard for each node and review the Running Machines section.
  2. If there are tenants running on any of the nodes, log into those tenant environments and gracefully shut down all running workloads.
  3. Power off all running workloads on each node, including VMs, tenant nodes, VMware backup services, and NAS services (if applicable).

vNet Containers

There is no need to manually stop any running vNet containers; they will be gracefully stopped automatically in the subsequent steps.

  1. After stopping all running workloads, navigate to the Cluster dashboard for the cluster you wish to power off.
  2. Select Power Off from the left-hand menu to begin shutting down each node in the cluster.
  3. Finally, navigate to System -> Clusters and select Power Off in the left menu to power off the entire cluster.

IMPORTANT

If an environment contains multiple clusters, ALWAYS shut down the cluster containing the controller nodes (Node1 & Node2) LAST.

Proper Power On Sequence for a VergeOS Environment

To properly power on a VergeOS environment, perform the following steps:

  1. Power on Node1.
  2. Once Node1 is online, power on Node2.
  3. Power on all other nodes, waiting approximately 1 minute between power actions.
  4. On the main dashboard, verify that the environment is Green and Online.

main-dash-stoplights.png

Document Information

  • Last Updated: 2024-08-29
  • vergeOS Version: 4.12.6