Skip to content

System Administration#

vSAN Diagnostics Guide

Overview

This guide provides comprehensive information about the vSAN diagnostic options available in the user interface. These diagnostic tools enable system administrators to monitor, troubleshoot, and maintain vSAN deployments effectively.

Critical Warning

The diagnostic commands detailed in this guide are powerful administrative tools. Improper usage can result in: - System outages - Service interruptions - Potential data loss

Exercise extreme caution and ensure proper understanding before execution.

Prerequisites

To use these diagnostic tools, you must have:

  • Root-level access to your VergeIO cluster
  • Note: Tenants do not have a vSAN.

Accessing vSAN Diagnostics

  1. Navigate to vSAN Diagnostics using either method:
  • From the home screen: Select the vSAN Tiers count box → vSAN Diagnostics (left menu)

  • Alternative path: Home screen → System (left menu) → vSAN Diagnostics 2. Command execution:

  • Select desired command from the dropdown menu

  • Configure available options if applicable
  • Click SEND→ to execute

Command Visibility

Enable the "Show Command" option to view the exact command being executed. This can be valuable for: - SSH execution - BASH script integration - Advanced command automation

Diagnostic Commands

Add Drive to vSAN

Running this command allows you to manually add a drive via the UI. Drives are normally added either during the installation, or via the Nodes > Nodes Drives page. However adding them in that way does not allow for the addition of drives to Tier0.

Prerequisites:

  • Drive must be physically present in the system
  • Drive must be visible from Nodes > Nodes Drives page

Usage Parameters:

  • Selecting Add Drive to vSAN
  • From the right menu, select the Node that we will be adding the drive to.
  • Enter the appropriate path, E.G. `/dev/nvme0n1```<br>
  • You can use the "Click here to view devices" to get the path. !!! warning "The contents of this drive will be overwritten."
  • Select the Tier you want to assign the drive to.
  • Check the Swap box if you want Swap enabled on this drive. !!! info "This will use the cluster settings for the Swap size."
  • Verify. You will need to TYPE Yes I know what I'm doing in the Verify box.
  • Select SEND →

CLI Syntax:

vcmd newdevice --path=PATH [OPTIONS]
  --path=PATH    Path to target device
  --tier=NUM     Tier number assignment

Cancel Integrity Check

Terminates any active integrity check operations. See Integrity Check for additional information.

CLI Syntax:

vcmd cancelintegcheck

Clear Reference Counts

Reference counts are how the vSAN tracks the number of times a file is referenced in the vSAN. Clearing this count will force a full vSAN walk and a refresh of the Reference Counts.

Function:

  • Clears existing reference counts
  • Initiates full vSAN traversal
  • Rebuilds reference count data

Support Authorization Required

Execute only under direct support guidance.

Usage Parameters:

  • Verify. You will need to TYPE Yes I know what I'm doing in the Verify box.
  • Select SEND →

CLI Syntax:

vcmd clearrefcounts

Find Inode

Running this query will allow you to find out what an Inode (Index Node) referrences. Inode is a data structure that stores information about a file or directory, such as its owner, access rights, date and time of creation and modification, size and location on the vSAN. Each file or directory in the system has its own unique index node number (inode number), which can be used to perform various operations with a file or directory. This can be used to troubleshoot errors in the vSAN.

Purpose:

  • Retrieves inode reference information
  • Maps inode numbers to filesystem entities
  • Assists in vSAN troubleshooting

CLI Syntax:

find /vsan -inum inode_number_here -printf /%P\n

Get Cache Info

Retrieves detailed cache information for specified nodes.

Output Information:

  • Total cache capacity
  • Available cache space
  • Cache page statistics
  • Performance metrics

CLI Syntax:

vcmd getcacheinfo

Get Clients

Retrieves client connection information for specified nodes.

Output Information:

  • Connected node information
  • IP address mappings
  • Worker thread statistics

CLI Syntax:

vcmd getclients

Get Cluster Rates

Retrieves cluster-wide performance metrics.

Output Information:

  • Read/write rates
  • Throttle status
  • Performance statistics

CLI Syntax:

vcmd getclusterrates

Get Cluster Usage

Provides cluster-wide storage utilization information.

Output Information:

  • Maximum storage capacity
  • Current utilization
  • Repair operation counts

CLI Syntax:

vcmd getclusterusage

Get Current Master

Retrieves master node information from each cluster member.

Output Information:

  • Master node identification
  • Online status
  • Transaction logging information

CLI Syntax:

vcmd getcurmaster

Get Device Integrity

Retrieves integrity check results for specified nodes.

CLI Syntax:

vcmd getdeviceinteg

Get Device List

Provides comprehensive device inventory.

Output Information:

  • Device identifiers
  • System paths
  • Tier assignments

CLI Syntax:

vcmd getdevicelist

Get Device Status

Retrieves detailed device status information.

Output Information:

  • Device paths
  • Operational status
  • Capacity metrics
  • Performance statistics

CLI Syntax:

vcmd getdevicestatus

Get Device Usage

Provides device utilization metrics.

Output Information:

  • Total capacity
  • Current utilization
  • Usage trends

CLI Syntax:

vcmd getdeviceusage

Get File Status

Retrieves detailed file metadata.

Output Information:

  • Inode information
  • File type
  • Tier assignment
  • Hash key data

CLI Syntax:

vcmd stat /path/to/file.raw

Get Fuse Info

Retrieves FUSE (Filesystem in Userspace) statistics.

Output Information:

  • Mount point information
  • Thread statistics
  • Throttling metrics

CLI Syntax:

vcmd getfuseinfo

Get Integrity Check Status

Retrieves results from the most recent integrity check.

Output Information:

  • Check status
  • Path information
  • Temporal data
  • Verification results

CLI Syntax:

vcmd getintegcheckstatus

Get Journal Status

Retrieves journal system status information.

Output Information:

  • Operational status
  • Redundancy status
  • System metadata

CLI Syntax:

vcmd getjournalstatus

Get Node Device List

Retrieves detailed hardware information for storage devices.

Output Information:

  • Driver information
  • Model specifications
  • Firmware versions
  • Physical attributes

CLI Syntax:

vcmd getnodedevicelist

Get Node Info

Retrieves comprehensive node configuration data.

Output Information:

  • Node identification
  • Cluster configuration
  • System parameters
  • Operational status

CLI Syntax:

vcmd getnodeinfo

Get Node List

Provides cluster-wide node inventory.

Output Information:

  • Node identification
  • Online status
  • Version information
  • Tier utilization

CLI Syntax:

vcmd getnodelist

Get Path from Inode

Resolves filesystem paths from inode numbers.

CLI Syntax:

vcmd getpathfromino $1

Get Read Ahead

Retrieves read-ahead buffer statistics.

Output Information:

  • Queue statistics
  • Thread utilization
  • System status

CLI Syntax:

vcmd getreadahead

Get Repair Status

Monitors ongoing repair operations.

Output Information:

  • Device repair status
  • Operation progress
  • System health

CLI Syntax:

vcmd getrepairstatus

Get Running Configuration

Retrieves active system configuration.

Output Information:

  • Worker thread allocation
  • System throttles
  • Operational parameters

CLI Syntax:

vcmd getrunningconf

Get Sync List

Monitors synchronization operations.

Output Information:

  • Operation frequency
  • Start times
  • File processing status

CLI Syntax:

vcmd getsynclist

Get Tier Device Maps

Retrieves tier-to-device mapping information.

Output Information:

  • Physical device mappings
  • Tier assignments
  • System configuration

CLI Syntax:

vcmd gettierdevicemaps

Get Tier Node Maps

Retrieves tier-to-node mapping information.

Technical Details:

  • Base-0 indexing (0=Node1, 1=Node2, etc.)
  • 65536 buckets per tier map
  • Primary (tier_x.0) and redundant (tier_x.1) mappings

CLI Syntax:

vcmd gettiernodemaps

Get Tier Status

Retrieves comprehensive tier health information.

Output Information:

  • Redundancy status
  • Walk statistics
  • Transaction data
  • Health metrics

CLI Syntax:

vcmd gettierstatus

Get Top Usage Rates

Monitors real-time I/O statistics.

Real-time Data

Multiple executions may be necessary for trend analysis.

CLI Syntax:

vcmd getfhlist | grep -Eo '(ino|rrate|wrate)\b.*'

Get Volume Usage

Retrieves detailed volume utilization statistics.

Parameters:

  • Path specification (optional)
  • Recursive flag
  • Human-readable output
  • Preferred tier display

CLI Syntax:

vcmd getvolusage --path=/ --recursive=1 --human=1

Integrity Check

Initiates system integrity verification.

Parameters:

  • Path specification (required)
  • Recursive operation
  • Fix mode (destructive)
  • Meta-tier only option

Data Loss Risk

Fix mode zeros bad blocks. THIS IS DESTRUCTIVE. Use only under support guidance.

CLI Syntax:

vcmd integcheck /vol

Integrity Check Device

Performs device-level integrity verification.

Parameters:

  • Node selection
  • Device ID (-1 for all devices)

CLI Syntax:

vcmd integcheckdevice --id=x

Summarize Disk Usage

Generates storage utilization summaries.

Parameters:

  • Path specification
  • Recursive operation
  • Preferred tier display
  • Deduplication analysis
  • Fast deduplication option

CLI Syntax:

vcmd du /vol

Additional Resources

Feedback

Need Help?

If you need further assistance or have any questions about this article, please don't hesitate to reach out to our support team.

Document Information

  • Last Updated: 2024-12-27
  • VergeOS Version: 4.13.2

Updating the VergeOS System

Overview

Key Points

  • System updates should be performed during a maintenance window
  • Updates can be performed with zero downtime when adequate resources are available
  • System updates are only run from the host system (top-level parent)
  • Tenant systems are automatically updated from their host system
  • Updates can be scheduled or performed on-demand
  • The system automatically handles workload migration during updates

This guide provides detailed instructions for performing system updates in VergeOS, whether on-demand or scheduled.

Prerequisites

  • Administrative access to the VergeOS Cloud Dashboard
  • Adequate system resources to allow workload migration during updates
  • A maintenance window (recommended, though not required due to zero-downtime capability)

Performing On-Demand Updates

1. Check for Updates

  1. Navigate to System > Updates in the Cloud Dashboard
  2. Click Check For Updates in the left menu
  3. Click Yes to confirm
    • The Packages section will show available updates
    • A cloud icon indicates downloadable packages
    • Version information displays current and available versions

2. Download Updates

  1. Click Download in the left menu
  2. Click Yes to confirm
  3. Wait for the download to complete

3. Install Updates

  1. Click Install in the left menu
  2. Click Yes to confirm
  3. Wait for installation to complete
    • Status will show "Idle - Reboot Required" when ready
    • The Reboot option will become enabled

Note

Updates that don't include VergeOS package changes won't require full node reboots, but still need the Reboot option to apply changes.

4. Apply Updates

  1. Click Reboot in the left menu
  2. Click Yes to confirm - The system will process one node at a time:
    • Node enters maintenance mode
    • Workloads migrate to other nodes
    • Application restarts/node reboots
    • Node exits maintenance mode
    • Progress shows in the Status field
    • Nodes Updated status tracks completion

Tip

Use Cancel Reboot to halt automatic reboots if needed (e.g., for workload rebalancing)

Scheduling Updates

1. Create Update Task

  1. Navigate to System > Updates > Tasks
  2. Click New in the left menu

2. Configure Schedule

  1. Choose scheduling option:
    • One-time: Keep default "Does Not Repeat"
    • Recurring: Select frequency (weekly, bi-weekly, monthly)
  2. Set Start Date and time
  3. For recurring tasks, optionally set end date

3. Configure Task Details

  1. Enter required Name
  2. Add optional Description
  3. Select Task Type:
    • Choose "Download, Install, and Reboot" for complete update
  4. Optional: Enable Delete After Running
  5. Click Submit to save

Best Practices

  • Schedule updates during low-usage periods and during maintenance windows
  • Ensure adequate system resources for workload migration
  • Monitor system during update process
  • Keep regular backups before major updates
  • Review available updates before applying

Troubleshooting

Common Issues

  • Issue: Workloads fail to migrate

  • Solution: Verify adequate resources on target nodes

  • Issue: Update process hangs

  • Solution: Check system logs and contact support if needed

  • Issue: Node fails to rejoin after reboot

  • Solution: Review logs and network connectivity

Feedback

Need Help?

If you encounter any issues during the update process or have questions, please reach out to our support team.

Document Information

  • Last Updated: 2024-12-19
  • VergeOS Version: All

Device Passthrough Advanced Configuration (Manual Creation/Editing of Resource Rules)

Although allowing auto-generation of resource rules (e.g. when you select a device and use the Make Passthrough menu option) is easiest and usually recommended, there may be situations where it may be useful to manually create a resource rule or to modify an auto-generated resource rule.

It is important to read and be familiar with PCI Passthrough Risks and Precautions before making passthrough configurations.

Manually Create a New Resource Rule

  1. From the main dashboard, click Resources.
  2. Click Rules (ui card or on the left menu).
  3. Click New on the left menu.
  4. Provide a Name for the Rule; it is recommended to use a descriptive name can be helpful in future administration.
  5. Select the Resource Group to which the resource rule will apply.
  6. Select a specific Node or select --None-- to apply the rule to all nodes.
  7. Select the Type (PCI, USB, SR-IOV, or NVIDIA vGPU).
  8. Leave the default value set to --None-- in the field labeled Automatically created based on PCI Device.
  9. Configure device filters as desired; filter fields will vary depending on the device type selected; see below. (Advanced Entry 1 option also available)

Information on installed PCI devices, for use in filters, you can use the PCI devices listing: from the Main Dashboard, navigate to the Resources -> PCI Devices. To show additional fields, right-click in the heading section to select from the full list of available columns that can be displayed.

Edit an Existing Resource Rule

  1. Navigate to the Associated Resource Group dashboard (Main Dashboard > Resources > Groups > double-click the particular group).
  2. In the Rules section, locate and click the desired resource rule.
  3. Click Edit on the left menu.
  4. Node selection and PCI Filters can be modified as needed. (Advanced Entry 1 option also available)

  1. The Advanced Entry section allows you to manually input filter syntax rather than using the filter entry fields. Generally, it is preferable to allow system-generated syntax based on your filter field selections. ↩↩↩↩

Terraform VergeIO Provider

The Terraform VergeIO Provider enables the integration and automation of VergeOS infrastructure with Terraform. It allows users to define, manage, and scale VergeOS resources as part of Infrastructure as Code (IaC) workflows.

For the latest provider documentation and examples, please refer to the following:

Example Usage

For more detailed usage examples, check the docs folder in the GitHub repository.

Example Configuration

provider "vergeio" {
  host     = "https://some_url_or_ip"
  username = "my_user"
  password = "my_password"
  insecure = false  # Use true if using self-signed SSL certificates
}

resource "vergeio_vm" "new_vm" {
  name        = "NEW VM"
  description = "NEW TF VM"
  enabled     = true
  os_family   = "linux"
  cpu_cores   = 4
  machine_type = "q35"
  ram         = 8192
}

Initializing and Applying

To apply the configuration:

terraform init && terraform apply

Configuration Reference

host (Required): URL or IP address for the VergeOS system or tenant.
username (Required): Username for the VergeOS system or tenant.
password (Required): Password for the provided username.
insecure (Optional): Set to true for systems using self-signed SSL certificates.

Resources

The following VergeOS resources can be managed via Terraform:

vergeio_drive
vergeio_member
vergeio_network
vergeio_nic
vergeio_user
vergeio_vm

Data Sources

The following data sources are available for querying VergeOS resources:

vergeio_clusters
vergeio_groups
vergeio_mediasources
vergeio_networks
vergeio_nodes
vergeio_version
vergeio_vms

Testing a Sample Configuration

To test your configuration, create a main.tf file in your Terraform workspace:

terraform {
  required_providers {
    vergeio = {
      source = "vergeio/cloud/vergeio"
    }
  }
}

provider "vergeio" {
  host     = "https://someURLorIP"
  username = "username"
  password = "password"
}

resource "vergeio_vm" "new_vm" {
  name        = "NEW VM"
  description = "NEW TF VM"
  enabled     = true
  os_family   = "linux"
  cpu_cores   = 4
  machine_type = "q35"
  ram         = 8192
}

Then, run the following command:

terraform init && terraform apply

Document Information

  • Last Updated: 2024-09-03
  • VergeOS Version: 4.12.6

Updating a VergeOS System with Airgap License

Overview

Key Points

  • System updates should be performed during a maintenance window
  • This guide details the process of manually updating a VergeOS system using an air-gap license.
  • The update is performed using an ISO file, ensuring that systems without internet access can be kept up-to-date.
  • Ensure you have a valid air-gap license and the latest ISO file before starting.

This guide provides a step-by-step process to manually update your air-gapped VergeOS system using an ISO file.

Prerequisites

  • Access to the VergeOS Cloud Dashboard.
  • The latest VergeOS update ISO file.
  • A valid air-gap license.
  • A recent backup of your VergeOS system.

Steps

  1. Download the Update ISO - Visit the VergeOS updates page at https://updates.verge.io/download. - Download the latest VergeOS release ISO file.

!!! tip "Pro Tip" Ensure that the ISO file corresponds to your current VergeOS version to avoid compatibility issues.

  1. Upload the ISO to VergeOS - Log in to your VergeOS Cloud Dashboard. - Navigate to Media Images in the left-hand menu. - Upload the downloaded ISO file to the Media Images section.

!!! note The upload process may take a few minutes depending on your network speed.

  1. Configure Update Settings - Go to System > Updates > Edit Settings. - In the Update Source dropdown menu, select -- Update ISO --. - Choose the ISO file you just uploaded from the Media Images. - Click Submit to save the settings.

  2. Perform the Update - Return to the Updates section and click Check For Updates. - Once the update is detected, click Download. - After the download completes, click Install. - Follow the prompts to Reboot the system to apply the updates.

!!! warning "Important" Do not interrupt the update process. Ensure that the system remains powered on and connected during the update.

Troubleshooting

Common Issues

  • Issue: Update not detected after uploading the ISO.

  • Solution: Ensure the ISO was uploaded correctly and reselect it in the Update Source settings.

  • Issue: Errors during the update process.

  • Solution: Check system logs for detailed error messages and verify that your air-gap license is valid.

  • Issue: System fails to reboot after the update.

  • Solution: Contact Verge support for assistance.

Additional Resources

Feedback

Need Help?

If you encounter any issues during the update process or have any questions, please reach out to our support team.

Document Information

  • Last Updated: 2024-08-19
  • VergeOS Version: 4.12.6

Requesting an Airgap License for VergeOS

Air-gap licensing is not common and requires justification. Please see Licensing and Software Updates for more information.

Overview

Key Points

  • VergeOS requires a valid license for operation
  • Air-gapped environments need a special airgap license
  • The process involves generating a license request file and emailing it to Verge

This guide walks you through the process of requesting an airgap license for VergeOS systems in environments without outbound Internet access.

Prerequisites

  • Access to the VergeOS Cloud Dashboard
  • A working email client on a machine that can send external emails
  • Understanding of your system's airgapped status

Steps

  1. Navigate to System Settings - From the Cloud Dashboard, click System on the left menu - Click Settings on the left menu

  2. Initiate License Request - In the License section, click the Request License button

  3. Generate License Request File - A popup window titled "Request Generated" will appear - This window displays information about the license request file

  4. Download Request File - Click the Download Request File button - Save the license request file to your local machine

  5. Prepare Email to Verge - Click the Email license@Verge.io button - This opens your default email client with a pre-addressed email

  6. Send License Request - Attach the downloaded license request file to the email - Provide additional information in the email body (e.g., company name, purpose of license) - Send the email to Verge's licensing team

What Happens Next

  1. Verge processes your request and generates an airgap license file
  2. You receive a reply email with the airgap license file attached
  3. Upload the received license file to your VergeOS system (covered in a separate guide)

Processing Time

If you haven't received a response within 2 business days, please follow up with Verge's support team.

Important Considerations

  • Ensure the system requesting the license is the one you intend to license
  • Keep the license request file secure
  • For multiple systems, repeat this process for each system individually

Troubleshooting

Common Issues

  • Problem: Unable to generate license request file

  • Solution: Verify your access permissions in the VergeOS Cloud Dashboard

  • Problem: Email client doesn't open automatically

  • Solution: Manually compose an email to license@Verge.io and attach the downloaded request file

Additional Resources

  • [Understanding VergeOS Licensing]
  • [Uploading an Airgap License to VergeOS]
  • [VergeOS System Requirements]

Feedback

Need Help?

If you encounter any issues while requesting an airgap license or have questions about this process, please don't hesitate to contact our support team.

Document Information

  • Last Updated: 2024-09-09T15:41:14.296Z
  • VergeOS Version: 4.12.6