Install vLLM on RHEL 9 Linux

We’re using a derivative of RHEL 9 Linux being AlmaLinux 9, but this should work for RHEL 9 itself and others like Rocky Linux 9 etc.

First we need an up to date Python 3 version installed, anything older than Python 3.11 is ok and up to and including Python 3.13, we’re going for 3.12 here:

dnf install -y python3.12 python3.12-devel

# Create a virtual environment
python3 -m venv ~/venvs/vllm

# Activate the environment
source ~/venvs/vllm/bin/activate

Before you can start the install of vllm you need to ascertain your CUDA version level, which can be found from the output of nvidia-smi | grep CUDA
And you’ll see something like “CUDA UMD Version: 13.4“, so that corresponds to the variable “cuversion” in the following command being set to “134

pip install --upgrade pip
pip install vllm --extra-index-url https://download.pytorch.org/whl/cu134

This command takes some time to run, does lots of the heavy lifting of the vllm install.

Verify the vLLM install with the command:

python -c "import vllm; print(vllm.__version__)"

Configure vLLM by editing the /etc/systemd/system/vllm.service file to be:

[Unit]
Description=vLLM OpenAI Compatible API Server
After=network.target nvidia-persistenced.service
Wants=nvidia-persistenced.service

[Service]
Type=simple
User=your_username
Group=your_username
Environment variables for CUDA and Cache directories

Environment="HF_HOME=/home/your_username/.cache/huggingface"
Environment="HF_TOKEN=hf_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
Environment="CUDA_VISIBLE_DEVICES=0"
Path to your virtual environment's python binary and the vLLM execution module

ExecStart=/home/your_username/venvs/vllm/bin/python3 -m vllm.entrypoints.openai.api_server
--model Qwen/Qwen2.5-7B-Instruct
--host 0.0.0.0
--port 8000

## Another model known to work is: ornith-ai/Ornith-1.5-9B

Restart=always
RestartSec=10
Security and resource tuning configurations for RHEL/AlmaLinux

LimitNOFILE=65536
LimitCORE=infinity

[Install]
WantedBy=multi-user.target

Start the vllm service and manage and examine it with the following:

systemctl edit vllm
systemctl start vllm
systemctl status vllm
tail -100 /var/log/messages

I’ll update this post shortly as my vLLM efforts continue..

Posted in Sales | Leave a comment

Install AWX on RKE2 after installing AWX Operator

We’re going to build on an AWX Operator pod install in a RKE2 based Kubernetes environment.

Quickly, AWX Operator is installed and running, its install is:

git clone --branch "2.19.1" --depth 1 https://github.com/ansible/awx-operator.git
cd awx-operator

We can update the manifests/awx-operator.yaml file to correct the rbac_proxy source, via:

sed -i 's%gcr.io/kubebuilder/kube-rbac-proxy%quay.io/brancz/kube-rbac-proxy%g' manifests/awx-operator.yaml
kubectl apply -f manifests/awx-operator.yaml

Once its pod is running, as seen via: kubectl get pods -n awx

We can then get the AWX web, Postges & tasks pods installed and running.

There’s a few updates needed for the manifests/awx.yaml

sed -i 's/redis_image_version: "7"/redis_image_version: "latest"/g' manifests/awx.yaml

We need to create the Storage needed by AWX pods before we start them:

kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.37/deploy/local-path-storage.yaml
kubectl patch storageclass local-path -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Now we can start the AWX install itself with:

kubectl apply -f manifests/awx.yaml

Watch it deploy and start up with:

kubectl get pods -n awx

Posted in Sales | Leave a comment

Install RKE2 to Ubuntu 26

We have an Ubuntu 26.04 host which is not connected to the Internet, so can’t do the online/default RKE2 install and needs the required install files downloaded and transferred to it.

If you’re running in a VM, ensure you have all the CPU flags exposed to the VM. eg: CPU pass-through

This post will be cleaned up/updated asap..

Commands are:

apt install -y curl ca-certificates apparmor apparmor-utils
swapoff -a
grep -n swap /etc/fstab
vi /etc/fstab ## disable swap


modprobe overlay
modprobe br_netfilter


cat >/etc/modules-load.d/rke2.conf <<'EOF'
overlay
br_netfilter
EOF

cat >/etc/sysctl.d/90-rke2.conf <<'EOF'
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

sysctl --system


RKE2_VERSION='v1.36.4+rke2r1'
curl -sfL https://get.rke2.io -o install.sh
chmod +x install.sh

curl -fLO "https://github.com/rancher/rke2/releases/download/${RKE2_VERSION/+/%2B}/rke2.linux-amd64.tar.gz"
curl -fLO "https://github.com/rancher/rke2/releases/download/${RKE2_VERSION/+/%2B}/sha256sum-amd64.txt"
curl -fLO "https://github.com/rancher/rke2/releases/download/${RKE2_VERSION/+/%2B}/rke2-images.linux-amd64.tar.zst"


Copy install.sh, rke2-linux-amd64.tar.gz, rke2-images-linux-amd64.tar.zst to the destination host, into directory /root/rke2-offline in our commands listed below.


cd /root/rke2-offline
INSTALL_RKE2_ARTIFACT_PATH=/root/rke2-offline sh ./install.sh
which rke2
rke2 --version
ls -l /usr/local/bin/rke2*
mkdir -p /var/lib/rancher/rke2/agent/images
cp /root/rke2-offline/rke2-images.linux-amd64.tar.zst /var/lib/rancher/rke2/agent/images/
touch /var/lib/rancher/rke2/agent/images/.cache.json
ls -la /var/lib/rancher/rke2/agent/images/
mkdir -p /etc/rancher/rke2

systemctl enable rke2-server
systemctl start rke2-server
systemctl status rke2-server
export PATH=$PATH:/var/lib/rancher/rke2/bin
export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
cat >/etc/profile.d/rke2.sh <<'EOF'
export PATH=$PATH:/var/lib/rancher/rke2/bin
export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
EOF

cat >/etc/rancher/rke2/config.yaml <<'EOF'
write-kubeconfig-mode: "0644"
tls-san:
- rke2
- rke2.yourdomain.name
- 1.2.3.4
EOF

Some diagnostic and information commands are:

kubectl get nodes -o wide
kubectl get pods -n kube-system
kubectl get helmcharts -n kube-system
kubectl get ingressclass
kubectl get pods -A -o wide

Posted in Sales | Leave a comment

Upgrade Ubuntu 24 to 26.04

Command to run as root:

apt update
apt upgrade -y
apt autoremove -y
apt install update-manager-core -y

Reboot here if there were updated packages installed.

Then run the main upgrade command, answering all questions with ‘y’

do-release-upgrade -d


Posted in Sales | Leave a comment

Use Hermes to install Ansible AWX – Part 1: The Operator

Here I describe customisation / settings needed for Hermes Agent to be able to be used to install the Ansible AWX server system to a RHEL based host by root ssh commands and sessions.

Firstly, setup the SSH key for your hermes user to be able to ssh to the target host as the root user.

ssh-keygen -t ed25519

Copy the resulting ~/.ssh/id_ed25519.pub to /root/.ssh/authorized_keys file on the target host and ensure the permissions of root’s authorized_keys file is correct with chmod 600 /root/.ssh/authorized_keys

Now your hermes user can ssh as root to the target AWX host and we can run some prompts through Hermes to action an install of the Ansible AWX software requirements and environment.

The initial prompt to prepare the chat session and config what Hermes will do is:

I want you to install Ansible AWX on this server.

You are executing terminal commands on the target through the configured
Hermes SSH backend.

The SSH account is root.

Before changing anything:

1. Determine the Linux distribution and version.
2. Determine CPU, RAM and free disk space.
3. Check whether Kubernetes, K3s, Docker, Podman, kubectl, Helm or an
   existing AWX installation is present.
4. Check listening TCP ports.
5. Check firewall configuration.
6. Check the server hostname and IP addresses.

Do not modify the system yet.

Produce an installation plan for deploying AWX using the AWX Operator
on a single-node K3s Kubernetes installation.

Use AWX Operator 2.19.1 / AWX 24.6.1 unless compatibility checks show
that this is inappropriate.

Show me the proposed commands and identify any destructive operations
before proceeding.

Assuming that responds well, the prompt to start to action the install is:

Proceed with the AWX installation plan.

Requirements:

- Install K3s if Kubernetes if not already installed.
- Verify the Kubernetes node becomes Ready before continuing.
- Install kubectl access for root if its not already installed.
- Install or update AWX Operator 2.19.1.
- Create a namespace for AWX.
- Deploy an AWX instance called awx.
- When installing AWX, note the issue around the gcr.io repository, where if you need it, the fix is to update this pkg to use its current quay.io location with the command: kubectl -n awx set image deployment/awx-operator-controller-manager kube-rbac-proxy=quay.io/brancz/kube-rbac-proxy:v0.15.0
- Configure persistent storage.
- Configure the AWX service so it is accessible from my LAN.
- Wait for all AWX pods to become Ready.
- Retrieve the generated AWX admin password.
- Verify the AWX HTTP interface responds.

After every major stage, verify that the previous stage succeeded.

If a command fails, diagnose the error rather than blindly continuing.

Do not disable SELinux or the firewall globally.
Do not delete existing filesystems, containers, Kubernetes resources or firewall configuration without asking me first.

At completion report:

- AWX URL
- admin username
- command used to retrieve the admin password
- Kubernetes namespace
- kubectl commands for checking AWX status
- commands for starting/checking the system after a reboot

If you hit a default Hermes limit with a message like:

Iteration budget exhausted (90/90) — asking model to summarise

Then run the following hermes commands in a new shell and resume the hermes chat or terminal session:

# Increase the maximum iteration budget per turn
hermes config set agent.max_turns 400

# Enable the bounded auto-continuation feature
hermes config set agent.auto_continue_on_max_iterations.enabled true
hermes config set agent.auto_continue_on_max_iterations.max_auto_continues 5

I found I had to set the following in my Ollama Server by editing its /etc/systemd/system/ollama.service.d/environment.conf file and adding in its [Service] settings:

Environment=OLLAMA_NUM_PARALLEL=1
Environment=OLLAMA_KEEP_ALIVE=-1
Environment=OLLAMA_FLASH_ATTENTION=1
Environment=OLLAMA_CONTEXT_LENGTH=65536

And then restart your Ollama service with:

systemctl stop ollama
systemctl daemon-reload
systemctl restart ollama

If there’s a problem with the process and for some reason Hermes stops the activity towards completing and verifying the AWX install, then you can often get it to continue by resuming the hermes chat or terminal session and telling it to “Proceed with the remaining steps towards completing and verifying the AWX install” or “Restart your previous activities and actions towards the AWX install and validation on the ssh host“.

This results in a functional AWX Operator install on the IP Address of the target host.

Posted in Sales | Comments Off on Use Hermes to install Ansible AWX – Part 1: The Operator

Adding Areca storage controller drivers to RHEL 9 based Linux

Adding an Areca ARC-188x series PCIe 2.0/3.0 to SAS/SATA 6/12Gb RAID Controller to a server involved installing the arcmsr driver to the existing CentOS 9 based Linux OS.

This install from source from the vendor was required because the package install for kmod-acrmsr failed with an error on a current kernel:

nothing provides kernel(_dev_info) = 0x749cbadd needed by kmod-arcmsr-1.50.00.05-20210429.12.el9_8.elrepo.x86_64 from elrepo

The steps to obtain build and install the arcmsr driver from Areca’s archive were:

a) Determine the driver to download it and download it to /usr/local/src/
Use the output of “lspci | grep Areca” to identify the card’s model # (“ARC-188x” in this case) and download the relevant zip file from https://www.areca.us/products/_driver_raidcard_1110.html#linux


In our case we downloaded the RHEL 9.4 based zip file and got a file named 9_4.zip from the Areca site, which we extracted to /usr/local/src/areca-download-rhel9_4/


b) In the /usr/local/src/areca-download-rhel9_4/ directory, create a new directory for the extracted contents of the arcmsr-1.51.0X.16-20231226.zip file:

mkdir arcmsr-1.51.0X.16-20231226 && cd arcmsr-1.51.0X.16-20231226 && unzip ../arcmsr-1.51.0X.16-20231226.zip

c) In your extracted arcmsr-1.51.0X.16-20231226/ directory, we need to apply some patches to the arcmsr.c file for modern RHEL 9 based kernels eg: from 5.14.0-6xx onwards kernel levels

The diff output of the patching required to arcmsr.c is:

201,202d200
< 	#include <linux/pci.h>
< 	#include <linux/timer.h>
1284c1282
< 		nvec = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_INTX);
---
> 		nvec = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_LEGACY);
4614c4612
< 	ACB *acb = container_of(t, ACB, eternal_timer);
---
> 	ACB *acb = from_timer(acb, t, eternal_timer);
5317c5315
< 	ACB *pacb = container_of(t, ACB, refresh_timer);
---
> 	ACB *pacb = from_timer(pacb, t, refresh_timer);

d) Now you can begin the DKMS Add, Build, Install process with:

Create dkms.conf file in your extracted arcmsr-1.51.0X.16-20231226/ directory:

cat << EOF > dkms.conf
PACKAGE_NAME="arcmsr"
PACKAGE_VERSION="1.51.00.16"
BUILT_MODULE_NAME[0]="arcmsr"
DEST_MODULE_LOCATION[0]="/kernel/drivers/scsi/"
AUTOINSTALL="yes"
EOF

Setup DKMS source directory:

mkdir /usr/src/arcmsr-1.51.00.16/
cp /usr/local/src/areca-download-rhel9_4/arcmsr-1.51.0x.16-20231226/* /usr/src/arcmsr-1.51.00.16/

Run the DKMS commands to build and install the kernel module in the /usr/src/arcmsr-1.51.00.16/ directory:

dkms add -m arcmsr -v 1.51.00.16
dkms build -m arcmsr -v 1.51.00.16 ## if needed add: -k `uname -r`
dkms install -m arcmsr -v 1.51.00.16

Check with: dkms status

Reboot to confirm.

Posted in Sales | Comments Off on Adding Areca storage controller drivers to RHEL 9 based Linux

Ending commercial hosting services through Network Presence

After some 15 years of operation, I’ve decided to shutdown the public
retail cloud/VPS operations side of Network Presence.

The first phase of this is to quickly shutdown the Sydney POP, with the last day of service from Network Presence in Sydney being February 28th 2026.

I apologise for the inconvenience and I ask you to please relocate your services provided by Network Presence asap and by February 28th, as any service not migrated away from Network Presence in Sydney by February 28th will no longer be available.

The Adelaide POP will continue to operate, but I won’t be offering commercial-grade QoS and products in Adelaide from April 2026.

Any payments made to date for service beyond February will be refunded and please cancel any Paypal Subscriptions or scheduled payments to Network Presence.

I’ve enjoyed working with you and thank you for your business over these years and I wish you all the best for your future endeavours.

Regards,
Richard.

Posted in Sales | Comments Off on Ending commercial hosting services through Network Presence

Installing OpenMPI across CentOS/RHEL 9 based Nodes

OpenMPI is available for install from the AppStream Repo and EPEL Repo is also best enabled for its required associated packages in a CentOS 9 (RHEL) environment of separate like installed compute nodes in a cluster.

Install OpenMPI with:

dnf -y install openmpi

The package in RHEL AppStream installs OpenMPI and its shell module file to the system, and given OpenMPI is a runtime based environment, there’s no server or such to run, the OpenMPI command line toolkit uses SSH to connect to the nodes in the OpenMPI cluster of hosts.

So ensure that the users who will be running OpenMPI commands have SSH Key based authentication configured on all hosts in the cluster. ie: populate the authorized_keys files on all relevant hosts with the user’s SSH Public Key text.

Once OpenMPI is invoked on a host in the cluster, there are dynamic network port based communications undertaken by the OpenMPI commands running on each node and in a RHEL FirewallD environment, the OpenMPI commands need to be configured to use a known range of network ports, which can be opened via firewall-cmd commands.

In our example here, we’ll set OpenMPI commands to use ports from 50000 to 51999 and these ports are opened using the following FirewallD based commands on each node in the cluster:

firewall-cmd --permanent --zone=public --add-rich-rule='rule family="ipv4" source address="10.0.0.0/24" port protocol="tcp" port="50000-51999" accept' && firewall-cmd --reload

Note: assuming the nodes are in a network 10.0.0.0/24 in this example.

Then these network port ranges can be used on the OpenMPI “mpirun” command line with the following parameters:

--mca btl_tcp_port_min_v4 50001 --mca btl_tcp_port_range_v4 30 --mca oob_tcp_dynamic_ipv4_ports 51001-51031

The standard shell user that’s to run the OpenMPI commands in their shell needs a MPI Hosts file which lists the hostname and other settings of each node in the OpenMPI cluster, an example file could be named “mpi_hosts” in the user’s home directory and a sample of it for a 3 node cluster, where each node’s hostname and IP Address is resolvable via DNS or is listed in each host’s /etc/hosts file is:

node1 slots=2
node2 slots=2
node3 slots=2

An example complete mpirun command line to run the “hostname” command 6 times across a 3 node cluster is:

/usr/lib64/openmpi/bin/mpirun --mca btl_tcp_port_min_v4 50001 --mca btl_tcp_port_range_v4 30 --mca oob_tcp_dynamic_ipv4_ports 51001-51031 --hostfile ~/mpi_hosts --path /usr/lib64/openmpi/bin -np 6 hostname

Note: the –path is required so the binaries of OpenMPI can be found on each node.

The above command will results in the hostname command being run 6 times across 3 nodes, with sample output looking like:

node1
node3
node2
node1
node2
node3

Posted in Sales | Tagged | Comments Off on Installing OpenMPI across CentOS/RHEL 9 based Nodes

Get all the logs and output from a systemctl based service

Use the following command to get the full output from a recent service start attempt:

SYSTEMD_LESS=”FRXMK” journalctl -xeu $SERVICENAME

eg: # SYSTEMD_LESS=”FRXMK” journalctl -xeu httpd

Posted in Sales | Comments Off on Get all the logs and output from a systemctl based service

tcpdump params to find the initial connection packets to a port

‘tcp[tcpflags] & (tcp-syn) != 0 and tcp[tcpflags] & (tcp-ack) == 0 and dst port $PORTNUM’

Posted in Sales | Comments Off on tcpdump params to find the initial connection packets to a port