HPC SLURM Management System
A web-based GUI and Ansible automation layer for SLURM-managed High-Performance Computing (HPC) clusters, eliminating CLI complexity for users and administrators.
- HPC
- Cloud
- Full-Stack
Context
SLURM is the industry-standard workload manager for High-Performance Computing (HPC) clusters. However, its pure CLI interface creates significant operational friction for both cluster administrators configuring multi-node environments and researchers submitting batch jobs.
HPC-Slurm-Hex was built as an integrated GUI and automation platform that layers seamlessly over SLURM. It combines Ansible cluster provisioning with a containerized web interface for zero-CLI job submission, resource quota enforcement, user management, and security scanning.
The problem
HPC Administrators face complex, error-prone manual setup steps when distributing configuration files, munge keys, compilers, and monitoring services across master and worker nodes.
HPC Users (researchers and engineers) are required to manually write batch scripts, memorize CLI commands (sbatch, squeue, sacct), and manage SSH file transfers to monitor jobs and retrieve computational outputs.
Unvetted user job submissions also pose severe security threats to shared cluster environments if malicious code, crypto miners, reverse shells, or fork bombs are uploaded into job packages.
Product strategy
The strategy was to complement SLURM without modifying its core codebase, creating an abstraction layer that handles setup automation, user interactions, security screening, and monitoring off the main compute path.
Zero-CLI operations
Enable researchers to upload code, run jobs, track progress, and download output files entirely through a clean web dashboard.
Automated infrastructure
Provision full OpenHPC + SLURM clusters, MPI stacks, compilers, and exporters with a single Ansible execution.
Pre-submission security
Screen uploaded job archives against 30+ threat signatures before files reach the FTP server or SLURM scheduler.
AuraNode turns that strategy into one native workspace with a provider-neutral backend.
Browser job submission
Upload ZIP archives or Git URLs with automated shell script detection (run.sh, main.sh) and sbatch execution.
Malicious code scanning
Inspect uploaded files against 30+ threat detection rules covering reverse shells, crypto miners, fork bombs, and privilege escalation.
RBAC & resource quotas
Enforce per-user and per-group CPU, GPU, and memory limits while controlling per-tab dashboard permissions.
Real-time cluster monitoring
Visualize live CPU, memory, GPU, and SLURM queue metrics via Prometheus and integrated Grafana dashboards.
System architecture
A web-based React/Vite dashboard communicates with Dockerized Express backend services, proxying job execution, metrics collection, and malware scanning across a multi-node SLURM HPC cluster.
React + Vite Dashboard
Single-page interface for admins and users providing browser job submission, real-time node status, resource quota allocation, and interactive security scan reports.
- React 19
- Vite
- MUI
- Axios
Nginx Reverse Proxy & Express Backend
Unifies frontend and REST endpoints, handles JWT auth, session rotation, per-file threat scanning, FTP handling, Nodemailer alerts, and CSV exports.
- Express.js
- Nginx
- SQLite
- JWT
SLURM Master API Agent
Runs on the SLURM controller node, wrapping sbatch, sacct, and scontrol commands into REST endpoints with Redis caching to avoid CLI latency.
- Express.js
- Redis
- SLURM CLI
SLURM Worker API Agents
Lightweight Flask services on compute nodes handling job execution, node status health checks, and automated output zipping for results download.
- Python
- Flask
- APScheduler
Prometheus + Grafana
Collects system and cluster metrics via Node, SLURM, and DCGM exporters, rendering real-time CPU, memory, and GPU graphs in Grafana.
Redis Cache
Caches cluster job statuses (sacct) on the controller node with 3s TTL to reduce CLI execution overhead and improve response times.
Ansible Playbooks
Automates OpenHPC, SLURM, MPI stacks, compilers, and monitoring exporter installation across master and worker nodes with a single command.
Request lifecycle
When a user submits a job, the React frontend uploads the archive to the Express backend where an inline security scanner checks files against 30+ threat signatures. Upon passing validation, the backend forwards the job specification to the Slurm Master API on the cluster controller node. The controller executes sbatch to schedule the execution on worker nodes. Background worker agents archive output directories upon job completion, allowing users to download results directly through the browser.
Separation of concerns
The Web GUI layer handles authentication, RBAC, session policies, and malware scanning off the HPC compute cluster. The Slurm Master agent isolates CLI execution (sbatch, sacct, scontrol) behind a clean REST interface with Redis caching, while Slurm Worker agents manage local task execution and result packaging on compute nodes. Prometheus and Grafana provide independent operational metrics collection across all cluster nodes.
Key experiences
The dashboard turns complex HPC administration and batch job submission into direct visual workflows.
Unified job dashboard
Track active, completed, failed, and cancelled jobs with real-time status indicators, allocated resources, and one-click output zipping.
Interactive security feedback
Failed uploads generate detailed threat reports displaying matched rules, affected files, line numbers, and severity levels.
Live resource telemetry
Filter live CPU load, RAM usage, GRES partitions, and GPU utilization graphs per node with configurable refresh rates.
Admin control panel
Manage users, groups, automated welcome email dispatch, node connections, and CSV job history exports for auditing.
Engineering decisions
React 19 & Vite for responsive management UI
Vite-powered single-page application built with MUI, Lucide icons, and cross-tab idle timeout protection.
Express REST backend with httpOnly JWT sessions
Handles authentication, RBAC, Nodemailer integration, and session lifecycle with SQLite session persistence.
Controller & worker API agents
Express + Redis on the master controller wraps sbatch/sacct; Flask + APScheduler on worker nodes handles local execution and result archiving.
Pre-submission rule scanner
AdmZip-based static analysis engine scanning scripts and config files prior to staging or execution.
Prometheus + Grafana telemetry pipeline
Scrapes Node Exporter, SLURM Exporter, and DCGM GPU metrics into embedded Grafana dashboards.
Ansible for end-to-end cluster setup
Automated provisioning of OpenHPC, SLURM, munge, NTP, compilers (GNU), MPI (OpenMPI/MPICH), and monitoring tools.
A modern GUI and automation layer for HPC clusters.
HPC-Slurm-Hex bridges the gap between HPC infrastructure and modern web usability, providing an end-to-end management platform for research teams and system administrators.
The project is fully containerized via Docker Compose for the web GUI and includes battle-tested Ansible playbooks for cluster deployment under GNU GPLv3.
- Ansible automated provisioning for OpenHPC + SLURM multi-node clusters
- Containerized React 19 + Express web GUI with Nginx reverse proxy
- Pre-submission malware security scanner with 30+ threat detection rules
- Redis-cached Slurm master API and Flask compute node agents
- Prometheus and Grafana real-time cluster monitoring integration