HPC & AI Infrastructure Engineer
I am the person who finds out that node 14 disagrees with the other thirty-three about which NVIDIA driver it is running. Sole administrator of three production clusters at Temple University — compute, storage and networking for 78+ researchers — plus a summer tuning GPU allocation across 356 nodes including H100 and A100 on Mount Sinai's 11-petaflop Minerva supercomputer.
The other half: I am a demanding user of the same hardware. VIEB, my GPU pipeline, grinds through 22M+ frames of mouse video hunting for behaviors nobody thought to name. Running a cluster and needing one teach different things. I refuse to pick.
Three production clusters end to end — compute, storage, networking. Bare-metal PXE provisioning through Cobbler, configuration management in Ansible, monitoring in Ganglia, on AlmaLinux 9 / RHEL. Sole administrator: I own the full node lifecycle from rack to production scheduling.
Multi-partition Slurm with GPU scheduling, fairshare and prolog/epilog. I ran an internal security assessment treating the scheduler as an attack surface — prolog/epilog injection, munge key handling, module system and SUID escalation paths — and hardened the cluster configuration against what it found.
I work extensively with Kubernetes, including k3s for GPU clusters — GPU scheduling through the NVIDIA device plugin, node lifecycle, cordon and drain for maintenance, and reconciliation-based deployment of long-running services. Batch and orchestration are different tools for different shapes of work, and running both means I can put a workload where it belongs instead of where the scheduler forces it.
NVIDIA A100 and H100 fleets across heterogeneous hardware. Diagnosed and resolved a fleet-wide driver generation mismatch (570 vs 595), iDRAC event storms, and recurring GPU node scheduling failures — the unglamorous work that keeps utilization up.
I run the cluster and I am also a demanding user of one. GPU-accelerated ML at scale: JAX/CUDA and cuML/RAPIDS debugging, chunked UMAP transforms in 200K-row batches to work around GPU memory limits. I know what the workloads on my nodes actually need.
A GPU-accelerated behavioral analysis pipeline for rodent recordings. DeepLabCut pose estimation feeds a geometry and quality-control layer, then a model-free recurrence measurement — an exact GPU nearest-neighbor search across 22M+ frames across 3,846 recordings asking whether a stretch of one animal's movement has a close match in a different animal's, not which cluster a frame falls into. Against a fitted autoregressive null, cross-animal behavioral recurrence is detected at +1.64% [+1.21%, +2.08%].
Unsupervised segmentation is easy to fool, so the result stands on controls, not on a clustering diagram:
The same instrument retracted one of its own headline claims: an earlier report of behavioral discreteness in 298 of 298 animals turned out to be reproducible from surrogate data through the identical fitted model, and was withdrawn. That is the audit working as intended, not failing.
It runs end to end from an interactive GUI — DeepLabCut extraction through the audit, no command line required — which is in active use in live neuroscience experiments and has to survive real data, real GPUs and real deadlines, not just a benchmark.
Extensible object-oriented scientific visualization software for neuronal modeling, supporting dynamic addition and removal of ion channel components. I implemented the Euler method solver for Hodgkin-Huxley action potential dynamics and cross-validated it against the Yale NEURON solver.
Batch. Jobs arrive with resource requests and wait in
PENDING until the backfill pass finds a node with room. Multi-GPU jobs
block until enough contiguous devices free up — which is exactly why a badly
balanced partition sits idle with a full queue. Drain a node and watch the
scheduler route around it.
Declarative. The same fleet under Kubernetes. You do not submit
work here — you declare that you want N replicas and a controller loop
reconciles reality toward that number. Kill a pod and it comes back. Cordon a node
and its pods are evicted and rescheduled somewhere with free
nvidia.com/gpu. Scale past the fleet's capacity and the surplus sits
Pending forever with Insufficient nvidia.com/gpu — the
scheduler will not queue it the way Slurm would, because a Deployment has no
concept of waiting its turn.
Occasional writing on cluster operations, scheduler internals and scientific computing. All notes →