A shared GPU cloud for student machine learning

Research Paper (PDF)

Machine learning coursework needs a GPU, most students cannot buy one, and nobody needs a whole machine all the time. So we pooled the lab hardware into an OpenStack cloud and handed out slices of it, reachable from anywhere through a browser. Undergraduate work, published in April 2022.

The problem

Anything interesting in machine learning wants a GPU, and a machine with one costs more than most undergraduates have. The demand is also lumpy. A student needs the whole machine for an afternoon, or a small slice of it for a fortnight, and almost never the thing they would have had to buy outright. That argues for pooling the hardware and handing out allocations that can be resized and given back.

What we built

A private cloud on a departmental server running OpenStack, brought up with DevStack on Ubuntu. The hardware underneath was Nvidia 3000-series GPUs, 128 GB of RAM and a terabyte of disk. OpenStack handles the plumbing: Nova for compute, Glance for images, Keystone for identity, Neutron for networking, Cinder for block storage, and Horizon for the dashboard an administrator actually clicks. A student gets an instance sized to what they asked for, reached over SSH with a key pair, with security groups and a floating IP in front of it.

Reaching any of it from outside was the awkward part. The server sat behind the college NAT, so an instance with a perfectly good address on the internal network was invisible from a dorm room. We ran Jupyter on the instance and exposed it through an ngrok tunnel, which gives you a public URL without anyone having to reconfigure the campus network. The student opens that link and enters a token the administrator hands out.

What it is, and what it is not

This is a systems paper. It documents a build in enough detail to repeat it, down to the local.conf settings and the IP-forwarding rules that made the tunnel work. It is not an evaluation. There are no throughput numbers, no measurement of utilisation against the dedicated machines it was meant to replace, and no scheduling policy beyond an administrator deciding who gets what. Scheduling is the part I would build next, and the part worth measuring.

It is also the earliest thing I built that was infrastructure for other people's research rather than for a grade, which is why it stays on this list. The work was joint, with four collaborators, and appeared in the International Research Journal of Engineering and Technology, volume 9, issue 4 (April 2022), pages 1045 to 1055.