Backend.AI is an AI infrastructure operations solution developed by Lablup. To support various types of infrastructure including on-premises, cloud, and hybrid environments, Backend.AI includes diverse features designed to optimize AI development and operational environments. While users configure virtual hardware resources and run instances and programs through the Backend.AI WebUI similar to other cloud services, behind the convenient UI, multiple Backend.AI components work together in complex combinations.

Lablup hosts Backend.AI Hands-on training sessions 2-4 times annually to allow more users to experience Backend.AI quickly and easily. Through these Hands-on sessions, participants understand Backend.AI's system architecture and learn about the functions and roles of components such as Manager, Agent, and Proxy, as well as how to actually utilize Backend.AI. The Hands-on is approximately a half-day course where participants directly install Backend.AI in an Ubuntu Linux environment with GPU and Docker pre-installed, download models available on Hugging Face, and run them as services through the Backend.AI WebUI.

While installing Backend.AI in an actual enterprise operational environment requires separate procedures and review processes, a single installation file is provided at https://www.backend.ai/download for easy installation with minimum specifications in environments similar to the Hands-on session.

One of the biggest challenges in Hands-on training is providing high-performance development environments such as GPU instances to each participant. GPU resources in particular are costly and difficult for individuals to prepare directly, which is why many participants prefer remote environments. While company internal GPU resources are sometimes utilized, company resources are often limited in size, making it difficult to respond flexibly to different situations.

To address this issue, Lablup received over 30 L40s GPU instances from Vultr, a cloud computing company establishing a partnership with Lablup, for this Hands-on session. During preparation, Lablup staff accessed each instance to complete basic setup and deployed unified environments to participants. Each participant used prepared instances to conduct exercises in independent GPU environments, enabling a smoother and more efficient training session than ever before.

Many people participated in this Hands-on session, and participants were able to successfully run the google/gemma-2-9b-it model from Huggingface on their directly installed Backend.AI. Participants not only installed Backend.AI but also had time to consider how to better use Backend.AI in practical work settings and how to incorporate AI into their professional activities.

The reason Lablup actively utilized Vultr's cloud GPU resources for this Hands-on was to increase accessibility and practicality, allowing anyone to easily experience high-performance AI infrastructure. Recently, Lablup, together with Vultr, installed Backend.AI on Vultr Cloud GPU environments configured with NVIDIA's latest B200 hardware to perform various benchmark projects, and discovered areas where Vultr Cloud GPU and Backend.AI could create synergy in the synthetic data generation process for creating domain-specific LLMs. More information about this can be found at the following link.