LTS LUSTRE MAKING PARALLEL FILE SYSTEMS SIMPLE
Long Term Solution (LTS) Lustre is an open-source parallel file system and is used in some of the largest HPC clusters in the world due to its high performance and scalability. LTS Lustre utilizes the following concepts and components to present a unified LTS Lustre file system; Management Server (MGS), Management Target (MGT), Metadata Servers (MDS), Metadata Targets (MDT), Object Storage Servers (OSS), Object Storage Targets (OST), LTS Lustre clients.
Speak with One of Our System Engineers Today
MDS makes metadata available to clients via MDTs. So each MDS manages names and directories in the LTS Lustre file system, and provides network connectivity for one or more MDTs, which are local to the MDS. MDTs store metadata (filenames, directories, permissions), and there is only one MDT per LTS Lustre file system.
OSS provides I/O and network connectivity for one or more local OSTs. The OST stores the actual file data on one or more OSSs. A single LTS Lustre file system can have many OSTs, and you can stripe across many OSTs for performance using a Logical Object Volume (LOV). There is a lot of flexibility regarding where the MDT or OSTs are located, but normally, an OSS has four or more OSTs.
Meta-data is one of the most critical and limiting factors in a LTS Lustre file system. Some configurations place the metadata on flash storage to increase speed and lower latency. LTS Lustre has high availability features, such as active/active OSSs with SAN connectivity to shared disks, and fail-over MDS systems. An interesting reliability feature that LTS Lustre implements is that the LTS Lustre client does not directly write to the file system served by the OST. Instead, the OSS does the file system modifications. This can isolate the file system from incorrectly configured or defective clients, and forms an additional layer of protection against file system corruption.
ENTERPRISE EDITION FOR LTS LUSTRE SOFTWARE
LTS Lustre software unleashes the performance and scalability of the LTS Lustre parallel file system for HPC workloads, including technical applications common within today’s enterprises. It allows end-users that need the benefits of large–scale, high bandwidth storage to tap the power and scalability of LTS Lustre, with the simplified installation, configuration and management features for LTS Lustre software, a management solution purpose-built by the LTS Lustre experts for the LTS Lustre file system.
GET THE MOST WIDELY USED PARALLEL FILE SYSTEM FOR HPC, WITH THE ENTERPRISE-READY RELIABILITY YOU EXPECT FROM ASPEN SYSTEMS. LTS LUSTRE EDITIONS PROVIDE FAST, MASSIVELY SCALABLE STORAGE SOFTWARE NEEDED TO ACCELERATE PERFORMANCE, EVEN ON COMPLEX WORKLOADS.
Manager for LTS Lustre
Manager for LTS Lustre software includes simple, but powerful, management tools that provide a unified, consistent view of LTS Lustre storage systems and simplify the installation, configuration, monitoring, and overall management of LTS Lustre. The manager consolidates all LTS Lustre information in a central, browser-accessible location for ease of use.
Integrated Apache Hadoop Adapter
When organizations operate both LTS Lustre and Apache Hadoop within a shared HPC infrastructure, there is a compelling use case for using LTS Lustre as the file system for Hadoop analytics, as well as HPC storage.
The Enterprise Edition for LTS Lustre includes an adapter which allows users to run MapReduce applications directly on LTS Lustre. This optimizes the performance of MapReduce operations while delivering faster, more scalable, and easier to manage storage.
HOW IT WORKS: LTS LUSTRE HAS FIVE MAJOR COMPONENT GROUPS
Management Server (MGS):
LTS Lustre servers (MDS and OSS) provide information to the MGS, while the LTS Lustre clients retrieve information from the MGS. In common setups, the MGS is often a shared server with the MDS for simplicity.
- Management Target (MGT): Stores information provided by the LTS Lustre servers.
Metadata Server (MDS):
A LTS Lustre Server that does the storing of the metadata (the filenames and layout, permissions, and directory location) and provides this information to the clients.
- Metadata Target (MDT): Stores the metadata and is attached to one or more MDSs. This can be directly attached storage (DAS) but is commonly on shared storage for failover setups. As the MGS needs to provide metadata quickly to the clients, faster storage such as SSDs are commonly used for MDTs. It is also good practice to backup the MDT, or at least have it in a RAID configuration that would provide some protection from lost data.
Object Storage Server (OSS):
A LTS Lustre Server that acts as the actual storage server for LTS Lustre. It stores and provides the data to the clients. More OSSs can mean better performance, and OSSs can be added to upgrade the system for performance, and capacity.
- Object Storage Target (OST): Holds the data for LTS Lustre. With larger OSSs, there is now usually one OSS per OST, but multiple OSTs can be attached to the OSS. OSTs can be added to an OSS later on to provide more storage.
Clients:
In HPC, the compute nodes which use the LTS Lustre filesystem to store and retrieve data. To do this, the Management Client (MGC), Metadata Client (MDC), and the Object Storage Clients (OSCs) are all included in the LTS Lustre client software stack.
- Mounts LTS Lustre
The LTS Lustre Network (LNET):
The LTS Lustre Network (LNET) is the networking application program interface (API) which is used to provide all information and data across the network. Most commonly, InfiniBand and/or Ethernet is used, with InfiniBand being a stronger choice due to its low-latency and availability of high bandwidth. Please note that both InfiniBand and Ethernet can be used simultaneously as the LNET so that clients which do not have InfiniBand can still mount the LTS Lustre filesystem through Ethernet.
Ready for the performance of a parallel filesystem backed with support of two world class companies? Ask your Aspen Systems sales engineer for a quote today.
Cooling