SCALE-OUT, FLASH-NATIVE
DATA CACHE COST-EFFECTIVELY
SOLVES I/O SCALE &
BOTTLENECK CHALLENGES

PRODUCT FAMILY

Infinite Memory Engine®

BREAK FREE FROM THE CHALLENGES AND INEFFICIENCIES CAUSED BY I/O BOTTLENECKS

DDN’s Infinite Memory Engine® (IME®) is a scale-out, software-defined, flash storage platform that streamlines the data path for application I/O. IME interfaces directly to applications and secures I/O via a data path that eliminates file system bottlenecks. With IME, architects can realize true flash-cache economics with a storage architecture that separates capacity from performance.

THE NEED FOR A NEW APPROACH TO FAST I/O

Several key factors, both technological and commercial are creating demand for a new approach to high performance I/O. New non-volatile memory (NVM) device technologies are proliferating and media capacities are increasing rapidly. Diverse many-core processor strategies are pushing higher I/O volumes and more demanding I/O profiles. A new generation of high-business-value markets are taking advantage of analytics and machine learning and further stressing performance boundaries. Traditional file systems only crudely manage flash at scale whereas HDD performance degrades as concurrency increases. IME introduces a new layer, above the file system that:

  • Maximizes flash media performance and longevity
  • Improves the efficiency of network and storage through adaptive I/O and data coalescing
  • Efficiently feeds the diverse I/O requirements of modern workloads and mixed CPU/GPU environments
  • Delivers the data rates needed across commercial analytics in machine learning

IME delivers predictable job performance, provides faster computation against data sets too large to fit in memory, accelerates I/O-intensive applications and provides a safe, cost and space-efficient landing space for bursty data.

“Infinite Memory Engine: HPC in the FLASH Era”
In this RichReport podcast, James Coomer from DDN discusses IME.

IME is well suited for a variety of use cases

  • Machine Learning and Analytics: Massive, flash cache with over 600K read IOPs in 2U and 20GB/s to support large scale data sets at speed
  • Application Ensembles: Multiple, simultaneous applications use cache to support communication throughout the job
  • Application Workflows: Cache stores a common dataset used by a succession of independent applications
  • In-situ Analysis: Real-time output of an application’s data in cache analyzed in-situ
  • Application Pre-Processing: Pre-processing job places output in cache, ready for main run
  • Application Post-Processing: Main run outputs to cache, post-processing commences in-situ
  • Checkpointing: Stores checkpoints faster than a parallel file system and keeps them in cache to support restarts

IME Test Beds and Major Deployments

jcahpc-210
ornl-210
ichec-210
tacc-210
tokyoTech-210
julich-210
kisti-210
cscs-210
sdsc-210
ncar-210
nersc-210
ohio-210
univ-tokyo-210
agronne-210
  • Software-Defined Flash Platform
  • Scale-Out Cache from TBs to PBs
  • Reduces Performance Infrastructure Footprint
  • Removes File System Bottlenecks
  • Delivers Predictable Run Times
  • Accelerate One Application or all I/O
  • Compute, Storage, and Network Agnostic
  • Supports Infiniband, Omni-Path, and Ethernet
  • Flexible Data Protection Options

USE CASES

USER VIDEOS

IME in 2017 - HPCXXL in Bologna (CINECA)
James Coomer presents: "IME in 2017" at HPCXXL in Bologna (CINECA)
1.5TB/s File System Built On DDN’s IME Burst Buffer Powers Japan’s Fastest Supercomputer
Professor Osamu Tatebe, Ph.D presents the Oakforest-PACS 25 Pflops System at SC16.
Jesse Hanley - DDN IME Early Analysis at OLCF - Video
Jesse Hanley - DDN IME Early Analysis at Oak Ridge National Laboratory
Application Experiences on DDN’s IME Burst Buffer
In this video from the 2015 HPC Advisory Council Switzerland Conference, Toine Beckers from DDN and Marco Grossi from ICHEC present: Application Experiences on DDN’s IME Burst Buffer.
DDN User Group at SC15: Early Evaluation of IME as a Burst Buffer and Application Accelerator
In this video from the DDN User Group at SC15, Dirk Pleiter from the Jülich Supercomputing Centre presents: Early Evaluation of IME as a Burst Buffer and Application Accelerator.
IME Update: I/O Profiling of Applications - Video
In this video from the DDN User Group from ISC 2016, James Coomer from DDN presents: IME Update: I/O Profiling of Applications.

IME DEPLOYMENT OPTIONS

ime-software-box-r

IME SOFTWARE-ONLY

For environments that want the maximum deployment flexibility, IME is available as software-only. IME software-only is ideal for those environments with IT-based limitations on supported hardware; that want the flexibility to repurpose existing hardware; and that have open-platform requirements. Most importantly, whether you decide to use IME240, or software-only, IME is the industry’s only flash-native cache that is not locked into a specific server or storage hardware vendor. With DDN’s IME, the choice is yours.

IME240-Front-View

I/O PERFORMANCE-OPTIMIZED COMMODITY SERVERs

For an integrated appliance, DDN offers the IME140™ and IME240™ The IME240 is built on a best-of-breed 2U commodity storage server that delivers over 20GB/s of bandwidth and over 1M IOPs direct to applications. IME240 has up to 23 NVMe SSD drives for data and flexible N+K erasure coding. The IME140 supports extreme file performance in a small 1U flash data appliance. Each appliance can deliver more than 11GB/s write and 20GB/s read throughputs and more than 1M file IOPs (read and write). Starting with a resilient solution as small as 4 units, the IME140 allows organizations to cost-effectively scale performance independent of amount of capacity required.

SPECIFICATIONS

 

IME140

IME240

SYSTEM FEATURES

Dual redundant power and cooling

FABRIC PORTS

Up to 2x InfiniBandTM EDR/FDR; or 2x 10/40/100GbE Ethernet; or 2x Intel® Omni-Path

2x InfiniBandTM EDR/FDR; or 2x 10/40/100GbE Ethernet; or 2x Intel® Omni-Path

WEIGHT

Maximum weight:  48lbs / 21.8KG

Maximum weight:  55lbs / 24KG

DIMENSIONS

1RU; (WxHxD) 17.2 x 1.7 x 27.82 in. (437 x 43 x 707 mm)

2RU; (WxHxD) 17.2 x 3.5 x 27.76 in. (437 x 89 x 705 mm)

POWER

800W total system power consumption  (AC input requirement, max. config. with 8 data drives)

796W total system power consumption  (AC input requirement, max. config. with 23 data drives)

DATA DRIVES

NVMe data drives up to 8 per system. Capacities from 1.2TB per NVMe SSD

NVMe data drives up to 23 per system. Capacities from 1.2TB per NVMe SSD

SYSTEM DRIVE

2x 2.5”, 1TB, SAS3.0 12GB/S, 7.2K RPM

2x 2.5”, 1TB, SAS3.0 12GB/S, 7.2K RPM

CPU, MEMORY

Dual  Intel® Xeon® Scalable Processors (skylake), 8x 8GB DDR4-2400 RAM

Dual Intel® E5 Series Processor, 8x 16GB
DDR4-2400 RAM

OUT-OF-BAND MANAGEMENT

Dedicated IPMI LAN RJ45

DRIVE BAYS

10, tool-less 2.5” drive bays

24, tool-less 2.5” drive bays

SEQUENTIAL READ AND WRITE PERFORMANCE

Up to 20GB/s (EDR/OPA)

Up to 23GB/s (EDR/OPA)

ERASURE CODING

N (data)+K (parity) Flexible Erasure Coding options with K=1, 2 or 3. Supports (3+1)...(15+1), (6+2)... (15+2), (9+3)...(15+3)

*Specifications are subject to change.