‘Charliecloud’ simplifies Big Data supercomputing
2017-06-13 20:50
274 查看
Los Alamos releases elegantly simple high-performance ‘convenience bubble’
LOS ALAMOS, N.M., June 7, 2017—At Los Alamos National Laboratory, home to more than 100 supercomputers since the dawn of the computing era, elegance and simplicity of programming are highly valued but not always achieved. In the case of a new product, dubbed“Charliecloud,” a crisp 800-line code helps supercomputer users operate in the high-performance world of Big Data without burdening computer center staff with the peculiarities of their particular software needs.
“Charliecloud lets users easily run crazy new things on our supercomputers,” said lead developer Reid Priedhorsky of the High Performance Computing Division at Los Alamos. “Los Alamos has lots of supercomputing power, and we do lots of simulations that are
well supported here. But we’ve found that Big Data analysis projects need to use different frameworks, which often have dependencies that differ from what we have already on the supercomputer. So, we’ve developed a lightweight ‘container’ approach that lets
users package their own user defined software stack in isolation from the host operating system.”
To build container images, Charliecloud sits atop the open-source Docker product that users install on their own system to customize the software choices as they wish. Users then import the image to the designated supercomputer and execute their application
with the Charliecloud runtime, which is independent of Docker. This maintains a “convenience bubble” of administrative freedom while protecting the security of the larger system. “This is the easiest container solution for both system administrators and users
to deal with,” said Tim Randles, co-developer of Charliecloud, also of the High Performance Computing Division. “It’s not rocket science; it’s a matter of putting the pieces together in the right way. Once we did that, a simple and straightforward solution
fell right out.”
The open-source product is currently being used on two supercomputers at Los Alamos, Woodchuck and Darwin, and at-scale evaluation on dozens of nodes shows the same operational performance as programs running natively on the machines without a container. “Not
only is Charliecloud efficient in compute time, it’s efficient in human time,” said Priedhorsky. “What costs the most money is people thinking and doing. So we developed simple yet functional software that’s easy to understand and costs less to maintain.”
Charliecloud is very small, only 800 lines of code, and built following two bedrock principles of computing, that of least privilege and the Unix philosophy to “make each program do one thing well.” Competing products range from 4,000 to over 100,000 lines
of code. Charliecloud is described in detail in a technical report online.
Los Alamos National Laboratory and supercomputing have a long, entwined history. Los Alamos holds many “firsts,” from bringing the first problem to the nation’s first computer to building the first machine to break the petaflop barrier. Supercomputers are integral
to stockpile stewardship and the national security science mission at Los Alamos.
4
TOPICS
CommunityComputing
Earth
Energy
Environmental Stewardship
Health
Materials
Operations
Science
Space
Technology
The Lab
JUNE
Rover findingsindicate stratified lake on ancient Mars
Girls in STEM aims to
boost interest in science careers
DOE Small
Business Program Manager of the Year
‘Charliecloud’ simplifies
Big Data supercomputing
LGBTQ+ art installation supports
mission focus on diversity
» More
FAST FACTS
PeoplePlace
Budget
Demographics
Awards
Directions
About Los Alamos National Laboratory
Los Alamos National Laboratory, a multidisciplinary research institution engaged in strategic science on behalf of national security, is operated by Los Alamos National Security, LLC, a team composed of Bechtel National, the University of California, BWXT GovernmentGroup, and URS, an AECOM company, for the Department of Energy’s National Nuclear Security Administration.
Los Alamos enhances national security by ensuring the safety and reliability of the U.S. nuclear stockpile, developing technologies to reduce threats from weapons of mass destruction, and solving problems related to energy, environment, infrastructure, health,
and global security concerns.
相关文章推荐
- 大数据与云计算(Big Data And Cloud Computing)
- Using In-Memory Computing to Simplify Big Data Analytics (zz)
- Putting Spark to Use: Fast In-Memory Computing for Your Big Data Applications
- Distributed SQL Query Engine for Big Data
- 翻译-In-Stream Big Data Processing 流式大数据处理
- Recommended Books on Machine Learning and Big Data
- Thinking in BigData(五)大数据之统计学与数据挖掘
- Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Other Huge Engineering Effo
- A record--Offline deployment of Big Data Platform CDH Cluster
- [CFNews] ComputerWorld:IBM melds crime-fighting, big data analytics in one security package
- Big Data Analytics Beyond Hadoop
- Big data and its developer fallout
- what we benefit from big data
- Thinking in BigData(六)大数据技术核心之ETL
- 翻译-In-Stream Big Data Processing 流式大数据处理
- Hadoop, Python, and NoSQL lead the pack for big data jobs
- In-Stream Big Data Processing
- Big data defined
- [文章摘要]Visual Exploration of Big Spatio-Temporal Urban Data: A Study of New York City Taxi Trips
- mount: wrong fs type, bad option, bad superblock on 125.64.41.244:/data/img