The intersection of artificial intelligence and scientific research is a burgeoning frontier. With its newly announced DeepSpeed4Science initiative, Microsoft Research is poised to redefine how AI assists in large-scale scientific discovery. This initiative builds upon Microsoft’s open-source DeepSpeed system, a framework that has already proven its mettle in enabling large-scale deep learning models. DeepSpeed4Science aims to provide unique capabilities for unlocking some of science’s most intriguing mysteries, spanning domains like climate science, drug design, and molecular dynamics simulation.
DeepSpeed’s Evolution into DeepSpeed4Science
DeepSpeed has earned its reputation as an industry-leading open-source AI system framework, specifically designed to enable deep learning training and inference at unprecedented scale and speed. Its core technology pillars—training, inference, and compression—form the foundation of the new DeepSpeed4Science initiative.
The initiative is designed to go beyond generic large language models (LLMs) and tailor AI technologies for scientific discoveries. The DeepSpeed4Science platform will serve as a unified repository for sharing advanced AI technologies that can accelerate scientific research, in line with Microsoft’s AI for Good commitment.
Collaborative Efforts in Science
DeepSpeed4Science has attracted a diverse group of collaborators both internally and externally. On the internal front, the initiative is supporting key science models from Microsoft Research AI4Science and Microsoft WebXT/Bing. Externally, it’s working with U.S. DoE National Labs and other academic institutions on cutting-edge scientific models.
Internal Partnerships
Scientific Foundation Model (SFM)
SFM aims to create a unified large-scale foundation model to empower natural scientific discovery. It supports diverse inputs and multiple scientific domains, including drugs, materials, biology, and health. DeepSpeed4Science will provide new training and inference technologies to empower ongoing research projects like Microsoft’s Distributional Graphormer.
ClimaX
ClimaX is the first foundation model designed for a wide variety of weather and climate modeling tasks. As the world faces increasingly frequent extreme weather events, the model aims to improve weather forecasting by absorbing different datasets with varying variables and resolutions.
External Collaborations
The journey began with partnerships on LLM-based AI models for structural biology research, such as OpenFold from Columbia University and GenSLMs from Argonne National Laboratory. These models represent common AI system challenges facing today’s AI-driven structural biology research. The initiative is also expanding its scope to support a more diverse range of science models, including projects focused on clean energy research and cancer surveillance.
Tackling Systemic Challenges
DeepSpeed4Science is making strides in addressing critical systemic challenges in structural biology research. For example, it has eliminated memory explosion problems for scaling Evoformer-centric protein-structure prediction models. It also enables very-long sequence support for genome-scale foundation models, thus allowing scientists to explore relationships that were previously inaccessible.
Memory Efficiency
A new solution from DeepSpeed4Science called DS4Sci_EvoformerAttention significantly reduces peak memory requirements for training by 13X without a loss in accuracy. This addresses a common challenge in structural biology research, particularly in protein structure prediction.
Long-Sequence Support
Through DeepSpeed4Science’s new designs, scientists can now build and train models with significantly longer context windows. This drastically improves model quality and expands the scope of scientific discovery.
The Path Forward
DeepSpeed4Science is not just a technological initiative; it’s a collaborative platform designed to generalize AI system technologies that broadly address major system pain points in large-scale scientific discoveries. Microsoft is inviting the global scientific community to participate, contribute, and utilize these open-sourced software tools to expedite scientific advancements.
Let’s Keep Exploring
By developing and sharing advanced AI system technologies through DeepSpeed4Science, Microsoft is setting the stage for the next wave of scientific discoveries. As this initiative evolves, it promises to unlock new capabilities and opportunities for scientists around the world. The fusion of AI and science has never looked so promising.
