The NESDIS Cloud Framework A Common, Data-Agnostic Cloud

The NESDIS Cloud Framework A Common, Data-Agnostic Cloud Computing Solution Satellite and Information Service 28 February 2020 Kathryn Shontz NOAA Office of System Architecture and Advance Planning Department of Commerce // National Oceanic and Atmospheric Administration // 1

NESDIS Strategic Objectives The next generation ground service capability will need to scale to accommodate ever increasing data source volumes and be flexible enough to bring in any type of data Lean into innovation solutions by asking: Why not the commercial cloud? Department of Commerce // National Oceanic and Atmospheric Administration // 2

The NESDIS Cloud Vision Centralize all NESDIS data holdings and bring the processing to the data Department of Commerce // National Oceanic and Atmospheric Administration // 3

NESDIS Cloud Framework Architecture Framework - Enables a set of services to work together to deliver mission value Service - How to meet the core NESDIS IT functions Tool - Cloud software application(s) used to implement the service Data Department of Commerce // National Oceanic and Atmospheric Administration // 4

Cloud Framework Use Case – Secure Data Ingest Workflow Continuous Integration Continuous Deployment Data Consolidated Ingest Data Storage Software and Release Management Science and Development Sandbox Compute Environment Data This depicts how to leverage a common ingest service securely to bring in non-NOAA partner data Results Developers Data Metadata Catalog Data Distribution and Access Users Configuration Management and Program Office Department of Commerce // National Oceanic and Atmospheric Administration // 5

Cloud Framework Use Case – Satellite Data Processing Workflow This represents a nominal Continuous Integration Continuous Deployment Data Consolidated Ingest Data Metadata Storage Metadata Software and Release Management Science and Development Sandbox Compute Environment Data workflow of operational ingest, processing, and distribution of low-latency satellite data Results Metadata Query Developers Data Data Metadata Catalog Metadata Query Distribution and Access Users Configuration Management and Program Office Department of Commerce // National Oceanic and Atmospheric Administration // 6

Cloud Framework Use Case – Science Development Workflow Continuous Integration Continuous Deployment Consolidated Ingest Data Metadata Storage Science and Development Sandbox Compute Environment Metadata This workflow shows how a NESDIS Developer creates a scientific algorithm Software and Release Management Data Metadata Query Developers Data Data Metadata Catalog Metadata Query Distribution and Access Users Configuration Management and Program Office Department of Commerce // National Oceanic and Atmospheric Administration // 7

Future NESDIS Cloud Development Department of Commerce // National Oceanic and Atmospheric Administration // 8

NESDIS Cloud Transformation Roadmap We are here today! NESDIS demonstrated that there is an advantage to bringing the processing to the data through a lean-in approach to innovation Adoption of an enterprise cloud framework fundamentally alters the business and processes we use today, so we are tackling that head on with a dedicated change management team Department of Commerce // National Oceanic and Atmospheric Administration // 9

A NESDIS Solution for All of NOAA NESDIS Cloud Framework NESDIS developed the Cloud Framework from High Performance Computing (HPC) Industry best practices Working with OAR to test viability of the NESDIS Cloud Framework for Numerical Weather Prediction Department of Commerce // National Oceanic and Atmospheric Administration // 10

A Better Infrastructure Future Secure – FISMA-compliant FedRAMP Moderate Cloud supports all Cloud Framework Services, tailoring controls to High in applicable areas Fault-Tolerant – redundant and highly available services provide robust applications Scalable – capacity to accommodate all current and future workloads Data Agnostic – enable any data type and workflow within the framework Decoupled – services are independent of each other and are interchangeable Cloud Agnostic – workloads and services run in any cloud service provider Resources On-Demand – rapid provisioning of cloud framework services based on business needs Agile – support faster transition to operations with DevOps Department of Commerce // National Oceanic and Atmospheric Administration // 11