National Data Platform

NDP-EP

National Data Platform - EndPoint

Accelerate Your Data Workflows

Process Once, Access Many Times

Press Next or use arrow keys

Case Study

GOES-18 Satellite Imagery Analysis

Scientific Goal

Generate a visible band reflectance map of Utah using GOES-18 geostationary satellite data for:

  • Cloud cover monitoring
  • Land surface analysis
  • Weather pattern visualization
  • Environmental research
🛰️
GOES-18 ABI
Band C02 (0.64 µm)
Red Visible Channel
CONUS
Full Continental US
→
Utah
37°N - 42°N, 109°W - 114°W

The Problem

Every researcher repeats the same work

☁️
AWS S3
Public Data
→
⬇️
Download
57 MB
→
⚙️
Process
~30 sec
→
📊
Visualize

Every user downloads 57 MB and processes from scratch

10 users = 570 MB downloaded, 10x processing time wasted

The Solution: NDP-EP

Process once, share with everyone

👤 First User (One Time)

☁️
AWS S3
→
⚙️
Process
→
📤
Upload to NDP-EP
→
📝
Register

👥 All Other Users (Forever)

🔍
Search
→
⬇️
Download
1 MB
→
📊
Visualize

Real Results

GOES-18 Satellite Data for Utah Region

❌ Without NDP-EP

Download Size 57.3 MB
Total Time ~45 seconds
Processing Required every time
Data Discovery Manual search

✓ With NDP-EP

Download Size 1 MB
Total Time ~0.3 seconds
Processing Already done
Data Discovery Searchable catalog
99%
Faster
98%
Less Data

Technical Benefits

Unified API for Multiple Technologies

👤
Your Application
🔌
NDP-EP API
Single Interface
📚
CKAN
Catalog
💾
MinIO/S3
Storage
📡
Kafka
Streaming
📓
JupyterLab
Analysis

Key Benefits

⚡

Faster Access

Download pre-processed data directly from local S3 instead of raw data from slow external sources.

🔄

No Redundant Work

Process data once, use many times. Share processed datasets with your entire team.

🔍

Data Discovery

Search the catalog before downloading. Find datasets others have already prepared.

🛠️

Simplified Tech Stack

One API for catalog, storage, streaming, and analysis. Focus on your research, not infrastructure.

Live Comparison

Download Time & Size

Without NDP-EP (AWS → Process) 57.3 MB / 45 sec
With NDP-EP (Local S3) 1 MB / 0.3 sec

Get Started

# Install the library
pip install ndp-ep

# Connect to NDP-EP
from ndp_ep import Client
client = Client(base_url="https://your-api.com", token="your-token")

# Search for data
results = client.search_datasets(terms=["satellite", "utah"])

# Download processed data
data = client.download_object("bucket", "file.nc")
📦
Python Library
📚
Tutorials
🔌
API Docs
1 / 9
NDP SCI Institute University of Utah ndp@sdsc.edu
Funded by NSF 2333609 under CI, CISE Research Resources programs