Managed Spark

Serverless Apache Spark.

Serverless Spark with no cluster management. Submit Spark jobs that auto-scale. Pay only for compute used.

Submit job View docs
SERVERLESS SPARK ENGINE1. UNSTRUCTURED LAKE📄Raw App LogsJSON files🛰️IoT TelemetryParquet Format💧CDC StreamsIceberg / DeltaDATA READ⚡2. AUTO-SCALING CLUSTERSERVERLESS COMPUTE ENGINEDriver NodeW-1W-2W-3+df = spark.read.json("s3://logs")df.groupBy("user").count().write()ZERO INFRASTRUCTURE OPS 🛠️DATA WRITE3. BUSINESS VALUE🥇Gold Tables (DWH)Cleaned & Joined🤖ML Feature StoreTraining Ready📈BI / DashboardsAggregated Views💸Cost-Optimized Spark100% Open Source API

Serverless

Infra

< 10 sec

Startup

Auto

Scaling

Per-second

Cost

Executor instances

Choose the right executor sizing for your Spark jobs.

Spark-S

Standard Executors

Standard compute-to-memory ratio for typical ETL and data processing jobs.

Daily ETLData prepLog parsingBatch processing
View all configurations

vCPUs

2 - 16

Memory

8 GB - 64 GB

Scale

Up to 1000 nodes

Startup

< 10 sec

Spark, serverless.

No clusters. 10s startup. Per-second billing.

No clusters

Submit jobs. No cluster management.

10-second startup

Warm pools for instant Spark startup.

Auto-scaling

Scale from 1 to 1000 executors automatically.

Per-second billing

Pay only for compute time used.

PySpark & SQL

PySpark, Spark SQL, and Scala support.

Delta Lake

Built-in Delta Lake for ACID transactions.

Getting started

Launch your first instance in three steps. CLI, console, or API — your choice.

Terminal
ur data spark submit \
  --script=etl.py \
  --input=s3://data/raw/ \
  --output=s3://data/processed/

Spark patterns.

Serverless ETL and ad-hoc analysis.

Serverless ETL

No-cluster ETL with per-second billing.

View tutorial

Suggested configuration

Serverless · Auto-scale · Per-second

Estimate your costs

Create detailed configurations to see exactly how much your architecture will cost. Pay for what you use, down to the second.

Configuration 1

Estimated: $569.50/mo

Spark Cluster

Compute Resources

TB

Storage & Output

GB
GB
Config 1 cost$569.50

Cost details

$569.50

Serverless and cluster-based Apache Spark.

Configuration 1
$569.50
10 Processing Unit(s)$500.00
Data Processed$50.00
Storage$11.50
Egress$8.00
Share

Works seamlessly with

S3/Delta
Airflow
Catalog
IAM
Monitoring
BI

Frequently asked questions

Spark, serverless.

No clusters. 10s startup. Per-second billing.

Submit job View docs