Role Description
UST Tittle: Lead I - Data Engineering
Who We Are
At UST, we help the world’s best organizations grow and succeed through transformation. Bringing together the right talent, tools, and ideas, we work with our client to co-create lasting change. Together, with over 30,000 employees in 30+ countries, we build for boundless impact—touching billions of lives in the process. Visit us at .
Summary
UST is looking for Sr. Big Data Engineer
The Opportunity
• Hands-on senior engineer and technical lead who has personally built and shipped large-scale, production-grade ETL pipelines on AWS.
• The resource will architect and deliver batch/streaming pipelines, optimize cost and performance at scale, and set the team's engineering standards.
Required
• Minimum 10 years in data engineering / big data is mandatory.
• Mandatory Qualifications Production experience (not POCs) required in all of: -
• AWS Glue — Glue ETL jobs, Data Catalog, crawlers, job bookmarks, dynamic frames; tuning for cost/runtime. -
• Amazon EMR — Cluster lifecycle (transient,
EMR Serverless / on EKS), Spark tuning, instance fleets / spot. -
• PySpark — DataFrame APIs, UDFs, partitioning, join strategies, skew handling, and Spark memory/executor tuning. -
• AWS Step Functions — Orchestrating multi-stage ETL with retries, error handling, and parallel/map states. -
• Production ETL delivery — Delivered ETL at scale with data quality controls, schema evolution, idempotency, and backfills.
Core Skills
• Apache Spark internals; Parquet/ORC/Avro and Iceberg/Hudi/Delta; partitioning and small-file management - S3 data lake, Athena, Redshift/Spectrum, Lake Formation; Lambda, SQS/SNS, Event Bridge; Kinesis/MSK (plus)
• Expert Python and solid SQL; IAC (Terraform / CDK / CloudFormation); CI/CD; Git; Docker - Dimensional modelling, Lakehouse / medallion architectures, batch and CDC ingestion
• Production-Readiness Expectations - Reliability (retries, checkpointing, safe re-runs, SLAs/SLOs)
• Ob
📌 Sr. Bigdata Engineer (Hyderabad)
🏢 UST
📍 Hyderabad