21 Nov
|
RADcube - A NLogix
|
Shaikpet
21 Nov
RADcube - A NLogix
Shaikpet
Senior Data Engineer
AWS Glue / Databricks
LOCATION: Hyderabad, India
EXPERIENCE: 6–10 years (preferred)
ROLE OVERVIEW
Build and operationalize robust data engineering pipelines using AWS Glue, PySpark and Delta Lake to create governed, high-quality enterprise datasets.
KEY RESPONSIBILITIES
- Develop AWS Glue / PySpark transformations from Raw to Refined datasets in Amazon S3.
- Implement schema enforcement, type conversion, PII tokenization, deduplication and merges.
- Develop Delta Lake pipelines and, when required, Gold datasets and Redshift loads.
- Orchestrate jobs using Airflow or AWS Step Functions with completeness and failure checks.
- Register datasets in Unity Catalog and support governance, metadata and consumer access.
- Perform reconciliation,
performance tuning, unit testing and production troubleshooting.
- Prepare CI/CD artifacts, deployment configuration, runbooks and technical documentation.
RequirementsREQUIRED SKILLS
- Python, PySpark, Spark SQL and robust SQL
- AWS Glue, Amazon S3, Delta Lake and Databricks
- ETL/ELT, data modelling and data-quality validation
- Airflow or AWS Step Functions and failure recovery
- Git, CI/CD, data reconciliation and production support
PREFERRED EXPERIENCE
Unity Catalog, Amazon Redshift, Immuta, ServiceNow integration and regulated-data experience.
📌 Senior Data Engineer (Shaikpet)
🏢 RADcube - A NLogix
📍 Shaikpet