21 Nov
|
RADcube - A NLogix
|
India
21 Nov
RADcube - A NLogix
India
Job Description
Senior Data Engineer
AWS Glue / Databricks LOCATION: Hyderabad, India EXPERIENCE: 6–10 years (preferred) ROLE OVERVIEW
Build and operationalize robust data engineering pipelines using AWS Glue, PySpark and Delta Lake to create governed, high-quality enterprise datasets.
KEY RESPONSIBILITIES
- Develop AWS Glue / PySpark transformations from Raw to Refined datasets in Amazon S3.
- Implement schema enforcement, type conversion, PII tokenization, deduplication and merges.
- Develop Delta Lake pipelines and, when required, Gold datasets and Redshift loads.
- Orchestrate jobs using Airflow or AWS Step Functions with completeness and failure checks.
- Register datasets in Unity Catalog and support governance, metadata and consumer access.
- Perform reconciliation, performance tuning, unit testing and production troubleshooting.
- Prepare CI/CD artifacts, deployment configuration,
runbooks and technical documentation.
RequirementsREQUIRED SKILLS
- Python, PySpark, Spark SQL and strong SQL
- AWS Glue, Amazon S3, Delta Lake and Databricks
- ETL/ELT, data modelling and data-quality validation
- Airflow or AWS Step Functions and failure recovery
- Git, CI/CD, data reconciliation and production support
PREFERRED EXPERIENCE
Unity Catalog, Amazon Redshift, Immuta, Service Now integration and regulated-data experience.
Requirements
Python, PySpark, Spark SQL and robust SQL AWS Glue, Amazon S3, Delta Lake and Databricks ETL/ELT, data modelling and data-quality validation Airflow or AWS Step Functions and failure recovery Git, CI/CD, data reconciliation and production support
📌 Senior Data Engineer (India)
🏢 RADcube - A NLogix
📍 India