Notice Period: Immediate or 10 days
Budget: 8-12 LPA
Must Have Skills: SQL, PySpark, Databricks, AWS, Python
Positive To Have Skills: PySpark, AWS CloudWatch, AWS S3, Streaming
External Description:
This is a technical role. No shift timings Job Description:
1) Experience in AWS Databricks Data pipelines Development - DLT, Streaming & Data ingestion patterns, Unity Catalogue migration
2) Independently develop AWS data extraction and integration code modules for one-time historical, incremental data flows from Databricks Delta Lake using PySpark, Python, SQL, AWS Databricks, and S3.
3) Develop scalable data ingestion pipelines as needed
4) Implement advanced transformation logics, schedule, orchestrate, and validate pipelines
5) Perform data validation, data reconciliation
6) Deployment of Components
7) Perform unit testing and document test results.