18 Nov
|
Advance Career Solutions
|
Pune
18 Nov
Advance Career Solutions
Pune
AWS Databricks Data Engineer
Experience: 312 Years
Location: Pune
Job Summary
We are looking for an experienced Data Engineer with strong hands-on experience in AWS, Databricks, PySpark, and SQL. The candidate will be responsible for designing, developing, and maintaining scalable data pipelines and data processing solutions using cloud-based technologies.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using AWS and Databricks.
- Develop data processing solutions using PySpark and Python.
- Write, optimize, and maintain complex SQL queries for data extraction and transformation.
- Work on data ingestion, transformation, cleansing, and processing from multiple sources.
- Build and maintain ETL/ELT workflows for large-scale data processing.
- Work with AWS services for data storage, processing, and integration.
- Perform data validation, troubleshooting, and performance optimization.
- Ensure data quality, reliability, and efficient pipeline execution.
- Collaborate with cross-functional teams to understand business requirements and deliver data solutions.
- Support deployment, monitoring, and maintenance of data engineering workflows.
Required Skills
- Hands-on experience with AWS cloud services.
- Strong experience in Databricks platform.
- Proficiency in PySpark and Python for distributed data processing.
- Strong SQL skills, including complex queries and performance optimization.
- Good understanding of ETL/ELT processes and data pipeline development.
- Experience working with large datasets and distributed data processing.
- Understanding of data warehousing and data lake concepts.
- Solid analytical, troubleshooting, and problem-solving skills.
Preferred Skills
- Experience with AWS services such as S3, Glue, Lambda, EMR, and Redshift.
- Knowledge of Delta Lake and Lakehouse architecture.
- Experience with workflow orchestration and job scheduling.
- Familiarity with CI/CD and version control tools such as Git.
- Understanding of data modelling and data governance concepts.
📌 AWS Databricks Engineer (Pune)
🏢 Advance Career Solutions
📍 Pune