03 Oct
|
Nous Infosystems
|
Hyderabad
03 Oct
Nous Infosystems
Hyderabad
We are looking for an experienced Data Engineer with robust hands-on expertise in AWS, PySpark, Python, and Databricks. The ideal candidate should have experience in designing, developing, and maintaining scalable data pipelines, implementing data processing solutions, and building robust ETL/ELT workflows on cloud platforms.
Roles & Responsibilities
- Design, develop, and maintain scalable data pipelines using Python, PySpark, and Databricks.
- Develop and optimize ETL/ELT workflows for processing large volumes of structured and unstructured data.
- Work extensively with AWS services such as S3, Glue, Lambda, EMR, Redshift, and Athena.
- Develop data transformation and processing logic using PySpark and Python.
- Build and manage data engineering solutions using Databricks.
- Work with Delta Lake, Delta Tables, and Lakehouse architecture.
- Implement data ingestion, cleansing, transformation, and aggregation processes.
- Optimize Spark jobs and troubleshoot performance issues.
- Implement data quality checks, error handling, and monitoring mechanisms.
- Collaborate with data architects, data analysts, and business stakeholders.
- Ensure data security, governance, and compliance across data pipelines.
- Participate in code reviews, technical discussions, and deployment activities
📌 Excellent Opportunity For Data Engineers (Hyderabad)
🏢 Nous Infosystems
📍 Hyderabad