Roles and Responsibilities:

- Develop and maintain scalable data engineering pipelines using Python and PySpark.

- Develop data processing workflows using Databricks and Apache Spark.

- Ingest data from Teradata, databases, APIs, files, and cloud storage into AWS-based data platforms.

- Write and optimize SQL queries for data extraction, transformation, and analysis.

- Build and maintain data pipelines using AWS services such as S3, Glue, Lambda, and Redshift.

- Develop and manage Databricks notebooks, jobs, clusters, and workflows.

- Extract and transform data from Teradata and migrate or integrate it with cloud-based data platforms.

- Use Git for version control and collaborative development.

Must-Have Skills

- Solid proficiency in Python for data processing and automation.

- Strong hands-on experience with PySpark and Apache Spark.





- Hands-on experience with Databricks for data engineering and analytics.

- Solid knowledge of SQL, including complex queries, joins, CTEs, window functions, and optimization.

- Strong experience with AWS cloud services, particularly S3, Glue, Lambda, and Redshift.

- Good hands-on experience with Teradata and Teradata SQL.

- Strong understanding of ETL/ELT concepts and data pipeline development.

- Experience working with large-scale data processing and distributed computing.

- Knowledge of AWS data lake and data warehouse architecture.

- Experience with Databricks workflows, jobs, notebooks, and cluster management.

- Strong analytical, problem-solving, and communication skills.

📌 Aws Data Engineer (Bengaluru)
🏢 CGI
📍 Bengaluru