Roles and Responsibilities:
Develop and maintain scalable data engineering pipelines using Python and PySpark.
Develop data processing workflows using Databricks and Apache Spark.
Ingest data from Teradata, databases, APIs, files, and cloud storage into AWS-based data platforms.
Write and optimize SQL queries for data extraction, transformation, and analysis.
Build and maintain data pipelines using AWS services such as S3, Glue, Lambda, and Redshift.
Develop and manage Databricks notebooks, jobs, clusters, and workflows.
Extract and transform data from Teradata and migrate or integrate it with cloud-based data platforms.
Use Git for version control and collaborative development.

Must-Have Skills
Solid proficiency in Python for data processing and automation.
Strong hands-on experience with PySpark and Apache Spark.




Hands-on experience with Databricks for data engineering and analytics.
Solid knowledge of SQL, including complex queries, joins, CTEs, window functions, and optimization.
Solid experience with AWS cloud services, particularly S3, Glue, Lambda, and Redshift.
Positive hands-on experience with Teradata and Teradata SQL.
Strong understanding of ETL/ELT concepts and data pipeline development.
Experience working with large-scale data processing and distributed computing.
Knowledge of AWS data lake and data warehouse architecture.
Experience with Databricks workflows, jobs, notebooks, and cluster management.
Strong analytical, problem-solving, and communication skills.

📌 Aws Data Engineer Bengaluru
🏢 CGI
📍 Bengaluru