19 Nov
|
Recognized
|
Bengaluru
19 Nov
Recognized
Bengaluru
Data Engineer L2
Founded in 1976, CGI is among the largest independent IT and business consulting services firms in the world. With 94,000 consultants and professionals across the globe, CGI delivers an end-to-end portfolio of capabilities, from strategic IT and business consulting to systems integration, managed IT and business process services and intellectual property solutions. CGI works with clients through a local relationship model complemented by a global delivery network that helps clients digitally transform their organizations and accelerate results. CGI Fiscal 2024 reported revenue is CA$14.68 billion and CGI shares are listed on the TSX (GIB.Job Title: Data Engineer L2
Experience: 4- 6 years
Software Development/ Engineering
Employment Type: Full Time
Bachelor's degree in Computer Science or related field or higher with minimum 4 years of relevant experience.
5–8 years of experience in data engineering, with very strong hands on expertise in at least one major cloud platform: AWS, Azure, or GCP.
- Design, develop, implement, and maintain scalable data pipelines and data processing solutions using cloud native services and modern data engineering technologies.
- Develop and maintain ETL/ELT pipelines to ingest, transform, validate, and load structured, semi structured, and unstructured data from multiple sources.
- Strong proficiency in SQL, including complex queries, joins, aggregations, window functions, query optimization, and performance tuning.
- Solid programming skills in Python and experience with PySpark or Apache Spark for large scale data processing.
- Hands on experience with cloud data services such as AWS Glue, Amazon Redshift, S3, Azure Data Factory,
Azure Databricks, ADLS, Synapse Analytics, GCP Dataflow, BigQuery, and Cloud Storage, depending on the cloud platform.
- Experience working with data warehouses, data lakes, lakehouse architectures, and analytical data platforms.
- Develop and optimize data transformation logic, reusable data processing components, and business data models.
- Implement data quality checks, data validation, reconciliation, error handling, and exception management within data pipelines.
- Troubleshoot pipeline failures, data inconsistencies, performance bottlenecks, and production data processing issues.
- Monitor data pipelines, job execution, resource utilization, and processing performance; Optimize SQL queries, Spark jobs, partitioning, file formats, and data storage to improve performance and reduce cloud costs.
- Implement incremental data loads, CDC (Change Data Capture), batch processing, and streaming data ingestion where required.
- Implement data orchestration and scheduling using tools such as Apache Airflow, Azure Data Factory, AWS Step Functions, or equivalent services.
- Apply data security, encryption, access control, secrets management, and data governance practices.
- Implement CI/CD practices, version control using Git, code reviews, and automated testing for data engineering solutions.
- Support infrastructure and pipeline deployments across development, test, and production environments.
- Perform L2 production support, incident troubleshooting, root cause analysis, defect resolution, and preventive maintenance for data pipelines.
- Collaborate with data architects, analysts, BI developers, application teams, and business stakeholders to understand data requirements and deliver reliable data solutions.
- Maintain technical documentation, data mappings, pipeline dependencies, operational runbooks, and support procedures.
- Identify opportunities to improve pipeline reliability, automation, data quality, processing efficiency, and maintainability.
Very strong hands on expertise in at least one cloud platform (AWS, Azure, or GCP); cloud experience is mandatory.
- Strong SQL and Python skills, with practical experience in PySpark/Spark.
- Proven experience building and supporting production grade ETL/ELT pipelines.
- Experience with cloud data storage, data integration, data processing, and data warehousing services.
- Knowledge of data modeling, dimensional modeling, partitioning, performance tuning, and data quality practices.
- Experience with Git, CI/CD, orchestration, monitoring, and production support.
- Here, you’ll reach your full potential because…
That’s why we call ourselves CGI Partners rather than employees. To support our recruitment team, AI tools may be used to help assess applications though they never replace human judgement. Come join our team—one of the largest IT and business consulting services firms in the world.
📌 Data Engineer (AWS, SQL) (Bengaluru)
🏢 Recognized
📍 Bengaluru