Job Summary
Job Title: Data Engineer with AI
Position: Software Engineer
Experience: 6-12 Years
Category: Data & Analytics / Data Engineering
Location: Bangalore
Employment Type: Full Time
Position Description: We are looking for an experienced Data Engineer with AI to join our team. The ideal candidate should have 6-8 years of experience in data warehousing, data engineering, or data warehouse operations. The role involves supporting and leading data migration projects, working with ETL tools such as Informatica PowerCenter, and managing data workflows using Python, SQL, PySpark, and Databricks. The candidate should also have an understanding of job scheduling, Apache Airflow, Delta Lake, and the responsible use of LLM-based coding assistants.
Responsibilities
- Lead and support data migration projects.
- Work with Informatica PowerCenter or comparable ETL tools such as SSIS, DataStage, and Talend.
- Read and understand ETL mappings and workflow XML.
- Work with data transformation concepts, including data types, NULL values, duplicates, and rounding differences.
- Understand and manage job scheduling concepts, including dependencies, calendars, and retries.
- Work with Python, PySpark, and SQL for data engineering activities.
- Work with Databricks notebooks, Delta Lake, and Databricks Workflows.
- Understand and work with Apache Airflow DAGs, operators, and sensors.
- Support and understand Databricks job-triggering mechanisms.
- Leverage LLM-based coding assistants responsibly to support development activities.
Required qualifications to be successful in this role
Must-Have Skills
- 6-8 years of experience in data warehousing, data engineering, or data warehouse operations.
- Working-level knowledge of Bash, Python, and SQL.
- Experience leading data migration projects.
- Hands-on experience with Informatica PowerCenter or comparable ETL tools such as SSIS, DataStage, or Talend.
- Ability to read and understand ETL mappings and workflow XML.
- Understanding of ETL concepts, data types, NULL values, duplicates, and rounding differences.
- Understanding of job scheduling concepts, including dependencies, calendars, and retries.
- Experience with schedulers such as Control-M, AutoSys, Robot, or Cron.
- Basic to working-level knowledge of Python, PySpark, Databricks notebooks, Delta Lake, and Databricks Workflows.
- Basic understanding of Apache Airflow, including DAGs, operators, and sensors.
- Experience using LLM-based coding assistants responsibly.
Positive-to-Have Skills
- Exposure to Databricks job-triggering mechanisms.
- Experience with multiple ETL tools such as Informatica PowerCenter, SSIS, DataStage, or Talend.
- Familiarity with Apache Airflow and workflow orchestration.
- Exposure to AI-assisted development and LLM-based coding tools.
Skills
- Databricks
- Python
- Artificial Intelligence
- AWS AI Services
- Analytical Thinking
📌 Data Engineer with AI (Bengaluru)
🏢 CGI
📍 Bengaluru