Job Summary
We are looking for a Data Engineer with AI to join our team. The ideal candidate should have 1-4 years of experience in data warehousing, data engineering, or data warehouse operations. The role involves working with ETL tools such as Informatica PowerCenter, SSIS, DataStage, or Talend, along with Bash, Python, and SQL. The candidate should have a basic understanding of data processing, job scheduling, Databricks, PySpark, Delta Lake, and Apache Airflow. Experience using LLM-based coding assistants responsibly is also required.
Your future duties and responsibilities
- Work on data warehousing, data engineering, and data warehouse operations.
- Work with Informatica PowerCenter or comparable ETL tools such as SSIS, DataStage, and Talend.
- Read and understand ETL mappings and workflow XML.
- Understand and work with ETL concepts, including data types, NULL values, duplicates, and rounding differences.
- Work with job scheduling concepts, including dependencies, calendars, and retries.
- Support data engineering activities using Bash, Python, and SQL.
- Work with Python and PySpark for data processing activities.
- Work with Databricks notebooks, Delta Lake, and Databricks Workflows.
- Understand Apache Airflow concepts, including DAGs, operators, and sensors.
- Understand how Databricks jobs are triggered.
- Use LLM-based coding assistants responsibly to support development activities.
Required qualifications to be successful in this role
Must-Have Skills:
- 1-4 years of experience in data warehousing, data engineering, or data warehouse operations.
- Working-level knowledge of Bash, Python, and SQL.
- Hands-on experience with Informatica PowerCenter or comparable ETL tools such as SSIS, DataStage, or Talend.
- Ability to read and understand ETL mappings and workflow XML.
- Understanding of ETL concepts, data types, NULL values, duplicates, and rounding differences.
- Understanding of job scheduling concepts, including dependencies, calendars, and retries.
- Experience with schedulers such as Control-M, AutoSys, Robot, or Cron.
- Basic knowledge of Python and PySpark.
- Beginner-to-working-level knowledge of Databricks notebooks, Delta Lake, and Databricks Workflows.
- Basic understanding of Apache Airflow, including DAGs, operators, and sensors.
- Experience using LLM-based coding assistants responsibly.
Valuable-to-Have Skills:
- Exposure to Databricks job-triggering mechanisms.
- Familiarity with multiple ETL tools such as Informatica PowerCenter, SSIS, DataStage, or Talend.
- Exposure to Apache Airflow and workflow orchestration.
- Familiarity with AI-assisted development tools and LLM-based coding assistants.
Skills
- Analytical Thinking
- Artificial Intelligence
- AWS AI Services
- Communication
- Database
- Machine Learning
- Python
📌 Data Engineer with AI (Bengaluru)
🏢 CGI
📍 Bengaluru