08 Nov
|
Valam Solutions
|
Chennai
08 Nov
Valam Solutions
Chennai
AI-Augmented Data EngineerRole : AI-Augmented Data EngineerEmployment Type : ContractualJob Summary :We are looking for an AI-Augmented Data Engineer to design, develop and maintain scalable data pipelines and infrastructure while leveraging AI-powered tools to improve engineering productivity and data processing efficiency.The role focuses on building robust ETL/ELT pipelines, integrating data from multiple sources, optimizing data workflows and implementing automated data processing solutions.The ideal candidate should have strong expertise in Python, SQL, data engineering frameworks and cloud-based data platforms, along with practical experience using AI-assisted development tools to accelerate coding, debugging, documentation and pipeline optimization.Key Responsibilities :- Design, develop and maintain scalable data pipelines for processing structured and unstructured data.- Build and manage ETL/ELT workflows to extract, transform and load data from multiple sources.- Leverage AI-powered coding assistants to accelerate pipeline development, SQL generation, debugging and code optimization.- Validate and optimize AI-generated code to ensure data accuracy, reliability, security and maintainability.- Develop data transformation workflows using Python, SQL and modern data engineering frameworks.- Implement batch and real-time data processing solutions based on business requirements.- Build and maintain data integrations across APIs, databases,
cloud storage and enterprise applications.- Develop and manage data workflows using orchestration tools such as Apache Airflow.- Work with data warehouses and lakehouse platforms to support analytics and reporting requirements.- Implement data quality checks, validation rules and monitoring mechanisms.- Optimize database queries, pipeline performance and resource utilization.- Collaborate with data scientists, AI engineers, analytics teams and software developers to support data requirements.- Implement data security, governance, access control and documentation standards.- Monitor pipeline execution, troubleshoot failures and resolve data inconsistencies.- Contribute to continuous improvement of data engineering practices through automation and AI-assisted workflows.Mandatory Skills & Qualifications:- Strong experience in Data Engineering, Data Pipeline Development or Data Platform Engineering.- Proficiency in Python and advanced SQL.- Hands-on experience developing ETL/ELT pipelines and data transformation workflows.- Practical experience using AI-assisted development tools for coding, debugging, documentation or query optimization.- Understanding of AI-generated code validation and responsible use of AI coding assistants in production environments.- Experience with Apache Airflow or similar workflow orchestration tools.- Strong understanding of data modelling,
data warehousing and database concepts.- Experience working with relational and non-relational databases.- Knowledge of batch processing and real-time data processing concepts.- Experience integrating data through REST APIs and external data sources.- Understanding of data quality, data validation and pipeline monitoring.- Familiarity with Git, version control and CI/CD practices.- Knowledge of cloud-based data engineering environments.- Robust analytical, troubleshooting and problem-solving skills.Preferred Skills :- Experience with Snowflake, Databricks, Google BigQuery or Amazon Redshift.- Familiarity with Apache Spark, PySpark and distributed data processing.- Knowledge of dbt for data transformation and modelling.- Exposure to Kafka or similar streaming technologies.- Experience with AWS, Microsoft Azure or Google Cloud Platform.- Familiarity with AI coding assistants such as GitHub Copilot, Cursor, Claude Code or equivalent tools.- Understanding of Generative AI and LLM-based developer productivity solutions.- Exposure to Docker, Kubernetes and containerized data workloads.- Knowledge of data governance, metadata management and data observability tools.- Familiarity with data lake and lakehouse architectures.Key Skills :- Data Engineering, AI-Augmented Development, Python, SQL, ETL, ELT, Apache Airflow, Snowflake, dbt, Databricks, Apache Spark, PySpark, Data Pipelines, Data Modelling, Data Warehousing, Data Integration, REST APIs, Kafka, AWS, Azure, BigQuery, Git, CI/CD, GitHub Copilot, Cursor, Claude Code, Data Quality, Data Governance (ref:hirist.tech)
📌 AI Augmented Data Engineer - ETL Pipelines (Chennai)
🏢 Valam Solutions
📍 Chennai