16 Nov
|
Intraedge Technologies
|
Pune
16 Nov
Intraedge Technologies
Pune
About the Role:We are looking for an experienced Senior Data Engineer with strong expertise in designing, developing, and optimizing enterprise-scale ETL/ELT pipelines and data integration solutions.Key Responsibilities:- Design, develop, and maintain scalable ETL/ELT pipelines for extracting, transforming, and loading data from multiple source systems.- Build robust data ingestion and integration frameworks to process structured and unstructured data.- Develop and optimize complex SQL queries, stored procedures, and data transformation logic.- Design and manage automated data workflows using Apache Airflow.- Implement batch and incremental data processing solutions to handle large volumes of enterprise data.- Develop and maintain data models, data warehouses, and data processing frameworks.- Monitor pipeline execution, troubleshoot failures, and optimize performance and resource utilization.- Implement data validation, reconciliation, and quality checks to ensure accuracy and consistency.- Collaborate with data architects, analysts, business stakeholders, and engineering teams to understand data requirements.- Establish best practices for data pipeline development, code quality, documentation, and deployment.- Support production data pipelines, resolve technical issues, and perform root cause analysis.- Provide technical guidance and mentorship to junior data engineers.Required Skills & Qualifications:- 10 - 12 years of experience in Data Engineering, ETL Development, or Data Integration.- Strong hands-on experience designing and developing enterprise-scale ETL/ELT pipelines.- Advanced proficiency in SQL,
including complex queries, joins, stored procedures, and query optimization.- Strong experience with Apache Airflow for workflow orchestration, scheduling, and pipeline monitoring.- Experience building data ingestion, transformation, and processing frameworks.- Strong understanding of data warehousing concepts, dimensional modelling, and database architecture.- Experience handling large datasets and complex data transformations.- Knowledge of batch processing, incremental data loading, and data integration patterns.- Experience with data quality, validation, reconciliation, and pipeline monitoring.- Familiarity with version control systems such as Git and software development best practices.- Strong analytical, troubleshooting, and problem-solving abilities.Good to Have:- Programming experience in Python or PySpark.- Exposure to cloud-based data platforms such as AWS, Azure, or GCP.- Familiarity with Snowflake, Databricks, Redshift, or similar data warehouse platforms.- Experience with DBT or other data transformation frameworks.- Knowledge of big data technologies such as Apache Spark and Hadoop.- Experience implementing CI/CD practices for data engineering workflows.Preferred Candidate Profile:- Solid technical ownership of end-to-end data engineering solutions.- Ability to work with cross-functional teams and translate business requirements into technical solutions.- Experience reviewing code, defining engineering standards, and mentoring team members.- Strong communication and stakeholder coordination skills. (ref:hirist.tech)
📌 Senior Data Engineer - ETL Pipelines (Pune)
🏢 Intraedge Technologies
📍 Pune