24 Aug
|
Indus Net Technologies
|
Kolkata
24 Aug
Indus Net Technologies
Kolkata
Job Summary
Data Engineer - Azure / AWS / Databricks / Microsoft Fabric. About the Role: We are looking for a Data Engineer with 2-3 years of experience to join our Data Analytics team and contribute to the development of modern, cloud-based data platforms and analytics solutions. The role involves designing and developing scalable data pipelines, data lake/lakehouse solutions, data transformations and curated data layers that support Business Intelligence, Advanced Analytics, AI/ML and Decision Intelligence use cases. The engineer will work closely with Data Architects, Business Analysts, BI Developers, Data Scientists and Project Managers to convert business requirements into reliable and production-ready data solutions.
Key Responsibilities
- Design, develop and maintain scalable ETL/ELT data pipelines.
- Build data ingestion and transformation pipelines across databases, APIs, enterprise applications and file-based sources.
- Develop data processing solutions using SQL, Python and PySpark/Spark.
- Work with cloud data platforms, Data Lakes and Lakehouse architectures.
- Develop and maintain Bronze, Silver and Gold/curated data layers.
- Implement data transformations, cleansing, validation, reconciliation and data quality checks.
- Work with Databricks, Microsoft Fabric and/or Azure data services.
- Develop productive and reusable data engineering frameworks and components.
- Optimize SQL queries, Spark jobs and data pipelines for performance and cost.
- Implement appropriate error handling, logging, monitoring and pipeline recovery mechanisms.
- Support data modelling for analytical and reporting use cases.
- Collaborate with BI teams to provide trusted and analytics-ready datasets.
- Support Data Scientists and AI teams with appropriately prepared datasets and feature-ready data.
- Follow engineering practices around Git, version control, CI/CD, testing and deployment.
- Contribute to technical documentation, data lineage and operational runbooks.
- Troublesho
📌 Data Engineer (Kolkata)
🏢 Indus Net Technologies
📍 Kolkata