SOTI is committed to providing its employees with endless possibilities; learning recent things, working with the latest technologies and making a difference in the world.
Key Responsibilities
- Design, build, and maintain production grade ETL/ELT pipelines using SQL, Python, PySpark, and Spark SQL.
- Own end to end data pipelines from ingestion through curated, analytics ready datasets.
- Maintain and enhance existing integrations to ensure stability, performance, and scalability.
- Troubleshoot complex pipeline failures and act as a key escalation point for production support.
- Collaborate with Lead Engineers and Data Architects on on-prem to cloud migration initiatives.
- Assess legacy pipelines and integrations, contributing to migration sequencing, refactoring, and risk mitigation.
- Ensure continuity of business-critical data flows during phased migrations and platform transitions.
- Build and operate solutions across hybrid data platforms, including SQL Server, Azure Synapse, and Microsoft Fabric.
- Develop and maintain Fabric Lakehouses, Warehouses, Pipelines,
and Notebooks aligned with architectural standards.
- Optimize workloads for performance, reliability, and cost efficiency.
- Implement data validation, reconciliation, monitoring, logging, and alerting to ensure trusted data products.
- Apply disciplined engineering practices: version control, CI/CD, testing, and documentation.
- Improve the operational maturity and production readiness of data solutions.
- Mentor junior data engineers through code reviews, guidance, and best practice sharing.
- Promote consistent engineering standards and support team onboarding.
- Collaborate closely with BI teams, data owners, and governance stakeholders to ensure alignment with analytics needs.
- Identify and drive improvements in pipeline efficiency, migration effectiveness, and developer productivity.
Required Qualifications
- 7+ years of experience in Data Engineering, Analytics Engineering, or a related techni
📌 Senior Data Engineer (Kochi)
🏢 Soti
📍 Kochi