Key Responsibilities :1. Data Pipeline Ownership :- Design, build, and maintain scalable ETL pipelines for tick, OHLC, fundamentals, and alternative data.- Develop automated ingestion systems for both real-time and historical market data.- Ensure fault-tolerant, high-availability data workflows.2. Data Quality & Integrity :- Detect and fix data anomalies: stale ticks, bad prints, missing data, timestamp issues.- Implement automated QA, validation checks, and reconciliation workflows.- Own daily data quality reporting.3. Data Architecture & Storage :- Manage storage layers.- Optimize data retrieval for quants and live trading systems.- Implement data versioning and point-in-time correctness for backtesting.4. Vendor & Exchange Data Management :- Integrate and manage feeds from Bloomberg, Refinitiv, FactSet, Tick data providers, NSE/BSE, global exchanges.- Be primary contact for vendor issue resolution.- Ensure compliance with vendor licensing and data usage policies.5.



Cross-Team Collaboration :- Work closely with quant researchers to understand data needs and create feature-ready datasets.- Support engineering teams in live feed integration, API access, and latency monitoring.- Work with product and compliance on data governance and documentation.6. Leadership & Mentorship :- Lead a small team of data engineers/analysts.- Review code, enforce best practices, and drive technical excellence.- Build roadmap for data capabilities and automation.Required Skills & Experience :- 6 - 10 years of experience in market data, data engineering.- Hands-on experience with :1. Python (mandatory)2. SQL and distributed databases3. ETL frameworks4. Message queues (Kafka/Pulsar a plus)- Robust debugging skills for live and historical data issues (ref:hirist.tech)

📌 Magna - Lead Data Engineer - SQL/Python (Maharashtra)
🏢 Magna International
📍 Maharashtra