Key Responsibilities :

1.

Data Pipeline

Ownership :

- Design, build, and maintain scalable ETL pipelines for tick, OHLC, fundamentals, and alternative data.
- Develop automated ingestion systems for both real-time and historical market data.
- Ensure fault-tolerant, high-availability data workflows.

2. Data Quality &

- Integrity :
- Detect and fix data anomalies: stale ticks, bad prints, missing data, timestamp issues.
- Implement automated QA, validation checks, and reconciliation workflows.
- Own daily data quality reporting.

3. Data Architecture &

- Storage :
- Manage storage layers.
- Optimize data retrieval for quants and live trading systems.
- Implement data versioning and point-in-time correctness for backtesting.

4. Vendor &

- Exchange Data Management :
- Integrate and manage feeds from Bloomberg, Refinitiv, FactSet, Tick data providers, NSE/BSE, global exchanges.
- Be primary contact for vendor issue resolution.
- Ensure compliance with vendor licensing and data usage policies.





5. Cross-Team Collaboration :

- Work closely with quant researchers to understand data needs and create feature-ready datasets.
- Support engineering teams in live feed integration, API access, and latency monitoring.
- Work with product and compliance on data governance and documentation.

6. Leadership &

- Mentorship :
- Lead a small team of data engineers/analysts.
- Review code, enforce best practices, and drive technical excellence.
- Build roadmap for data capabilities and automation.

Required Skills &

Experience :

- 6 - 10 years of experience in market data, data engineering.
- Hands-on experience with :
- Python (mandatory)
- SQL and distributed databases
- ETL frameworks
- Message queues (Kafka/Pulsar a plus)
- Robust debugging skills for live and historical data issues

📌 Magna - Lead Data Engineer - SQL/Python (India)
🏢 Magna International
📍 India