Essential Responsibilities:
Building, refining, tuning, and maintaining our real-time and batch data infrastructure Daily use technologies such as Python, Spark, Airflow, Snowflake, Hive, Scylla, Django, FastAPI, etc
Maintaining data quality and accuracy across production data systems
Working with Data Engineers to optimize data models and workflows
Working with Data Analysts to develop ETL processes for analysis and reporting
Working with Product Managers to design and build data products
Working with our DevOps team to scale and optimize our data infrastructure
Participate in architecture discussions, influence the road map, take ownership and responsibility over recent projects
Participating in on-call rotation in their respective time zones (be available by phone or email in case something goes wrong)

Desired Characteristics:
Minimum 5 years of software engineering experience.
Proven long term experience and enthusiasm for distributed data processing at scale, eagerness to learn recent things.




Expertise in designing and architecting distributed low latency and scalable solutions in either cloud and onpremises setting.
Exposure to the whole software development lifecycle from inception to production and monitoring
Fluency in Python or solid experience in Scala, Java
Proficient with relational databases and Advanced SQL
Expert in usage of services like Spark and Hive
Experience with web frameworks such as Flask, Djang
Experience in adequate usage of any scheduler such as Apache Airflow, Apache Luigi, Chronos etc
Experience in adequate usage of cloud services (AWS) at scale
Experience in agile software development processes
Excellent interpersonal and communication skills Nice to have:
Experience with large scale / multi-tenant distributed systems
Experience with columnar / NoSQL databases - Vertica, Snowflake, HBase, Scylla, Couchbase
Experience in real team streaming frameworks - Flink, Storm
Experience in open table formats such as Iceberg, Hudi or Deltalake

📌 Lead Software Engineer Bengaluru
🏢 Zeta Global
📍 Bengaluru