02 Oct
|
spark eighteen
|
New Delhi
02 Oct
spark eighteen
New Delhi
We’re passionate about building seamless digital experiences powered by robust and intelligent data systems. We’re on the lookout for a Data Engineer - Senior to architect and maintain high-performance data platforms that fuel decision-making and innovation.
If you enjoy designing scalable pipelines, optimising data systems, and leading with technical excellence, you’ll thrive in our fast-paced, outcome-driven culture.
You’ll take ownership of building reliable, secure, and scalable data infrastructure - from streaming pipelines to data lakes. Working closely with engineers, analysts, and business teams, you’ll ensure that data is not just available, but meaningful and impactful across the organization.
Requirements
• 5+ years of experience in Data Engineering and production-scale data platforms.
• Design and manage Apache Airflow pipelines for high-volume sensor, satellite, and third-party data ingestion.
• The project requires experience with Object Storage (such as AWS S3 or MinIO), Kafka streaming, and familiarity with an Ubuntu workplace.
• Build and optimize Apache Spark workloads for batch processing, geospatial analytics, and large-scale aggregations.
• Develop and maintain PostgreSQL, TimescaleDB, PostGIS, and DuckDB-based data solutions.
• Implement efficient data ingestion, transformation, and bulk-loading pipelines.
• Containerize and deploy applications using Docker and Docker Compose, including air-gapped environments.
• Collaborate with customer teams to translate business requirements into data products and analytical solutions.
• Ensure data quality, lineage, observability, performance, and SLA compliance.
• Troubleshoot production issues and optimize pipeline, database, and query performance.
• Create and maintain technical documentation, data models, and operational run-books.
• Demonstrate strong proficiency in Python, SQL, Airflow, Spark, and PostgreSQL.
• Experience with geospatial and time-series data, S3-compatible storage, a
📌 Data Engineer (New Delhi)
🏢 spark eighteen
📍 New Delhi