16 Nov
|
Paisabazaar
|
Gurugram
16 Nov
Paisabazaar
Gurugram
Job Details Role: Data Engineer
Location:
Gurgaon
- Sector 44 (Work from Office)
Experience:
2-5 Years Technical Skills
• Hands-on experience with big data technologies such as Apache Spark, Hadoop, Hive and YARN.
• Strong programming skills in Python/PySpark and proficiency in SQL, including complex queries and performance optimization.
• Experience with relational and NoSQL databases such as MySQL, PostgreSQL, HBase, MongoDB and Cassandra.
• Experience in designing, developing and maintaining scalable batch and real-time data ingestion pipelines using Apache Kafka, NiFi and Debezium.
• Experience with workflow orchestration and scheduling tools such as Apache Airflow.
• Experience with Azure cloud services, particularly Azure Data Lake Storage (ADLS Gen2) and Azure Blob Storage.
• Understanding of distributed data processing, partitioning, data formats (ORC, Parquet, Avro) and performance tuning.
• Experience with Spark Streaming, Change Data Capture (CDC) and incremental data processing.
• Understanding of data quality, data governance, access management and security.
• Proficiency in Linux, shell scripting, Git and debugging distributed applications. Good-to-Have Technical Skills
• Exposure to up-to-date data lakehouse architectures and technologies such as Apache Iceberg, Delta Lake and Trino.
• Experience with Databricks, Microsoft Fabric or Google BigQuery.
• Exposure to monitoring and observability tools such as Grafana and Prometheus.
• Understanding of data modelling, data warehouse concepts and data marts.
• Familiarity with Docker, Kubernetes and CI/CD practices.
📌 Data Engineer (Gurugram)
🏢 Paisabazaar
📍 Gurugram