16 Nov
|
Paisabazaar
|
Gurugram
16 Nov
Paisabazaar
Gurugram
Job Details
Role: Data Engineer
Location: Gurgaon - Sector 44 (Work from Office)
Experience: 2-5 Years
Technical Skills
- Hands-on experience with big data technologies such as Apache Spark, Hadoop, Hive and YARN.
- Strong programming skills in Python/PySpark and proficiency in SQL, including complex queries and performance optimization.
- Experience with relational and NoSQL databases such as MySQL, PostgreSQL, HBase, MongoDB and Cassandra.
- Experience in designing, developing and maintaining scalable batch and real-time data ingestion pipelines using Apache Kafka, NiFi and Debezium.
- Experience with workflow orchestration and scheduling tools such as Apache Airflow.
- Experience with Azure cloud services, particularly Azure Data Lake Storage (ADLS Gen2) and Azure Blob Storage.
- Understanding of distributed data processing, partitioning, data formats (ORC, Parquet, Avro) and performance tuning.
- Experience with Spark Streaming, Change Data Capture (CDC) and incremental data processing.
- Understanding of data quality, data governance, access management and security.
- Proficiency in Linux, shell scripting, Git and debugging distributed applications.
Good-to-Have Technical Skills
- Exposure to up-to-date data lakehouse architectures and technologies such as Apache Iceberg, Delta Lake and Trino.
- Experience with Databricks, Microsoft Fabric or Google BigQuery.
- Exposure to monitoring and observability tools such as Grafana and Prometheus.
- Understanding of data modelling, data warehouse concepts and data marts.
- Familiarity with Docker, Kubernetes and CI/CD practices.
📌 Data Engineer (Gurugram)
🏢 Paisabazaar
📍 Gurugram