Key Responsibilities

- Design end-to-end data solutions, including data lakes, data warehouses, ETL/ELT pipelines, APIs, and analytics platforms.
- Architect scalable and low-latency data pipelines using tools like Apache Kafka, Flink, or Spark Streaming to handle high-velocity data streams.
- Design /Orchestrate end-to-end automation using orchestration frameworks such as Apache Airflow to manage complex workflows and dependencies.
- Design intelligent systems that can detect anomalies, trigger alerts, and automatically reroute or restart processes to maintain data integrity and availability.
- Define and implement data governance, metadata management, and data quality standards.
- Lead architectural reviews and technical design sessions to guide solution development.
- Partner with business and IT teams to translate business needs into data architecture requirements.
- Ensure security, compliance, and regulatory requirements are addressed in all data solutions.
- Evaluate and recommend improvements to existing data architecture and processes.

Technical Expertise





- Bachelors or masters degree in computer science, Information Systems, Data Science, or related field.
- Extensive experience with high level programming languages - Python, Java or Scala
- 5+ years of experience in data architecture, data engineering, or a related field.
- Proficient in data pipeline tools such as Apache Spark, Kafka, Airflow, or similar.
- Experience with data governance frameworks and tools (e.g., Collibra, Alation, OpenMetadata).
- Strong knowledge of cloud platforms (Azure or Google Cloud), especially with cloud-native data services.
- Experience working in Agile or DevOps environments.
- Experience with up-to-date data stack tools (e.g., dbt, Snowflake, Databricks).
- Experience with Hive, Oozie, Airflow, HBase, MapReduce, Spark along with working knowledge of Hadoop/Spark Toolsets.
- Extensive Experience working with Git and Process Automation
- In depth understanding of relational database management systems (RDBMS) and Data Flow Development.

📌 Data Engineer (Noida)
🏢 hackajob
📍 Noida