Roles and Responsibilities-

● Design and develop scalable data pipelines and workflows for data ingestion, transformation, and integration.

● Build and maintain data storage systems, including data warehouses, data lakes, and

relational databases.

● Ensure data accuracy, integrity, and consistency through validation and quality

assurance processes.

● Collaborate with data scientists, analysts, and business teams to understand data

needs and deliver tailored solutions.

● Optimize database performance and manage large-scale datasets for efficient

processing.

● Leverage cloud platforms (AWS, Azure, or GCP) and big data technologies

(Hadoop, Spark, Kafka) for building robust data solutions.

● Automate and monitor data workflows using orchestration frameworks such as

Apache Airflow.

● Implement and enforce data governance policies to ensure compliance and data

security.

● Troubleshoot and resolve data-related issues to maintain seamless operations.

● Stay updated on emerging tools, technologies, and trends in data engineering.

Skills and Knowledge-

1. Core Skills:





● Proficient in Python (libraries: Pandas, NumPy) and SQL.

● Knowledge of data modeling techniques, including:

β—‹ Entity-Relationship (ER) Diagrams

β—‹ Dimensional Modeling

β—‹ Data Normalization

● Familiarity with ETL processes and tools like:

β—‹ Azure Data Factory (ADF)

β—‹ SSIS (SQL Server Integration Services)

2. Cloud Expertise:

● AWS Services: Glue, Redshift, Lambda, EKS, RDS, Athena

● Azure Services: Databricks, Key Vault, ADLS Gen2, ADF, Azure SQL

● Snowflake

3. Big Data and Workflow Automation:

● Hands-on experience with big data technologies like Hadoop, Spark, and Kafka.

● Experience with workflow automation tools like Apache Airflow (or similar).

Qualifications and Requirements-

● Education: Bachelor’s degree (or equivalent) in Computer Science, Information

Technology, Engineering, or a related field.

● Experience:

β—‹ Freshers with robust understanding, internships and

📌 Data Engineer(Experience 2-5 year) (Indore)
🏢 ZecData Technology
📍 Indore