Key Responsibilities:

Data Architecture & Solution Development

- Translate business and functional requirements into robust, scalable data solutions aligned with overall architecture.
- Design and implement data integrations and data quality frameworks.
- Participate in full development lifecycle including design, implementation, testing, documentation, delivery, support, and maintenance.

Pipeline & API Development

- Develop and maintain scalable data pipelines.
- Build new API integrations to support data flow and analytics requirements.
- Implement large-scale, high-volume, low-latency applications for efficient data processing.

Big Data & ETL Development

- Design, develop, and maintain ETL pipelines.
- Apply dimension modeling and data warehousing best practices.
- Write complex SQL queries for data transformation, extraction, and validation.

Streaming & Advanced Processing

- Build stream processing platforms using Kafka and Spark Streaming.




- Optimize data pipelines for performance and scalability.
- Independently deliver end-to-end projects with minimal supervision.

Required Skills & Experience:

- 5+ years of development experience in big data using Spark.
- Solid programming skills in Python.
- Expertise in SQL and complex query writing.
- Experience with dimension modeling, data warehousing, and ETL pipelines.
- Knowledge of streaming platforms like Kafka and Spark Streaming.

Nice to Have:

- Experience with job orchestration frameworks like Airflow, Oozie, or Luigi.
- Familiarity with AWS services such as S3, EMR, RDS.
- Understanding of cloud data warehouses like Snowflake.
- Knowledge of SQL distribution engines like Presto, Druid.
- Familiarity with streaming frameworks like Flink.
- Exposure to NoSQL databases such as HBase or Cassandra.

📌 Data Engineer 2 (Secunderabad)
🏢 Phenom People
📍 Secunderabad