Contract Duration : 3 months

Skills : - PySpark or Scala with Spark, Spark Architecture, Hadoop, SQL
- Streaming Technologies like Kafka etc.- Proficiency in Advanced SQL (Window functions)- Airflow, S3, and Stream Sets or similar ETL tools.- Basic Knowledge on AWS IAM, AWS EMR and Snowflake.

Responsibilities :
- Data Pipeline Development: Design, implement, and maintain scalable and productive data pipelines to collect, process, and store large volumes of structured and unstructured data.
- Data Modeling: Develop and maintain data models, schemas, and metadata to support the organization's data initiatives. Ensure data integrity and optimize data storage and retrieval processes.
- Data Integration: Integrate data from various sources, including databases, data warehouses, APIs, and streaming platforms, ensuring compatibility, consistency, and quality.
- Performance Optimization: Optimize data pipelines and processing systems for performance, scalability, and reliability. Identify and resolve bottlenecks and inefficiencies in data workflows.
- Data Quality Assurance:



Implement data quality checks, validation rules, and monitoring mechanisms to ensure the accuracy, completeness, and consistency of data across different systems.
- Infrastructure Management: Collaborate with DevOps and IT teams to provision, configure, and manage infrastructure components such as databases, clusters, and cloud services required for data processing and storage.
- Security and Compliance: Implement data security best practices and compliance standards to protect sensitive information and ensure regulatory compliance (e.g., GDPR, HIPAA). Monitor and audit data access and usage to prevent unauthorized activities.
- Documentation and Communication: Document data pipelines, systems architecture, and technical processes. Communicate effectively with cross-functional teams to gather requirements, provide updates, and address issues.
- Continuous Learning: Stay updated on emerging technologies

📌 Data Engineer (Noida)
🏢 Forward Eye Technologies
📍 Noida