Key Responsibilities

- Design, develop, and maintain scalable ETL/ELT data pipelines using PySpark and Databricks.
- Develop batch and streaming data processing solutions for structured and unstructured datasets.
- Build and optimize data workflows on cloud platforms.
- Implement Delta Lake architecture for productive data storage and processing.
- Integrate data from multiple sources including databases, APIs, cloud storage, and third-party systems.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Develop reusable data engineering frameworks and utilities.
- Work closely with Data Architects, Data Scientists, Business Analysts, and application teams.
- Ensure data quality, governance, and security standards are maintained.
- Troubleshoot production issues and provide timely resolutions.
- Participate in code reviews and follow CI/CD best practices.
- Create technical documentation for data pipelines and processes.

📌 Data Engineer (Hyderabad)
🏢 Lorven Technologies
📍 Hyderabad