16 Nov
|
LPL Financials
|
Hyderabad
16 Nov
LPL Financials
Hyderabad
Responsibilities:
- Design, develop, and maintain distributed data pipelines on AWS that ingest, process, and deliver data at scale. Implement both batch and event-driven data processing patterns using AWS-native services.
- Build and support event-driven data solutions using asynchronous, decoupled architectures to enable near real-time processing and system scalability.
- Provide hands-on engineering using Python and AWS Glue with pyspark to process large datasets efficiently. Apply best practices in distributed systems, performance optimization, and fault tolerance.
- Ensure end-to-end data quality, implement validation and monitoring checks, establish data lineage, and manage orchestration workflows to ensure reliable and auditable data movement.
- Design and develop RESTful and event-driven APIs to expose data and data services to internal and external consumers.
- Build and deploy data services using Docker containers and manage workloads on Kubernetes following cloud-native and DevOps best practices.
- Collaborate with data consumers, architects, platform teams, and business stakeholders to gather requirements,
design scalable solutions, and deliver high-quality outcomes.
- Contribute to monitoring, logging, alerting, and incident response for data platforms. Ensure systems meet reliability, performance, and security standards.
- What are we looking for?
We are looking for strong data engineers who can deliver reliable, scalable, and high-quality data solutions. The ideal candidate thrives in a quick-paced environment, is passionate about data engineering, and is comfortable working across teams to solve complex data problems.
Requirements:
- B.E in Computer Science, Engineering, or equivalent practical experience
- 5+ years of experience in data engineering or distributed systems development
- Strong hands-on experience with AWS cloud services
- Advanced proficiency in Python
- Hands-on experience with Apache Spark and AWS AWS services such as Glue, Lambda, Step functions, Event Bridge, S3, Athena, RDS Aurora Postgresql
- Experience building event-driven data pipelines
📌 Data Engineer (Hyderabad)
🏢 LPL Financials
📍 Hyderabad