Position: Lead Developer (Data Integration)

Job Description

Required Skill Set:

Must Have -
- Expertise and hands-on experience on Python/Node JS/Java
- Expertise knowledge ETL and Tools - SparkQL/Spark (PySpark) Dataframe / AWS Glue/ Snowflake
- Good knowledge of SQL, NoSQL (DynamoDB, Cassandra etc.)
- AWS Compute and Database (AWS Lambda, API Gateways SQS, Step Functions, RDS, S3)
- Able to write basic queries

Positive to Have
- Versioning tool – GIT
- Good knowledge of Agile Development
- Experience in developing data pipelines to transform, aggregate and or process data
- Experience in Data Lake, Databricks, Data Mart
- Understanding of Analytics and Machine Learning
- Experience in CI / CD principles (Jenkins/GitHub actions/any other equivalent)
- Experience in preparing design documents
- Awareness of design principles, design patterns, performance parameters like memory / CPU utilization, profiling, connection pooling, multi-threading

Responsibilities:
- Work closely with Architects, Project Manager, Business Analyst




- Working on Coding, Unit Testing, Test case development, Integration testing, Performance testing, Troubleshooting & Bug fixing
- Document use cases, flow chart, sequence diagrams, class diagrams
- Work with distributed Team following Agile methodologies
- Develop, build data engineering pipelines using Scala/PySpark in AWS using services like EKS, Lambda, S3, Glue, Step Functions
- Build ingestion as a service using the AWS Native and PySpark to facilitate the data gravity shift from on-premises to the cloud through migration of data assets and workloads to the cloud
- Build components in the data engineering following Test Data Driven development practices
- SQL skills for data profiling and analysis, and ability to reverse engineer existing databases with minimal assistance
- Support data requests/projects and helps to interpret data sets
- Perform ad-hoc data/process analysis as needed

📌 Lead Developer (Bengaluru)
🏢 Schneider Electric
📍 Bengaluru