Job highlights

- Requirements:Bachelors degree in Computer Science,Information Technology,or a related field . Masters degree is a plus .

- Experience with Spark SQL,Spark Streaming,and related Spark libraries .

Unify Technologies is looking for Sr SDE-Spark to join our dynamic team and embark on a rewarding career journey

- We are seeking a highly skilled and experienced Senior Software Development Engineer with expertise in Apache Spark to join our dynamic engineering team
- The ideal candidate will have a strong background in big data processing, distributed computing, and software development

- As a Senior SDE specializing in Apache Spark, you will play a key role in designing, implementing, and optimizing data processing workflows to support our organization's data-driven initiatives

Responsibilities

- Spark Application Development:Design, develop, and maintain Apache Spark applications for large-scale data processing

- Implement efficient data transformations, aggregations, and analytics using Spark RDDs and DataFrames

- Distributed Computing:Work on distributed computing frameworks and architectures to ensure scalability and performance of Spark applications

- Optimize Spark jobs for resource utilization and execution speed

- Data Processing Workflows:Design end-to-end data processing workflows, integrating Spark with other components of the data pipeline

- Collaborate with data engineers and data scientists to implement data processing requirements

- Optimization and Performance Tuning:Identify and address performance bottlenecks in Spark applications

- Implement optimizations to enhance the efficiency of data processing jobs





- Data Integration:Integrate Spark applications with various data sources and sinks, including databases, data lakes, and external APIs

- Ensure data quality and reliability in integration processes

- Monitoring and Debugging:Implement monitoring solutions to track the performance and health of Spark applications

- Debug and troubleshoot issues to ensure the reliability of data processing workflows

- Code Reviews and Collaboration:Conduct code reviews and provide constructive feedback to peers

- Collaborate with cross-functional teams, including data engineers, data scientists, and software developers

- Documentation:Maintain accurate and up-to-date documentation for Spark applications, workflows, and development processes

- Provide documentation for troubleshooting and knowledge transfer

- Requirements:Bachelor's degree in Computer Science, Information Technology, or a related field

- Master's degree is a plus

- Proven experience as a Software Development Engineer with a focus on Apache Spark

- In-depth knowledge of Spark architecture, internals, and best practices

- Proficiency in programming languages such as Scala or Python for Spark development

- Strong understanding of big data processing concepts and distributed computing

- Experience with Spark SQL, Spark Streaming, and related Spark libraries

- Familiarity with data pipeline orchestration tools (eg, Apache Airflow)

- Knowledge of data integration and ETL processes

- Robust problem-solving and analytical skills

- Excellent communication and collaboration skills

- Ability to work independently and contribute to a collaborative team environment

📌 Sr SDE-Spark (Hyderabad)
🏢 UNIFY TECHNOLOGIES
📍 Hyderabad