Role: Mid-level Big Data Engineer

Position Type: Full-Time Contract (40hrs/week)

Contract Duration: 6 months + extendable

Work Schedule: 8 hours/day (Mon-Fri)

Location: Hybrid - Wed/Thursday to be in onsite in Hyderabad, India

We are seeking a Mid-Level Big Data Engineer with 5+ years of experience developing scalable Big Data solutions and distributed data-processing applications in AWS environments.

The ideal candidate will have strong hands-on experience with Scala, Python, Apache Spark/PySpark, AWS, Linux, and shell scripting, along with experience building batch and streaming data pipelines and optimizing large-scale distributed workloads.

Key Responsibilities

- Design, develop, test, and deploy scalable Big Data solutions on AWS.
- Build and maintain batch and streaming data pipelines using Scala, Python, Spark, and PySpark.
- Process and transform large volumes of structured and unstructured data.
- Develop and support distributed data-processing and data-ingestion platforms.
- Integrate recent data sources and technologies into existing Big Data ecosystems.
- Optimize Spark, Hadoop, EMR, and distributed computing workloads.
- Build cloud-native APIs and microservices supporting data platforms.
- Automate data workflows, deployments, monitoring, and operational processes.
- Collaborate with architects, product owners, QA teams, and business stakeholders.
- Apply security, governance,



reliability, and software engineering best practices.

Requirements
- 5+ years of experience in Big Data development and data/software engineering.
- Strong hands-on experience with Scala and Python.
- Strong experience with Apache Spark / PySpark.
- Experience with the Hadoop ecosystem, including HDFS, MapReduce, and Hive.
- Experience with Kafka or similar streaming technologies.
- Strong hands-on AWS experience, particularly with EMR, S3, EC2, ECS/EKS, MWAA/Airflow, Step Functions, API Gateway, Lambda, DynamoDB, and/or RDS/Aurora.
- Experience building large-scale batch and API-based data-processing solutions.
- Strong Linux and shell scripting skills.
- Experience with data ingestion, transformation, data modeling, schema design, and data lifecycle management.
- Strong understanding of performance tuning for Spark, Hadoop, EMR, and distributed systems.
- Experience working in Agile/Scrum environments.
- Strong troubleshooting and problem-solving skills.

Preferred
- Generative AI / AI-assisted development
- LLMs, RAG, Vector Databases, and AI/ML integrations
- AI-enabled data platforms
- Git, GitHub, GitLab, Bitbucket, Jenkins, Maven, Gradle, Artifactory
- Docker / Kubernetes
- Automated testing and data quality tools
- Enterprise-scale cloud environments
- AWS or Databricks certifications

📌 Big Data Developer (Hyderabad)
🏢 Sky Systems, Inc. (SkySys)
📍 Hyderabad