Job Responsibilities -

- Architect and implement a scalable, offline Data Lake for structured, semi-structured, and unstructured data in an on-premises, air-gapped environment.
- Collaborate with Data Engineers, Factory IT, and Edge Device teams to enable seamless data ingestion and retrieval across the platform.
- Integrate with upstream systems like MES, SCADA, and process tools to capture high-frequency manufacturing data efficiently.
- Monitor and maintain system health, including compute resources, storage arrays, disk I/O, memory usage, and network throughput.
- Optimize Data Lake performance via partitioning, deduplication, compression (Parquet/ORC), and implementing effective indexing strategies.
- Select, integrate, and maintain tools like Apache Hadoop, Spark, Hive, HBase, and custom ETL pipelines suitable for offline deployment.
- Build custom ETL workflows for bulk and incremental data ingestion using Python, Spark, and shell scripting.
- Implement data governance policies covering access control, retention periods, and archival procedures with security and compliance in mind.
- Establish and test backup, failover, and disaster recovery protocols specifically designed for offline environments.
- Document architecture designs, optimization routines, job schedules, and standard operating procedures (SOPs) for platform maintenance.




- Conduct root cause analysis for hardware failures, system outages, or data integrity issues.
- Drive system scalability planning for multi-fab or multi-site future expansions.

Essential Attributes (Tech-Stacks) -

- Hands-on experience designing and maintaining offline or air-gapped Data Lake environments.
- Deep understanding of Hadoop ecosystem tools: HDFS, Hive, Map-Reduce, HBase, YARN, zookeeper and Spark.
- Expertise in custom ETL design, large-scale batch and stream data ingestion.
- Robust scripting and automation capabilities using Bash and Python.
- Familiarity with data compression formats (ORC, Parquet) and ingestion frameworks (e.g., Flume).
- Working knowledge of message queues such as Kafka or RabbitMQ, with focus on integration logic.
- Proven experience in system performance tuning, storage efficiency, and resource optimization.

Qualifications -

- BE/ ME in Computer science, Machine Learning, Electronics Engineering, Applied mathematics, Statistics.

Desired Experience Level -

- 4 Years relevant experience post Bachelors
- 2 Years relevant experience post Masters
- Experience with semiconductor industry is a plus

Skills: Yarn, Zookeeper, Data Lake, Hbase, Python, Hadoop

Experience: 1.00-4.00 Years

📌 Platform Engineer (Rajkot)
🏢 Tata Electronics (TEPL)
📍 Rajkot