The Big Data Engineer is responsible for development and automation of Big Data ingestion, transformation and consumption services; adopting new technology; and ensuring contemporary operations in order to deliver consumer driven Big Data solutions.
The role • Implement request for ingestion, creation, and preparation of data sources
• Develop and execute jobs to import data periodically/ (near) real-time from an external source
• Setup a streaming data source to ingest data into the platform
• Delivers data sourcing approach and data sets for analysis, with activities including data staging, ETL, data quality, and archiving
• Design a solution architecture to meet business, technical and user requirements
• Profile source data and validate fit-for-purpose
• Works with Delivery lead and Solution Architect to agree pragmatic means of data provision to support use cases




• Understands and documents end user usage models and requirements
Preferred skills
• Experience in data profiling, source-target mappings, ETL development, SQL optimisation, testing and implementation
• Expertise in streaming frameworks (Kafka/Spark Streaming/Storm) essential
• Experience in building Microservices, Rest APIs, Data as a Service architectures. • Experience managing structured and unstructured data types
• Experience in requirements engineering, solution architecture, design, and development / deployment
• Experience in creating big data or analytics IT solution
• Track record of implementing databases and data access middleware and high-volume batch and (near) real-time processing

📌 Big Data Developer (Pune)
🏢 Important Group
📍 Pune