Data Engineer (35+ years)

Responsibilities

- Build connectors: enterprise DBs, APIs, CSV/Excel, SFTP, ERP, GIS, SCADA, meter-data, IoT
- Develop batch and incremental ingestion pipelines; CDC and streaming pipelines
- Create reusable source-to-canonical mapping configurations
- Normalize inconsistent identifiers, timestamps, units, entity names
- Implement entity resolution, deduplication, master-data reconciliation
- Build pipelines loading data into PostgreSQL, Apache Jena/Fuseki, Apache AGE/Neo4j, OpenSearch, object storage
- Support disconnected/air-gapped ingestion from files and internal systems
- Build automated data-quality rules (schema conformity, missing values, duplicates, invalid IDs, unit inconsistencies, timestamp gaps, outliers, referential integrity, freshness, reconciliation)
- Implement quality scoring at dataset/record/attribute level
- Integrate validation with SHACL-based semantic validation
- Quarantine invalid data, produce validation reports
- Track lineage from source records to findings/model outputs
- Build graph-ready datasets and graph-loading pipelines

Core Skills

- Solid Python, advanced SQL/PostgreSQL, pandas/Polars, NumPy, Apache Arrow




- Apache Airflow, NiFi, Kafka/Redpanda, Debezium, Spark, Trino, Iceberg, Parquet/Avro
- MinIO/S3-compatible storage
- Schema evolution, data contracts
- Docker, Linux fundamentals, Git/CI/CD
- Graph data modelling basics (Cypher, Neo4j/AGE)

Deliverables

- Source-system profiling reports
- Data-ingestion and transformation pipelines
- Data-quality rules and scorecards
- Entity-resolution logic
- Graph-loading pipelines
- Data lineage documentation
- Offline/air-gapped data movement processes

Preferred domain experience Experience in one or more of the following would be valuable:

- Power and utilities
- Smart metering
- SCADA or industrial IoT
- Energy accounting
- Fraud or leakage detection
- Banking and financial-risk analytics
- Telecom network analytics
- Manufacturing and asset intelligence
- Government or critical-infrastructure systems Domain experience should be preferred, not mandatory. A strong data professional can learn utility terminology with help from the FDE and domain SME.

📌 Data Engineer (Telangana)
🏢 Nxtwave Disruptive Technologies (Hiring for a client)
📍 Telangana