21 Sep
|
MathCo India
|
India
21 Sep
MathCo India
India
Job Responsibilities • Responsible for designing, storing, processing, and maintaining of large scale data and related infrastructure. • Can drive multiple projects both from operational and technical standpoint. • Ideate and build PoV or PoC for new product that can help drive more business. • Responsible for defining, designing, and implementing data engineering best practices, strategies, and solutions. • Is an Architect who can guide the customers, team, and overall organization on tools, technologies, and best practices around data engineering. • Lead architecture discussions, align with business needs, security, and best practices. • Has robust conceptual understanding of Data Warehousing and ETL, Data Governance and Security, Cloud Computing, and Batch & Real Time data processing • Has strong execution knowledge of Data Modeling, Databases in general (SQL and NoSQL), software development lifecycle and practices, unit testing, functional programming, etc. • Understanding of Medallion architecture pattern • Has worked on at least one cloud platform. • Has worked as data architect and executed multiple end-end data engineering project. • Has extensive knowledge of different data architecture designs and data modelling concepts. • Manages conversation with the client stakeholders to understand the requirement and translate it into technical outcomes.
Required Tech Stack
• Strong proficiency in SQL
• Experience working on any of the three major cloud platforms i.e., AWS/Azure/GCP
• Working knowledge of an ETL and/or orchestration tools like IICS, Talend, Matillion, Airflow, Azure Data Factory, AWS Glue, GCP Composer, etc.
• Working knowledge of one or more OLTP databases (Postgres, MySQL, SQL Server, etc.)
• Working knowledge of one or more Data Warehouse like Snowflake, Redshift, Azure Synapse, Hive, Big Query, etc.
• Proficient in at least one programming language used in data engineering, such as Python (or Scala/Rust/Java)
• Has strong execution knowledge of Data Modeling (star schema, snowflake schema, fact vs dimension tables)
• Proficient in Spark and related applications like Databricks, GCP DataProc, AWS Glue, EMR, etc. • Has worked on Kafka and real-time streaming. • Has strong execution knowledge of data architecture design patterns (lambda vs kappa architecture, data harmonization, customer data platforms, etc.) • Has worked on code and SQL query optimization. • Strong knowledge of version control systems like Git to manage source code repositories and designing CI/CD pipelines for continuous delivery. • Has worked on data and networking security (RBAC, secret management, key vaults, vnets, subnets, certificates)
📌 Senior Data Engineer (India)
🏢 MathCo India
📍 India