28 Sep
|
AGAM Technologies Solutions Private
|
Coimbatore
28 Sep
AGAM Technologies Solutions Private
Coimbatore
n
nExperience: 6+ Years
nRole: Senior Data Engineer
nLocation: Coimbatore- Work from Office
nEmployment Type: Full-time
nAvailability: Immediate Joiner Preferred
n
nRole Summary
n
nWe are seeking an experienced Data Engineer with 6+ years of experience in designing, developing, and maintaining scalable data pipelines and data warehouse solutions. The ideal candidate should have solid expertise in ETL/ELT, Python, dbt, Snowflake, workflow orchestration, job scheduling, and Git.
n
nKey Responsibilities
n
n Design, develop, test, and maintain scalable ETL/ELT data pipelines.
nBuild and manage enterprise data warehouse solutions using Snowflake.
n Develop reusable data transformation models using dbt.
nWrite efficient and maintainable Python programs for data ingestion, transformation, validation, and automation.
n Extract and integrate data from databases, APIs, flat files, and other structured or semi-structured sources.
nDevelop dimensional data models using concepts such as:
n Star and snowflake schemas
nFact and dimension tables
n Slowly Changing Dimensions
nSurrogate keys
n Incremental and historical data loads
nCreate and maintain ETL jobs, schedules, dependencies, alerts, and recovery procedures.
n Work with orchestration and scheduling tools such as Apache Airflow, Control-M, Azure Data Factory, AWS Glue, or similar platforms.
nImplement data-quality checks, reconciliation processes, audit controls, and exception handling.
n Monitor pipeline performance and troubleshoot failed or delayed data-processing jobs.
nOptimize Snowflake queries, storage, warehouses, and overall pipeline performance.
n Use Git for source-code management, branching, merging, pull requests, code reviews,
and release management.
nSupport CI/CD processes for deploying data pipelines and dbt models across environments.
n Collaborate with business analysts, data analysts, reporting teams, architects, and other stakeholders to understand data requirements.
nPrepare and maintain technical documentation, data mappings, lineage, and operational support procedures.
n Participate in Agile ceremonies, sprint planning, estimation, peer reviews, and production-support activities.
n
nRequired Skills and Qualifications
n
n6+ years of professional experience in data engineering, ETL development, or data warehousing.
n Strong understanding of ETL/ELT processes and data warehousing concepts.
nHands-on experience with Python for data processing and automation.
n Strong hands-on experience with dbt, including:
nModels and materializations
n Sources and seeds
nTests and documentation
n Macros and Jinja
nIncremental models
n Snapshots
nHooks and packages
n Strong experience with Snowflake, including:
nDatabases, schemas, tables, and views
n Virtual warehouses
nStages and file formats
n Snowpipe
nStreams and Tasks
n Time Travel and zero-copy cloning
nQuery and warehouse performance optimization
n Role-based access control
nAdvanced SQL skills, including joins, CTEs, window functions, stored procedures, and query optimization.
n Experience creating, scheduling, monitoring, and troubleshooting ETL jobs.
nExperience with workflow orchestration tools such as Apache Airflow or an equivalent platform.
n Strong working knowledge of Git, including branching strategies, merge conflict resolution, pull requests, and code reviews.
nExperience working with structured and semi-structured data formats such as CSV, JSON, XML, Avro, and Parquet.
n Knowledge of data-quality frameworks, logging, monitoring, reconciliation, and error-handling practices.
nGood understanding of software development life cycle, Agile methodologies, and production-support processes.
n Strong analytical, troubleshooting, communication, and problem-solving skills.
n
nPreferred Qualifications
n
nExperience with cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform.
n Experience implementing CI/CD pipelines using GitHub Actions, GitLab CI, Jenkins, Azure DevOps, or similar tools.
nFamiliarity with BI and reporting tools such as Power BI, Tableau, or Looker.
n Understanding of data governance, metadata management, lineage, security, and access control.
nRelevant Snowflake, dbt, cloud, or data-engineering certifications would be an advantage.
n
nExpected Competencies
n
n Ability to independently design and deliver end-to-end data pipelines.
nStrong ownership of data accuracy, reliability, and performance.
n Ability to analyse complex data issues and provide sustainable solutions.
nEffective collaboration with technical and business stakeholders.
n* Ability to mentor junior engineers and contribute to engineering best practices.
📌 Senior Data Engineer (ETL Developer) (Coimbatore)
🏢 AGAM Technologies Solutions Private
📍 Coimbatore