Data Engineer, Python, AWS, Data Pipelines
Job Description:
- Design and maintain scalable ETL/ELT pipelines in Python, integrating structured and semi-structured data (JSON, CSV, XML) across Snowflake, MongoDB, Postgres, and AWS services (S3, Glue, Lambda, EC2, EMR, Redshift, RDS).
- Build and optimize transformation and orchestration workflows using DBT, Airflow, Prefect, or Dagster.
- Implement data governance, quality checks, and security best practices throughout data pipelines.
- Extend and optimize the Elastic Hierarchy framework to harmonize financial data from various systems.
- Collaborate with analysts, ML engineers, and product teams to deliver business-ready datasets and data solutions.
Requirements:
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
- Strong Python programming skills with experience in data processing and automation.
- Proven experience with the AWS data ecosystem, including S3, Glue, Lambda, EMR, EC2, Redshift, and RDS.
- Hands-on experience with Snowflake, MongoDB, and Postgres databases.
- Proficiency with DBT or similar data transformation tools, and with orchestration frameworks such as Airflow, Prefect, or Dagster.
- Knowledge of data mapping, attribution, or reconciliation (experience in financial services is a strong plus).
- Understanding of hybrid/on-premise deployment models within enterprise environments.
- Excellent English communication skills and ability to collaborate effectively within distributed teams.
Benefits: