Big Data Developer

Quarry Consulting
Quarry Consulting

Software Engineering

Posted 6+ months ago

Quarry Consulting is looking for a Data Developer with extensive experience in Python and data analytics.

Contract: 1 year

Location: Remote within Canada

Requirements:

  • Data warehousing experience with dimensional and data vault data models.
  • Proficient in SQL, PL/SQL, and Python
  • Hands-on experience creating data pipeline ETL jobs using AWS Glue with PySpark.
  • Creating and testing pipeline jobs locally using AWS glue interactive session.
  • Efficient in using PySpark Data frame API and Spark SQL.
  • Performance tuning of PySpark jobs.
  • Using AWS Athena to perform data analysis on Lake data populated into AWS glue data catalog through AWS glue crawlers.
  • Knowledge of AWS services e.g. DMS, S3, RDS, Redshift, Step Function.
  • ETL development experience with tools e.g. SAP BODS, Informatica.
  • Efficient in writing complex SQL queries to perform data analysis.
  • Good understanding of version control tools like Git, GitHub, TortoiseHg.