AWS data Engineer with pyspark

long term Contract //Toronto ON//Canada . Please share resumes to Amarjeet.Kumar@akkodisgroup.com

Role:- AWS data Engineer with pyspark
Location-Toronto ON
Employment Type: Contract

Industry: Life Science / Pharma

Role: AWS Data Engineer

Roles & Responsibilities
Design, develop, and maintain scalable data pipelines for data ingestion and processing using Python, Spark, and AWS services.
Implement and manage ETL processes using AWS Glue / EMR for batch / streaming data.
Develop and maintain data storage solutions using Medallion Architecture in S3, Redshift
Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs.
Monitor and optimize data workflows using Airflow / other orchestration tools.
Ensure data quality and integrity throughout the data lifecycle.
Implement / understanding CI/CD practices for data pipeline deployment
Utilize monitoring and logging tools such as CloudWatch /Datadog to ensure system reliability and performance.
Communicate effectively with stakeholders to gather requirements and provide updates on project status.

Must Have Technical/Functional Skills
Proficient in Python for data processing and automation.
Experience with Spark for large-scale data processing.
Familiarity with AWS S3 for data storage and management.
Knowledge of Redshift for data warehousing solutions.
Experience with AWS Glue for ETL processes.
Familiarity with Airflow / other tool for workflow orchestration.
Experience with EMR for big data processing.
Understanding / experience in other AWS services such as Athena, SSM,Secret Manager,Step fucntion, AWS Batch
Strong SQL knowledge with experience in SQL based transformation using CTE/Stored procedure