Job Description
Lead Python Data Engineer
Mississauga Canada (Hybrid)
Fulltime
Note:- looking for candidates those who are presently in Canada
Must Have :
10+ years of hands-on experience in a Data Engineering role, with a strong proficiency in Python (version 3.6+).
Extensive experience working with relational databases and writing complex SQL queries.
Proven expertise with SQLAlchemy or similar ORM libraries.
Experience with workflow management tools like Airflow (experience with PySpark or PyFlink is a major plus).
Solid understanding of data warehousing concepts and experience working with large datasets.
Ability to guide and mentor junior developers, fostering a collaborative team environment.
Strong communication skills, both written and verbal, with the ability to explain complex technical concepts to both technical and non-technical audiences.
Experience deploying and managing applications on cloud platforms like Openshift, ECS, or Kubernetes.
Design, architect, and implement robust and scalable data pipelines using Python and related technologies (Airflow, PySpark, PyFlink are a plus).
Champion best practices for data engineering, code quality, testing, and deployment.
Mentor and guide a team of talented data engineers, fostering a collaborative and high-performing team culture.
Collaborate closely with Data Scientists, Data Analysts, and business stakeholders to translate complex business requirements into efficient data solutions.
Continuously research and implement new technologies and best practices to improve the efficiency and scalability of our data platform.
Take ownership of the deployment and monitoring of data pipelines and related infrastructure on cloud platforms such as Openshift, ECS, or Kubernetes.
It would be great if you also had:
Experience with data visualization tools and techniques.
Familiarity with agile development methodologies.
Contributions to open-source projects or active participation in the data engineering community.
Banking/ Financial experience is preferred