Sr. Data Engineer (ETL, Databricks, PySpark)

📢 We’re Hiring🎯: Sr. Data Engineer (ETL, Databricks, PySpark)
🔍 Position: Sr. Data Engineer (ETL, Databricks, PySpark)
📍 Location: Canada Local (Remote)
🕒 Type: Full-Time

Job Description
🗝️ Key Responsibilities:
📌 Design, build, and maintain scalable ETL and data streaming pipelines (Databricks, PySpark), ensuring performance, reliability, and traceability across systems.
📌Resolve data production and pipeline issues, including troubleshooting legacy ETL (e.g., Informatica) with minimal disruption to live systems.
📌Implement and optimize data lake architecture from RAW to Curated
📌layers to support analytics, financial modeling, and predictive use cases.
📌Validate data transformations and business logic to ensure accuracy across cost and margin datasets.
📌Define and enforce data architecture standards, covering schema evolution, normalization, data governance, and performance.
📌Collaborate with cross-functional teams (e.g., finance, category management) to deliver trusted, actionable data insights.
📌Maintain documentation of data flows, architecture, and use cases using tools like JIRA and Confluence.
📌Manage source control and CI/CD pipelines using GitHub and GitHub Actions in a DevOps environment for automated integration and deployment.
📌Leverage AI development tools (e.g., GitHub Copilot, Databricks Assistant) to boost code quality and engineering efficiency.

📍 Required Skillset & Tool Knowledge
📌 Databricks, PySpark, Azure Data Factory, Azure Storage Account
📌Informatica Data Cloud Management
SQL, NoSQL, DB2, JDBC, MQ
📌Microsoft Excel (with automation is a plus), Notepad++
📌GitHub, GitHub Actions, Jenkins, QMetry
📌JIRA, Confluence, OneDrive (Cost-Item storage)

You can share your resume on madhuri.rane@techedoquest.com or DM me Madhuri Rane !!