Senior Data Engineer

W3Global is hiring for “Senior Data Engineer” – Mississauga, ON. Let me know if you are interested. Feel free to reach me @manish.jupaka@w3global.com

Role – Senior Data Engineer
Location – Mississauga, ON, CA

Full Description
Mandatory Skills: Data Lake Storage, Azure Data Factory, RDD/DataFrame/SQL
Onsite- Mississauga, ON

Required Skills & Qualifications:

Bachelor’s or Master’s degree in Computer Science or related field.
6+ years of hands-on experience in software engineering/data engineering roles.
Strong experience with Databricks platform for big data analytics.
Proficient in Scala programming language; solid understanding of Spark framework (RDD/DataFrame/SQL).
Hands-on experience working on Azure cloud services (e.g., Data Lake Storage, Azure Data Factory).
Strong background in Database design & development; excellent SQL skills (T-SQL/PLSQL etc.).
Understanding of CI/CD pipelines for automated deployments on cloud platforms.
Experience working within Agile/Scrum methodologies.
• Advanced knowledge of Python for scripting, automation, and AI/ML development.
• Familiarity with Langchain for integrating and orchestrating large language model workflows.
• Hands-on experience with Azure OpenAI to leverage generative AI capabilities in applications.
• Understanding of Vector Databases for efficient storage and retrieval of high-dimensional data embeddings, essential for semantic search and recommendation systems.
• Knowledge of RAG (Retrieval-Augmented Generation) techniques to combine information retrieval with generative AI models for enhanced application intelligence.