AI/ML Engineer (Databricks)
Description
ABOUT DYSRUPIT
DysrupIT is a leading global-based cloud services company. We support clients across the globe in
their adoption of the cloud and their transformation to as-a-Service business models. DysrupIT is
dedicated to making a positive impact in the communities it serves.
COMPANY CULTURE
We are constantly looking for talented people to join our team. We offer a unique and
extraordinary career experience to motivate and inspire talents like you to make a difference in
our customers, clients, and the entire organization. At DysrupIT, you’ll join a team of dedicated
and brilliant professionals who share the same goal of pursuing integrity and excellence. We are
committed to promoting a culture where employees are valued and respected. We invest in our
employees and make sure that they will have the right training to secure their growth and
advancement towards their career aspirations.
JOB SUMMARY
We are seeking an AI/ML Engineer to deliver production-grade AI/ML solutions on Databricks for
enterprise clients. You will build robust pipelines, productionize machine learning and Generative
AI use cases, and help clients move from notebook experimentation to reliable, governed, at-scale
production systems. You will work closely with data scientists, ML scientists, and solution
architects, translating client requirements into reliable, governed, and scalable solutions.
This role is the right fit if you love hands-on AI/ML engineering, enjoy direct client collaboration in
a consulting environment, and stay current with the latest in Databricks tooling, MLOps, and
GenAI.
JOB RESPONSIBILITIES
- Build and productionize feature pipelines and training/serving pipelines for ML and GenAI
use cases, using MLflow for experiment tracking and model management. - Support the delivery of Generative AI and agentic systems on Databricks, including data
preparation for RAG pipelines, vector search integration, and AI/BI tooling such as
Databricks Genie Spaces. - Help stand up new AI use cases end to end on Databricks, from data preparation through
to a working prototype, working alongside data scientists to move ideas from notebook to
production. - Build reusable Databricks templates and accelerators for common AI/ML use cases (RAG,
agentic assistants, forecasting, personalisation), reducing time-to-development for new
client engagements. - Optimize Databricks jobs, clusters, and workflows for cost, performance, and reliability at
production scale. - Work directly with client engineering teams and stakeholders to gather requirements,
troubleshoot issues, and hand over documentation and runbooks.
REQUIRED TECHNICAL SKILLSET
- 3+ years of experience as an ML Engineer or in a similar role, with meaningful hands-on
time on the Databricks platform. - Strong Python and PySpark skills, with the ability to write production-quality Spark jobs,
not just notebooks. - Practical experience delivering production Generative AI or agentic systems, including RAG
architectures, LLM orchestration frameworks (e.g., LangChain, LangGraph), and AgentOps
tooling (evaluation, monitoring, governance). - Extensive hands-on industry data science and ML experience using common tools such as
pandas, scikit-learn, and PyTorch. - Background supporting large-scale consumer or enterprise ML systems (recommenders,
ranking, forecasting, personalisation). - Experience with MLflow or a comparable tool for experiment tracking, model registry, and
model serving. - Solid SQL skills and experience with cloud data platforms (Azure, AWS, or GCP); Azure
Databricks experience is a plus. - Comfortable working directly with clients or cross-functional stakeholders to scope and
deliver technical work.
PREFERRED EXPERIENCE
- Practical experience with Delta Lake and lakehouse/medallion architecture patterns.
- Working knowledge of Unity Catalog or equivalent data governance and access-control
tooling. - Prior consulting or professional services experience in AI/ML delivery.
- Familiarity with experimentation and A/B testing pipelines built on Databricks.
- Exposure to causal inference or applied statistics (e.g., uplift modelling, Double ML) to
support AI use case development.
CERTIFICATIONS PREFERRED
- Databricks Certified Machine Learning Associate or Professional
- Databricks Certified Generative AI Engineer Associate
WHAT WE OFFER
- Competitive compensation package commensurate with experience
- Government-mandated benefits plus supplemental Health Insurance
- Collaborative and professional work environment
- Career growth opportunities within a growing organization
- Remote work arrangement
Skills
- A/B Testing
- Agentic AI
- AI
- AWS
- Azure
- Causal Inference
- Cloud
- Data Governance
- Data Science
- Databricks
- Delta Lake
- GCP
- Generative AI
- Lakehouse
- LangChain
- LangGraph
- LLM
- Machine Learning
- Medallion Architecture
- MLflow
- MLOps
- pandas
- PySpark
- Python
- PyTorch
- RAG
- scikit-learn
- Spark
- SQL
- Statistics
- Unity
- Unity Catalog
- Vector Search