LLMOps & AI Evaluation · Live online · 8 weeks · Mentor-led

LLMOps & AI Evaluation in Production.

An eight-week, part-time live program for engineers who have already built a prototype with large language models and now need to evaluate, observe, deploy and harden it. A natural follow-on if you have already shipped a RAG or agent prototype — including graduates of our GenAI program.

Design an evaluation set and a release bar, not just a demo notebook
Add traces, quality checks and basic cost and latency visibility
Practise deploying an LLM service with a simple API and a repeatable release path

Talk to a counsellor Download syllabus

Part-time, live online · 8 weeks · 8–10 hours a week · Fees and EMI options shared on a counsellor call.

Live online · Mentor-led

8 weeksDuration
8–10 hours a weekTime commitment
Part-time, live onlineFormat
Recordings if you miss a sessionAttendance

Designed with the US and the UK in mind; join live from anywhere.

Outcomes

What you will be able to do.

These are skills and work products from the program — not placement, salary or promotion promises.

01
Design a golden set, a regression check and a human-review sample for an LLM feature
02
Instrument traces and basic quality, latency and cost visibility
03
Version prompts and application code so a change can be reviewed before release
04
Evaluate retrieval quality on a RAG-style path, not only final answers
05
Expose the system behind a simple FastAPI service with a repeatable deploy path
06
Document failure modes, guardrails and a rollback plan for a capstone service

Who it's for

A good fit, and when it is not.

For engineers who already have an LLM or RAG prototype

This program is for

Software, data and ML engineers who need to take an LLM app beyond a prototype
People who have completed a GenAI or similar program and want a production follow-on
Engineers in the US, UK and other regions working on evaluation, observability or deployment

This program is not for

Beginners who have not yet built a first LLM or RAG project — start with a foundations program such as Applied Agentic AI or GenAI & Agentic AI Engineer
Leaders who want strategy without writing code — see AI Product & Strategy for Leaders

Starting point

Comfortable writing Python
You have built, or can bring, an LLM, RAG or agent prototype — a personal project is enough
Familiarity with Git and a local development workflow

Curriculum

Week by week, 8 weeks.

8–10 hours a week, live and mentor-led. Open a week for topics, labs and what you ship.

- What breaks when a notebook becomes a service: quality drift, latency, cost, prompt changes
- A reference architecture for an LLM application
- Choosing the prototype you will harden during the program
- Writing down current behaviour before you change it
LabRun your prototype against a first slice of representative tasks and record failures.
You shipA short baseline report of quality issues.
ToolsPython, Jupyter, Git

Projects

Work you can take with you.

Evaluation harness
A golden set, a scoring script and a short report on your starting prototype. This is the backbone of the later weeks.
Instrumented service
Traces plus a simple view of quality, latency and cost, with prompts and code under version control.
Capstone: a hardened LLM service
A FastAPI-wrapped service with an evaluation report, a release checklist, a safety note and a short runbook. Depth matters more than a new model demo.

Tools

What you will practise with.

These are tools used in the sessions. Product and company names are trademarks of their respective owners and are shown for identification only; no affiliation is implied.

Language & serving

Python
FastAPI
Docker
pytest

Evaluation & observability

LangSmith
Langfuse
OpenTelemetry

Delivery

GitHub
GitHub Actions

Format and schedule

Part-time, live, with regional batches.

Part-time and live online, in a small mentor-led cohort. Weekend-morning batches are offered for learners in the US. Evening batches (India time) are offered for learners in the Gulf and Africa. Europe-oriented groups typically meet after work and on Saturdays. Recordings are shared if you miss a live session. Exact days, times and the next cohort date are confirmed on a counsellor call.

Designed with the US and the UK in mind; join live from anywhere.

Fees and EMI options shared on a counsellor call. A Datamarcos certificate of completion is awarded when you finish the program work. Cohort start dates are shared on a counsellor call — we do not publish a date until then.

Lead mentor

Mentor

Deepesh Mishra

Founder, Datamarcos · Solution Architect · Corporate Trainer · BITS Pilani alumnus

Deepesh is the founder and lead mentor at Datamarcos. He has worked as a solution architect and corporate trainer in data and AI, and has trained teams at 200+ organizations. Live sessions on this program are mentor-led, with practical, project-based delivery.

LinkedIn profile

FAQ

Questions, answered.

Can't find yours? A counsellor will answer it on a short call.

Ask a counsellor

Do I need prior LLM experience?

Yes. You should already be comfortable in Python and have built, or be able to bring, an LLM, RAG or agent prototype. This is a production follow-on, not a first course in generative AI. Applied Agentic AI or GenAI & Agentic AI Engineer are better starting points if you are new to this work.

How long is the program?

Eight weeks, part-time and live online, at about 8–10 hours a week including live sessions, labs and your capstone work.

What will I build?

You will harden a prototype: an evaluation harness, tracing, a release checklist, retrieval and safety notes, and a FastAPI-wrapped capstone service with a short runbook.

Are the sessions live?

Yes. Sessions are live and mentor-led, with recordings if you miss a class. Weekend-morning batches are offered for learners in the US, with other regional times available. Exact times are confirmed on a counsellor call.

Which tools are covered?

Python, pytest, FastAPI, Docker, GitHub Actions, and an observability stack such as LangSmith or Langfuse, plus OpenTelemetry. You work on a prototype you bring or rebuild, rather than a single vendor platform.

What about fees?

Fees and EMI options shared on a counsellor call.

Is there a refund policy?

Yes. Datamarcos has a 7-day refund policy. If the program is not right for you, ask within 7 days of enrollment.

The full program guide

Read the 8 weeks curriculum.

Modules, projects, tools and format in one PDF. No prices are listed; fees are shared on a counsellor call.

Download PDF

Full program guide