Lecture 27: Build LLM Evaluation script to test Faithfulness

RAG-LLM Evaluation & Test Automation for Beginners

Section 1: Introduction to AI concepts - LLM's & RAG LLM's

Lecture 1: What this course offers? FAQ"s -Must Watch (9:12)
Lecture 2: Course outcome - Setting the stage of expectation
Lecture 3: Introduction to Artificial Intelligence and LLM's - How they work (6:17)
Lecture 4: Overview of popular LLM"s and Challenges with these general LLM's (6:15)
Lecture 5: What is Retrieval Augmented Generation (RAG)? Understand its Architecture (12:39)
Lecture 6: End to end flow in RAG Architecture and its key advantages (12:07)

Section2: Understand RAG (Retrieval Augmented Generation) - LLM Architecture with Usecase

Lecture 7: Misconceptions - Why RAG LLM's - cant we solve problem with traditional methods? (5:26)
Lecture 8: Optional - Overview how code looks in building RAG LLM's applications (6:51)

Section 3: Getting started with Practice LLM's and the approach to evaluate /Test

Lecture 9: Course resources download
Lecture 10: Demo of Practice RAG LLM's to evaluate and write test automation scripts (6:51)
Lecture 11: Understanding implementation part of practice RAG LLM's to understand context (8:36)
Lecture 12: Understand conversational LLM scenarios and how they are applied to RAG Arch (5:47)
Lecture 13: Understand the Metric benchmarks for Document Retrieval system in LLM (8:12)

Section 4: Setup Python & Pytest Environment with RAGAS LLM Evaluation Package Libraries

Lecture 14: Install and set the path of Python in windows OS (10:16)
Lecture 15: Install and set the path of Python in MAC OS (10:26)
Lecture 16: Install RAGAS Framework packages and setup the LLM Test project (9:35)
Lecture 17: Python & Pytest Basics - Where to find them in the tutorial?

Section 5: Programmatic solution to evaluate LLM Metrics with Langchain and RAGAS Libraries

Lecture 18: Making connection with OpenAI using Langchain Framework for RAGAS (15:49)
Lecture 19: End to end -Evaluate LLM for ContextPrecision metric with SingleTurn Test data (20:38)
Lecture 20: Metrics document download
Lecture 21: Communicate with LLM's using API Post call to dynamically get responses (9:51)
Lecture 22: Evaluate LLM for Context Recall Metric with RAGAS Pytest Test example (13:22)

Section 6: Optimize LLM Evaluation tests with Pytest Fixtures & Parameterization techniques

Lecture 23: Build Pytest fixtures to isolate OpenAI and LLM Wrapper common utils from test (7:56)
Lecture 24: Introduction to Pytest Parameterization fixtures to drive test data externally (10:13)
Lecture 25: Reusable utils to isolate API calls of LLM and have test only on Metric logic (13:19)

Section 7: Evaluate LLM Core Metrics and importance of EvalDataSet in RAGAS Framework

Lecture 26: Understand LLM's Faithfulness and Response relevance metrics conceptually (4:56)
Lecture 27: Build LLM Evaluation script to test Faithfulness benchmarks using RAGAS (9:42)
Lecture 28: Reading Test data from external json file to LLM evaluation scripts (9:58)
Lecture 29: Understand how Metrics are used at different places of RAG LLM Architecture (10:34)
Lecture 30: Factual Correctness - Build a single Test to evaluate multiple LLM metrics (12:02)

Section 8: Upload LLM Evaluation results & Test LLM for Multi Conversational Chat History

Lecture 31: Understand EvaluationDataSet and how it help in evaluating Multiple metrics (9:41)
Lecture 32: Upload the LLM Metrics evaluation results into RAGAS dashboard portal visually Lesson (8:22)
Lecture 33: How to evaluate RAG LLM with multi conversational history chat (7:59)
Lecture 34: Build LLM Evaluation Test which can evaluate multi conversation - example (17:42)

Section 9: Create Test Data dynamically to evaluate LLM & Generate Rubrics Evaluation Score

Lecture 35: How to Create Test Data using RAGAS Framework to evaluate LLM (15:02)
Lecture 36: Load the external docs into Langchain utils to analyze and extract test data (8:52)
Lecture 37: Install and configure NLTK package to scan the LLM documents & generating tests (20:11)
Lecture 38: Generate Rubrics based Criteria Scoring to evaluate the quality of LLM responses (11:46)

Section 10: Conclusion and next steps!

Section 11: Optional - Learn Python Fundamentals with examples

Lecture 41: Python hello world Program with Basics (8:35)
Lecture 42: Datatypes in python and how to get the Type at run time (5:17)
Lecture 43: List Datatype and its operations to manipulate (12:47)
Lecture 44: Tuple and Dictionary Data types in Python with examples (8:28)
Lecture 45: If else condition in python with working examples (3:10)
Lecture 46: How to Create Dictionaries at run time and add data into it (7:55)
Lecture 47: How loops work in Python and importance of code idendation (8:58)
Lecture 48: Programming examples using for loop - 1 (4:17)
Lecture 49: Programming examples using While loop - 2 (10:28)
Lecture 51: OOPS Principles : Classes and objects in Python (7:38)
Lecture 50: What are functions? How to use them in Python (10:46)
Lecture 52: What is Constructor and its role in Object oriented programming (13:38)
Lecture 53: Inheritance concepts with examples in Python (12:12)
Lecture 54: Strings and its functions in python (9:53)

Section 12: Optional - Overview of Pytest Framework basics with examples

Lecture 55: What are pytest fixtures and how it help in enhancing tests (10:29)
Lecture 56: Understand scopes in Pytest fixtures with examples (11:59)
Lecture 57: Setup and teardown setup using Python fixtures with yield keyword (9:04)

Lesson content locked

If you're already enrolled, you'll need to login.