BroncoBot

Analytics Dashboard

Query volume, search quality, and response-time visibility for the competition demo.

Back to Chat

Total Queries

Avg Response Time

16229 ms

Success Rate

100.0%

No-Answer Rate

0.0%

Helpful Rate

N/A

0/0 ratings

Retrieval Quality

Avg Top Search Score

0.81

Avg Results Returned

7.5

Search Modes Seen

BM25: 0

Hybrid: 32

Semantic: 0

Tool Usage

search_corpus32

Retrieval Evaluation — Blind A/B Test (10 queries)

We validated our search pipeline with objective metrics, not subjective judgment.

Metric	limit=8	limit=15
Mean Top-1 Score	0.7388	0.7472
Mean Reciprocal Rank (MRR)	0.9500	0.8833
Queries with correct answer in top-3	9/10	8/10

Key finding: Expanding the semantic candidate pool from 24 to 45 (via match_count: limit * 3) pulled in tangentially related chunks that distorted hybrid score normalization. 6/10 queries returned different top-3 rankings. We chose limit=8 for higher MRR despite marginally lower top-1 scores — correct ranking matters more than marginal relevance gains.

Recent Queries

Latest analytics events captured from the chat route.

Time	Query	Response	Search	Tools	Sources
4/19/2026, 5:34:56 PM	the economics department Session: 39e66246-9038-446e-b0f5-33ab98388937	Success 16647 ms Answer returned	Results: 8 Top score: 0.75 Modes: hybrid	search_corpus	https://www.cpp.edu/programs/class/economics/economics.shtml https://www.cpp.edu/programs/class/economics/general-economics.shtml
4/19/2026, 5:33:57 PM	What are Dr. Rod Smith’s office hours? Session: 39e66246-9038-446e-b0f5-33ab98388937	Success 17083 ms Answer returned	Results: 8 Top score: 0.77 Modes: hybrid	search_corpus	https://www.cpp.edu/cba/finance-real-estate-and-law/current-students/office-hours.shtml http://www.cpp.edu/class/history/faculty-staff/office-hours.shtml
4/19/2026, 4:43:31 PM	What courses does Dr. Hien Nguyen teach at Cal Poly Pomona? Session: 37fcac5f-66e5-4be1-8b65-93cde9d8d38b	Success 10777 ms Answer returned	Results: 8 Top score: 0.80 Modes: hybrid	search_corpus	https://www.cpp.edu/programs/class/economics/economics.shtml http://www.cpp.edu/class/economics/undergraduate-program/curriculum.shtml
4/19/2026, 4:42:47 PM	What are Dr. Hien Ngyue's office hours? Session: 37fcac5f-66e5-4be1-8b65-93cde9d8d38b	Success 10486 ms Answer returned	Results: 8 Top score: 0.80 Modes: hybrid	search_corpus	http://www.cpp.edu/class/economics/faculty/facultyandadmin.shtml https://www.cpp.edu/cba/finance-real-estate-and-law/current-students/office-hours.shtml
4/19/2026, 4:41:47 PM	What are Dr. Mark Generous' office hours? Session: 37fcac5f-66e5-4be1-8b65-93cde9d8d38b	Success 6931 ms Answer returned	Results: 8 Top score: 0.85 Modes: hybrid	search_corpus	http://www.cpp.edu/faculty/magenerous http://www.cpp.edu/cba/technology-and-operations-management/faculty-and-staff/office-hours.shtml
4/19/2026, 4:41:01 PM	What are Dr. Smith’s office hours? Session: 37fcac5f-66e5-4be1-8b65-93cde9d8d38b	Success 2840 ms Answer returned	Results: 0 Top score: N/A Modes: N/A	N/A	N/A
4/19/2026, 4:26:30 PM	How long does it take to receive my financial aid award letter? Session: 37fcac5f-66e5-4be1-8b65-93cde9d8d38b	Success 14101 ms Answer returned	Results: 8 Top score: 0.84 Modes: hybrid	search_corpus	https://www.cpp.edu/financial-aid/credentials/newly-admitted/award-notification.shtml http://www.cpp.edu/financial-aid/credentials/current/award-notification.shtml
4/19/2026, 4:23:44 PM	What should I do after I apply? Session: 37fcac5f-66e5-4be1-8b65-93cde9d8d38b	Success 21402 ms Answer returned	Results: 8 Top score: 0.85 Modes: hybrid	search_corpus	http://www.cpp.edu/financial-aid/resources/aid-offer-notification.shtml https://www.cpp.edu/financial-aid/credentials/newly-admitted/getting-your-aid.shtml
4/19/2026, 4:21:41 PM	How do I apply for financial aid at CPP? Session: 37fcac5f-66e5-4be1-8b65-93cde9d8d38b	Success 25643 ms Answer returned	Results: 8 Top score: 0.84 Modes: hybrid	search_corpus	https://www.cpp.edu/financial-aid/undergraduate/prospective/how-to-apply-for-aid-fafsa.shtml https://www.cpp.edu/financial-aid/credentials/newly-admitted/after-you-apply.shtml
4/19/2026, 4:16:27 PM	What’s CS 2400? Session: 37fcac5f-66e5-4be1-8b65-93cde9d8d38b	Success 15087 ms Answer returned	Results: 8 Top score: 0.78 Modes: hybrid	search_corpus	https://www.cpp.edu/~lrc/our-services/tutoring.shtml https://www.cpp.edu/lrc/our-services/tutoring.shtml
4/19/2026, 4:23:49 AM	What are the key differences between MSBA and MSIS programs? Session: 671a9fe3-d0e7-48bb-ad23-36aed565942c	Success 20088 ms Answer returned	Results: 8 Top score: 0.83 Modes: hybrid	search_corpus	https://www.cpp.edu/cba/graduate-business-programs/faqs.shtml https://www.cpp.edu/programs/cba/gp/masters-business-analytics-msba.shtml
4/19/2026, 4:22:08 AM	What is the office of Dr. Mohammad Husain? Session: 671a9fe3-d0e7-48bb-ad23-36aed565942c	Success 9658 ms Answer returned	Results: 8 Top score: 0.84 Modes: hybrid	search_corpus	https://www.cpp.edu/polysec/project/iap https://www.cpp.edu/polysec/project/cpsiot
4/19/2026, 4:13:11 AM	How to do a course substitution for MSIS students? Session: 671a9fe3-d0e7-48bb-ad23-36aed565942c	Success 27776 ms Answer returned	Results: 8 Top score: 0.81 Modes: hybrid	search_corpus	http://www.cpp.edu/cba/technology-and-operations-management/current-students/minor.shtml http://www.cpp.edu/class/psychology/advising/advising-faq.shtml
4/19/2026, 4:08:49 AM	What financial aids and assistance are available to MSIS students? Session: 4d106481-738c-4a5b-abda-9ea426fb2937	Success 15248 ms Answer returned	Results: 8 Top score: 0.83 Modes: hybrid	search_corpus	https://www.cpp.edu/cba/graduate-business-programs/costs/msis-costs.shtml https://www.cpp.edu/cba/graduate-business-programs/faqs.shtml
4/19/2026, 4:04:21 AM	What are the major core courses in the CIS department? Session: 4d106481-738c-4a5b-abda-9ea426fb2937	Success 12358 ms Answer returned	Results: 8 Top score: 0.77 Modes: hybrid	search_corpus	https://www.cpp.edu/cba/management-and-human-resources/curriculum https://www.cpp.edu/cba/accounting/curriculum
4/19/2026, 4:03:25 AM	How many career tracks or emphases does the CIS department have? Session: 4d106481-738c-4a5b-abda-9ea426fb2937	Success 11017 ms Answer returned	Results: 8 Top score: 0.86 Modes: hybrid	search_corpus	https://www.cpp.edu/cba/computer-information-systems/academics/undergraduate-program/cis-curriculum.shtml https://www.cpp.edu/cba/computer-information-systems/students/advising.shtml
4/19/2026, 3:56:56 AM	What titles Dr. Faheem Shaikh has? Session: 4d106481-738c-4a5b-abda-9ea426fb2937	Success 7183 ms Answer returned	Results: 8 Top score: 0.85 Modes: hybrid	search_corpus	https://www.cpp.edu/cba/faculty/msis-faculty.shtml https://www.cpp.edu/cba/graduate-business-programs/programs/msis-faculty.shtml
4/19/2026, 3:54:59 AM	What are the prerequisites for CIS 4290? Session: 4d106481-738c-4a5b-abda-9ea426fb2937	Success 6622 ms Answer returned	Results: 8 Top score: 0.78 Modes: hybrid	search_corpus	https://www.cpp.edu/faculty/dhwang https://www.cpp.edu/cba/information-assurance/academics/academics.shtml
4/19/2026, 3:54:04 AM	What is the title of CIS 4290? Session: 4d106481-738c-4a5b-abda-9ea426fb2937	Success 13183 ms Answer returned	Results: 8 Top score: 0.78 Modes: hybrid	search_corpus	https://www.cpp.edu/faculty/dhwang https://www.cpp.edu/cba/computer-information-systems/students/certificate.shtml
4/19/2026, 3:52:39 AM	Who is the director of MSIS program? Session: 4d106481-738c-4a5b-abda-9ea426fb2937	Success 14739 ms Answer returned	Results: 8 Top score: 0.80 Modes: hybrid	search_corpus	https://www.cpp.edu/cba/faculty/msis-faculty.shtml https://www.cpp.edu/cba/graduate-business-programs/programs/msis-faculty.shtml