BroncoBot
Analytics Dashboard
Query volume, search quality, and response-time visibility for the competition demo.
Total Queries
57
Avg Response Time
15005 ms
Success Rate
96.5%
No-Answer Rate
0.0%
Helpful Rate
N/A
0/0 ratings
Retrieval Quality
Avg Top Search Score
0.82
Avg Results Returned
6.9
Search Modes Seen
BM25: 0
Hybrid: 49
Semantic: 0
Tool Usage
Retrieval Evaluation — Blind A/B Test (10 queries)
We validated our search pipeline with objective metrics, not subjective judgment.
| Metric | limit=8 | limit=15 |
|---|---|---|
| Mean Top-1 Score | 0.7388 | 0.7472 |
| Mean Reciprocal Rank (MRR) | 0.9500 | 0.8833 |
| Queries with correct answer in top-3 | 9/10 | 8/10 |
Key finding: Expanding the semantic candidate pool from 24 to 45 (via match_count: limit * 3) pulled in tangentially related chunks that distorted hybrid score normalization. 6/10 queries returned different top-3 rankings. We chose limit=8 for higher MRR despite marginally lower top-1 scores — correct ranking matters more than marginal relevance gains.
Recent Queries
Latest analytics events captured from the chat route.
| Time | Query | Response | Search | Tools | Sources |
|---|---|---|---|---|---|
| 10/20/2026, 5:23:04 PM | what Session: e01135f9-8ed8-4af0-8e55-e65b798eb364 | Error 92 ms Answer returned | Results: 0 Top score: N/A Modes: N/A | N/A | N/A |
| 10/20/2026, 5:22:59 PM | hey Session: e01135f9-8ed8-4af0-8e55-e65b798eb364 | Error 89 ms Answer returned | Results: 0 Top score: N/A Modes: N/A | N/A | N/A |
| 4/21/2026, 3:48:43 AM | What time is it? Session: 4e4fe715-9b6e-4f0f-8d65-b2756246cf63 | Success 2346 ms Answer returned | Results: 0 Top score: N/A Modes: N/A | N/A | N/A |
| 4/21/2026, 3:48:18 AM | How many years has it been since Cal Poly Pomona founded? Session: 4e4fe715-9b6e-4f0f-8d65-b2756246cf63 | Success 3489 ms Answer returned | Results: 0 Top score: N/A Modes: N/A | N/A | N/A |
| 4/21/2026, 3:47:52 AM | When was Cal Poly Pomona founded? Session: 4e4fe715-9b6e-4f0f-8d65-b2756246cf63 | Success 11274 ms Answer returned | Results: 8 Top score: 0.73 Modes: hybrid | search_corpus | |
| 4/21/2026, 3:47:11 AM | how many years since Dr Sandeep Krishnamurthy has been dean? Session: 4e4fe715-9b6e-4f0f-8d65-b2756246cf63 | Success 15219 ms Answer returned | Results: 8 Top score: 0.83 Modes: hybrid | search_corpus | |
| 4/21/2026, 3:45:59 AM | What is the college of business deans email addressa Session: 4e4fe715-9b6e-4f0f-8d65-b2756246cf63 | Success 7467 ms Answer returned | Results: 8 Top score: 0.83 Modes: hybrid | search_corpus | |
| 4/21/2026, 3:45:25 AM | If I want a minor in CS and major in CIS what would be the requirements? Session: 4e4fe715-9b6e-4f0f-8d65-b2756246cf63 | Success 23907 ms Answer returned | Results: 8 Top score: 0.83 Modes: hybrid | search_corpus | |
| 4/21/2026, 3:43:37 AM | What are requirements to receive CIS degree? Session: 4e4fe715-9b6e-4f0f-8d65-b2756246cf63 | Success 33162 ms Answer returned | Results: 8 Top score: 0.83 Modes: hybrid | search_corpus | |
| 4/19/2026, 6:43:27 PM | who is the director of the MSIS program? Session: a40caabc-187b-45cf-b568-634278ce2da4 | Success 7281 ms Answer returned | Results: 8 Top score: 0.82 Modes: hybrid | search_corpus | |
| 4/19/2026, 6:41:47 PM | what are the main differences between CS and CIS departments? Session: a40caabc-187b-45cf-b568-634278ce2da4 | Success 19794 ms Answer returned | Results: 8 Top score: 0.81 Modes: hybrid | search_corpus | |
| 4/19/2026, 6:40:51 PM | what are differentces between the msba and msis programs Session: f502d8fb-e86f-460a-b1cc-247730ee3d47 | Success 24280 ms Answer returned | Results: 8 Top score: 0.85 Modes: hybrid | search_corpus | |
| 4/19/2026, 6:35:09 PM | Rod Smith in the economics department Session: a40caabc-187b-45cf-b568-634278ce2da4 | Success 8856 ms Answer returned | Results: 8 Top score: 0.78 Modes: hybrid | search_corpus | |
| 4/19/2026, 6:34:37 PM | What are Dr. Smiths office hours? Session: a40caabc-187b-45cf-b568-634278ce2da4 | Success 5707 ms Answer returned | Results: 0 Top score: N/A Modes: N/A | N/A | N/A |
| 4/19/2026, 6:33:28 PM | What should i do after i apply? Session: a40caabc-187b-45cf-b568-634278ce2da4 | Success 20000 ms Answer returned | Results: 8 Top score: 0.83 Modes: hybrid | search_corpus | |
| 4/19/2026, 6:32:38 PM | How do i apply for finacnial aid Session: a40caabc-187b-45cf-b568-634278ce2da4 | Success 15214 ms Answer returned | Results: 8 Top score: 0.80 Modes: hybrid | search_corpus | |
| 4/19/2026, 6:25:35 PM | what is the office location of dr. mohammed husain Session: f502d8fb-e86f-460a-b1cc-247730ee3d47 | Success 7347 ms Answer returned | Results: 8 Top score: 0.85 Modes: hybrid | search_corpus | |
| 4/19/2026, 6:25:05 PM | What are the office hours of Dr. Zhongming ma Session: f502d8fb-e86f-460a-b1cc-247730ee3d47 | Success 8093 ms Answer returned | Results: 8 Top score: 0.84 Modes: hybrid | search_corpus | |
| 4/19/2026, 6:24:33 PM | What are the admission requirements for CSU LA? Session: f502d8fb-e86f-460a-b1cc-247730ee3d47 | Success 5026 ms Answer returned | Results: 0 Top score: N/A Modes: N/A | N/A | N/A |
| 4/19/2026, 6:23:52 PM | Does Cal Poly Pomona require SAT or ACT scores for admission? Session: f502d8fb-e86f-460a-b1cc-247730ee3d47 | Success 9339 ms Answer returned | Results: 8 Top score: 0.82 Modes: hybrid | search_corpus |