BroncoBot

Analytics Dashboard

Query volume, search quality, and response-time visibility for the competition demo.

Back to Chat

Total Queries

57

Avg Response Time

15005 ms

Success Rate

96.5%

No-Answer Rate

0.0%

Helpful Rate

N/A

0/0 ratings

Retrieval Quality

Avg Top Search Score

0.82

Avg Results Returned

6.9

Search Modes Seen

BM25: 0

Hybrid: 49

Semantic: 0

Tool Usage

search_corpus49

Retrieval Evaluation — Blind A/B Test (10 queries)

We validated our search pipeline with objective metrics, not subjective judgment.

Metriclimit=8limit=15
Mean Top-1 Score0.73880.7472
Mean Reciprocal Rank (MRR)0.95000.8833
Queries with correct answer in top-39/108/10

Key finding: Expanding the semantic candidate pool from 24 to 45 (via match_count: limit * 3) pulled in tangentially related chunks that distorted hybrid score normalization. 6/10 queries returned different top-3 rankings. We chose limit=8 for higher MRR despite marginally lower top-1 scores — correct ranking matters more than marginal relevance gains.

Recent Queries

Latest analytics events captured from the chat route.

TimeQueryResponseSearchToolsSources
10/20/2026, 5:23:04 PM

what

Session: e01135f9-8ed8-4af0-8e55-e65b798eb364

Error

92 ms

Answer returned

Results: 0

Top score: N/A

Modes: N/A

N/AN/A
10/20/2026, 5:22:59 PM

hey

Session: e01135f9-8ed8-4af0-8e55-e65b798eb364

Error

89 ms

Answer returned

Results: 0

Top score: N/A

Modes: N/A

N/AN/A
4/21/2026, 3:48:43 AM

What time is it?

Session: 4e4fe715-9b6e-4f0f-8d65-b2756246cf63

Success

2346 ms

Answer returned

Results: 0

Top score: N/A

Modes: N/A

N/AN/A
4/21/2026, 3:48:18 AM

How many years has it been since Cal Poly Pomona founded?

Session: 4e4fe715-9b6e-4f0f-8d65-b2756246cf63

Success

3489 ms

Answer returned

Results: 0

Top score: N/A

Modes: N/A

N/AN/A
4/21/2026, 3:47:52 AM

When was Cal Poly Pomona founded?

Session: 4e4fe715-9b6e-4f0f-8d65-b2756246cf63

Success

11274 ms

Answer returned

Results: 8

Top score: 0.73

Modes: hybrid

search_corpus
4/21/2026, 3:47:11 AM

how many years since Dr Sandeep Krishnamurthy has been dean?

Session: 4e4fe715-9b6e-4f0f-8d65-b2756246cf63

Success

15219 ms

Answer returned

Results: 8

Top score: 0.83

Modes: hybrid

search_corpus
4/21/2026, 3:45:59 AM

What is the college of business deans email addressa

Session: 4e4fe715-9b6e-4f0f-8d65-b2756246cf63

Success

7467 ms

Answer returned

Results: 8

Top score: 0.83

Modes: hybrid

search_corpus
4/21/2026, 3:45:25 AM

If I want a minor in CS and major in CIS what would be the requirements?

Session: 4e4fe715-9b6e-4f0f-8d65-b2756246cf63

Success

23907 ms

Answer returned

Results: 8

Top score: 0.83

Modes: hybrid

search_corpus
4/21/2026, 3:43:37 AM

What are requirements to receive CIS degree?

Session: 4e4fe715-9b6e-4f0f-8d65-b2756246cf63

Success

33162 ms

Answer returned

Results: 8

Top score: 0.83

Modes: hybrid

search_corpus
4/19/2026, 6:43:27 PM

who is the director of the MSIS program?

Session: a40caabc-187b-45cf-b568-634278ce2da4

Success

7281 ms

Answer returned

Results: 8

Top score: 0.82

Modes: hybrid

search_corpus
4/19/2026, 6:41:47 PM

what are the main differences between CS and CIS departments?

Session: a40caabc-187b-45cf-b568-634278ce2da4

Success

19794 ms

Answer returned

Results: 8

Top score: 0.81

Modes: hybrid

search_corpus
4/19/2026, 6:40:51 PM

what are differentces between the msba and msis programs

Session: f502d8fb-e86f-460a-b1cc-247730ee3d47

Success

24280 ms

Answer returned

Results: 8

Top score: 0.85

Modes: hybrid

search_corpus
4/19/2026, 6:35:09 PM

Rod Smith in the economics department

Session: a40caabc-187b-45cf-b568-634278ce2da4

Success

8856 ms

Answer returned

Results: 8

Top score: 0.78

Modes: hybrid

search_corpus
4/19/2026, 6:34:37 PM

What are Dr. Smiths office hours?

Session: a40caabc-187b-45cf-b568-634278ce2da4

Success

5707 ms

Answer returned

Results: 0

Top score: N/A

Modes: N/A

N/AN/A
4/19/2026, 6:33:28 PM

What should i do after i apply?

Session: a40caabc-187b-45cf-b568-634278ce2da4

Success

20000 ms

Answer returned

Results: 8

Top score: 0.83

Modes: hybrid

search_corpus
4/19/2026, 6:32:38 PM

How do i apply for finacnial aid

Session: a40caabc-187b-45cf-b568-634278ce2da4

Success

15214 ms

Answer returned

Results: 8

Top score: 0.80

Modes: hybrid

search_corpus
4/19/2026, 6:25:35 PM

what is the office location of dr. mohammed husain

Session: f502d8fb-e86f-460a-b1cc-247730ee3d47

Success

7347 ms

Answer returned

Results: 8

Top score: 0.85

Modes: hybrid

search_corpus
4/19/2026, 6:25:05 PM

What are the office hours of Dr. Zhongming ma

Session: f502d8fb-e86f-460a-b1cc-247730ee3d47

Success

8093 ms

Answer returned

Results: 8

Top score: 0.84

Modes: hybrid

search_corpus
4/19/2026, 6:24:33 PM

What are the admission requirements for CSU LA?

Session: f502d8fb-e86f-460a-b1cc-247730ee3d47

Success

5026 ms

Answer returned

Results: 0

Top score: N/A

Modes: N/A

N/AN/A
4/19/2026, 6:23:52 PM

Does Cal Poly Pomona require SAT or ACT scores for admission?

Session: f502d8fb-e86f-460a-b1cc-247730ee3d47

Success

9339 ms

Answer returned

Results: 8

Top score: 0.82

Modes: hybrid

search_corpus