Case studies.

A public research university with a strong institutional research office.

NJIT is one of the most diverse public research universities in the country, with an active IR practice that produces a Common Data Set, a comprehensive Factbook, and an internal "Cookbook" of methodology notes that lives in equal parts on disk and in the heads of the analysts who wrote it.

The IR team is small, careful, and exactly the kind of team that should not be losing time to information retrieval.

Fragmented documentation. Inconsistent definitions. Real turnover risk.

Knowledge in IR offices accretes over time: a Factbook published every spring, methodology notes added during the year, policy changes documented in scattered PDFs. The result is a corpus that is comprehensive but not navigable.

Analysts were spending non-trivial fractions of their week answering questions whose answers existed already, somewhere, and reconciling small definitional inconsistencies between departments. The risk concentrated on two or three senior analysts who held the working memory in their heads.

IRIS, deployed inside NJIT's AWS environment.

We ingested the working institutional corpus: the Common Data Set; the most recent Factbook; the Cookbook; published policies; methodology notes accumulated across the past four reporting cycles. IRIS is grounded only on this set.

The interface is a single conversational surface, accessible to authorized IR and academic-affairs staff via single sign-on. Every answer cites the specific page of the specific document it came from. Out-of-scope questions are refused, by design.

Three weeks from kickoff to production.

Week 1: document ingestion and knowledge-base setup in single-tenant AWS resources, with SSO connected to NJIT's identity provider. Week 2: tuning against a private test set assembled with the OIE team; evaluation against faithfulness, citation accuracy, and refusal behavior. Week 3: pilot to a small group inside OIE, daily check-ins, then a quiet broader launch on the Friday.

All models run via AWS Bedrock under enterprise terms that prohibit using customer data for training. No NJIT document left the AWS network.

Time recovered. Definitions consistent. Methodology preserved.

It is early to publish specific numbers, and we won't until NJIT signs off on a metric publicly. Patterns observed in production use include:

• — A Materially less time spent on retrieval, particularly for the questions that come in from non-IR offices ahead of board cycles.
• — B Greater consistency in how definitions are quoted across the institution, with the source document attached to the answer.
• — C A working corpus of institutional methodology that the next generation of analysts can interrogate directly.

What worked, what surprised us, what we'd do differently.

What worked. Starting narrow. IRIS was scoped to IR only, with a published list of documents in-scope and a published list of categories out-of-scope. Trust was earned by being unambiguous about what IRIS would and wouldn't answer.

What surprised us. The most useful questions were not the ones we expected. Analysts use IRIS less to find numbers and more to find the methodology behind a number they already have. The most-asked question is some form of "why is it computed that way?"

What we'd do differently. Bring the eval set in earlier. By Week 1 we should have had thirty real questions in hand. Next deployment, we will.