Section 1This page
This chapter is the work still in progress. The steps below are the plan. When a notebook has been run, that section is replaced with the result from the export.
The local pipeline and the four Power BI pages are finished. Databricks already holds the RFI tables and two registered classifiers. What remains is one registration, then one notebook each for a short summary, a meaning search, and a copilot. Those notebooks read the Databricks tables. They do not rewrite the workbook, the local CSVs, or rfi.duckdb.
Section 2Already in place
| Piece | State |
|---|---|
| Clean register and DuckDB star schema | 107 RFIs, four dimensions, six SQL checks passed |
| Power BI report rif_vis | Four pages: executive, document impact, coordination, register |
bronze.rfi_raw | 112 rows, the workbook as loaded |
silver.rfi_clean and gold.rfi_analytics | 107 rows each |
digital_engineering.ml.rfi_discipline_classifier | Version 1, alias champion. Input is a GTE embedding of the question. Accuracy 0.70, macro F1 0.50 on 107 RFIs. |
digital_engineering.ml.rfi_issue_type_classifier | Version 1, alias champion. Input is a GTE embedding of the question plus the notes. Accuracy 0.72, macro F1 0.47 on 105 RFIs. |
The document-impact comparison is also finished. On the 70 RFIs whose notes say Yes or No, the GTE classifier reached accuracy 0.84 and Yes recall 0.68, with 5 false Yes calls. The zero-shot run caught every Yes and also called 39 of the 51 No rows Yes, so the GTE classifier is the one to keep. Its logged model id is m-9977de1f28c84f769c44b4a3c28aafcd.
Section 3Register document impact
databricks/20_register_document_impact_model.py is written and has not yet been confirmed in a run export. It registers digital_engineering.ml.rfi_document_impact_classifier in Unity Catalog and sets the alias champion. The model scores one embedding vector from databricks-gte-large-en, built from the question. It does not take raw text.
The run to do: import the notebook, run all cells, and keep the printed lines. This page will then record the version number, the champion alias, and the catalog path. Until those lines are back, the classifier stays logged in the rfi_document_impact experiment and is not yet a registered model in this report.
Section 4Summarise each question
One notebook, after the registration. It reads the question from gold.rfi_analytics and writes a one- or two-sentence summary back beside that row in Databricks, in a field named rfi_short_summary. The summary is for the register preview and for the copilot. The full question stays in the table.
The summary uses only the words in that RFI. It keeps the technical point, says when the note is unclear, and leaves out cost, schedule, and any fact the row does not contain. A check on RFI-002 should come back close to this: the contractor asks which battery energy storage capacity governs, because the solicitation and the drawings disagree.
When the export is back, this section will show how many of the 107 summaries were written, two or three examples, and any row the model left blank.
Section 5Search by meaning
One notebook. It embeds each question with databricks-gte-large-en, the same 1024-number vector used by the classifiers, and once the summaries exist it embeds those too. A search phrase is embedded the same way. The notebook ranks the 107 RFIs by closeness and returns the nearest ids. The ranking stays in the notebook. This workspace is not set up for a managed vector index.
The first searches to run, and the rows they should surface:
| Search phrase | Rows that should appear |
|---|---|
| battery capacity discrepancy | RFI-002, Electrical, Drawing Conflict, document impact Both |
| SCADA integration | RFI-005, Controls / SCADA, Interface Coordination |
| cybersecurity requirement | RFI-006, Cybersecurity, Compliance Requirement |
| special inspections | RFI-009, Quality / Inspection, Inspection / Quality |
When the export is back, this section will show the top ids for each phrase and whether those four RFIs were among them.
Section 6RFI copilot
One notebook, after the summary and the search exist. A question in ordinary language is answered from the 107 RFIs by calling a fixed set of tools:
| Tool | What it is allowed to do |
|---|---|
| SQL counts | Read gold.rfi_analytics for a count, a rate, or a list of ids |
| Meaning search | Return the nearest RFIs from the notebook ranking |
| Summary | Return rfi_short_summary for the ids already retrieved |
| Classifiers | Call the three champion models on an embedding, for discipline, issue type, and document impact |
The first questions to run are ones the tables already answer: how many Electrical RFIs there are (40), which RFIs are still open (the eight tasked to CT), and which RFIs change both a drawing and a specification (the 16 Both rows). A question about project profit, or about any fact outside these 107 rows, is answered as out of scope. The reply names the ids or the count it used.
When the export is back, this section will show each question, the tool that ran, and the answer checked against the gold table.
Section 7Check the answers
After the copilot notebook has a few answers, a short check list is scored by hand against the gold table. Each line has the question, the expected count or the expected RFI ids, and whether the question can be answered from this register. The scores to keep are: the count matches, the ids match, the wording stays inside the row, and an out-of-scope question is refused.
The same note records the model versions in use, that a person reviews the answer before it is treated as project direction, and that the classifiers and the summary support the team. They do not close an RFI or change a drawing.
Section 8How this page will change
The order stays as it is: registration, summary, search, copilot, then the check list. A finished step keeps its heading and replaces the plan paragraph with the printed result. A step that has not been run stays written as the work still to do.