Report ยท Chapter 15

Work still ahead

The remaining Databricks steps. This page is updated when each notebook has been run.

In progress Update as each step finishes 8 October 2026
An isometric view of site records moving through documents and a database into dashboards.

Section 1This page

This chapter is the work still in progress. The steps below are the plan. When a notebook has been run, that section is replaced with the result from the export.

The local pipeline and the four Power BI pages are finished. Databricks already holds the RFI tables and two registered classifiers. What remains is one registration, then one notebook each for a short summary, a meaning search, and a copilot. Those notebooks read the Databricks tables. They do not rewrite the workbook, the local CSVs, or rfi.duckdb.

Section 2Already in place

PieceState
Clean register and DuckDB star schema107 RFIs, four dimensions, six SQL checks passed
Power BI report rif_visFour pages: executive, document impact, coordination, register
bronze.rfi_raw112 rows, the workbook as loaded
silver.rfi_clean and gold.rfi_analytics107 rows each
digital_engineering.ml.rfi_discipline_classifierVersion 1, alias champion. Input is a GTE embedding of the question. Accuracy 0.70, macro F1 0.50 on 107 RFIs.
digital_engineering.ml.rfi_issue_type_classifierVersion 1, alias champion. Input is a GTE embedding of the question plus the notes. Accuracy 0.72, macro F1 0.47 on 105 RFIs.

The document-impact comparison is also finished. On the 70 RFIs whose notes say Yes or No, the GTE classifier reached accuracy 0.84 and Yes recall 0.68, with 5 false Yes calls. The zero-shot run caught every Yes and also called 39 of the 51 No rows Yes, so the GTE classifier is the one to keep. Its logged model id is m-9977de1f28c84f769c44b4a3c28aafcd.

Section 3Register document impact

databricks/20_register_document_impact_model.py is written and has not yet been confirmed in a run export. It registers digital_engineering.ml.rfi_document_impact_classifier in Unity Catalog and sets the alias champion. The model scores one embedding vector from databricks-gte-large-en, built from the question. It does not take raw text.

The run to do: import the notebook, run all cells, and keep the printed lines. This page will then record the version number, the champion alias, and the catalog path. Until those lines are back, the classifier stays logged in the rfi_document_impact experiment and is not yet a registered model in this report.

Section 4Summarise each question

One notebook, after the registration. It reads the question from gold.rfi_analytics and writes a one- or two-sentence summary back beside that row in Databricks, in a field named rfi_short_summary. The summary is for the register preview and for the copilot. The full question stays in the table.

The summary uses only the words in that RFI. It keeps the technical point, says when the note is unclear, and leaves out cost, schedule, and any fact the row does not contain. A check on RFI-002 should come back close to this: the contractor asks which battery energy storage capacity governs, because the solicitation and the drawings disagree.

When the export is back, this section will show how many of the 107 summaries were written, two or three examples, and any row the model left blank.

Section 6RFI copilot

One notebook, after the summary and the search exist. A question in ordinary language is answered from the 107 RFIs by calling a fixed set of tools:

ToolWhat it is allowed to do
SQL countsRead gold.rfi_analytics for a count, a rate, or a list of ids
Meaning searchReturn the nearest RFIs from the notebook ranking
SummaryReturn rfi_short_summary for the ids already retrieved
ClassifiersCall the three champion models on an embedding, for discipline, issue type, and document impact

The first questions to run are ones the tables already answer: how many Electrical RFIs there are (40), which RFIs are still open (the eight tasked to CT), and which RFIs change both a drawing and a specification (the 16 Both rows). A question about project profit, or about any fact outside these 107 rows, is answered as out of scope. The reply names the ids or the count it used.

When the export is back, this section will show each question, the tool that ran, and the answer checked against the gold table.

Section 7Check the answers

After the copilot notebook has a few answers, a short check list is scored by hand against the gold table. Each line has the question, the expected count or the expected RFI ids, and whether the question can be answered from this register. The scores to keep are: the count matches, the ids match, the wording stays inside the row, and an out-of-scope question is refused.

The same note records the model versions in use, that a person reviews the answer before it is treated as project direction, and that the classifiers and the summary support the team. They do not close an RFI or change a drawing.

Section 8How this page will change

The order stays as it is: registration, summary, search, copilot, then the check list. A finished step keeps its heading and replaces the plan paragraph with the printed result. A step that has not been run stays written as the work still to do.