AutomatedReasoningPolicyFidelityReport¶
Structure Class¶
dataclass
¶
A comprehensive analysis report that measures how accurately a generated policy represents the source documents. The report includes coverage and accuracy scores, detailed grounding information linking policy elements to source statements, and annotated document content.
Attributes¶
instance-attribute
¶
accuracy_score: float
A score from 0.0 to 1.0 indicating how accurate the policy rules are relative to the source documents. A higher score means the policy rules more faithfully represent the source material.
instance-attribute
¶
coverage_score: float
A score from 0.0 to 1.0 indicating how well the policy covers the statements in the source documents. A higher score means more of the source content is represented in the policy.
instance-attribute
¶
document_sources: list[AutomatedReasoningPolicyReportSourceDocument]
A list of source documents with their content broken down into atomic statements and annotated with line numbers for precise referencing.
instance-attribute
¶
rule_reports: dict[str, AutomatedReasoningPolicyRuleReport]
A mapping from rule identifiers to detailed fidelity reports for each rule, showing which source statements ground each rule and how accurate it is.
instance-attribute
¶
variable_reports: dict[str, AutomatedReasoningPolicyVariableReport]
A mapping from variable names to detailed fidelity reports for each variable, showing which source statements ground each variable and how accurate it is.