demo/after/)Scores are calculated across five weighted reliability domains totaling 100 points: (1) Input, Schema & Circuit-Breaker Validation (25 pts), (2) Spatial Reference (CRS) & Unit Integrity (20 pts), (3) Transaction, Cursor & Lock Safety (20 pts), (4) Error Handling, Logging & Run Receipts (20 pts), and (5) Configuration Hygiene & Offline Test Seams (15 pts). Each verified Critical finding deducts 12-18 domain points, High deducts 7-10 points, Medium deducts 4-6 points, and Low deducts 2-4 points (floored at 0 per domain).
| ID | Reliability Domain | Baseline | Post-Remediation | Findings | Scoring Rationale |
|---|---|---|---|---|---|
DOM-01 | Input, Schema & Circuit-Breaker Validation | 5 / 25 | 23 / 25 | F-01, F-06 | Baseline deducts 20 pts for unconditional production table wipe (F-01, -15 pts) and missing schema/null checks (F-06, -5 pts). Refactored package enforces 11 preflight checks and a 25% drop-ratio circuit breaker (+18 pts). |
DOM-02 | Spatial Reference (CRS) & Unit Integrity | 3 / 20 | 18 / 20 | F-02 | Baseline deducts 17 pts for setting WKID 3857 (meters) while dividing area by 43,560 sq ft/acre (F-02). Refactored package enforces explicit WKID 2229 + Foot_US preflight validation (+15 pts; remaining 2 pts reserved for Month 2 auto-reprojection). |
DOM-03 | Transaction, Cursor & Lock Safety | 7 / 20 | 19 / 20 | F-04 | Baseline deducts 13 pts for unmanaged UpdateCursor/SearchCursor objects and in-place target mutation (F-04). Refactored package isolates writes in Staging.gdb and uses context-managed cursors (+12 pts). |
DOM-04 | Error Handling, Logging & Run Receipts | 5 / 20 | 18 / 20 | F-03, F-07 | Baseline deducts 15 pts for six bare except: pass blocks and zero run receipts (F-03). Refactored package logs all checks, returns exit code 1 on blocked runs, and writes deterministic JSON run receipts (+13 pts). |
DOM-05 | Configuration Hygiene & Offline Test Seams | 4 / 15 | 14 / 15 | F-05, F-07 | Baseline deducts 11 pts for hardcoded C:\Users\bjenkins paths (F-05) and 0 unit tests (F-07). Refactored package externalizes config/workflow.example.json and adds 14 offline pytest tests (+10 pts). |
Operational Consequence: If the \\fileserver02\gis_drop share is offline or S_Fld_Haz_Ar_LATEST.shp contains 0 SFHA rows, L35/L48 swallow the copy/select error and L54 wipes LandRecords.DBO.Flood_Notifications_Active in production SDE, leaving web maps and counter lookups empty.
EV-01 (demo/before/nightly_parcel_flood_notification.py, L53-L54, confidence: HIGH): arcpy.DeleteFeatures_management(TARGET_PROD_FC) -> Script truncates the live production SDE feature class unconditionally before verifying whether fema_sfha exists or contains any features.EV-02 (demo/before/nightly_parcel_flood_notification.py, L32-L36, L40-L49, confidence: HIGH): try: arcpy.CopyFeatures_management(FEMA_FEED_SHP, local_fema) except: pass -> Upstream network copy and SFHA selection failures are swallowed by bare except: pass blocks prior to the L54 production table wipe.EV-08 (demo/after/sample_output/sample_run_receipt.json, L1-L133, confidence: HIGH): "status": "dry_run_passed", "production_target_modified": false, "flagged_for_notification": 4 -> Refactored workflow executes 11 preflight checks and 1 drop-ratio circuit-breaker check in dry-run mode without modifying the production target.Recommended Remediation (Phase 1 (Immediate Stop-Gap) & Phase 2 (Month 1 Retainer)): Enforce non-destructive preflight checks (validate_minimum_rows) before geoprocessing, write intersect candidates to an isolated Staging.gdb feature class, and block promotion via validate_candidate_drop_threshold() if candidate rows drop by > 25% compared to active production.
Status in Refactored Package (demo/after/): Resolved and verified in demo/after/src/parcel_flood_workflow/domain.py and runner.py (tested in test_empty_fema_feed_trips_circuit_breaker_and_prevents_delete and test_candidate_drop_ratio_circuit_breaker_blocks_partial_feed).
Operational Consequence: Setting arcpy.env.outputCoordinateSystem = arcpy.SpatialReference(3857) on L59 causes planar geometry area (SHAPE@AREA) to be computed in square meters while L89 divides by 43,560.0 (square feet per acre). Every parcel's flood acreage is under-reported by 10.7639x, causing properties between 0.25 and 2.68 actual flood acres to be omitted from statutory notices.
EV-03 (demo/before/nightly_parcel_flood_notification.py, L59, L88-L90, confidence: HIGH): arcpy.env.outputCoordinateSystem = arcpy.SpatialReference(3857) ... acres = raw_area / 43560.0 -> Global output CRS is set to Web Mercator (WKID 3857, meters), while L89 divides SHAPE@AREA by 43,560 (sq feet per acre), under-calculating flood acreage by 10.7639x.EV-08 (demo/after/sample_output/sample_run_receipt.json, L1-L133, confidence: HIGH): "status": "dry_run_passed", "production_target_modified": false, "flagged_for_notification": 4 -> Refactored workflow executes 11 preflight checks and 1 drop-ratio circuit-breaker check in dry-run mode without modifying the production target.Recommended Remediation (Phase 1 (Immediate Stop-Gap) & Phase 2 (Month 1 Retainer)): Remove the hardcoded WKID 3857 environment override and enforce validate_crs_compatibility() during preflight so both County_Parcels and FEMA_NFHL_Drop are verified as WKID 2229 (Foot_US) before dividing planar area by 43,560.0.
Status in Refactored Package (demo/after/): Resolved and verified in demo/after/src/parcel_flood_workflow/domain.py (tested in test_crs_mismatch_blocks_web_mercator_3857_and_wgs84_4326 and test_acreage_and_ve_zone_classification).
Operational Consequence: Network timeouts, schema mismatches, and SDE lock errors are discarded without logging a traceback, and the script exits with code 0 so Windows Task Scheduler reports a successful run.
EV-02 (demo/before/nightly_parcel_flood_notification.py, L32-L36, L40-L49, confidence: HIGH): try: arcpy.CopyFeatures_management(FEMA_FEED_SHP, local_fema) except: pass -> Upstream network copy and SFHA selection failures are swallowed by bare except: pass blocks prior to the L54 production table wipe.EV-04 (demo/before/nightly_parcel_flood_notification.py, L35, L48, L70, L76, L102, L131, confidence: HIGH): except: pass (6 occurrences across copy, filter, intersect, append, cursor, and share export) -> Six bare except: pass blocks suppress exceptions and allow the process to terminate with exit code 0.EV-09 (sales/intake.md (Synthetic Client Intake Record), Intake Q4-Q7, confidence: MEDIUM): Scheduled task runs Mondays 02:00 AM under DOMAIN\bjenkins; zero unit tests or staging dry-run flags exist. -> Operational schedule and lack of pre-deployment test harness confirmed by client intake submission.Recommended Remediation (Phase 2 (Month 1 Reliability Retainer)): Remove all bare except: pass blocks, replace print() calls with structured timestamped logging, and return exit code 1 (sys.exit(1)) whenever any preflight check, circuit breaker, or runtime exception occurs.
Status in Refactored Package (demo/after/): Resolved in demo/after/src/parcel_flood_workflow/runner.py and scripts/run_workflow.py.
Operational Consequence: Opening arcpy.da.UpdateCursor (L82) and SearchCursor (L112) without with context managers means any unhandled exception (such as a NULL OWNER_NAME on L122 before del search_cur on L125) leaves persistent .lock files or SDE locks.
EV-05 (demo/before/nightly_parcel_flood_notification.py, L82, L105, L112, L125, confidence: HIGH): upd_cur = arcpy.da.UpdateCursor(TARGET_PROD_FC, upd_fields) ... del upd_cur -> UpdateCursor and SearchCursor are opened without Python with context managers, risking stale FileGDB and SDE schema locks if an exception occurs before del.EV-07 (demo/before/nightly_parcel_flood_notification.py, L118-L123, confidence: HIGH): full_addr = srow[2] + ", " + srow[3] + " " + str(srow[4]); writer.writerow([srow[0], srow[1].upper(), ...]) -> SearchCursor export loop calls .upper() on OWNER_NAME and concatenates MAIL_ADDR/MAIL_CITY without null checks or try/except protection, crashing mid-export on NULL values.Recommended Remediation (Phase 2 (Month 1 Reliability Retainer)): Wrap all arcpy.da.SearchCursor and UpdateCursor calls in with context managers inside gis_adapter.py and clean up temporary feature layers in a finally block.
Status in Refactored Package (demo/after/): Resolved in demo/after/src/parcel_flood_workflow/gis_adapter.py (ArcPyGisAdapter).
Operational Consequence: If IT disables or cleans up the departed contractor's C:\Users\bjenkins profile folder on the server, workspace initialization and CSV exports fail immediately.
EV-06 (demo/before/nightly_parcel_flood_notification.py, L19-L26, L31, L39, L60, confidence: HIGH): arcpy.env.workspace = r"C:\Users\bjenkins\Desktop\Flood_Update_FINAL_v3\Scratch.gdb" -> Eight path assignments hardcode a departed contractor's C:\Users\bjenkins profile and UNC shares directly in source code.EV-09 (sales/intake.md (Synthetic Client Intake Record), Intake Q4-Q7, confidence: MEDIUM): Scheduled task runs Mondays 02:00 AM under DOMAIN\bjenkins; zero unit tests or staging dry-run flags exist. -> Operational schedule and lack of pre-deployment test harness confirmed by client intake submission.Recommended Remediation (Phase 2 (Month 1 Reliability Retainer)): Externalize all dataset paths, staging paths, WKID contracts, and row-count thresholds into a validated JSON configuration file (config/workflow.example.json loaded by config.py).
Status in Refactored Package (demo/after/): Resolved in demo/after/config/workflow.example.json and demo/after/src/parcel_flood_workflow/config.py.
Operational Consequence: In the L118 SearchCursor loop, calling srow[1].upper() or concatenating srow[2] + srow[3] raises an uncaught AttributeError or TypeError whenever a newly split parcel has NULL OWNER_NAME or MAIL_ADDR, aborting the CSV export midway.
EV-07 (demo/before/nightly_parcel_flood_notification.py, L118-L123, confidence: HIGH): full_addr = srow[2] + ", " + srow[3] + " " + str(srow[4]); writer.writerow([srow[0], srow[1].upper(), ...]) -> SearchCursor export loop calls .upper() on OWNER_NAME and concatenates MAIL_ADDR/MAIL_CITY without null checks or try/except protection, crashing mid-export on NULL values.EV-08 (demo/after/sample_output/sample_run_receipt.json, L1-L133, confidence: HIGH): "status": "dry_run_passed", "production_target_modified": false, "flagged_for_notification": 4 -> Refactored workflow executes 11 preflight checks and 1 drop-ratio circuit-breaker check in dry-run mode without modifying the production target.Recommended Remediation (Phase 2 (Month 1 Reliability Retainer)): Validate required field schemas upfront (validate_required_schema) and use null-safe formatting in classify_parcel_notification() that substitutes 'UNKNOWN OWNER (VERIFY ASSESSOR ROLL)' and records a structured warning in the run receipt.
Status in Refactored Package (demo/after/): Resolved and verified in demo/after/src/parcel_flood_workflow/domain.py (tested in test_missing_required_fields_detected and test_null_owner_name_graceful_fallback_with_warning).
Operational Consequence: Analysts cannot test a new FEMA NFHL feed or ArcGIS Pro patch without mutating the live Flood_Notifications_Active table in production SDE.
EV-08 (demo/after/sample_output/sample_run_receipt.json, L1-L133, confidence: HIGH): "status": "dry_run_passed", "production_target_modified": false, "flagged_for_notification": 4 -> Refactored workflow executes 11 preflight checks and 1 drop-ratio circuit-breaker check in dry-run mode without modifying the production target.EV-09 (sales/intake.md (Synthetic Client Intake Record), Intake Q4-Q7, confidence: MEDIUM): Scheduled task runs Mondays 02:00 AM under DOMAIN\bjenkins; zero unit tests or staging dry-run flags exist. -> Operational schedule and lack of pre-deployment test harness confirmed by client intake submission.Recommended Remediation (Phase 2 (Month 1 Reliability Retainer)): Decouple pure-Python domain logic (domain.py) from spatial I/O (gis_adapter.py), default the CLI runner to --dry-run, require --production --confirm-production CONFIRM_PRODUCTION_WRITE for live writes, and emit a deterministic JSON run receipt on every run.
Status in Refactored Package (demo/after/): Resolved and verified in demo/after/ with 14 passing offline pytest tests and sample_output/sample_run_receipt.json.
| Phase | Budget | Commercial Mapping | Findings | Deliverables | Verification Gate |
|---|---|---|---|---|---|
| Phase 1 Immediate Stop-Gap Containment (Pre-Cutover Safety Patch) | 0.50 hrs | Included in Month 1 Reliability Retainer Kickoff (0.50 hr hands-on block) | F-01, F-02 |
| Confirm SHAPE@AREA / 43560.0 matches known parcel acreage on 3 control parcels before next Monday 02:00 AM run. |
| Phase 2 Month 1 Structural Refactor, Dry-Run Harness & Offline Test Suite | 1.50 hrs | Covered by Month 1 Reliability Retainer ($249/mo introductory rate; 1.00 hr hands-on + 0.50 hr monthly inspection/report overhead = 2.00 hrs all-in Month 1 total) | F-01, F-02, F-03, F-04, F-05, F-06, F-07 |
| All 14 offline unit tests pass (`pytest demo/after -v -rs`) and client analyst approves staging `--dry-run` receipt (`status == dry_run_passed`). |
| Phase 3 Month 2 Proactive Hardening (Auto-Reprojection Gate & Receipt Diffs) | 1.50 hrs | Covered by Month 2 Reliability Retainer ($249/mo introductory rate; up to 1.50 hrs hands-on + 0.50 hr monthly check/report = 2.00 hrs all-in cap) | F-02, F-07 |
| Synthetic and staging dry-run tests verify automatic reprojection from WKID 4269 to WKID 2229 and accurate APN diff summary. |