Open Source Data Quality Tool

Purpose & User Intent: The tool provides a rule-based scoring engine to quantify data quality for open source datasets. It targets data engineers, data scientists, data governance professionals, and ML practitioners who must decide whether a dataset meets minimum reliability before ingestion, modeling, or public release. It enables objective measurement, reproducible comparisons across sources, and documentation suitable for audits or governance gates.
Core Logic & Features: The system computes a composite quality score from four core components: completeness, accuracy, consistency, and timeliness. Required inputs include dataset_name and total_rows; optional inputs enable configuration of weights, per-field profiling, sampling methods, and export options. Distinctive capabilities include transparent weighting, dimension-level breakdown, and reproducible scoring that remains independent of individual data values.
Inputs & Outputs: Inputs capture dataset identity, size, and quality indicators. Outputs provide overall_quality_score (0-100), a breakdown by dimension (each 0-100), and remediation recommendations. Outputs can be exported as JSON or CSV and integrated into data catalogs or governance dashboards. The system logs inputs and weights to support traceability.
Algorithms & Calculations: Default weights are completeness 0.30, accuracy 0.30, consistency 0.20, timeliness 0.20. Completeness = clamp(100 - missing_rate, 0, 100); Accuracy = clamp(100 - invalid_rate, 0, 100); Consistency = clamp(100 - inconsistent_rate, 0, 100). Timeliness may be supplied as timeliness_score or derived from last_update_date if available. If custom weights are provided, they must sum to 1; otherwise defaults apply. All outputs are bounded 0-100 with two-decimal precision.
Error & Edge Cases: Missing required inputs yields an error state with no score. Rates outside [0,100] are clipped. If total_rows is 0, completeness signals limitation and overall score reflects available metrics. Mis-specified weights default to the predefined distribution. External data availability does not affect internal calculations.
Industry/Region & Localization: Metrics use percent units (0–100) and counts for rows. Time-based metrics may reference ISO 8601 dates if provided. Default language is English; numeric formats follow locale-neutral representations unless configured otherwise.
Assumptions & Exclusions: Assumes inputs reflect observed indicators rather than content inspection. No data cleaning, value transformation, or semantic validation. The tool does not evaluate proprietary data sources beyond provided metrics and does not dictate UI or presentation layers.