Build a web-based Dataset Relationship and Compatibility Explorer for NYC Open Data using HTML, CSS, and JavaScript.
The tool should be designed for urban data scientists and researchers who often need to combine multiple NYC Open Data datasets but have difficulty understanding whether datasets are spatially, temporally, and structurally compatible.
Users should be able to select or search for an NYC Open Data dataset. The application should inspect its metadata and sample records and suggest other datasets that could potentially be analyzed together.
For each dataset, automatically generate a data profile showing:
number of records and variables
available geographic fields, such as latitude/longitude, ZIP code, borough, census tract, or community district
available temporal fields and the dataset's time coverage
missingness of important variables
common categorical values
potential identifiers or join keys
When two datasets are selected, provide a compatibility report showing:
whether their geographic coverage overlaps
whether their time periods overlap
geographic units they have in common
possible ways to join or spatially aggregate the datasets
differences in spatial or temporal granularity
missing-data issues that could affect the analysis
Include an interactive map that visualizes the spatial coverage of each dataset and a timeline showing their temporal coverage.
Add a “Prepare for Analysis” feature where users can choose a common geographic unit (such as ZIP code, census tract, community district, or borough) and time unit (day, month, or year). Preview how both datasets could be aggregated into a common analysis-ready structure.
Clearly warn users about potential methodological problems. For example, do not imply that two datasets can be directly joined simply because both contain geographic information. Highlight differences in geographic units, missing coordinates, temporal coverage, and aggregation levels.
Use NYC Open Data metadata and Socrata APIs dynamically rather than hard-coding dataset information.
The goal is not simply to visualize NYC Open Data, but to reduce the work urban data scientists must do before analysis: discovering related datasets, evaluating data quality, determining whether datasets can be combined, and identifying appropriate spatial and temporal units of analysis.
Create a functional prototype with a clean interface suitable for exploratory urban data analysis.