Agreement needs
more than one number.
A credible model result needs evidence about its equations, implementation, inputs and performance. A conservation check answers a different question from a comparison with a buoy. Both are useful, and neither replaces the other.
Verification
Does the implementation solve the intended mathematical problem sufficiently accurately? Use analytical solutions, conservation identities, limiting cases and convergence studies. Chapter 20 is an example.
Calibration
Estimate uncertain parameters using a designated dataset and a stated objective. Document the choices and plausible parameter range. A good fit to that dataset is not an independent performance test.
Validation
Evaluate performance against independent evidence for the intended application. Hold out complete events, locations or time periods where appropriate; nearby samples from one storm are not independent replications.
Diagnose four synthetic comparisons.
These ten reference values and their model counterparts are invented teaching data, not buoy observations. The second plot compares representative spectra normalized to the same m0 = 0.36 m²; it is a separate shape diagnostic, not a spectrum reconstructed from the ten height values.
Coral: synthetic reference. Blue: model example. Sample number has no assigned real date or sampling interval.
Both spectra have Hm0 = 2.4 m. Equal area does not require equal peak frequency, width or mean period.
None of these scores is a universal pass/fail threshold. Acceptable errors depend on the use, observed uncertainty and conditions being tested.
Metric definitions
ei = Mi − Oi; bias = mean(ei); RMSE = √mean(ei²)
Centered RMSE = √mean[(ei − bias)²]
Scatter index here = centered RMSE / mean(Oi)
Pearson correlation describes linear co-variation, not equality. A consistently scaled and biased model can have correlation one. Definitions of scatter index vary, so state the chosen convention. RMSE² = bias² + centered RMSE² for the same paired sample and population-mean normalization used here. These metrics retain their units except correlation and scatter index.
A ten-point demonstration cannot establish uncertainty bounds or rare-event skill. Correlated errors reduce the effective amount of information; resampling should respect storm or temporal dependence. A confidence interval derived as though every nearby sample were independent can be misleadingly narrow.
Compare like with like.
Before computing a score, align time stamps, locations, water levels, averaging intervals and quality-control flags. Confirm whether directions are from or toward, whether spectra use hertz or angular frequency, and whether reported heights are spectral Hm0 or wave-by-wave H1/3. Instrument response, sampling limits and processing choices can contribute to disagreement.
Use several diagnostics: time series, paired errors, spectral shapes, frequency and directional marginals, partitions and performance across sea-state ranges. Direction errors require circular differences, for example wrap(M − O) into [−180°,180°), rather than ordinary subtraction across north. A single mean direction can conceal two crossing systems.
Keep an evidence record.
Record the model version, bathymetry, forcing, boundary spectra, numerical settings, calibration dataset, independent evaluation dataset and acceptance criteria. Test sensitivity to plausible changes in uncertain inputs. Numerical error, observational uncertainty and physical model inadequacy should be discussed separately, even when their effects overlap.