USGS -- SMIG --
Surface-water quality and flow Modeling Interest Group

Comparison of Nine Uncalibrated Runoff Models to Observed Flows in Two Small Urban Watersheds

by Phillip J. Zarriello

USGS, Water Resources Division
903 Hanshaw Road
Ithaca, NY 14850
Internet: pzarriel@usgs.gov
Phone: (607) 266-0217 x3014
FAX: (607) 266-0521

Editor's note:
This paper was written for the First Federal Interagency Hydrologic Modeling Conference, held in Las Vegas on April 19-23, 1998. This paper is also available in Postscript format (gzipped, 210k).

Citation:
Zarriello, P.J., 1998, Comparison of nine uncalibrated runoff models to observed flows in two small urban watersheds, in Proceedings of the First Federal Interagency Hydrologic Modeling Conference, April 19-23, 1998, Las Vegas, NV: Subcommittee on Hydrology of the Interagency Advisory Committee on Water Data, p. 7-163 to 7-170.

Contents

Abstract

Nine uncalibrated runoff model results -- CASC2D, CUHP, CUHP/SWMM, DR3M, HEC-1, HSPF, PSRM, SWMM, and TR20 were compared to observed flows in two small urban watersheds with distinctly different climatic and physiographic settings; a 3.10 mi2 semiarid moderately-sloped watershed near Denver, CO, and a 0.14 mi2 coastal watershed with steep slopes near Seattle, WA. All models were run by experienced modelers using identical data provided on selected basin characteristics and rainfall that are readily available to the engineering community. Observed streamflow was not provided, and thus the results are dependent on modelers' judgment and the ability to conceptually and quantitatively represent the hydrologic system. Data for six storms in each watershed (antecedent conditions, rainfall volume, intensity, and duration) were provided. Simulated peak flows differed from observed peak flows by as much as 260 percent and simulated storm volumes differed from observed storm volumes by as much as 240 percent. The average root mean square model error (RMS) was slightly larger for peak flows in the coastal watershed (68) than in the semi-arid watershed (55), whereas the RMS for storm volume was about the same in the two watersheds (54 and 56, respectively). In general, the models based on the SCS curve number had the poorest fit. Results indicate simulated flows from uncalibrated models have wide variability for both types of watersheds; this could result in over- or underdesign of stormwater-management structures.

Introduction

Planning and design of stormwater-drainage systems, culverts, detention basins, and other stormwater facilities requires information on storm peak flows and runoff volumes. Rainfall-runoff models, often utilized to estimate this information, vary in complexity, functionality, and applicability to a given region or storm type, but only through a process of calibration and verification can reliable results be acquired. Uncalibrated models are often used, however, because either the information needed for calibration and verification are unavailable, or the expense of these procedures can not be justified. Little information is available on the reliability and error associated with the use of an uncalibrated model.

To determine the variability of uncalibrated flow simulations across a range of model types, a simple comparison of nine selected, nonproprietary, runoff models were made for six storms in each of two small urban watersheds with distinctly different climatic and physiographic settings; Harvard Gulch, a semi-arid watershed near Denver, CO, and Surrey Downs, a coastal watershed in the Pacific Northwest near Seattle, WA. Each of the models were run by experienced modelers using only data provided on selected basin characteristics, antecedent storm conditions, and storm rainfall that are typically available to the engineering community. Observed streamflow were not provided and the models were run without being calibrated. Thus, model results are dependent on modelers' judgment and the ability to conceptually and quantitatively represent the hydrologic system. This paper describes the results of the uncalibrated models.

Acknowledgments

Numerous people provided their time and expertise to perform model simulations -- Dr. Gret Aron, Penn State Univ. (PSRM); Darryl Davis, Arlen Feldman, Gary Brunner, Dr. David Goldman, and Stephen Breithaupt, U.S. Army Corps of Engineers (HEC-1); Richard Dinicola, U.S. Geological Survey (HSPF); Dr. Pierre Julien, Scott Hogan, Aaron Egbert, James White, and James Light, Colorado State Univ. (CASC2D); Tania McNutt, Colorado State Univ. (CUHP); Michael Schmidt, Camp, Dresser and McKee (SWMM). The ASCE Urban Water Resources Research Council, particularly Ben Urbonas, provided the initial idea and support for this study.

Rainfall-Runoff Model Characteristics

Runoff models differ mainly in the methods used to generate runoff and to route it through a basin; they also differ in the control options available, data handling, and user interface, but these differences generally have little or no effect on how the model computes runoff. The test models (summarized in table 1) calculate runoff (excess precipitation) by one of the following; (1) SCS curve number, (2) Horton's equation, or (3) continuous soil moisture accounting. The SCS curve number is the most widely used method because of its relative simplicity; it defines the watershed storage and is determined for a watershed or sub-watershed predominantly from the types of soils, vegetative cover, and land-use characteristics (Soil Conservation Service, 1986). Horton's equation assumes that the soil infiltration rate decreases exponentially as a function of time since the storm began. Some models account for soil-moisture storage and infiltration using either the Green-Ampt or Phillips equation, or a variation thereof. The PSRM model uses the SCS curve number for determining soil infiltration, but uses soil moisture accounting to determine available storage. These models are either continuous or quasi-continuous (soil-moisture accounting is continuous, but routing is only performed only for a specified storm period), but continuous meteorologic data was not made available for this test, thus modelers were required to estimate initial starting conditions for each storm. Soil moisture accounting and infiltration procedures generally are more data-intensive than the SCS curve and Horton methods, and require a number of parameters corresponding to physical soil-water storage and infiltration characteristics.

(table 1)

Once excess precipitation is determined, surface runoff is calculated for overland flow and channel flow by one of the following methods; (1) unit hydrograph, (2) SCS triangular unit hydrograph, or (3) by solving equations for flow. The unit-hydrograph procedure derives a hydrograph by assuming a specific shape that represents land-use, soil, and geometry characteristics of the watershed, although techniques are available to derive the unit hydrograph from observed rainfall-runoff data, these data were not made available in this study. The SCS triangular unit hydrograph is an approximation of a nonlinear runoff distribution that is assumed to be constant in a unit hydrograph method. A number of methods exist for solving equations for flow. The Muskingum method is used for channel routing by determination of a wedge-shape channel storage in relation to inflow and outflow channel volume. Overland flow and channel routing is performed in some models by kinematic wave to solve the continuity equation for flow or by diffusive wave, which includes an additional pressure-differential term (Miller, 1984). The cascade method is a two-dimensional kinematic wave approximation for routing overland flow (Julien and others, 1995). Models that use the kinematic or diffusive wave routing differ by how overland flow and channel characteristics are specified.

Model Data Description

Watershed Characteristics

Two small urban watersheds were selected for model simulations that differed in size, climate, and drainage characteristics. Information about the watershed that is typically readily available to the engineering community was provided to each modeler; this included a base map and tabular data on topography, buildings, roads, soils, and drainage characteristics.

Surrey Downs watershed (fig. 1A) is a 0.14 mi2 area of mostly single-family residential land-use. The basin is about 2.5 times as long as wide and is drained entirely by a closed-pipe storm-sewer system. Base flow was given as 0.02 ft3/s. Most streets have curb and gutter that drain to the storm-sewer system. Relief is about 150 feet, with slopes upwards of 20 percent. Impervious area is about 30 percent of the watershed, with about 20 percent being effective impervious area. Modelers were provided with details on the types of impervious areas, but not by subbasin. Figure 1A does not show details of the storm-sewer system including pipe diameter, length, invert elevations, slope, material, and soil characteristics that were provided to the modelers. Soils are gravelly-sandy loam to fine-sandy loam. Additional details on the watershed are reported by Ebbert and others (1985).

Harvard Gulch watershed (fig. 1B) is a 3.10 mi2 area of mixed urban land-use. The basin is about twice as long as wide and is drained by a combination of storm sewers and open channels. Base flow ranges from 2.1 and 5.6 ft3/s. Relief is about 150 feet with slopes generally ranging from about 2 to 0.5 percent. Impervious area covers about 38 percent of the watershed. Modelers were provided delineations, area, and percent impervious area for 33 subbasins, but were required to determine the effective impervious area where appropriate to the model. This task was made somewhat more difficult by a map of poor reproductive quality, a problem not uncommon in the real world. Soils are SCS Group B (Ben Urbonas, Urban Drainage and Flood Control, Denver, CO, written commun., 1990).

(fig. 1)

Figure 1. Location and physical characteristics of (A) Surrey Downs and (B) Harvard Gulch watersheds.

Storm Characteristics

Harvard Gulch and Surrey Downs watersheds are in distinctly different climatic regions. Harvard Gulch is in a semi-arid region with typically dry antecedent conditions and storms that are usually of short duration and high intensity. Surrey Downs is in the coastal Pacific Northwest, where antecedent conditions are typically wet and storms are usually of low intensity and long duration.

Five-minute rainfall data for six storms for two gages at Surrey Downs (fig. 1A) and five gages at Harvard Gulch (fig. 1B) were provided. Information on antecedent conditions including the amount of precipitation in the past 1, 3, 7, and 14 days and the time since the last 0.0, 0.2, 0.5, and 1.0 in. of rain was provided for each storm at Surrey Downs. No antecedent conditions were reported for Harvard Gulch because of the semiarid climate, but it was reported that the area is routinely irrigated with 1.0 to 1.5 in/wk which is considered equal to the rate of evaporation. Daily evaporation at the Surrey Downs watershed ranged from about 0.30 inches in the summer to about 0.02 inches during the winter.

The rainfall and antecedent conditions for the storms simulated are summarized in table 2. Storms at the Harvard Gulch site averaged about 1.5 hours, and those at the Surrey Downs site averaged about 15 hours. Rainfall intensity in the Harvard Gulch watershed was generally 5 to 6 times greater on average than in the Surrey Downs watershed. Often, the 60-minute maximum intensity recorded in the Harvard Gulch watershed by a single raingage exceeded the average rainfall volume for all gages indicating a wide spatial variability. Rainfall in the Surrey Downs watershed was relatively uniform.

(table 2)

Comparison of Simulated and Observed Flows

Results of the model-simulated flows and storm volumes (figs. 2 and 3) are summarized in terms of the percent error (departure from observed values) in table 3 (peak flows) and table 4 (storm volumes). The percent error is computed as:

(equation 1)         (1)

where predicted is the simulated value of flow or volume, and observed is the measured value of flow or volume.

CASC2D results were obtained only for Harvard Gulch for the May 8, 1981 storm; the results are not included in the tables, but are shown in figure 2. Error for CASC2D simulations was -10 percent for peak flow and -39 percent for storm volume. Also, results of CHUP/SWMM, the distributed version of the CUHP model linked using SWMM, is only available for the Harvard Gulch watershed because no simulations were made of the Surrey Downs watershed.

Simulated peak flows differed from observed peak flows (table 3) by -100 to 260 percent at Harvard Gulch and by -100 to 200 percent at Surrey Downs. The RMS peak flow error ranged from 19 to 171 at Harvard Gulch and from 40 to 162 at Surrey Downs. The average RMS peak flow error for all models was 55 for Harvard Gulch and 49 for Surrey Downs. The standard deviation of the error in the predicted peak flow ranged from 17 to 92 at Harvard Gulch and from 20 to 85 at Surrey Downs. The average standard deviation in peak flow model error is about 40 percent greater at Surrey Downs (40) then at Harvard Gulch (31). In general, simulated peak flows at the Surrey Downs site tend to be overpredicted, whereas simulated peak flows for some models are overpredicted and some are underpredicted at Harvard Gulch.

(fig. 2)

Figure 2. Simuated versus observed peak flows at (A) Surrey Downs, WA and (B) Harvard Gulch, CO for six storms. [CASC2D results only available for one storm at Harvard Gulch; Locations shown in figure 1.]

(table 3)

Simulated storm volumes differed from observed storm volumes (table 4) by -100 to 190 percent at Harvard Gulch and by -100 to 240 percent at Surrey Downs. The RMS storm volume error ranged from 17 to 101 at Harvard Gulch and from 15 to 142 at Surrey Downs. The average RMS storm volume error is 56 at Harvard Gulch and 54 at Surrey Downs. The standard deviation of the error in the predicted storm volume ranged from 19 to 70 at Harvard Gulch and from 16 to 98 at Surrey Downs. The average standard deviation in storm volume error is also about 40 percent greater at Surrey Downs (62) than at Harvard Gulch (38). In general, simulated storm volumes tended to be overpredicted for larger storms and underpredicted for smaller storms at both sites, except for the largest storm at Surrey Downs which was underpredicted.

(fig. 3)

Figure 3. Simuated versus observed storm volume at (A) Surrey Downs, WA and (B) Harvard Gulch, CO for six storms. [CASC2D results only available for one storm at Harvard Gulch; Locations shown in figure 1.]

(table 4)

Models based on the SCS curve number (HEC-1 and TR20) for generating runoff generally had the poorest fit. HEC-1 simulations substantially overpredicted peak flows, and TR20 simulations substantially underpredicted peak flows; this may indicate the sensitivity of the simulations to user judgment of the SCS curve number. These HEC-1 results represented simulations based on 6 subbasins; results also were provided for a simplified model that used only 3 subbasins. The simplified model produced slightly smaller peak flow error, but about the same runoff volume error. An additional HEC-1 analysis of the May 1981 storm at Harvard Gulch, performed by the CASC2D modelers, had only a -22 percent peak flow error which underscores the sensitivity of the model to the SCS curve number. A comparison of runoff simulation techniques in west-central Florida indicated somewhat less, but comparable error, in simulated peak-flows and storm volumes for TR20 and HEC-1 simulations (Trommer and others, 1996). In that study, average uncalibrated-model peak-flow and storm-volume error averaged 45 and 43 percent, respectively, for TR20 simulations and 105 and 27 percent, respectively, for HEC-1 simulations.

The unit hydrograph method generally had about half the error for simulations of the Harvard Gulch watershed than for simulations of the Surrey Downs watershed; this is attributed to its design for application in the Denver area. DR3M simulations of Harvard Gulch indicated an extended hydrograph recession compared to the observed recession; this resulted in relatively large storm-volume error for some simulations.

Model error was generally smaller for complex-distribution models DR3M, HSPF, PSRM and SWMM than for simple distribution-models CUHP, HEC-1 and TR20. The average peak-flow RMS error for the complex-distribution models was 36 at Harvard Gulch and 50 at Surrey Downs, in contrast the average RMS for the simple-distribution models was 75 and 92, respectively. The RMS storm-volume error for complex-distribution models averaged 46 at Harvard Gulch and 27 at Surrey Downs, in contrast to an average RMS for the simple-distribution models of 65 and 90, respectively. The error associated with the CUHP was generally similar to the error associated with the complex-distribution models, however. Michaud and Sorooshian (1994) also demonstrated that an uncalibrated complex-distribution model (KINEROS) was more accurate than a simple uncalibrated SCS-curve model.

The results of this test are not conclusive and do not indicate that one model performs better than another. The results merely show that the error associated with uncalibrated models can be substantial. The design of structures such as culverts, storm sewers, detention basins, and other storm-water facilities based on uncalibrated model results could result in unnecessary costs when a model overpredicts peak flows and storm volumes, and perhaps result in even more severe consequences when these structures are underdesigned because the model underpredicts peak flows and storm volumes.

References

Alley, W.M. and Smith P.E., 1982, Distributed routing and rainfall-runoff model -- version II: U.S. Geological Survey Open-File Report 82-344, 201 p.

Aron, Gret, Smith, T.A.3rd., Lakatos, D.F., 1996, Penn State Runoff Model, PSRM C96, User Manual: Environmental Resources Research Institute, Penn State, PA, Penn State Univ., 54 p.

Bicknell, B.R., Imhoff, J.C., Kittle, J.L., Jr., Donigian, A.S., and Johanson, R.C., 1993, Hydrological Simulation Program--Fortran, Users Manual for Release 10: EPA-600/R-93/144, Environmental Research Laboratory, Athens, Ga., U.S. Environmental Protection Agency, Environmental Research Laboratory, 660 p.

Ebbert, J.C. Poole, J.E., and Payne, K.L., 1985, Data collected by the U.S. Geological Survey during a study of urban runoff in Bellevue, Washington, 1979-82: U.S. Geological Survey Open-File Report 84-064, 256 p.

Huber, W.C. and Dickinson, R.E. 1988, Storm water management model version 4, part A: Users manual: U.S. Environmental Protection Agency Report EPA/600/3-88/001a, 569 p.

Hydrologic Engineering Center, 1990, HEC-1 Flood hydrograph package user's manual: U.S. Army Corps of Engineers, Davis, CA, 410 p.

Julian, P.Y. and Saghafian, Bahram, 1991, CASC2D: A two-dimensional watershed rainfall-runoff model, CASC2D user's manual: Report CER90-91PYJ-BS-12, Colorado State University, Fort Collins, CO, 66 p.

Julian, P.Y. and Saghafian, Bahram, and Ogden, F.L., 1995, Raster-based hydrologic modeling of spatially-varied surface runoff: Water Resources Bulletin, Vol. 31, No. 3, p 523-536.

Michaud, Jene and Sorooshian Soroosh, 1994, Comparison of simple versus complex distributed runoff models on a midsized semiarid watershed: Water Resources Research, Vol. 30, No 3, p. 593-606.

Miller, J.E., 1984, Basic concepts of kinematic-wave models: U.S. Geological Survey Professional Paper 1302, 29 p.

Soil Conservation Service, 1986, Urban hydrology for small watersheds: Technical Release No. 55, 210-VI-TR-55, 156 p.

---, 1983, TR-20 Project formulation-hydrology (1982 version), Technical Release No. 20, 296 p.

Tommer, J.T., Loper, J.E., and Hammett,K.M., 1996, Evaluation and modification of five techniques for estimating stormwater runoff for watershed in West-Central Florida: U.S. Geological Survey Water-Resources Investigations Report 96-4158, 37 p.

Urban Drainage and Flood Control District, 1984, Urban storm drainage criteria manual, Vol. 1 Chps. Rainfall and Runoff, 1984 revision, Denver, C.O., 120 p.

Back to the SMIG Features Page

button bar

SMIG Home Mailing List Features Conferences Classes Reading Room Model Archives Feedback Home | Mailing List | Features | Conferences | Classes | Reading | Model Archives | Feedback

Stewart Rounds, SMIG coordinator <sarounds@usgs.gov>
U.S. Geological Survey
http://smig.usgs.gov/SMIG/features_0398/runoff.html
Last modified Wednesday, 17-Dec-2003 14:06:58 EST
Privacy Statement · Disclaimer · FOIA · Accessibility