Creating and Importing a New Dam Break Area with Baselines

Purpose:

This jobsheet is used for demonstrating how to create a new shapefile using baselines, and ingesting this as a new dam to test with the Hazard Services dam/levee flood recommender.

Tasks:

STEP 1 (of 3): Generate Shapefiles Using Baseline Tool

For focal points who do not have pre-defined polygons in WarnGen (or who just want to make a new shape) this process is a quick way of making new shapefiles. It uses the baseline tool in D2D to draw a shape, then calls a conversion script to export the baseline to a shapefile. The script will require information about which baseline to export, the filename/directory to save to, and a name for the new shape.

REQUIRED SCRIPT: baseline2shape.py (FOCAL POINT WORKSHOP: this will be on your desktop)

  1. In the D2D perspective, launch the baselines tool by clicking on its icon in the D2D toolbar, or selecting "Baselines" from the "Tools" menu
  2. Zoom out to see both endpoints of the "A" baseline by using the middle-mouse scroll wheel
  3. Find the "A" baseline to use for drawing
    1. You can use any baseline, but our commands later use baseline "A" when exporting coordinates, so you'll need to substitute your baseline at that time
  4. Form a rudimentary polygon with your baseline as follows:
    1. Left-click and drag on the vertices to move them, making sure to keep the vertices inside the CWA
    2. Middle-click on a portion of the baseline to create additional vertices (you must have at least three vertices)
    3. Use zoom and other aides like maps to help better localize your polygon to an area of interest
    4. It is not necessary to close the shape, but keep in mind that the unconnected ends will be bridged when converted to a shapefile
  5. Once satisfied with the shape of your baseline, open a terminal on your Linux, by right-clicking on the desktop and selecting "Open Terminal"
  6. Make sure a cwa-specific directory exists for exporting your dam shapefile to by running the command below in your terminal, editing the LLL to match your site
    mkdir -p /localapps/HazardServices/LLL/shapefiles/damBreak
    NOTE: A directory like /localapps/HazardServices/LLL/shapefiles/damBreak/ is recommended for consolidating shapefiles several reasons.
    • It isolates local shapefiles from any changes made to the /awips2 directory
    • It allows shapefiles to be grouped by site LLL, which facilitates storing shapes for offices other than your own, such as for backup
    • It separates shapefiles for damBreaks and burnScars into distinct folders, a necessary requirement for ingest later
  7. [DO NOT EXECUTE COMMAND YET] Prepare to run a conversion command (below) which turns your baseline into a shapefile. The command below accomplishes this after appropriate substitutions are made to the color-coded expressions
    /awips2/python/bin/python ~/Desktop/baseline2shape.py ${outputFilePath} ${baselineName} ${shapeName}
    1. ${outputFilePath}: Substitute with the full path for the shapefile to create (including the name)
      • EXAMPLES: /localapps/HazardServices/LLL/shapefiles/damBreak/MyExampleDam  ...where LLL should be your CWA
      • Use a full path, ending in the desired filename (no .shp or other extension needed), to save the files in the place you want them
    2. ${baselineName}: Substitute with the name of the baseline you used
      • EXAMPLE: A
      • Only the letter is needed, no quotes or other surrounding characters
    3. ${shapeName}: Substitute with the name you want for your shape in the database
      • EXAMPLE: "My Example Dam"
      • Wrap in quotes, especially if you use spaces in the name
      • WHAT NAME TO USE? This will be the name added to your maps table as well as a default name in products, however you'll have a chance to specify an alternate in MetaData files later
    4. FULL EXAMPLE OF COMMAND:
      /awips2/python/bin/python ~/Desktop/baseline2shape.py /localapps/HazardServices/LLL/shapefiles/damBreak/MyExampleDam A "My Example Dam"
      
  8. After typing a customized version of the command above in your terminal to suit your shape, hit ENTER to execute the conversion command
    1. If successful, there should be NO message displayed
    2. If you get a "can't open file 'baseline2shape.py'" error, that means the baseline2shape.py script cannot be found at ~/Desktop/baseline2shape.py. Adjust the path "~/Desktop/baseline2shape.py" in the previous command to point to the correct location and try running again.
  9. You're finished creating your shapefiles!
    1. Remember the output directory you used (e.g. "/localapps/HazardServices/...") ... that will be a crucial input when using ingestShapeFile script to load your shapefile into AWIPS

 

STEP 2 (of 3): Run Shapefile Import

Whether you have just run the parseWarnGen script (which produces shapefiles from your WarnGen configured dams) or you have local shapefiles you want to import, this next step is the crucial step which imports the geometries into your maps database.  The ingestShapeFiles script is capable of importing shapefiles from both sources (those parsed from WarnGen and other local ones) simultaneously.

  1. Open a terminal by right-clicking on your desktop and selecting "Open in Terminal"
  2. Now, execute the following command in your terminal (editing the LLL to match your site) to ingest the shapefiles you created into Hazard Services' map table 
    /awips2/edex/scripts/HazardServices/ingestshapefiles.sh -d -s /localapps/HazardServices/LLL/shapefiles/damBreak/
    1. This command uses a "-d" flag to indicate a new dambreak shape type, and an "-s" flag followed by the path to your shape (ANY shape at that path will also be imported).
    2. TECHNICAL NOTE: There are many other arguments and ways of using the ingestShapeFiles.sh script (run ./ingestShapeFiles.sh by itself to see usage). The scenarios above are intended to cover the most frequent need.
  3. Note any help messages produced by the ingestShapeFiles script, such as:
    1. "Some impact areas in mapdata.hazardservicesarea do not have an entry in any instance of DamMetaData.py": This message points to a helpful attempt by the ingest script to identify where new or existing shapes are missing critical MetaData. Every time the ingest script is run, a bare-bones DamMetaData.py file is auto-generated file with basic entries for these missing dams, which again encompasses any newly ingested shapes OR shapes already in the database which do not have an entry in the DamMetaData file. This auto-generated file (whose location and name is given after the above message) is suitable to use as or incorporate into an override file for DamMetaData.py, to provide at least minimal functionality for those shapes.
    2. "Existing impact areas are being preserved, unless they have a redundant name attribute with new shapes being ingested": This message lets us know that the default behavior of the ingestShapeFiles script, when it's run successive times, is to leave previously imported shapes in the database, UNLESS a shape with the same name is being imported again, in which case that shape is updated. This also means that you can run ingestShapeFiles multiple times, pointing at different shapefile directories (or perhaps shapes with different attribute structures, which it's good to keep separate), and each run will accumulate new shapes in the daminundation maps table.
    3. If you have any other issues, refer to Section 1.0.2.4 of the Focal Point Guide for troubleshooting
  4. Before moving to the next step, carefully look for the "Some impact areas in mapdata.hazardservicesarea" message mentioned in the prior step...
    1. If you see this mentioning a tmp/DamMetaData.py* file (with some numbers after it), that will be important! 
    2. This file, if created, is trying create a very basic metadata entry for any dams which do not yet have them
    3. Feel free to open it for viewing by using gedit (note the numbers in your filename will be different):
      gedit /tmp/DamMetaData.py.00000 
  5. Go on to the next step (checking what you've made), but keep in mind:
    1. It will be necessary to manage damMetaData to correct values, add values (like scenarios) and more, especially if the shapes you ingested have never existed in the system before.

 

STEP 3 (of 3): Check Results

The best way to check the result of this process is to try running the damBreak recommender and see if it has your new dams in it.

  1. Launch Hazard Services (either in D2D or Hydro perspective)
  2. Run the Dam/Levee Flood Recommender (you may need to load a "Hydrology_ALL" or other setting to make this recommender visible)
  3. Look for your new/updated dam in the dropdown at the top of the dialog which appears
  4. Click the Run button
  5. Check the results
    1. Does your polygon load and look the way you expected it to?  Be aware that Hazard Services will have rounded your polygon lat/lons to a precision of 0.01, so this might cause your resulting shape to differ slightly from the way you initially drew it.
    2. With a new shapefile, you will not have any pre-populated fields like "rivername" or scenarios. These will require dam-specific configurations in the DamMetaData.py file
  6. If the polygon and/or HID reflect the updates you expected, congratulations! If not, you might need to re-do the polygon creation and ingest, and for non-polygon errors you may need to fix or add a DamMetaData override.

 

 

DELETE A DAM BREAK IMPACT AREA

It may sometimes be necessary to remove a stored polygon (this may be more common for burnScars than damBreaks). This task demonstrates how to remove:

  1. Identify the dam whose polygon you wish to delete
    1. [FOCAL POINT WORKSHOP] If you previously created a dummy shapefile (e.g. "myDam"), test this on that dam by assigning damName="your dam name"
    2. Run the following command: 
    /awips2/edex/scripts/HazardServices/ingestshapefiles.sh -d -n $damName -w $siteID

NOTE: You may still need to delete the metadata for that removed dam