Wednesday, March 25, 2015

Spatial Join

I did a prior post on how to select and export points for any area, but in my case, I could use spatial info for my entire extent. Namely, I want to know what region BBS routes fall in, and I have a shape file of those regions. I need a column added to the BBS route shape file that says which BCR each route is in. My solution: spatial join.
  • Target feature: what I need columns added to
  • Join feature: where the columns are coming from
  • Output: name your new feature, because unlike some other programs, Arc knows you might not want your original messed with
  • match option: how should the values be assigned?
To me, it made most sense for the latter to be "have their center in," because routes cross BCR borders, etc. (It's messy.) This, for once, isn't my code block; it's straight from the Arc website:
 import arcpy  
 target_features = "C:/data/usa.gdb/states"  
 join_features = "C:/data/usa.gdb/cities"  
 out_feature_class = "C:/data/usa.gdb/states_cities"  
 arcpy.SpatialJoin_analysis(target_features, join_features, out_feature_class)  

Tuesday, March 24, 2015

Breeding Bird Survey

The state-level data has all years, and is summarized by 10 stop. As I mentioned in a prior post, it has a numeric ID for state, and a route number. Commonly, these are concatenated (always 3 digit for route) to produce a unique ID. Further, in the route file (weather), there is a unique ID for each route run (each year a route was surveyed), called RouteDataId. As you can imagine, that's pretty valuable, and someone thought of it, so why it's not in the BBS data I'm not sure. So, my best answer so far is a kinda messy merge.

Thanks to GME, I have predictors now in my DBF (we'll deal with that later).

Monday, March 23, 2015

BBS Points

I have this point layer, that I'm not entirely confident where it came from. The good news is, it seems to correspond to the coordinates from some other file (I think?). However, it didn't have any spatial information with it.

Note to self: I assigned it the datum WGS 1984 because I thought I read that somewhere with relation to BBS data.

Sunday, March 22, 2015

Last Week of March WI Rarities

  • Eurasian wigeon
  • cinnamon teal
  • king eider
  • smew
  • yellow rail
  • mew gull
  • burrowing owl
  • Lewis' woodpecker
  • American 3-toed woodpecker
  • Bewick's wren
  • curve-billed thrasher
  • green-tailed towhee
  • black-throated sparrow
  • lark bunting
  • black-headed grosbeak

Friday, March 20, 2015

Geospatial Modelling Environment Script

1st warning: GME summarizes variables by appending them to your shapefile's attribute table. If you don't want all those added columns in your "one good input file," first make a copy of your input file then use another copy to play with in GME. (I have a separate folder for all the work I do in GME, and I copy files there to use with the program. That way, I still have untarnished shapefiles left in my main folders for all my other purposes, especially if I'm doing something exploratory in GME.)

GME has a variety of input methods (common), from simple point-and-click to a scripting interface. You already know which direction I lean when interfacing with it. Save yourself some time and trouble! Your time is more valuable than that; let the computer do the repetitive stuff. Here's a small example of a loop in GME.
1:  for (i in 2000:2013) {  
2:  isectlinerst(in="shapefile", raster=paste("path",i), prefix=paste("datatype",i), thematic=FALSE, proportion=FALSE);  
3:  };  
I have rasters that are simply named by the year they were captured (newer remote sensing data).

2nd warning (already a caveat to my earlier post...and there's a follow-up caveat): for some reason the GME doesn't like underscores in the prefixes?! Annoying, as it's a pretty standard alternative to a space for computer science naming conventions. It also would allow for it to interface pretty seamlessly when importing that info into R, as it's specifically designed to do, but hey, I guess we live in an imperfect world. Since R is so built as a stats language, "-" looks too much like a minus sign for it. All the other operators are thus out too, and anything else special character-y (like ampersands) could get too format-y for me (check out how weird they parse in HTML) to be comfortable with (but maybe I'm wrong there...maybe that would be just fine). Anyway, to me, it looks like we have no good options for special character separators (ARGH...who's idea was that?).

3rd warning (but you'll also just have to deal with this): R fusses when you try to import a column name that starts with a number, so avoid it. In my case, since I want to import it straight from the DBF (because I'm difficult) it's fairly impossible to not at least throw warnings that way. So, even though the years are both...
  • the names of my rasters
  • the most important info I want
...I add a prefix that is the type of data that's been summarized (probably smart anyway).

Until 4th warning: GME fusses again if you're trying to do this for polygons, saying that the prefix "should be no longer than 4 characters." So guess what? I'm just doing my year, and read.dbf will have to throw me a long list. Oh well. Special bonus: don't try to do anything in Arc after you've made a col name starting with a number from GME...
 for (i in 2000:2013) {   
 isectpolyrst(in="your shapefile", raster=paste("path to rasters",i), prefix=i);   
I get the relevant info in the column names out with a regular expression in my R script. As mentioned, we can't take advantage of any convenient splitting characters this time (sad face). For an example specific to my data, I have "VCF2000MIN" as a resulting column name from GME. "VCF" is what I came up with to not have the column name start with a number, and reminds me what variable I'm summarizing. The year is 2000, and MIN is a summary stat generated by GME.

Thursday, March 19, 2015

Make Your Life Easier

A few tips, that will make you happier (some of these you probably already know). Think about your naming conventions as they may be used later (i.e. in a script or even standalone commands to a terminal).
  • no folder names with spaces. USUALLY "_" is still your best friend as a file name separator...but with some weird exception sometimes
  • no field names starting with numbers: this is a problem for both Arc and R
  • save pristine originals of things you get...then make copies to play with

Wednesday, March 18, 2015

Remembering Why I Hate Arc

I start with a philosophy: do you remember when we weren't picky consumers, especially when it came to technology? Computers were made the way they were, and long ago, it kind of made sense. Cutting edge software wasn't made by laypeople, so we kind of put up with it. It did stuff we previously didn't think possible, no matter how clunky. Worse, I think we even thought we were stupid for not being able to figure it out, so the least we could do was learn a command line. Now, I'm so glad we are. Things are better, and companies think about users. We've started to demand change in the market, and it will continue improving. Yet, the revolution is far from complete.

Arc exemplifies this. I wished that when I was a MS student taking a GIS class for the first time, I would have been more cognizant of what didn't make sense, but now I'm remembering. It starts from the beginning: adding "layers." What on earth is a layer? And why are we adding data? What are we making here?

Now, of course, it makes a little more sense AND I've gotten used to it. But, I want to get un-used to it. Maybe I need to start interviewing undergrads to get a better sense of how to reinterpret this beast.