Loading Shapefiles in to MongoDB

I’ve been playing a bit recently with a small geospatial/location based app. After haggling with a bunch of tools and MongoDB a bunch, here are a few tips on importing a set of ESRI Shapefiles in to a MongoDB. I’m looking at SF Street Sweeping data but you can use any Shapefiles you wish.

Get the shapefiles

For example, grab the SF Street Sweeping data, and download and unzip those.

Convert Shapefiles to WGS84 Projection

The SF shapefiles are in Northern California specific projection (2227). Lat/long coordinates are in WGS84 projection (4326). Download the GDAL tools to get access to the ogr2ogr tool to directly convert them. Or use QGIS to load the shapefile as a vector, then export it out using the WGS84 projection.

Convert Shapefile to GeoJSON

ogr2ogr -f geoJSON sweeping.json sfsweeproutes_in_wgs84.shp

Clean up the resulting GeoJSON

MongoImport doesn’t like the ogr2ogr generated GeoJSON. Remove the first two lines:

{
"type": "FeatureCollection",

and the last line:

}

and save that to `sweeping_clean.json`

Import the data to Mongo

mongoimport --db sfstreets --collection streets < sweeping_clean.json

Create a 2dsphere spatial index

Mongo needs an index to query on geospatial data. To create it from the mongo command line:


db.streets.ensureIndex({"geometry":"2dsphere"})

Where streets is your collection name and `geometry` is the object in your document that contains the GeoJSON location data.

That’s it!
You now have in the sfstreets database a streets streets collection. I’ll follow up in the next post on how to query this data.<

259 Words

We’ve raised $15M in funding from Kleiner Perkins

What an exciting morning — and really an exciting year here at Remind101. This morning we announced that we’ve raised $15M in funding Kleiner Perkins and John Doerr has joined our board of directors. This was the first time I was at all involved in a fund raise and I’d be lying if I didn’t say it was a learning experience. From the process of helping to put together the pitch deck, the models, help out on pitches and follow up it’s a pretty daunting task for any team to go through. I’m thrilled how it turned out for us and to have such an amazing partner at Kleiner in John Doerr.

If you’re interested in reading any of the coverage, here’s a list I’ll try to keep up to date:

138 Words

Music Top 10 from 2013

Like last year, here’s the list of music I’ve listened to most, as recorded by my scrobbling to Last.fm.

Top Artists

  1. Above & Beyond – 135 listens
  2. Avicii – 98 listens
  3. Daft Punk – 96 listens
  4. Jay-Z – 77 listens
  5. The Delfonics – 71 listens
  6. Alt-J – 69 listens
  7. Delorean & Hardwell – tied at 60 listens
  8. Morgan Page & Toro y Moi –  tied at 54 listens
  9. Hall and Oats & CHVRCHES – tied at 52 listens
  10. Bastille – 50 listens

Top Songs

  1. Bastille – Pompeii – 31 plays
  2. Hardwell – Spaceman – 28 plays
  3. Zedd – Clarity – 27 plays
  4. Swedish House Mafia – Don’t You Worry Child – 16 plays
  5. Avicii – Addicted To You – 16 plays
  6. Hardwell – Apollo – 15 plays
  7. Stevie Nicks – Edge of Seventeen – 13 plays
  8. CHVRCHES – The Mother We Share and  Walking Def – Running All My Life – tied at 12 plays
  9. Fastball – The Way and Frank Sinatra – I Love Paris and Swedish House Mafia – Greyhound and Alt-J – Something Good and Hall and Oats – Private Eyes and Avicii – Wake Me up – tied at 11 plays
  10. Kansas – Carry on Wayward Son – 10 plays

Observations:

  • Clearly the trend that started towards more electronica (specifically house and trance) continues this year.
  • Jay Z made a strong showing, largely due to me going to the Manga Carta world tour which sparked a re-interest in his epic catalog
  • Couple of really old songs and artists showed up, like Kansas, Hall and Oats and Frank Sinatra. This is because I started using Spotify in addition to iTunes/Amazon which has made getting to older music much easier and fun to rediscover stuff I haven’t heard in a long time.
276 Words

Static Site Hosting on Heroku with Node.js

I’ve been moving a lot of my web content off of a personal server which has been kept in to my apartment to various hosting services while on break this year.  Sites like Ask An Asian Person and other small inside jokes I used to host on a Windows 2003 Server with IIS on a Dell machine that ran in my closet. That setup is/was so very, well, 2003. In addition, it’s always a good move to reduce and remove any ingress points to my home network.

So for a bunch of the silly small sites I have, I’ve moved them over to one-dyno free hosting on Heroku. To do that, I made up a little template to use called static-heroku-node. It’s a tiny 10 line Node.js + Express application that deploys applications out of the /public/ folder in the app. Quick and easy to use, I managed to move a few sites over in short order.

As an aside, I moved my blog over to DreamHost. I looked at Heroku for hosting WordPress — there are a bunch of options on how to do it, but any production setup (e.g. > 1 dyno and any of their production level Postgres databases) would cost something like $25-$50 per month which is a bit rich for just a blog. DreamHost’s 1-click WordPress setup is much cheaper and more flexible than trying to scaffold the same thing up on Heroku/Dotcloud/etc.

238 Words

Track of the Week: Hey Now by London Grammar

This week’s track is Hey Now by London Grammar off of Metal and Dust, and the Arty Remix of the same. Not a new song by any measure, but I heard it for the first time on KCRW earlier last week, and then again in a house podcast in a house remix. So today’s track of the week is presented in two parts. First, the haunting yet slightly electro-poppy original, and then the dance remix which ups the tempo, adds in a classic four on the floor dance line and synths.

91 Words

You’re No Good by Major Lazer

This week’s track if You’re No Good by Major Lazer, featuring Santigold, Vybz Kartel, Danielle Haim and Yasmin. There’s been a dearth of good music the last few weeks, hence the lack of tracks of the week. I’ve finally gotten around to going through a backlog of albums that came out in the last month plus (Yeezus? Blerg, what a joke). I’ve finally made it to the latest Major Lazer album and the opening track from the album stood out, mostly since it sounds like a James Bond title track: “It’s for your eyes only, living for you solely.”

 

100 Words

Track of the Week: Wake Me Up by Avicii

This week’s track is Wake Me Up by Avicii. Originally found on his 2013 promo mix, which debuted a lot of new material, this new track is a totally different direction from traditionally house and dance/electronica. Much more pop, much more designed for mainstream listening, and definitely a tad country (stripped guitar, anyone?). Aloe Blacc (who, coincidentally, went to my high school a year ahead of me) lends his vocals to the track, giving it a touch of Blacc’s typical soul flair as well. I’m excited to see what else is coming in the new Avicii album, it could be an interesting and exciting new direction for electronica.

108 Words

Getting Maven to Work with Android 17

I just suffered through this for a good part of an afternoon, so figured I’d blog it for anybody else. Apparently a bunch of stuff moved around between some versions of Android, so make soft links to:

./sdk/tools/aapt to ./sdk/build-tools/17.0.0/aapt
./sdk/tools/dx to ./sdk/build-tools/17.0.0/dx
./sdk/tools/lib/dx.jar to ./sdk/build-tools/17.0.0/lib/dx.jar

So, if you’re in your ./sdk/tools directory, you can:

ln -s aapt ../build-tools/17.0.0/aapt
ln -s dx ../sdk/build-tools/17.0.0/dx
ln -s lib/dx.jar ../build-tools/17.0.0/lib/dx.jar

92 Words