Showing posts with label public data. Show all posts
Showing posts with label public data. Show all posts

Wednesday, March 19, 2014

IPEDS Revisited

File under "how to break every Tableau rule in the book and thus create a poorly performing dashboard". Armed with a 70,000 row dataset from nces.ed.gov (specifically, the IPEDS longitudinal data set) I used parameters to allow the end user to select their desired metrics for correlation. I also have way too many filters on the filter shelf. The basic problem is that NCES has hundreds of metrics, all poorly named, all arcane, all requiring an excel file to look up their data definition. Many of them are ragged, meaning not all schools reported in all years. A brief scan of the metrics also shows me that definitions of metrics themselves change over the years.

I'm curious as to how other might solve this problem. I considered reshaping the 70k rows but with the number of metrics, it would end up in the tens of millions. Further, I would have a data model that is "too reshaped" or "too tall and narrow" for correlations.

If you are familiar with the common data set around IPEDS, you will be comfy using this dashboard. If you are not familiar with it... good luck! There's also a second tab with "additional filters" to refine your IPEDS criteria.

Wednesday, February 19, 2014

Divvy Data Challenge Using Tableau

I had a couple of free hours the other day, so I decided to create and submit the following "mini application" to divvybikes.com and their recently announced data challenge. I am always reticent to enter contests, lotteries, or what have you - because I never win dammit! ... anyways, let's see what those Chicago startup folks think. Yet another fun waste of time using Tableau. Enjoy!


I want to:

See when and where riders are going

See station mapping info

Review rider demographics

Looks at a calendar

Research the bike I am using

Look at station distances

Study capacity planning

Friday, January 31, 2014

Tableau Boeing 12th Man Flight Data

Boeing Inc. took their 747-8 out for a superbowl jaunt yesterday. Based upon flight tracking data from FlightAware located here, I ran the data through Tableau, added a couple of seahawks logo, and bam! 12th Man Viz... download this book and use the page shelf playback. Enjoy!

Tuesday, January 21, 2014

Tableau: Vaccine-preventable outbreaks

This simple dashboard is a reproduction of an existing visualization located here. This was a simple experiment to see how I could reduce screen real estate as well as eliminate the classic "Best viewed with the following specific browsers" warning. Mission achieved! It's also an interesting data set being mined in a rather manual fashion, all by catching news/media articles as they are released.

Wednesday, January 15, 2014


Exploits Dashboard

I'm starting a new theme in 2014 which is to have guest visuals on this site. This is partly because I am lazy (read: 100% because I am lazy). For a modest fee of exactly zero dollars and zero cents, you too can rent this space. Act soon to get in on this hot, hot visual action! This week we have a great dashboard from Brian Deely, one of the expert Product Consultants at Tableau Software.

At first glance you might think, "hey this viz is kind of simple. Where's the crazy Alan Smithee ridiculousness that I am used to seeing?". But here's the thing: I LOVE THIS VIZ. It's so simple. There's an elegance to it. It was so easy to create - it literally took Brian all of 2 minutes to build this. The goal of the viz is crystal clear. He left almost all of the default formatting untouched (except for the black. Nice one, Brian. I love black)

Those of us who are viz nerds - the vizarati of the BI world so to speak - get all worked up over advanced features and solutions. We sometimes forget what the whole point is: Fast. Easy. Fun.

With that, I give you Brian's exploit dashboard:

Monday, April 22, 2013


IPEDS School Data

The U.S. Department of Education is nice enough to collect a vast trove of demographic information on over 7,000 schools in the US. I've created this little mini-app to help you explore the metrics involved. These are just a sliver of all of the available metrics. You can build your own output data and then run it through Tableau. Warning: it's a very rough ride to format this data to something which is beautiful!

Enjoy!

Wednesday, March 27, 2013


Fedscope Data

This data set comes from the US federal government "Office of Personnel Management" - the data is the latest reported as of march 2012. The direct link is here.

All the panes in the dashboard below are "hot". Start clicking around to see where our federal salary dollars are being spent. Click on any whitespace to clear the selection for that pane. Enjoy!

 

Friday, August 31, 2012


USA Un-insured rates and statistics

The map below is of course an interactive viz. The rest of the tabs are web pages from the census website which describe the sampling methods, the field names, etc.

Relatively depressing information, overall...


Tuesday, August 28, 2012


Summer Olympics 2012 Medal Search

Thursday, August 9, 2012


Seattle 911 Calls 2011-2012




Step 1: Enter a Seattle address to see 911 calls near that address






Over two years ago I created a thematic map of Seattle Crime data, here. Well, a lot has changed since then. The details of 911 police calls and sometimes incident reports are available from the Seattle Police Department directly, so I decided to create a full blown interactive map to explore the data. Since the department does not report on postal codes or anything other than their district data, I had to implement a google address lookup using javascript. The result is that you can your address and see 911 calls with NNN miles of your specified location.

When you drill into a particular 911 call, look for the blue link in the tooltip to the incident report. This will take you to a Seattle Police website where once registered, you can pull up the incident report. If you get "no report available" it means no report was filed for the 911 call - this is a very normal result.

View source on this page to see the code I used to trigger the call to the Tableau viz. On the sending end, I used the google maps API to retrieve the latitude and longitude given a street address. On the receiving end inside of Tableau, we have parameters for latitude and longitude which can receive input from the URL. Finally, we have a mercator equation which allows you to specify a desired distance from your original address (download the workbook to see that calc).

Tuesday, November 29, 2011


No Taxation without Representation


I've been meaning to create a tax adjustment calculator. I started with the 2009 IRS summary data here and then created a "change the tax rate" scenario.

As I type this, I remain firmly neutral to this issue. However, it is interesting to note that even the smallest changes in tax percentage points add up to significant overall changes. Ask yourself: what can 1 billion dollars provide if handled properly by the US government? How about a 100 billion?

Download the workbook to pick away at it, change it, improve it, or scoff at it. It is by no means perfect. Just an idea that I wanted to mess around with. The last tab contains more references. Each tab has a quick comment on what's going on.

Thursday, August 25, 2011


MACD Technical Analysis with Tableau

Moving Average Convergence Divergence (MACD)

As a follow up to my previous post on Bollinger Bands, I took a stab at MACD, which derives from similar calculations.

This is a somewhat squirrely subject and I might be opening myself up to attack or a can of worms in this attempted analysis. As a preemptive strike against such attacks, let me start off by saying that as a stock trader, I fall firmly in the "fundamentals" camp, and this type of technical analysis does little for me. Second, allow me to graciously further admit that my math could very well be wrong. Finally, grant me the chance to comment on the fact that we can find on the internet dozens and dozens of permutations on the mathematics of this MACD subject.

Three web links which I found quite helpful in understanding the math behind MACD are these:

The Wikipedia Page on MACD
The Wikipedia Page on Moving Averages and Exponential Moving Averages
which are required components of MACD analysis, and lastly,
A Stock Charts Dot Com web page which has a nice discussion on the subject.

Feedback on the mathematics is welcome.

Tuesday, May 31, 2011


USDA National Nutrient Database

The USDA publishes a nutrition database once a year. It's an interesting data set to explore, so I decided to create and publish several different types of view. These are all very basic. This workbook was mostly an exercise of creating a simple, guided experience for exploring the data, as opposed to making a case for or against certain types of nutrients.

The website to get started is http://www.ars.usda.gov/services/docs.htm?docid=8964.

Enjoy!

Tuesday, April 5, 2011


FIPS Maps and Census Data

The purpose of this post is to demonstrate the ability to map census data in Tableau. I am at once both excited and disheartened about the United States census.gov website. I am excited because they do a great job of collecting the data - far better than many other countries, it seems. Disheartened because they then proceed to bury the data inside of massive amounts of excel files, with arcane naming conventions, master key lookup files, and what have you.

An ongoing project of mine will be to slowly amass this information into one unified view. This will exceed the capacity of Tableau Public, of course.

Anyway, the first step was to convert the FIPS codes as represented in shapefile format, to a valid polygon format that Tableau can understand. (This subject has been covered elsewhere in this blog as well as on the web). Notes: I provided Tableau with a copy of this polygon file, and you can download it here. FIPS code = US Counties. The Census download website is here. The income and poverty excel file is here. And the master key excel file is here.

Enjoy!

Tuesday, June 1, 2010


UK Fat Cat Pay Data Made Public

One of the first items David Cameron has released is "Top Pay" for public sector employees in the UK. The data comes from http://www.cabinetoffice.gov.uk/ - the exact link for the CSV file is:
http://download.cabinetoffice.gov.uk/transparency/high-earners-pay.csv

Monday, March 22, 2010


Public Data - Seattle Restaurant Inspections


Tuesday, January 19, 2010


EIMS - Water Samples for Snohomish Watershed using Tableau

This data came from EIMS, their website is here:

http://www.ecy.wa.gov/eim/

A few notes:
  • I culled data for only the last ten years, and only for the snohomish area.
  • The contractors who actually performed the contaminant samples often used different units of measure, as a result of using different testing methods. So I have placed a filter for UOM to help verify the test results. In plain english: the time trend and its 'median' value reported is bogus information if you mix your UOM types.
  • I removed any concept of what testing method they used, what the baseline observable measurement was, etc.
  • you can select a sample type, then lasso some data marks on the timeline to see individual sample reports.
  • The Tableau tooltips are nicely formatted; take a look at those!
  • clicking on the "links" item will show a bing map of the sample location. Hovering back over that same selected link item will also show two other links, to NIH.gov and wikipedia (for the contaminant in question)
  • In short: this was mostly an experiment with Tableau features, as opposed to a deep dive into water quality and contaminant analysis.