<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"><title>Bryan Brattlof - Datasets</title><link href="https://0x42.sh/" rel="alternate"/><link href="https://0x42.sh/feeds/datasets.atom.xml" rel="self"/><id>https://0x42.sh/</id><updated>2021-02-09T00:00:00+00:00</updated><entry><title>Boston Parking Tickets</title><link href="https://0x42.sh/boston-parking-tickets/" rel="alternate"/><published>2021-02-09T00:00:00+00:00</published><updated>2021-02-09T00:00:00+00:00</updated><author><name>bryan brattlof</name></author><id>tag:0x42.sh,2021-02-09:/boston-parking-tickets/</id><summary type="html">&lt;p&gt;Last year (mid December 2020) I created a &lt;a class="reference external" href="https://www.foia.gov/"&gt;Freedom of Information Act&lt;/a&gt; request for all parking tickets issued in Boston from
2011 to the end of 2020. Eventually I was given 40 CSV files that I've combined
into a simple torrent you can download here:&lt;/p&gt;
&lt;blockquote&gt;
&lt;a class="reference external" href="https://git.sr.ht/~bryanb/boston-parking-tickets/blob/canon/data/boston-parking-tickets-2011-2020.tar.gz.torrent"&gt;boston-parking-tickets-2011-2020.tar.gz&lt;/a&gt;&lt;/blockquote&gt;
&lt;p&gt;Please feel …&lt;/p&gt;</summary><content type="html">&lt;p&gt;Last year (mid December 2020) I created a &lt;a class="reference external" href="https://www.foia.gov/"&gt;Freedom of Information Act&lt;/a&gt; request for all parking tickets issued in Boston from
2011 to the end of 2020. Eventually I was given 40 CSV files that I've combined
into a simple torrent you can download here:&lt;/p&gt;
&lt;blockquote&gt;
&lt;a class="reference external" href="https://git.sr.ht/~bryanb/boston-parking-tickets/blob/canon/data/boston-parking-tickets-2011-2020.tar.gz.torrent"&gt;boston-parking-tickets-2011-2020.tar.gz&lt;/a&gt;&lt;/blockquote&gt;
&lt;p&gt;Please feel free to &lt;a class="reference external" href="https://0x42.sh/connect/"&gt;send me an email&lt;/a&gt;
if you don't wish to use BitTorrent, and I'll do my best to send you a copy
using a different protocol.&lt;/p&gt;
&lt;p&gt;These &lt;strong&gt;(very messy)&lt;/strong&gt; files have data on every ticket, time and date it was
issued, violation and fine total, how much was payed, the license plate number
and state, including the car's make, style and color on every parking ticket.
Also included is the hand-entered location of where the ticket was issued.&lt;/p&gt;
&lt;p&gt;I say messy, because each ticket is manually entered on very small screens,
often by parking attendants wearing gloves during winter while someone is
telling them about their very bad day. &lt;em&gt;Understandably there is a lot of typos
and cleaning to do.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;All of the code for these examples, along with the torrent file, is available in
a &lt;a class="reference external" href="https://git.sr.ht/~bryanb/boston-parking-tickets"&gt;git repository here&lt;/a&gt;.
My goal is to implement various data cleaning techniques to see how well I can
prepare this data for processing. What follows is the little bit of data
expiration.&lt;/p&gt;
&lt;div class="section" id="tickets-issued-in-each-year"&gt;
&lt;h2&gt;Tickets Issued in each Year&lt;/h2&gt;
&lt;img alt="tickets grouped by year." class="right" src="https://0x42.sh/boston-parking-tickets/tickets-per-year.png" /&gt;
&lt;p&gt;Boston police officers issued 13,023,114 parking tickets inside the Boston city
limits between January 1&lt;sup&gt;st&lt;/sup&gt; 2011 and December 31&lt;sup&gt;st&lt;/sup&gt; 2020. If we
exclude 2020 and its &lt;a class="reference external" href="https://www.bostonglobe.com/2020/03/26/metro/mayor-walsh-just-relaxed-some-boston-parking-rules-heres-what-they-are/"&gt;relaxed parking rules&lt;/a&gt;, Boston receives on average
1,367,606 (±51,848) parking tickets each year.&lt;/p&gt;
&lt;p&gt;Surprisingly there wasn't a significant change in the number of tickets issued
during each year even as Boston's &lt;a class="reference external" href="https://www.census.gov/quickfacts/fact/table/bostoncitymassachusetts,US/PST045219"&gt;population continues to grow&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;I had assumed this would correlate with Boston's population growth, however a
simple linear fit shows there are 3,358 fewer tickets being issued each year,
well within the margin of error of the 1.4 million tickets issued on average.&lt;/p&gt;
&lt;/div&gt;
&lt;div class="section" id="tickets-issued-in-each-month"&gt;
&lt;h2&gt;Tickets Issued in each Month&lt;/h2&gt;
&lt;img alt="tickets grouped by the month they where issued." class="right" src="https://0x42.sh/boston-parking-tickets/tickets-per-month.png" /&gt;
&lt;p&gt;Another interesting thing I noticed (for this southerner) if we plot the number
of tickets issued by month, we can clearly see a dip in tickets during the
winter months.&lt;/p&gt;
&lt;p&gt;I have no evidence to support this, however, I assume the snow covered streets
of Boston during the winter reduces the available metered parking spaces, or
removes the 20,000 people willing to park their car when snow plows are
actively roaming.&lt;/p&gt;
&lt;p&gt;On average there is a decline of 20,000 tickets during the winter months. If we
exclude 2020 again (shown here as red dots), there is on average 117,000 tickets
issued each month.&lt;/p&gt;
&lt;/div&gt;
&lt;div class="section" id="tickets-issued-by-day-of-month"&gt;
&lt;h2&gt;Tickets Issued by Day of Month&lt;/h2&gt;
&lt;p&gt;Drilling further into when tickets are issued, if we look at each day in a month
each ticket is issued (excluding the 31&lt;sup&gt;st&lt;/sup&gt; day of the 7 months that have
31 days) we can see a pretty steady ticketing rate. Again the red dots represent
data from 2020 and are not included in the white shaded standard deviation range.&lt;/p&gt;
&lt;img alt="tickets grouped by day of the month" src="https://0x42.sh/boston-parking-tickets/tickets-by-day-of-month.png" /&gt;
&lt;p&gt;We can also clearly see the 572 days with under 1,000 tickets issued. As we'll
see in &lt;a class="reference internal" href="#tickets-issued-by-day-of-week"&gt;the next section&lt;/a&gt;, this is mostly
due (80% of the 572 day) to the relaxed parking rules on Sundays when most
parking meters are turned off.&lt;/p&gt;
&lt;/div&gt;
&lt;div class="section" id="tickets-issued-by-day-of-week"&gt;
&lt;h2&gt;Tickets Issued by Day of Week&lt;/h2&gt;
&lt;img alt="tickets grouped by day of week" class="right" src="https://0x42.sh/boston-parking-tickets/tickets-by-day-of-week.png" /&gt;
&lt;p&gt;Like I was saying in the last section, &lt;a class="reference external" href="https://www.boston.gov/departments/parking-clerk/how-do-parking-meters-work"&gt;parking on Sundays and City holidays is
free&lt;/a&gt;.
When we split the tickets issued by day of week we can see just how great this
policy is for people who fail to feed their meters.&lt;/p&gt;
&lt;p&gt;It's also interesting to see the reduction (roughly 1,000 on average) in tickets
issued on Mondays. As of right now I don't have a good explanation as what could
cause this.&lt;/p&gt;
&lt;/div&gt;
&lt;div class="section" id="tickets-by-violation"&gt;
&lt;h2&gt;Tickets by Violation&lt;/h2&gt;
&lt;img alt="number of tickets for each violation" class="right" src="https://0x42.sh/boston-parking-tickets/tickets-by-violation.png" /&gt;
&lt;p&gt;When we see what type of violations people are breaking on average each year, we
can start to see why Sundays and City holidays have such a huge impact on the
number of tickets issued each day.&lt;/p&gt;
&lt;p&gt;First place with 25.7% or 3,348,515 of all tickets issued was from unpaid parking
meters, most of which are disabled on Sundays.&lt;/p&gt;
&lt;p&gt;Followed by an ever shrinking list of significantly less common violations.&lt;/p&gt;
&lt;/div&gt;
&lt;div class="section" id="the-to-do-list"&gt;
&lt;h2&gt;The To-Do List&lt;/h2&gt;
&lt;p&gt;There is still a large amount of cleaning work I would like to do in the future.
There is currently many misspelled states, vehicle makes and models, ticket
locations or cross streets indicated with different symbols all of which makes
classifying this data a fun and difficult task.&lt;/p&gt;
&lt;p&gt;As of right now though, I'll publish this dataset with the promise to see what
insights we can gleam from it in the future.&lt;/p&gt;
&lt;/div&gt;
</content><category term="Datasets"/></entry></feed>