In 2010, Stamen was awarded a Knight News Grant to build tools that help people tell better stories about cities. I'm pleased to announce that we've been awarded another grant from Knight to extend and support this project, as follows:
Terrain layers, at all zoom levels, outside the US
Transfer the hosting and processing to a cloud-based infrastructure to increase stability, scalability, server response and service levels, creating more reliability and confidence for those considering using the tile sets, & to allow for more frequent data updates
Improving the accessibility and openness of the code base for the whole system
All too often, foundations and granting organizations get excited about funding new projects, without adequately considering how to support them once they've been launched. It's great to see Knight stepping out of this mold; Chris Barr and John Bracken are thinking about the future and it's great to have their continued support.
The project is used all over the place, which is great to see. Seth reports that we've served 329 million tiles since July 25th of this year, and 277 million in August so far.
This announcement comes at the same time as we're announcing a fairly substantial update to the project, Toner in particular. We've moved the infrastructure for the project off of Tile Farm, the communal server in Zynga's basement, and it's now living entirely on production-quality servers "in the cloud". We've worked out how to reliably import a whole planet's worth of data from Open Street Map, so you'll start seeing much more up to date data moving forward.
The new Toner also contains some significant design updates. The repo is open source as usual, at https://github.com/stamen/toner-carto. But the real pop, for me, is to see how far OSM has come since we last published it. There are so. many. more. buildings! And lakes and rivers and streams and detailed canals and quays and on ramps. The project is just continuing to seriously kick ass, and it's amazing to really dig in to what the army of volunteer mappers who make it happen have been up to.
In the following screenshots, the old Toner tiles are on the left, and new Toner is on the right.
At lower zooms we increased the detail on the coastlines, which will look really sweet on retina displays. Also note South Sudan has been added:
We improved which borders are displayed. Old Toner wasn't showing the borders between England, Scotland, and Wales. Now, that's all we show at this zoom level, and we wait to display smaller subdivisions until you zoom in further:
We use OpenStreetMap at higher zoom levels, and there has been a lot of improvement in the last few years.
The old Toner data was from early 2012, before OpenStreetMap's license change from a Creative Commons Share-Alike license to the Open Database License (ODbL).
Tokyo, Japan shows a lot of improvement, both in the data and how we style it:
Ho Chi Minh City, Vietnam shows new roads, but also how the styling of existing roads has improved. The neighborhood streets are more subtle at this zoom level.
A lot of cities in the US now have complete coverage of buildings. We've adjusted Toner's settings so that large buildings show up sooner as you zoom in. For example, in San Francisco, California you can see many more buildings at zoom 14, especially in the former industrial district of Soma.
By zoom 16, you can see all the buildings.
Seattle, Washington also has every building in OSM.
Notice the improved detail along the waterfront.
New York City now has every building, too. And look at those piers!
Seoul, South Korea has more streets and buildings in the center of the city
Look at the detail in the harbor of Hamburg, Germany
Amsterdam had a lot of building footprints before, but look at how much detail there is now!
More detail of individual buildings in Rotterdam, Netherlands, too:
You can now see the Parthenon in the Acropolis of Athens, Greece, along with thousands of other buildings
I'm so very sorry to have to tell you that Zach passed away this afternoon. The doctors determined a few days ago that his coma would be permanent, and he was taken off of life support at 3:14pm.
It's appropriate that he would leave us then, right at the number pi. I personally knew the side of him that was deeply involved with math, having worked with him on so many wonderful projects while he was at Stamen. I had second hand knowledge of the other side of him, the part that was perhaps even more deeply involved with food, and dancing, and embracing life. He was a smart interesting curious man, quick to laughter and very much on his own path. The work he did with me and Stamen is some of the work that I'm proudest of in my life. I was thrilled for him when he decided to continue his career after Stamen at the Exploratorium, and I'm sorry not to have seen more of the work he did there. Today the world lost a great artist and thinker and bon vivant.
Finding water data is harder than I thought. Like detective Gittes in the movie Chinatown, I’m poking my nose around and asking everyone about water. Instead of murder and slimy deals, I am scouring the internet and working with city government. I’ve spent many hours sleuthing and learning about the water system in our city.
In San Francisco, where this story takes place, we have three primary water systems. Here’s an overview:
The Sewer System is owned and operated by the SFPUC. The DPW provides certain engineering services. This is a combined stormwater and wastewater system. Yup, that’s right, the water you flush down the toilet goes into the same pipes as the the rainwater. Everything gets piped to a state-of-the art wastewaster treatment plant. Amazingly the sewer pipes are fed almost entirely by gravity, taking advantage of the natural landscape of the city.
The Auxiliary Water Supply System (AWSS) was built in 1908 just after the 1906 San Francisco Earthquake. It is an entire water system that is dedicated solely to firefighting. 80% of the city was destroyed not by earthquake itself, but by the fires that ravaged the city. The fires rampaged through the city mostly because the water mains collapsed. Just afterwards, the city began construction on a separate this infrastructure for combatting future fires. It consists of reservoirs that feed an entire network of pipes to high-pressure fire hydrants and also includes approximately 170 underground cisterns at various intersections in the city. This incredible separate water system is unique to San Francisco.
The Potable WaterSystem, a.k.a. drinking water is the water we get from our faucets and showers. It comes from the Hetch Hetchy — a historic valley but also a reservoir and water system constructed from 1913-1938 to provide water to San Francisco. This history is well-documented, but what I know little about is how the actual drinking water gets piped into San Francisco. homes Also, the San Francisco water is amongst the most safe in the world, so you can drink directly from your tap.
Given all of this, where is the story? This is the question that I asked folks at Stamen, Autodesk and Gray Area during a hyper-productive brainstorming session last week. Here’s the whiteboard with the notes. The takeaways, as folks call it are, are below and here I’m going to get nitty-gritty into process.
(whiteboard brainstorming session with Stamen)
(1) In my original proposal, I had envisioned a table-top version of the entire water infrastucture: pipes, cisterns, manhole chambers, reservoirs as a large-scale sculpture, printed in panels. It was kindly pointed out to me by the Autodesk Creative Projects team that this is unfeasible. I quickly realized the truth of this: 3D prints are expensive, time-consuming to clean and fragile. Divide the sculptural part of the project into several small parts.
(2) People are interested in the sewer system. Someone said, “I want to know if you take a dump at Nob Hill, where does the poop go?” It’s universal. Everyone poops, even the Queen of England and even Batman. It’s funny, it’s gross, it’s entirely human. This could be accessible to everyone.
(4) Think about focusing on making a beautiful and informative 3D map / data-visualization of just 1 square mile of San Francisco infrastructure. Hone on one area of the city.
(5) Complex systems can be modeled virtually. Over the last couple weeks, I’ve been running code tests, talking to many people in city government and building out an entire water modeling systems in C++ using OpenFrameworks. It’s been slow, deliberate and arduous. Balance the physical models with a complex virtual one.
I'm still not sure exactly where this project is heading, which is to be expected at this stage. For now, I’m mining data and acting as a detective. In the meantime, here is the trailer for Chinatown, which gives away the entire plot in 3 minutes.
Outside our own kitchens, gas leaks aren’t something we’d ever thought much about here at Stamen. But being a studio filled with progressive San Francisco do-gooders and all, we do think about climate change and sustainability. We also think an inordinate amount about all the sensors out in the world gathering data – satellites, airplanes, cellphones, Jawbones, and, of course, those Google Street View cars.
Stamen worked with EDF and Google Earth Outreach to design and build a system that EDF staff could use to crunch complex data files used by scientists into easily digestible maps that give an immediate sense of the scale of leaks in various cities around the country. For now, the maps cover only Boston, Indianapolis, and Staten Island, but Google will keep gathering data in more cities and the system we built for EDF allows them to easily add more maps as more cities come online.
Here’s New York, which is totally covered in methane leak occurrences:
As is Boston:
You might ask, is this normal? Well, take a look at Indianapolis:
There’s a lot less going on.
It was important to EDF and Google that we show the paths the cars drove, as well as the leaks detected, since the absence of leaks could mean sound pipes, or just an area that hadn’t yet been sampled. We took our cue from the familiar blue lines of Google Street View maps to show the drive paths, and then desaturated the base map to make sure the focus stayed on the leaks and sample areas.
Among our design goals for the project was to balance the visual presentation against the nature of the data. The leaks detected are really something called “verified peaks” — elevated methane levels detected frequently enough and at high enough levels to be significant. But, given winds and the potential for leaks to be contained in buildings and a thousand other variables, a measured peak doesn’t necessarily mean there’s a leak at an exact location out to n-decimals of latitude and longitude. So we opted for larger dots and diffuse edges.
And all these leaks are much lower than anything that would cause your neighborhood to erupt in flames. So the overall palette is a bit muted — nothing’s going to explode!
But our vast natural gas system — from wells getting drilled across the county to the networks of pipes in our cities to the knob on your kitchen stove — does make a huge difference for the future of our climate. We’re delighted to have helped EDF and Google begin mapping the part of that infrastructure that most closely touches tens of millions of people across the country.
We’ve been quiet about something cooking here at Stamen, in part because it’s new and experimental, and in part because we’ve been so busy doing it that we haven’t made time to write yet. That something is our Education program, run by Beth.
Stamen has had internships before, but this summer we wanted to take it a step further. Rather than having students work on bits of client projects, we decided that we really wanted to create a program that affords both research and design on a single project. This approach gives fellows the opportunity to complete a work with us that they feel proud of and that advances the current state of knowledge in their area of interest.
Although each fellow is working on their own project, what unites them all is a focus on urban data and making something invisible about the city of San Francisco truly visible.
Meet the Fellows
Andreas Viglakis // Uncovering Bay Area Transit
Andreas is a recent graduate of the Harvard Graduate School of Design, and has worked as an architect and urban designer in Shanghai, New York, and Hong Kong. His work this summer, in partnership with urban planning think tank SPUR, focuses on San Francisco’s fragmented transit system and how it's affected by the changing economic climate of the Bay Area.
A scale study comparing the size of the Bay Area to the size of other major cities. Shown above: Bay Area over New York.
Kristin Henry // Storytelling with Sounds in San Francisco
Kristin Henry, a fellow in partnership with Gray Area, is a generative artist and computer scientist specializing in science and data visualization. She also founded GalaxyGoo.org, a small non-profit for science literacy, where she leads a group of volunteer tech + art + science enthusiasts. During mid-career graduate studies in Computer Science, she developed an Android application that collects sound and some Python scripts to analyze and visualize the resulting collection of audio recordings. This summer, she’s using this technology to develop a sound map of San Francisco.
Sketches of walking paths and recordings.
Scott Kildall // Water Works
Scott Kildall is a visual artist who writes algorithms that transform various datasets into 3D sculptures and installations, which imagine “what data looks like.” His project for the Creative Code Fellowship is called Water Works and will include a 3D-printed data visualization of the San Francisco water system, which includes drinking water, sewer and storm pipes and emergency fire-fighting systems, all of which interact in various ways to provide a dynamic city infrastructure. His fellowship this summer is in partnership with Gray Area and Autodesk's Pier 9 Workshop, who is providing access to their workshop and advanced machine training in conjunction with their Artist in Residence program.
Screenshot of a mesh of manholes of the San Francisco water system, z-axis exaggerated.
Moira Forberg is a current MBA student at Harvard Business School and is interested in the intersection of design and technology. Prior to school, she worked on hardware design for 3D printing projects at MIT and Disney Research, and worked as a strategy consultant at OC&C. This summer, she is helping Stamen design best practices for intellectual property (IP) management around internal research projects and productization.
About the classes
So this is all great, but why is Stamen getting involved in education in the first place?
Stamen has always been a place for experimentation and research, typically through our research projects. We’ve also been involved in teaching people how to do this kind of work for a number of years now, historically with a focus on developers. While that focus continues to be important, the world of visualization has changed and what’s new is a focus on mapmaking and data visualization literacy for all skill levels, and an effort to support education around truly creative coding (i.e. the beautiful universe that exists beyond charts and graphs).
Inspiration for developing this practice around education came from Maptime, which Beth started here at Stamen last year. In addition to helping even more people learn how to make maps, it’s also building a community through education. We aspire for Stamen’s education program to have a similar community building aspect to it as well, using educational and research activities as a way to engage the public, use public data, and when possible, to collaborate with some of the talented visualizers and cartographers and mathematicians and artists on our network map and beyond.
Additionally, we believe it’s important for businesses in our space to pay it forward. This sentiment has its roots in the spirit of open source software development, and in practice it also gives us an opportunity to meet new people and try new things. Thanks to our fellowship program, for example, we’re learning so much about visualization-based installations, physical computing and non-web visualization. It gives us a chance to be students again, as well as teachers.
Eric’s always talking about how he never wanted to work for a cigarette company during the day to pay the bills, and for Greenpeace at night to salve his conscience, but instead wanted to commit a studio towards relevant work that also pays for everyone’s 401(k). Beth’s always talking about how she dreams of starting her own experimental, forward-thinking school that focuses on art, design, technology, and sustainability. Together, we foresee some lovely opportunities and collaborations in our future. It’s starting here in the Bay, with Gray Area, Autodesk, and SPUR. We’re excited to see where this experiment goes next.
Interested in collaborating with Stamen on a educational activity? Want to get more information about next year’s fellowships and internships? Drop a line to education [at] stamen [dot] com.
Stamen has long aspired to make it easier for people everywhere to visualize data, particularly on and with maps. In our recent partnership with Tableau, we’ve helped to improve a tool that does just that. Tableau’s latest version, 8.2, comes complete with a mapping suite designed by us. Suddenly it’s that much easier for people all over to make beautiful maps with their data.
The maps come in three varieties: light, dark, and blue water.
Light maps are meant to be as subdued as possible, using hints of terrain to provide texture.
The dark maps are designed for bright colors, barely hinting at the outline of country borders.
Blue Waters are meant to provide a similar subtlety as the light maps, only with more natural tones.
Our goals for the project were:
A clean, modern look and feel.
Consistency across the three maps. They should be strong enough to stand alone but work together as a set.
That the maps be beautiful in themselves, but primarily built to display data.
Finding the right shades for the background, shades that work well with all of Tableau’s standard palettes, brought a level of science to the color process that we had simply never practiced before. Typically Stamen's approach to color is an intuitive, experimental and evocative one; color theory is present but not primary. Tableau’s maps, in contrast, required rigorous testing and iteration for even the most incremental tweaks in order to retain contrast. Tableau’s Visual Analyst, Maureen Stone, was our guide deep into the world of color science. In doing so, she taught us more about color calibration than we ever would have thought to learn. We now know that color is not just about trusting the eyes; it’s also about trusting the numbers. The numbers are what allow a wide range of colors – important for data layers – to have consistent levels of contrast and pizazz.
This connection of art and science is part of what made the project come out so well. The resulting map design system has a consistency and intensity to it that we haven’t seen with any other out-of-the-box analytic tools.
We applaud Tableau’s use of Leaflet and OpenStreetMap for creating and presenting these maps. We're proud every time we can help one of our clients make the switch to designing with open data.
This year, people across California began getting health insurance coverage in new ways, thanks to President Barack Obama’s signature Affordable Care Act, often called Obamacare.
As the new marketplaces, subsidies, and penalties got discussed and debated in the media this spring, we at Stamen were hard at work with our frequent client the California Healthcare Foundation, creating a system to visualize and communicate a range of key measures to assess the impact and effectiveness of Obamacare.
It was initially a bit daunting to develop a systematic, dynamic way to display all this raw data, but we were determined to help make the ACA understandable by applying some delight to the figures.
Within each indicator, the data is sliced in several ways: types of care, providers, demographics, enrollment, quality of care, income levels, even geographic averages. With so many categories, the data can give you a general overview or a very specific peek into health coverage. After relentless explorations, we’ve developed a system we think works pretty darn well. And it’s a solution flexible enough to accommodate new data in years to come.
The data is broken down into three categories: Domain, Topic, and Indicator. Once the user makes their selection, they are taken to the indicator’s page, which then has a variety of data points to look at.
By default, the page opens to the Total datapoint and the most recent year of data collected, with trending data displayed on the left. Indicators with data from more than one year also let you browse through the different years. Once you click on another tab, you have the ability to compare those specific data points to the trended totals, as well as the ability to export, share, or embed the data you’re looking at.
The combination of data points results in thousands of potential exports. Here are some examples of other charts you’ll find in the project:
The technical challenges in the project were in how to make the whole system run off a simple set of spreadsheets that anyone could edit in Excel. We worked hard to make sure that CHCF can manage the project into the future, adding new indicators and assigning chart types to them, all via simple text files they can publish through their existing content management system.
We’re looking forward to seeing those trend lines fill out for years to come — and, we hope, measure ever better health coverage and health care for California.
Climate change is one of those things that can be so invisible, and in some cases so subtle, that it can be hard to conceptualize. Over the years, our work with Climate Central has sought to make one symptom of this looming problem – sea level rise – much easier to see. The maps we've made together reveal what's lost as water makes its way upon our shores, rather than the mass of land that's left behind. Our most recent work with them, in partnership with New America Media, shows us very clearly what we'd lose in California: the homes of hundreds of thousands of people.
90,000 of those people live in San Mateo County alone, 50,000 of whom are ethnic and racial minorities. Climate Central estimates the cost of this damage to be in the realm of $22 billion. Cities all along California's coast will face similar losses.
This version of the map, released today, lets us see a very poignant and personal part of the story, one more specific and far more real than a population density map could reveal. Inspiration for this method came by way of the by the Cooper Center's Racial Dot Map.
It's not just the numbers that are blowing our minds. It's the difference that we're seeing in each city and state, each of which has different a different datasets, and different levees for that matter.
We can see in Hayward that a large Black and Asian population will be affected:
Here are some specific numbers:
And here is the overall social vulnerability for the area:
I especially love parks. I spent almost a decade telling stories of local parks and wildlife for Bay Nature Institute, and toward the end of my time there I was constantly trying to push the envelope of what a small nonprofit could do with technology.
Now that I’m at Stamen I’ve got a slightly different view of how park agencies and nonprofits can and should engage with technology.
Back at Bay Nature, I thought, “I’ll figure out precisely what we want and do all the research, and then I’ll just hire someone to make that exact thing!”
I’ve now seen this same thought process at work in a few RFPs for mobile websites and interactive maps for parks and trails. Figure out every possible feature you and your colleagues might want, put them in a giant list, throw in some house-made wireframes, and then send out the RFP for precisely what you know you need.
Makes perfect sense. Except:
It doesn’t work.
What you think you know at the beginning of the process may well turn out to be dead wrong, and, more importantly, you lose so much of the big win of hiring professional designers when you try to solve all the design problems ahead of time.
When Stamen took on the effort to redesign the maps of the Golden Gate National Parks Conservancy, we were fortunate to have flexible, and all-around awesome, project leads at the Conservancy. Michael Norelli and Mason Cummings were ready and willing to hear that the enormous feature set they’d assembled was too big to do well in one scope of work.
Instead, we focused on the key goal: Get people to the parks.
Being able to focus that way was incredibly important to making sure the effort was successful. It meant the Trip Planner was the most important thing to build out fully, complete with bespoke directions to obscure locations and special warnings if parking at Muir Woods was impossible.
We made some awesome Trail Views (and those weren’t in the RFP), but the key to success here was understanding that getting people to the parks was the top priority and the main measure of success.
What about the original RFP’s several other pages of requirements, from hawk tracking to editable management zones? We had to let those go for future phases of work. If we’d tried to do them all, we wouldn’t have finished anything well. Check out the Golden Gate National Parks Conservancy's open source code repository if you want to do a deep dive into the code that made that project possible.
Better Scoping Principles
Prioritize: define must-haves, then cut your list in half
Start simple, move quickly
Release early, release often
Be ready to change your priorities
Favor open data, standard formats, and existing tools
Avoid letting data management dictate design
A common thread in parks and mapping RFPs is that data pipelines, GIS workflows, and general data management sometimes receive a lot more attention than the ultimate goal, which is usually something like getting people to parks or helping people discover new trails or find programs in parks.
That's natural, since the people writing the RFPs are the ones who most often wrangle all that data. But unless you’re really doing a pure infrastructure project, resist the temptation to let data management and your internal needs dictate the thing you’re getting built.
Which leads to:
Use the kinds of software you aim to hire someone to build!
This might seem obvious, but it’s surprisingly rare, especially if you’re a longtime parks pro. Hey, you already know tons of trails and parks and birds and nature lore, why do you need to use an app or website?
Well, first, you might discover some things you don’t know or new ways of looking at things you do know. But more importantly: Just trying out existing tools and really using them is a huge source of insight. Heck, you might discover something that works so well that you want to just hire the people who built it to make a special version for your park or agency.
Keep it simple! (That is hard.)
You might notice that the best existing services and applications are often the simplest. Whether through dumb luck or, more likely, previous mistakes combined with persistence, they’ve figured out what people want most and they’ve given it to them. And they’ve made it easy.
You can do the same in your search for a technology firm to build your stuff: Keep it simple. Favor qualifications and past work over ability to navigate complex documents and matrices. Find a firm with which you can have a relationship based on collaboration and trust. Hire them, take some risks, be ready for small failures in the service of big wins. And have fun.
Recently the fine folks at WIRED's Map Lab asked for a checklist for maps. We have a list of interaction design-related things we like to ensure across all of our map projects which we thought you might be interested to see. Here goes!
URLs should contain and maintain state, by default. This feature means that you can deep-link into any place, any zoom and even any data layer's state.
Clicking UI elements affects the URL too, e.g. show/hide legend should also affect the URL.
The map should be integrated with any external displays, so one affects the other in real time. You can see this at work on the Parks Conservancy Trails Page.
The application should know when all required tiles are loaded, and act accordingly.
e.g. Make sure tiles are loaded before any animation begins.
e.g. Don't necessarily display data overlays before the tiles have loaded, unless they're easy to place geographically, which is rare.
Mouse scroll should not affect map zoom level when the map is embedded in a page. There's little more irritating than scrolling quickly down a page and getting stuck way zoomed out of a map you hadn't intended to interact with.
Double click should zoom into the click point on zoom, not the center of the map.
No flickering (unless it's intentional). Work hard to make map transitions as smooth as silk.
All data and tiles should load smoothly. Tiled data is good for this, as are compression and content delivery networks.
Wait until the new data is ready and loaded before hiding older data. (Avoid showing grey tiles.)
e.g. switching between map styles. Don't throw the old stuff away and show grey tiles in between the switch.
If you can, make a full screen version, or at least make the map as BIG AS POSSIBLE, like on the Chesapeake Bay Grasses project.
What's on your list? Anything you'd like to see added to ours? Tweet @stamen + @wiredmaps + #maplist, or send an email to info [at] stamen [dot] com.