Stamen is a design and technology studio in San Francisco.

You are at Stamen's first blog, mostly written by Eric Rodenbeck. Visit our new blog! or come visit us at our newly designed stamen.com.

Contact Stamen, or follow us on Twitter

Recent blog posts

Jul 10, 2013

New & Improved Field Papers!

by George

Field Papers is a tool to help you create a multi-page atlas of anywhere in the world. The first version was launched in May of 2012, in partnership with Caerus Associates. In early 2013, we were approached by the U.S. Agency for International Development (USAID) to extend Field Papers, and make it more robust.

USAID wanted us to work on the performance of the system. After attempting to use it in the field, they had been hampered by the amount of time it took to make an atlas, seeing some atlases take up to 30 minutes to be created - obviously not good enough. We were thrilled to be given the opportunity to improve it, and we're hoping that Field Papers Version 2 proves much more useful, and encourages broader uptake. While we worked on the infrastructure, we also took the opportunity to make some improvements to the front end of the site, add a couple of new features, and fix some old bugs.

Here's a summary of what's new in Version 2:

Performance

The main work we did on Version 2 was about performance. We installed a queue manager called Celery to make sure all the uploads and atlases were being managed properly and cleanly. We know our improvements worked because the weekly average atlas creation times showed more than a 10x reduction (from over an hour to just under 5 minutes). Not only that, but the number of atlases created per week has also increased, from about 190 in 2013 prior to this project to almost 300 each week since.

Cancelling stalled prints
As we poked deeper into the code, and monitored a few more operations, we soon realized that there were quite a few "zombie" tasks clogging things up; processes that hadn't worked but were still alive. Sweeping all of these away and tweaking the code so failed operations were killed instead of retained went a really long way to tightening things up.

Designing for delay
I'm not normally a fan of designing around technical constraints, but in this case, I had some fun doing a design sketch of how we could let people know that their atlas was in a queue, and when it would be done. I was desperate to have a "Ding!" sound incorporated into the UI when someone's atlas was "cooked." But! Since Seth did such a good job of improving performance, the design I made was no longer needed. (I remain quietly desperate to use more sound in interaction design.)

New Features

Along with the performance improvements, we took the opportunity to make a bunch of enhancements and fix a couple of niggly bugs, like character encoding.

Edit in iD / Potlatch
We've added a couple of simple links from the pages that display scans of an atlas that take you directly into the lovely new OSM editor, iD, or Potlatch. (That means it's quicker to get the new information back into Open Street Map.)

Copies
Now you can copy an existing atlas with the click of a button. Say you and your street team are working in the same general area, and can basically use the same atlas. You just have to create an atlas once, then your team mates can copy it into their accounts. Copies are also noted in the activity log for each atlas, so you can track them.

Refresh
It might be the case that you're working on improving the data in a single area. If you're using OSM as your basemap, and you've gone through a round or two of updates to the OSM data, you can refresh your atlas to bring any updates to the cartography into a new version of the atlas. Same place, same pages, same basemap, new data.

Private Atlases
There's a new checkbox in the atlas-making process that lets you mark an atlas as private. You can do this whether you're logged into your account, or using the site anonymously. The only trick is, if you're not logged in, you'd better make sure you record the atlas's URL somewhere, otherwise, you'll never find it again because it won't show up anywhere around the site. (If you are logged in, you'll see it in the page that lists all your atlases, which you can get to by clicking on your username in top right of the nav bar.) Here's a private atlas I made of Marin County. You can only see it because I want you to (and am sharing its URL):

Maps, Activity, Counts for Things
We knew we wanted to improve how we're showing activity around the site, so, we've added a new map and counts to any page that shows you a list of atlases. Now we can easily see that there are 12,983 atlases in Field Papers (at time of writing), 14 atlases in Collioure, 1,216 atlases were made in May 2013, and awiseman has made 13 atlases, mostly in the Caribbean.

Nearby
There's a new, simple display on the bottom of an atlas page too, labeled "Nearby," to show you atlases that are, well, nearby the one you're looking at. Like this atlas from Collioure, in France:

Or, Dublin...

Or a selection of atlases from a mapping party in the Rift Valley in Kenya...

Metrics
In addition to learning more about the performance of the software, we also implemented a very quick view on some simple metrics for the site too, like portrait vs landscape, and which basemap is used the most with D3.js, the very handy JavaScript library:

Github

The Field Papers repo is public on Github. By all means, fork it for your own use, or report any issues you see as you use the updated site!

Head over to Github

OSM Wiki

We're working with USAID to create better How To documentation too, which you can see unfolding on the Field Papers page in the OpenStreetMap wiki. Feel free to chip in if the mood takes you!

So please, have a look around at the new Field Papers Version 2, and enjoy the new features and faster performance. Do let us know if you're using Field Papers on an interesting project too. It's great to hear when people are enjoying it!

Jun 27, 2013

How to make 3d maps of Mars

by Mike Tahani

Stamen created maps.stamen.com to showcase our cartographic explorations. It's also a place to stash our mapping experiments (Burning Map; Trees, Cabs & Crime). We recently added a new map: a three-dimensional map of contour lines that compose the surface of Mars.

1. Motivations

I started looking for space-related data shortly after hearing news of the NASA sequester (PDF). I wanted to show the value of freeing data produced by NASA, and at the time felt that the sequester would prevent investment in the technical infrastructure necessary to engage developers and citizens by making the data public. After much searching, I stumbled onto a dataset containing contour lines for Mars. Having breached 3D-in-browser in prior work--the Here maps, I realized that creating an explorable, 3D map of Mars was a simple project that would allow me to visit and traverse the surface of another planet.

2. Data

The most difficult part of creating the map was finding the data. I had a general idea of what I was looking for when I began (some sort of geo-located data), but was unsure of what NASA had available.

I recently attended a JPL/Caltech data visualization symposium with Eric, who wrote about the experience here. At the symposium, we spoke to several people at JPL about their "data problem". What's interesting is that we percieve the problem differently-- NASA/JPL have petabytes worth of data and no way to get it to developers. Bulk downloads would take days if not weeks, and the current API offering is slim (but under active development). While they seem to be grappling with data storage and delivery, we're having trouble with discovery. We don't know what the petabytes of data contain, and there's no single location to browse the totality of NASA/JPL's data offerings. Furthermore, each NASA/JPL project has a different website, with a different navigation system, with different offerings in different formats.

The data I eventually dug up out of the FTP site PIGPEN is a shapefile of Mars contour lines (FTP) from the Mars Orbiter Laser Altimeter, or MOLA, an instrument aboard the Mars Global Surveyor. The MOLA dataset also contains height data, which is necessary for the conversion into 3D.

3. Method

I set up a Tilestache server to serve the shapefile as GeoJSON data, then seeded the entire tileset for several zoom levels since doing it live means a huge delay as the shapefile is re-read for every single tile. Once I had all the tile data cached in a locally-accessible directory, I used Polymaps to read the tiles into the frontend. Polymaps is also the map interaction and tile-handling layer, so that when a user pans or zooms on the (hidden) Polymap, the event handlers translate the motion into 3D coordinate-space. This means I don't have to re-implement map movement and interaction and can focus on the trickier parts.

Once the tiles load in the frontend, the data is read and the 2D coordinates are drawn as lines in WebGL using Three.js, with the z-coordinate of each line defined by the heights specified in the original shapefile.

4. Future

At a recent Mars hackathon, I had the chance to work with heightmaps, and am currently working on building an uninterrupted surface of Mars. We're hoping to add more data layers to the maps so that they can be used as interfaces for intergalactic exploration, or maybe even educational tools.

Stamen would love to work with more spacey data. Recently, we've been speaking to some folks at the JPL, attempting to understand their data and figuring out how to make it more accessible for developers, who in turn can build interesting projects that will engage the public and hopefully garner more awareness and support for NASA programs.

5. Resources

Jun 12, 2013

Map Stack: Maps for all

Good-looking maps used to be the domain of experts. That’s been changing quite a bit in the last few years, and it’s easier than ever now for developers to access mapping data (the recent State of the Map US conference was a great place to hear about this). Never content to leave well enough alone, we thought we’d kick this sideways a bit and make it easy for the rest of us to make some great stuff.

We call it Map Stack. It's a bit like Instagram for maps.

The maps.stamen.com project was designed to let people easily use Open Street Map data in their own applications, and to provide well-designed map styles that would raise the bar for what people expect from open data. We're taking this a step further now, with http://mapstack.stamen.com/. Map Stack is about putting your creativity on the map, making it radically simpler to design your own map styles, without having to know any code, sign up for anything, install any software, or do any typing. We provide different parts of the map stack: backgrounds, roads, labels, and satellite imagery and straightforward controls for manipulating things like color, opacity and masking.

Watercolor punched through by satellite imagery from MapBox.

You don't need to sign up for anything, know how to code, or know much about cartography to make great looking maps. You just need to decide where in the world you want to go, a web browser, and about five minutes.

So now you can make all kinds of cool maps — say, this map of the Pentagon (which appears aglow with all the PRISM data flowing through it) — in no time:

Opening Hours?

We’d originally talked about calling the project the “map sandwich,” since it’s all about the stacking of layers of maps. You know, like a sandwich! The first WhereCamp DC used a sandwich as its logo and the idea stuck. From there we went to “delicatessen,” 'cause that's where you go to have your sandwiches made. (We went through lots of names - “mappuccino kundalini” was a favorite for a while.)

The deli name went away but the idea of a neighborhood, brick-and-mortar shop with regular hours stayed. We’ve decided, for the sake of Seth’s sanity, that Map Stack will be open for business Monday to Friday, from 11 a.m. to 5 p.m. PST until we can sort out what kind of infrastructure the project will need given the level of interest we’re seeing. You can tweet us during those hours with questions or give us a shout out to show off your smorgastrata at: http://twitter.com/mapstackStamen.

In its first day, viewers to the site generated about 10GB of map tiles—there are 10GB of new maps in the world! Which is great, but that’s a lot of tiles. We’re not getting paid for this—yet—and part of the project is figuring out what the business model might be.

A few initial ideas: we might offer the map backgrounds as a hosted service, we might sell prints (like we do via Soft Cities, 20x200 (sadly down but apparently coming back soon), and as of today, DODOcase), we might integrate it with Pinterest or Twitter or Facebook better (there’s a conversation about this very thing happening on Twitter right now)—we’re not sure. One thing we do know is that the best way to find this information out is to release projects into the wild, so!

Here are some examples, a short menu of maps that we and others have worked up to get you started. You can also view a gallery of images that people are creating here. Enjoy!

Jun 5, 2013

Earthlings! We have built you 3D Mars maps.

Last week Mike and I were invited to speak at the Jet Propulsion Lab for "Visualization: From Data to Discovery," a conference about the overlap between data visualization and science.

Which would have been interesting enough on its own (because science is awesome), but the icing on the cake was a tour of the Labs. We got to spend some time in the control room where the Mars Exploration Rover is controlled and check out live updates from the Voyager space probe (64k of memory, 35 years old, almost interstellar, totally bad ass).

 

We also took a close look at Curiosity, which has paint chips attached to its back that the onboard camera can use to color correct itself. The probe also has holes in the tires so the dust can fall out. As it drives around, these holes are visible in the tracks the rover leaves behind.

These markings can be used to visually determine distance travelled. Originally the holes spelled out J-P-L. This didn't go over well with NASA, so now it's—wait for it—Morse Code that spells out…J-P-L.

 

As a small gesture of thanks to the scientists who are working hard to do science and communicate with the public despite the stupid sequester, we've put together a 3d contour map of the Martian surface, using data collected by the Mars Orbiter Laser Altimeter. You can take a look for yourself at http://maps.stamen.com/mars.

Mars!

May 28, 2013

Facebook: Engineering a Beatquake

by Zach

Today we want to draw back the curtain on our most recent piece for Facebook Stories: Beatquake.

More than 110 million songs, albums and radio stations have been played 40 billion times through apps integrated with open graph. So what does it look like when a song goes viral on Facebook?

Inspired by old-school graphic equalizers, this Beatquake maps the volume of listens for each of the top three most popular songs in the U.S. each day over a period of 90 days. Colored layers show the number of plays expanding over time, visualizing the response to each song simultaneously across the country, while the texture of the map is driven by BPMs (beats per minute) and changes as one song overtakes another in popularity.

As a data-driven 3D visualization, the piece features a number of moving parts all working together to drive the final look:

Heightmap Generator:
We started with a day's worth of listening data for a single song; we had about ninety days worth of data for each song, and ten different songs. This was plotted onto a simple heightmap, binning by pixel. Because places like Manhattan have significantly more than 255 song listens per day, we encoded the number of listens using all three channels:

This data is extremely spiky. We discovered that specific versions of the specific songs we're looking at were hard to find across the Open Graph (and various music services within). There was quite a lot of variety in song and artist names; a noisy dataset. In addition, because location is determined by city in the data, there were a lot of very tall spikes in places like Manhattan. To counteract these giant peaks, we applied some Gaussian filters to smooth things out into more recognizable shapes. The banding is a result of encoding data as RGB:

This pre-rendering gave us ten images per day, for ninety days, which we used as heightmaps in the final piece.

Beat Engine:
In addition to the listen data provided by Facebook, we used the tempo of each song to lend a little bit of texture. Using each song's B.P.M., we set up an engine to simulate the beat and fall-off of sound for each song on each frame. This creates the pulse of the piece.

Song Popularity:
Finally, we built a piece of code which determines the top three songs on any given day; we calculated total US listen counts and fade songs in and out as they rise into and fall from the top 3:

Particle Display:
These two pieces are folded together in a WebGL-based visualization. We used Three.js for a lot of the setup, and then ran everything through some custom GLSL shaders to determine particle height and size. There are four — or six, depending on how you look at it — different cycles which produce the final look of each frame:

  • Frames - Simple incremented value
  • Beats - One per song, these go up and down according to the Beat Engine
  • Days - Each day plays for about a second, and then switches to the next - each day loads one of the blurred heightmaps.
  • Crossfade - We're doing a sine-modulated crossfade between days. If the list of top 3 songs changes, we fade the old song(s) out and the new in.

Ultimately, a particle gets a height based on the two crossfaded heightmaps, and a size based on the song's beat:

Rendering Engine:
Working in WebGL is fantastic for quick iteration, but a little tricky for final rendering. We used a solution we came up with for our previous Facebook Stories piece Virality, which involves saving out the current state of the canvas via php. In JavaScript, we set up function to bundle up the data and send it out via Ajax:

function saveImage(title, canvas) {
  c = canvas || d3.select("canvas")[0][0];
  var rawImageData = c.toDataURL("image/png;base64");
  var f = frame;
  
  var ajax = new XMLHttpRequest();
  ajax.open('POST', 'image_catcher.php', true);
  ajax.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
  var req = "convert="+encodeURIComponent(rawImageData)+"&id="+title+"&frame="+f;
  ajax.send(req);
  }

The resultant data gets sent to a php script which catches it and saves out an image:

$imageData = $_POST['convert'];
$imageID = $_POST['id'];
$frameNumber = $_POST['frame'];
$filteredData=substr($imageData, strpos($imageData, ",")+1);

// Decode the base64 encoded data
$unencodedData=base64_decode($filteredData);

// Save file.
$fp = fopen( 'frames/img_' . $imageID . "_" . $frameNumber . '.png', 'wb' );
fwrite( $fp, $unencodedData);
fclose( $fp );

In addition, all of the text on screen is rendered in a separate canvas, which uses the same process to save a transparent overlay for each frame. Since these are all identified by frame number, we could simply use them as image sequences to produce the final video.

View Project

May 28, 2013

Facebook Beatquake: What millions of sound waves traveling through Facebook look like

Facebook Stories has just released their third visualization project with us, which we're affectionately calling "Beatquake." The project is designed to show listening activity across Facebook over the course of 90 days. We were all surprised how popular Thrift Shop by Macklemore & Ryan Lewis is. It blew out most of the investigative charting we worked on early in the process. Here's how the project blossomed over the course of four weeks.

You can read about how this piece was made here, see more Facebook/Stamen in-process work here, and a series of posts looking under the hood at how this kind of work gets made here, here and here.

View Project

May 22, 2013

New Work: All Over The Map - California Healthcare Foundation Report

by George

The California Healthcare Foundation (CHCF) released an update to their "All Over The Map" survey of elective surgery and other procedure rates across the country yesterday. Funded by CHCF, this research is designed to help Californians understand their local healthcare region performs against the California average. Stamen worked with the CHCF a second time on this project, taking the opportunity to update the first version built in Flash into HTML5 & JavaScript, including a new Leaflet map of California. (The first version was released in December 2011.)

The updated data included several new elective procedures like breast cancer treatment and spinal procedures, which we highlighted in the "menu" of procedures:

This update also allowed us to make some improvements in the readability of the data, like a dynamically updating graph at the bottom of the map that shows the range of performance per procedure for all the regions currently visible on the map:

And a new chart per region that allows you to glance at that region's performance by age relative to the California average, for example, you can see that more gallbladder-ectomies happen in Red Bluff comparative to other regions, in each age group:

It's always good to have the opportunity to try to improve on work we've done before, and we were very pleased to have this opportunity with the folks at CHCF. Making this healthcare-related information more accessible to Californians is important work, and we're proud to have been involved. The study has made the news too - you can see the press release on Reuters or this article in Silicon Valley's Mercury News.

May 4, 2013

Stamen, Maps, and Brightworks

Remember being a kid and daydreaming about what it would be like if school wasn't just rows of desks and boring classes? What if you could go to a school where, instead of memorizing facts you may never reference again, you learned how to garden, make art, build things (based on math equations!), and work on projects of your choice?

This daydream has come true just around the corner from Stamen at Brightworks, an exploratory school whose method is rooted in project-based learning. Right now the kids' curriculum is focusing on maps, wrapping up a month-long study covering wayfinding, ecosystem maps, map projections, topology, and more. Stamen was invited to Brightworks for an afternoon visit, where we got to have a show-and-tell about the kinds of maps we make.

In our presentation, we covered a range of works, starting with maps and moving into data visualizations, answering questions from the (well-behaved, sweet and attentive) kids throughout the talk. Not surprisingly, the students made incredible observations and asked terrific questions. For maps.stamen.com, they wanted to know if we had made a watercolor painting of the whole world, which led to a discussion of mixing art and code, and creating CartoCSS (which they totally got). For Trees, Cabs, and Crime in San Francisco, they pointed out that there were no trees in Golden Gate Park. True! That's because our tree data came from Friends of the Urban Forest, and most of their trees are planted along sidewalks. We'd ask them questions too, like "What is data?" and "What are maps?" garnering responses like "Data is code" and "Data is information" and "a map is a picture of information." They put it together themselves that maps are pictures of data, and data can be anything, so you can make a map of anything too.

That connection made it easy for us to talk about data visualization works like A Day of Financial Transactions on NASDAQ and the Photo Sharing Explosions from Facebook Stories. They loved the explosive animations the best and made us replay the AbFab one a couple of times. Their favorite part in that one was the final explosion at 0:24, which they referred to as Pink Panther Pooping and then exploded in laughter. We have been giggling about it ever since!

When we were done, the kids thanked us and then ran off to finish their day. We got to take a moment to explore their space, an open warehouse (no classrooms!) which includes a communal meeting area, a kitchen, an art-making space, and a tinkering lab.

Yes tinkering lab! Above is a student (maybe 6 or 7 years old) showing us the 3D printer and laser cutter they have on hand. This is just after one of the Brightworks staff, Justine, shows us the shipping container outfitted with a table saw, where kids learn how to make things.

It's just across the way from the artist in residence, who is on hand to make art and work with the kids, letting them ask questions about how she works.

But we digress. From the maps. The kids have been making some:

As we were leaving, they were making even more!

The whole school is covered in them! It's like a mini-Stamen HQ, full of kids.

All these maps serve as inspiration for their round phase of work, where the kids take a whole month to work on a map-related project of their very own.

6-14 years old. Making maps. We can't wait for our Age Reverser to be finished so we can all go to school there! In the meantime, we're excited to visit Brightworks in June to see final presentations.

Apr 22, 2013

Grasses, Watermen & Bathymetry

By George

Today we're announcing a new cartography and storytelling project called Chesapeake Bay Grasses with the Chesapeake Bay Program (CBP), a partnership of federal, state, non-profit organizations and academic institutions dedicated to the protection and restoration of the Bay. Stamen has been working with the CBP to develop a visual story about the health of the Bay, based on a ton of scientific data that has been collected over the last 40 years or so.

The Chesapeake Bay is the largest estuary in the United States. (An estuary is a body of water where fresh and salt water mix.) It's about 200 miles long, with a total surface area of about 4,480 square miles, an average depth of about 21 feet, and the Bay and its tributaries have a shoreline longer than the entire U.S. west coast. Thanks to its watermen, the Bay produces about 500 million pounds of seafood per year, despite increasing pollution levels. You can read lots more interesting facts and figures on the Chesapeake Bay Program website.

Here's a (west-facing) map of what it looked like in the 1600s.

Nova Virginiae Tabula.

And a more detailed view of the Bay created in 1840 by Fielding Lucas Jr. in Baltimore:

Explore the map in greater detail on davidrumsey.com

As you can see, humans have been interested in the Bay for ages. In addition to the fisherpeople, citizens and pirates (!) that live or have lived on its shallow waters, there is also a ton of scientific research going on, much of which is lead by the CBP. The Bay is home to some 80,000 acres of bay grasses, and bay grass density over time is a great indicator of the overall health of the system. We worked with CBP and, in particular, Dr. Robert Orth from the Virginia Institute of Marine Science to create a map that helps show grass density across the Bay from 1984-2012. There's a general challenge for scientists around the world to mine their gigantic research datasets for insight and stories, and this project is a first round at help the CBP do that.

a 2013 map of the Chesapeake Bay

Bathymetry

Knowing that the project was focussed on a body of water, and not the land ended up being a strong influence on the cartography we designed. But first, we had to make sure we could "operate" the remarkably comprehensive bathymetry data that CBP has. Here are a few of Seth's first drawings, where we were trying to work out why there were odd "water-y alien crop circles" appearing in the middle of the bay:

Turns out that the alien-y circles are probably data artifacts (dartifacts?) from a long-term Bay soundings dataset that had been interpolated into a raster. Nathaniel ended up fixing this more or less by hand, cutting and pasting non-alien pieces back into the larger picture.

In addition to a bunch of public datasets online at ftp://ftp.chesapeakebay.net, CBP partner, NOAA also has a selection of bathymetry datas available, if you're interested to experiment with it.

Here are some highlights of the bathymetry:

Washington DC

washington

We gave the roads a very light-handed treatment because the water is the main feature of the map. There are enough roads to help locals find their way, and we labelled only a few of the major cities in the area, again to help people orient themselves, but never to overwhelm the water.

Tangier Island has been inhabited for years and years by fishermen and women. Crabs grow happily in the shallows around the islands, and you can see by comparing year on year, that the bay grasses where the crabs frolic have covered the area for the last 20 or so years. Dr. Orth explained it like this:

Tangier Island is home to one of the biggest grass beds in the Chesapeake Bay, and many island residents continue to make their livelihoods fishing and crabbing among the eelgrass and widgeongrass that grows here. While both of these species are typically found in high-salinity areas, they continue to thrive in the medium to low salinity of the region.

tangier
tangier

We're excited that the CBP now host the tiles themselves, and look forward to seeing this bathymetry used to help tell other stories with their data into the future.

Go have a look around!

Apr 17, 2013

Found on campus: a Stamen map of Palo Alto in tablecloth form

News flash: maps transform from paper things to digital things to things all over the map with maps all over them.

We were delighted to receive an invitation to the Stanford Cool Product Expo, a yearly showcase of innovative companies and products produced by Stanford's MBA students.

Stamen was one of 45 producers invited to show on campus, alongside Sifteo (fun game cubes), Node (modular sensing devices), Stealth HD (360 cameras), and Oru Kayak (a kayak that folds like origami into itself) (no really!). Not exactly makers of products ourselves (yet!), we invited collaborators Soft Cities and DODOcase to join us. More on that second collaboration coming soon. Promise!

We spent the entire day talking about maps. About why pretty maps matter. About how maps are for more than wayfinding and how data visualizations are also maps. About why people love maps and how they connect us. About how maps.stamen.com connects people to making their very own maps. About all of the places a map can take us, and all of the places where maps can go.

Like tablecloths.

You know this place? It's Palo Alto mapped with Toner on a tablecloth, inked by Nikki at Soft Cities for our expo table (as well as the table in the studio, above).(She can also make you one of Peoria, Paris or Phalaborwa with napkins to match.)

It's always inspiring to see people take Stamen's maps into the tangible world (see: sneakers and cupcakes) and something we'll be doing more in the future, as we have with projects like the Watercolor prints.

Watch this space! Where will the maps go next?