tecznotes
Michal Migurski's notebook, listening post, and soapbox. Subscribe to 
this blog.
Check out the rest of my site as well.
Jul 28, 2016 11:07pm
openstreetmap: robots, crisis, and craft mappers
The OpenStreetMap community is at a crossroads, with some important choices on where it might choose to head next. I spent last weekend in Seattle at the annual U.S. State Of The Map conference, and observed a few forces acting on OSM’s center of gravity.
I see three different movements within OpenStreetMap: mapping by robots, intensive crisis mapping in remote areas, and local craft mapping where technologists live. The first two represent an exciting future for OSM, while the third could doom it to irrelevance.
The OpenStreetMap Foundation should make two changes that will help crisis responders and robot mappers guide OSM into the future: improve diversity and inclusion efforts, and clarify the intent of OSM’s license, the ODbL.
Robot Mappers
Engineers from Facebook showed up to talk about how machine-learning and artificial intelligence (“robot”) techniques might help them produce better maps. Facebook has been collaborating for the past year with another SOTMUS attendee, DigitalGlobe, to consume and analyze high-resolution satellite imagery searching for settled areas as part of its effort to expand internet connectivity.
However, there are still parts of the world in which the map quality varies. Frequent road development and changes can also make mapping challenging, even for developed countries. … In partnership with DigitalGlobe, we are currently researching how to solve this problem by using a high resolution satellite imagery (up to 30cm per pixel). … For small geographical areas, this technique has allowed our team to contribute additional secondary and residential roads to OSM, offering a noticeable improvement in the level of details of the map.
OSM has long had a difficult relationship with so-called “armchair mapping,” and Facebook’s efforts here are a quantum leap in seeing from a distance. This form of mapping typically requires the use of third party data. For sources such as non-public-domain satellite imagery, robot mappers must be sure that licenses are compatible with derived data and OSM’s own ODbL license. Copyright concerns can make or break any such effort. Fortunately, OSM has a sufficiently high profile that contributors rarely attempt to undermine the ODbL directly. Instead the choose to cooperate with its terms, to the extent they understand the license.
Crisis Mappers
Meanwhile, representatives of numerous crisis mapping organizations showed to talk about the use of OSM for mapping vulnerable, typically remote populations. Dale Kunce of the Red Cross Missing Maps project gave the second day keynote, while Lindsey Jacks gave an update on her work with Field Papers (Stamen’s ongoing product I originally called Walking Papers) and numerous members of organizations like Humanitarian OpenStreetMap Team (HOT), Digital Democracy, and others attending workshops on collecting data.
Disaster response and crisis mapping organizations take a more direct, on-the-ground approach to map features that can’t be seen remotely or require local knowledge to interpret. While these efforts have often used remote data, as in the satellite-aided Haiti earthquake response in 2010, that data has always been paired with ground efforts in the area concerned.
When major disaster strikes anywhere in the world, HOT rallies a huge network of volunteers to create, online, the maps that enable responders to reach those in need. … HOT supports community mapping projects around the world and assists people to create their own maps for socio-economic development and disaster preparedness.
The populations most in need of crisis-response mapping efforts are typically furthest from OSM’s W.E.I.R.D. founding core. Such efforts will succeed or fail based on their participation. OSM should do a better job of welcoming them.
Craft Mappers
Historically, OpenStreetMap activity took place in and around the home areas of OSM project members, as a kind of weekend craft gathering winding up in a local pub. OSM originated from a mapping party model. Western European countries like England, Germany, and France achieved high coverage density early in the project’s history due to active local mappers carrying GPS units, riding bikes to collect data, and getting together at a pub afterward. Mapping pubs and similar amenities was a cultural touchstone for the project’s founding participants. The project has always featured better-quality data in areas where these craft mappers lived, for example near universities with computer science or information technology programs.
Many historic OSM tendencies, such as aversion to large-scale imports and a distinctly individualist technical worldview, are the result of this origin. They mirror the histories of other open source and open data projects, which often start as itch-scratching projects by enthusiastic nerds. Former OSM Foundation board member Henk Hoff’s 2009 “My Map” keynote is a great example of OSM’s early focus on areas local to individual mappers. The founding work of this branch of the community treats the project as a kind of large-scale hobby, like craft brewing or model railroading.
At its current stage of development, OSM’s public communications channels seem to be divided amongst these communities. I heard much frustration from crisis mappers about the craft-style focus of the international State Of The Map conference in Brussels later this year, while the hostility of the public OSM-Talk mailing list to newcomers of any kind has been a running joke for a decade. The robot mappers show up for conferences but engage in a limited way dictated by the demands of their jobs. Craft mapping remains the heart of the project, potentially due to a passive Foundation board who’ve let outdated behaviors go unexamined.
It’s a big downer to see a fascinating community mutely sitting out important discussions and decisions about the community’s future. Left to the craft wing, OSM will slide into weekend irrelevance within 5-10 years.
Two Modest Proposals
Two linked efforts would help address the needs of the crisis response and robot mapping communities.
First, U.S. technical gatherings like PyCon have been invigorated over the past years by codes of conduct and other mechanisms intended to welcome new participants from under-represented communities. State Of The Map US has done a great job at this, but the international conference and foundation seem to be engaging only in passive efforts, if any. A prominent code of conduct for in-person and online events would bring OSM in-line with advances in other technology forums, who have chosen to value the contributions of diverse participants. With the appearance of crisis and disaster mapping on the OSM landscape over the past five years, we need to appeal to people of color disproportionately affected by crises and disasters, so they understand they are welcome within OSM’s core. Ignoring this or settling for “#FFFFFF Diversity” is a copout.
Ashe Dryden says this about CoC’s at events with longtime friends running the show:
We focus specifically on what isn't allowed and what violating those rules would mean so there is no gray area, no guessing, no pushing boundaries to see what will happen. "Be nice" or "Be an adult" doesn't inform well enough about what is expected if one attendee's idea of niceness or professionalism are vastly different than another's. On top of that, "be excellent to each other" has a poor track record. You may have been running an event for a long time and many of the attendees feel they are "like family", but it actually makes the idea of an incident happening at the event even scarier.
Second, the license needs to be publicly and visibly explained and defended for the benefit of large-scale and robot participants. ODbL FUD (“fear, uncertainty, and doubt”) is a grand tradition since the community switched licenses from CC-BY-SA to ODbL in 2012. I support the share-alike and attribution goals of the license. But passive communication about its intent and use has left the door wide open for unhelpful criticisms. OSM Foundation publishes community guidelines on a separate wiki rather than a proper website. It’s not enough: the license needs to be promoted, defended, and consistently reaffirmed. Putting it under active discussion may even make it possible to adapt to new needs via a mechanism like Steve Coast’s license ascent, where “work starts out under a restrictive and painful license and over time makes its way into the public domain.”
I’ve struggled to write this post without overusing the word “actively,” but it’s the heart of what I’m suggesting. OSMF Board has been at best a background observer of project progress, while OSM itself has slowly moved along Simon Wardley’s “evolution axis” from a curiosity to a utility:
A utility like today’s OSM requires a different form of leadership than the uncharted and transitional OSM’s of 2005-2010. There are substantial businesses and international efforts awkwardly balanced on a project still being run like a community garden, without visible strategy or leadership.
Thanks to Nelson, Mike, Kate, and Randy for their input on earlier drafts of this post. I am an employee of Mapzen, a Samsung-owned company that features OSM and other open data in our maps, search, mobility, and data products.
subscribe to this site.
|
contact Michal Migurski
Comments (24)
Why do robot mappers have to edit OpenStreetMap? Why not start a project of their own? OSM has already done the hardest parts - the technical groundwork for distributed editing, the tagging-scheme (much derided though it may be, it works) and proven the idea of a quality crowd-sourced geographic database. Craft mappers, if that somewhat-offensive term is what you wish to define the original core of OSM editors and their latter followers, have doubts about the quality of automated mapping efforts just as we have doubts about the quality of armchair edits. Imports give good coverage but introduce errors originating in other datasets. Armchair editors can make gross mistakes but their impact is limited in scope. Automated editing might be very good at discerning geographic features and will tend to have a high level of consistency due to their automated nature. However their scope will mean that errors when they inevitably do occur will be widespread, not limited in scope. If craft mappers really are such undesirables, and automated mapping is the future, why not fork and create a new project in which the backwards ways of the "craft mapper" are not permitted. Then everyone can see which project is more successful in the long term.
Posted by Alex on Friday, July 29 2016 4:34pm UTC
Thanks for your comment Alex! The power of OSM is its unified dataset. Forking off a new project with a similar remit would lead to a weaker position for both. Justin O'Beirne has written about the emergence of candidates for what he calls a "universal map" (http://www.justinobeirne.com/essay/the-universal-map) and I think that a unified OSM welcoming to participants at a variety of scales makes OSM a contender alongside commercial efforts like Apple and Google. During the license debates of 2007-2012, many angry mappers suggested forking off a CC-BY-SA project and their efforts essentially fizzled while OSM moved into the future with an improved license. I understand the doubts of about robot and armchair edits, and I’m making the argument that they are rooted in the concerns of 2005-2010 and should no longer take precedence over new opportunities. I hope it's clear that I’m pointing to _two_ constituencies rather than one: large-scale automated inputs combined with on-the-ground targeted editing from organizations like HOT or Red Cross. It’s not just Facebook showing up with a bag of machine-learned data.
Posted by Michal Migurski on Friday, July 29 2016 4:54pm UTC
Mike, the first thought I had when reading this is that this is just "Americans vs Europeans, episode 735". The overwhelming percentage of data in the US already comes from imports and not from what you belittle as yesterday's craft mappers. Perhaps your call for more diversity could also lead to a better understanding, on your part, of the rest of the world, rather than proclaiming that the American way is the future and everything else is stuff from yesterday. I don't know if you noticed, but everything that you identified as positive and forward-looking in your post comes from the USA. I'm also very tired of the drive-by-shooting style of discrediting mailing lists and other forms of established communication. There's tons of ways for OSM-interested people to meet and talk, and there are people on the OSM mailing lists for whom the list is their peer group and who like it there. What do you gain from ruining it for them? What does it do for diversity if you stand on your little soap box and shout out to the world: "Look how bad these people are, they don't even notice how they're bathing in poison", or whatever other "running joke" you circulate among your friends. If the divisiveness you're practicing here (some people = forward thinking, other people = stuck in the past; some people = inclusive, other people = stuck on evil mailing lists) is a hallmark of your brand of diversity then perhaps I'd rather have another one.
Posted by Frederik Ramm on Friday, July 29 2016 8:48pm UTC
Nice to hear from you Frederik! HOT OSM works around the world, so I don't believe that it's a US-exclusive organization. Regarding the mailing list, I think it’s absolutely an unfriendly environment for newcomers. Mikel Maron attempted to moderate it many years ago; I’m uncertain what the status of that effort is, maybe the list has improved? I’ve made occasional posts in recent years, but the list itself is not maintained as a welcoming community space. For an example of what that means, I would offer Metafilter.com’s proactive moderation activities, with a rotating group of fulltime staff explaining their decisions as they make them. Here are three examples from the recent multi-thousand-comment DNC thread series: http://www.metafilter.com/161242/430PM-On-a-Wednesday-Philadelphia-PA-The-Gang-Rigs-the-DNC-Primary#6639664 http://www.metafilter.com/161242/430PM-On-a-Wednesday-Philadelphia-PA-The-Gang-Rigs-the-DNC-Primary#6639717 http://www.metafilter.com/161219/Its-hot-as-hell-in-Philadelphia#6632798 While there are people on the OSM mailing lists for whom the list is their peer group and who like it there, it is also a public space and representative of the project as a whole. The choice to treat OSM as a friend group is precisely the risk that Ashe Dryden identifies in the CoC excerpt I included above, and a convenient example of how the project is choosing to risk its future. For the robot and crisis mappers of the world, OSM is work. The mailing lists are an unprofessional environment.
Posted by Michal Migurski on Friday, July 29 2016 11:12pm UTC
Different geographies require different approaches to mapping and this will be for each community to adopt what they feel works best. In India for instance, where safe water and reliable electricity are still a struggle for a large portion of the population, nobody really cares if some digital map is open or not. So who exactly is going to trace the 4.5 million kms of roads onto OSM? Machines should do what they are good at - automate repeated tasks at a lower error rate than a human being. Humans will always be needed to add our observations using intelligence with a lower error rate than robots. Maybe we should just call this computer aided mapping to avoid people confusing people that this will wipe out local mappers, if anything this should encourage building more grassroots mappers if done right.
Posted by Arun Ganesh on Sunday, July 31 2016 12:36pm UTC
This is an interesting post, Mike, but genuinely and surprisingly mistaken about some of the basics, and easily sidetracked by emotional ephemera (e.g. the implementation detail that the OSM Foundation uses Mediawiki rather than, say, Wordpress as their CMS). In 2016 the mailing lists are an irrelevance to most people. As a quick reflection of the amount of activity that's routed through them, look at the filesizes of each month's archives at https://lists.openstreetmap.org/pipermail/talk/ . For a long time it's been hovering around 100KB a month. In 2008/2009 it was 400-500KB a month. Most people simply don't use them or even know they exist. They are not an indicator of anything significant. More importantly, the OSM mapping community outside the US has never been limited to "technologists" (I grant you that in the US it largely is). It was the generalist university cities in Britain that were mapped first (Oxford and Cambridge), not those with strengths in IT (Warwick and St Andrews). "Craft brewing" and "model railroading" are much closer to the OSM demographic. The OSM map of Europe has been moved forward by enthusiasts who feel comfortable with technology: cyclists and hikers who want a map better than Google can give them, civic-pride types who want their town to be represented well, railfans and roadgeeks and canallers. Technologists, per se, are far from dominant in OSM in Europe, as you can tell by the fact we've been happy with the same "good enough" API since 2009. (This does mean that OSM is a very middle-class map, majoring on middle-class preoccupations in middle-class areas. The most interesting thinking on diversity in OSM has long come from Muki Haklay at UCL, who first noted a "digital divide" in 2008: https://povesham.wordpress.com/2008/08/07/osm-quality-evaluation/ . One of the biggest challenges for OSM is how, in a world where OSM is becoming part of real service delivery, we break out of that socio-economic bracket into poorer areas.) Robot mapping is terrifically interesting but no panacea. At best, robot mapping can only get you an openly-licensed clone of Google or TomTom's dataset, and probably a worse one. Trying to outspend Google on mapping technology and AI is a fool's game, as for that matter is trying to engage more Mechanical Turks than Google. It may produce a lucrative dataset for services-on-OSM firms and for those companies big enough to build their own solutions without engaging such a firm, but it's no future for OSM. Because OSM has always been more than just open map data: it's better map data. It's more than you can get simply by applying robots and manpower. It is local knowledge, agglomerated. I think it was Heather Leson who wrote a superb piece last year about how the real success for humanitarian mapping is when it progresses from "we map this for you" to "we enable you to map this for yourselves", from "we choose what your map shows" to "you map what's important to you". This is the point you make when writing about crisis mappers, so I am puzzled why you draw a distinction between that and the third point, because that is what craft mapping has always been about. Locals mapping what's important to them. That is why people in the UK, say, continue to build with OSM even though we now have a more permissively licensed dataset of excellent, consistent quality (Ordnance Survey OpenData): because OSM is the map that reflects their lives. Finally, I'd just make one simple comment. The way OSM functions might be counter-intuitive... but maybe it actually works. Maybe empowering people to build something good has proved the best way to create a comprehensive and rich map, rather than damning them with outdated stereotypes about mapping parties, pubs, and "community gardens". Maybe, to shape a better future for OSM, we should start by looking at what we've done right so far. Maybe we knew what we were doing.
Posted by Richard Fairhurst on Sunday, July 31 2016 1:18pm UTC
Richard, thanks so much for commenting! You’re saying a few related things. I think the two critical distinctions between crisis and craft mappers are that the former are more focused on partnerships than the latter, and they also work under unique constraints. Projects like the HOT Tasking Manager (http://tasks.hotosm.org), Field Papers, and the periodic donation of satellite imagery during disasters are all responses to the remote, large-scale, technologically limited, and complex needs of crisis response. Crisis mappers also coordinate with organizations like the World Bank or Red Cross, unlike local craft mappers. From a political and organizational perspective, this is a wholly different kind activity from the middle class history you describe. A few responses to my post have focused on the robot side of the story, which is too bad. These two new communities of robot and crisis mappers work in tandem, to extend the breadth and depth of OSM far beyond its traditional university town, middle class roots. This is the exact way in which OSM can rise to meet the challenge of breaking out into broader areas, and it’s a problem that OSM leadership does not effectively advocate for this shift. An openly-licensed TomTom clone, for example, would be amazing in certain areas as Arun says in the comment above. OSM was started by middle-class geeks who breathed life into the project, but that history is now holding OSM back by preventing it from growing the leadership and strategic abilities it needs. All OSM groups are “technologists” at some level, but their interests differ. To return to the Talk list as an example, right here we have current OSM Foundation board member Frederik Ramm saying it’s just a peer group, you as a former Foundation board member saying it’s shrunk by 80% anyway, and linked in the Facebook forum post former OSM Foundation chairman Simon Poole asking Facebook to take their announcement off of Forums and over to Talk. These are different and incompatible visions for the purpose served by Talk: Frederik’s “if you don’t like it, leave” vs. Simon’s suggestion that it’s still the correct point of entry. Lacking agreement on the strategic purpose of the Talk list makes it ungovernable, and also precludes the creation of an official channel that an organization like Facebook can use when approaching with news of a large-scale project. This lack of a consistent “face” is the precise risk to OSM that I’m trying to highlight. I’ve been a part of OSM since 2006, but it’s always been in the context of my work and I’ve never had the energy or desire to sustain participation in Talk. You mention the challenge of breaking out of a middle-class socio-economic bracket; getting serious about diversity & inclusion is exactly how it gets done. If Talk is indeed no longer significant, and no one is able to maintain quality of the discussion by means of a Code of Conduct, and it’s believed to be a hostile environment, it should be shut down. For the same reason, the choice of publishing system for the OSM Foundation (Mediawiki vs. Wordpress) is a significant signal. When I see a URL like http://wiki.osmfoundation.org/wiki/Main_Page with “wiki” included twice, I take it as a signal that the page contents should be treated with the same caution that I bring to a Wikipedia article. Maybe it’s been vandalized or abandoned? Maybe it’s not “official”? For risk-averse organizations who might be worried about the license because they’ve heard FUDdy talks at conferences, OSM Foundation should be presenting a clear, confident message about what the license supports and how it’s enforced. A page like http://wiki.osmfoundation.org/wiki/License is a great start, but I regularly encounter worried questions about what share-alike might truly mean in a variety of cases, and there’s very little precedent to draw on. Anyway, I’m really happy that people like yourself and Frederick are interested enough in the contents of this post to take the time to respond. Thank you.
Posted by Michal Migurski on Sunday, July 31 2016 4:22pm UTC
Dear Mr Migurski Could you please at least make an attempt to be factual? At no time did I ask or suggest to Facebook to "to take their announcement off of Forums". I did suggest that the talk mailing list and/or a OSM diary post would be more suitable place than the rather obscure forum they posted to (because of the larger and more diverse audience they could reach that way). Kind regards Simon Poole
Posted by Simon Poole on Sunday, July 31 2016 11:53pm UTC
Thanks for the correction Simon, I apologize for misinterpreting your intent.
Posted by Michal Migurski on Monday, August 1 2016 12:41am UTC
Others have addressed other parts of your post, so I am not trying to comment on everything here, just comment on one point, the diversity of OpenStreetMap. OpenStreetMap is probably the most diverse map we have on the planet. It has diversity built into its core. The openness of the data and the tagging scheme together with the openness of the software used to create and use OSM is unique in the world. This diversity is a result of the technology and of the ground rules built into OSM, but it is also a result of the craft mapper you so despise. OSM is not a manufactoring sweat shop and not a robot factory, it is created by individuals who care. And that's why it shines and that's why it is diverse. Sure, there is more we can do to make it even more diverse and we should work on that. But I can't understand why replacing this diversity with algorithms invented by predominantly white male Facebook engineers is better or why it is better to replace this by a mapping army directed by single-minded goals of some Red Cross project or another. I welcome the Red Cross or anybody else as participant in our diverse project and I welcome help from machines in our project. But just looking at our map I am confident, that we have done more things right then wrong and that our methods do work.
Posted by Jochen Topf on Monday, August 1 2016 1:08pm UTC
Hi Jochen, it’s nice to hear from you! I think you may be misunderstanding “diversity” as I use it in this post. There’s a stub article in the OSM wiki which defines it in terms of gender, race, age, and other protected classes: https://wiki.openstreetmap.org/wiki/Diversity — that’s the definition I’m using. The diversity of data in the OSM database is significant and important, but it’s not relevant to the involvement of under-represented minorities that I’d like to see change. My colleague and US Foundation board member Alyssa Wright has offered numerous talks and interviews on the topic linked from that page, such as this: http://www.fastcoexist.com/3032543/how-street-maps-can-be-sexist Right now, the diversity of OSM is terrible. Two OSM power centers that I’m familiar with are the board and the sysadmins, each of which overrepresent male (and I suspect white) participants by ratios of 22/25 and 7/8 respectively: http://wiki.openstreetmap.org/wiki/Foundation#History http://wiki.openstreetmap.org/wiki/System_Administrators I know that Andy Allan has made a very recent effort to open up the operations of OSM to newcomers (https://blog.gravitystorm.co.uk/2016/07/28/getting-involved-in-owg/ is excellent), and I think that it’s critically important to make Andy’s work central to the project through explicit expectations and outreach, ensuring a better variety of participants. The core technology teams in particular are made up of extremely old-school OSM project members; at this stage of growth I’d hope to see a mix of paid staff and junior interns as a sign that diversity is taken seriously. I don’t have any particular animosity toward the group I’ve called “craft mappers,” nor should we replace local knowledge with robots. As Richard points out above, the craft mappers have gotten us pretty far. The robots only make sense when paired with local knowledge, such as crisis mapper efforts. However, OSM is singularly closed to opinions from outside, and insists on propping up hostile spaces like Talk and hostile reactions to suggestions of change. I’ve been here ten years. Every time a new direction of any kind is proposed, arguments like Frederik’s and misunderstandings like yours grind the conversation to a halt. Tom Lee’s recent blog post in response to this one says more: “It is not a very democratic circumstance, but no one claimed otherwise: OpenStreetMap is often referred to as a ‘do-ocracy’, implying that those who contribute work have outsize influence. Fair enough, but this means that when the work that must be done includes discussion, conflict that amounts to a veto is the overwhelmingly likely outcome.” — http://tomlee.wtf/2016/07/31/introductory-openstreetmap-politics/ Thank you so much for your comment, I appreciate your input!
Posted by Michal Migurski on Monday, August 1 2016 4:07pm UTC
Hi Mike, what about POIs? Robot mappers can't do that (even if you feed them a street view for whole cities), crisis mappers don't do that (not visible on imagery). Craft mappers are obsolete. What other options do we have? Or POIs are irrelevant to the better OSM?
Posted by Ilya Zverev on Tuesday, August 2 2016 9:12am UTC
It is very interesting to read your piece, and the one by Tony Lee you linked to. The inroad to OSM seems to determine your vision. My inroad to OSM was experimentation, which probably determines mine. You seem to think we are making a map. I don't believe we are. We're building a mapping community. There is only a place for data to be introduced to OSM if it has a net effect of enlarging the community. For example: a nasty import in Mumbai killed growing a community of local craftmappers (I actually like the sound of that), until one very courageous local fixed it up entirely. Since then a local community started growing. A better import might attract more data users and not make contributing harder, and those are in fact welcome. Your focus on data (as opposed to community) ignores the empowering effect of building your own map. Some time ago, there was a very interesting (and "surprisingly civilized") discussion of how crisis mapping takes away the opportunity of local people to build their own map from scratch [1]. While crisis trumps such sensitivities IMHO, there really is something to be said about the colonial effect you could have when enlightened Westerners come in to build the map the locals can't seem to build themselves. People in European countries, having gone through a traumatizing de-colonization and introspection process, are bound to be more sensitive to issues like this, than people in self-confident superpowers. I cannot understand how taking the power away from locals mapping their own neighborhoods towards any group of AI developers or NGO people can be interpreted as better diversity. Not all data need to be incorporated into OSM, at least not right now. For example, I really like the idea of openadresses. It allows for rapid integration of open datasets into something global, without the hassle of integrating with OSM data and conflating changes in the resultant OSM and official dataset. If the local OSM community is strong enough, the imported data could be forked off from the official one. And with enough programmers, it might be possible to conflate updates automatically. But there is little advantage to letting that data live in OSM and have it updated completely automatically. That leads to a data desert where no-one bothers to update it, as it will be done automatically after a while anyway. And that is the exact opposite of what we are trying to achieve. We don't want to wait for the government to decide an address is now real. We want to have a map that is as up to data as possible. We don't want to copy their errors as quickly as possible - we only want to map things that are sure to be correct. One could imagine similar projects which are fed by both OSM and AI datasets. For example, an openbuildings project, containing AI buildings, open data buildings and some OSM buildings. There is definitely use for AI aided mapping within OSM too, but it would be a tool for local craft mappers using robots - not mighty robots providing dearly needed data. Just like humans decide today which Mapillary or Strava data to incorporate, there would be humans validating whatever more advanced techniques come up. The contrast you make between "mapping your own neighborhood" and "mapping the world" is quite strange to me. The people I know at HOT are focused on creating neighborhood mappers everywhere in the world, not making the map NGO's need now and stopping there. HOT remote mapping is always embedded in an effort to build local communities. That's exactly why (at least in Belgium) us craftmappers are so active in supporting and organizing Missing Maps events. Again and again, I see people trying to put wedges between remote and local mapping, but that really is a false dichotomy - they are a continuum where both extremes are quite rare. The measure of success of remote mapping is the birth of a local community, not something like road network completeness. The map of Bolivia, for instance, was first made largely by outsiders (though often with some local knowledge). But then it became the best map of the country and started getting more local attention. Now the community is growing autonomously, and is possibly more active than the Belgian community. OSM is mapping the world one neighborhood at a time, not just mapping our own Western-European neighborhood and stopping there. Could it possibly be that because that model is taking longer to materialize in the United States (blame Tiger?) you think that model is dead? The culture clash mentioned between Europeans and Americans (when Frederik said that, he obviously wasn't talking about the places mapped, but the nationality of the people who tend to agree with your post) is also visible in the focus on CoC. That might be something big in America, but it really is something alien to me and other local OSM people I know. My lack of understanding might stem from culture too. Here in Belgium, rules are one thing, morality something completely different. Maybe because we've always seen the law as coming from the powers that be, and the powers that be as "not to be trusted". We're willing to copy-paste that CoC and follow them, as it probably won't make a difference in what we're doing anyway. But it's not something we'd make a big fuss about. On the other hand, we would be interested in evidence-based policy adjustments we could make. Any scientific research on the subject is more than welcome. Talking about diversity, since SOTM is in Belgium, we would have thought it a good time to break the English language monopoly. Unfortunately that didn't happen, because of budget constraints. When discussing your article with some mapper friends, we jokingly said we were willing to engage in discussions on diversity with anyone, as long as the discussion was in any another language than English. As for the SOTM program, do the crisis mappers you've talked to have a problem with sessions about community building in Africa, in Asia, in Latin America? There could have been more, but there is the HOT summit the day before in Brussels, and some were better placed there. Is it craftmapping when we have sessions with Telenav, Mapbox, American and international Red Cross, Development Seed, Geofabrik, CartoD, United Nations Human Settlements Programme, SNCF, HOT, MongoDB, Mapillary, Google, eHealth Africa, Peace Corps and USAID, or the Worldbank? Please ask people to aim before they shoot. A final remark as to "don't use a wiki about legal stuff". If someone can't trust an official wiki, how are they going to trust wiki geodata? The same goes for having a unified voice or talking space. We are chaotic, but that is not a weakness, it's a strength. OSM doesn't have the best hiking network in Belgium because that was the plan. We have it exactly because there was no plan. Giving the freedom to people to do what they want to do is the best way to get them to do it. If that means we're missing some opportunities for quick growth, so be it. It would only bring false expectations if we were to present ourselves as a well organized bunch. In the case of Facebook lost in the forum-desert, no harm done. Their post was picked up by the weeklyOSM crew [2], so they can consider the community informed. [1]: https://lists.openstreetmap.org/pipermail/talk/2015-June/073264.html [2]: http://www.weeklyosm.eu/archives/7896
Posted by joost schouppe on Tuesday, August 2 2016 3:12pm UTC
Joost -- as a somewhat bi-cultural (Dutch - US) individual I am happy to help you work through the challenges of establishing a CoC. I was involved in creating the initial one for SOTM US. Let me know, happy to help in any way I can.
Posted by Martijn on Tuesday, August 2 2016 10:21pm UTC
Ilya, nice to hear from you! POI mapping is completely compatible with crisis mapping. A pattern I’ve seen since the Haiti earthquake response is to use satellite imagery donations for an immediate rough pass, and then on-the-ground mappers for local points of interest. HOT used locals to record building construction materials in Indonesia and supported the work of students in Ulaanbaatar, for example. Many of the techniques are identical to craft mapping, just augmented with remote data and subject to equipment limits. Joost, thank you for your thoughtful comment as well. I don’t believe that I’m focusing on data to the exclusion of community, since neither of my proposals are technical in nature. Both are intended to present a confident, welcoming project to new participants who might represent large companies with access to advanced technologies or small communities partnered with organizations like HOT, Digital Democracy, and others. I agree that the use of CoC’s and other rules is something that’s emerged from primarily U.S. technology organizations, typically advanced by under-represented populations looking for assurance that their participation is safe from harassment. Ashe Dryden explains the background effectively: “The people most affected by harassing or assaulting behavior tend to be in the minority and are less likely to be visible. As high-profile members of our communities, setting the tone for the event up front is important. Having visible people of authority advocate for a safe space for them goes a long way. Sadly, the occurence of these incidents is high.” (https://www.ashedryden.com/blog/codes-of-conduct-101-faq) I suspect that Silicon Valley has focused attention on the economic power to be gained through access to advanced technology forums; people want in and feel shut out. I certainly don’t believe the one-neighborhood-at-a-time model is dead, but the particular historic characteristics of OSM’s founding participants sets the stage for major problems today. Having a leaderless foundation with a volunteer board limits long-term participation to those with the free time to pursue a hobby and engenders suspicion of mappers representing companies. Tom Lee says this better than I in his post, calling out the lack of comfort and process that OSM has for decision making. Technical matters are typically easy to handle and there’s no shortage of developers ready to write code or tend to servers, but equally-important social and legal concerns are kicked down the road. Thank you both for leaving your thoughts, Joost and Ilya!
Posted by Michal Migurski on Wednesday, August 3 2016 2:05am UTC
> Technical matters are typically easy to handle and there’s no shortage of developers ready to write code or tend to servers So many competent people queuing for the privilege of shoveling shit at server consoles, such a glut of developers effortlessly churning out functionality and ready to do so under the rightful guidance of good governance... C'mon - you now better: trivializing the technical enablers and feigning to ignore how rare they are and how fragile their motivation is under user pressure is a distorted view of operational reality in free software projects. Contributions have a variety of motivations but they are all about autonomous collaboration and therefore making contributors feel welcome. So, yes - a code of conduct and a process for dealing with abuse are necessary within the frame of the Openstreetmap project as a whole, which is not much: not much happens in the Openstreetmap project as a whole because it happens in a variety of decentralized projects whose only common feature is the Openstreetmap database. So to reflect the diversity of communities, contributors and motivations, it is within the realm of each organization that it is most important to codify conduct - because diversity in the Openstreetmap community is not just a diversity of individuals but a diversity of organizations where each individual finds the environment most amenable to his thriving. Anyway, on the title's topic of automation, I believe that automation vs. craft is a strawman argument. Openstreetmap is about human intelligence, so full automation is most definitely outside of its scope - it is about geographic information, not geographic data. But caricatural "craft" is also out - the tooling is getting increasingly sophisticated: automation under human control. Yes, automated tools will soon recognize highways and trace them effortlessly - even for example suggesting classification on the basis of width or surface and max speed on the basis of GPS traces... But then a human will review and edit - I have done enough of that on various sources to know that the value lies in humans making sense of the data to turn it into useful information... There is no automating that because the nature of the task is subjective interpretation within a human frame of reference. Automation vs. craft ? Both - in an integrated manner ! Likewise, Tom Lee's "Finding the nearest coffeeshop is a modest benefit compared to being pulled from a collapsed building by rescue workers" is also a strawman argument: adding POI on the ground and classifying buildings from imagery, even semi-automatically are not mutually exclusive propositions - different contributors will contribute whatever they want... Cooking stone soup is what Openstreetmap is about: https://en.wikipedia.org/wiki/Stone_Soup - like any free software project. When cooking stone soup, new contributors bring random ingredients, but existing contributors might object to them being added to the existing soup... Should Openstreetmap be led by its contributors or should a "humanitarian" political action group that claims to represent what they believe to be important stakeholders in the field of geographical data take a lead role ? I have an opinion about that but it is subjective as the question is political in nature... So the answer is a power struggle - not a mere optimization because the very goal is in question. Also, about HOT, I disagree with Joost Schouppe's "HOT remote mapping is always embedded in an effort to build local communities" - my experience in Senegal and Mali is that HOT remote mapping is mostly haphazard drive-by addition through their Tasking Manager, focused on showing off number of additions and sorely lacking in quality control and local knowledge. I do not believe that HOT is a good example of community development - but that is just the view from my window: maybe they are locally well integrated in other parts of the world. Finally, I find very strange that you underline that "the license needs to be promoted, defended, and consistently reaffirmed" - and then casually mention that is could be possible of selling it short as public domain... This is a very important reason why existing contributors wish to retain control: to avoid a takeover by "a different form of leadership than the uncharted and transitional OSM’s of 2005-2010" that might be tempted to bend to outside pressure to make the data available under a license friendly to non-cooperative end-users.
Posted by Jean-Marc Liotier on Thursday, August 4 2016 11:51am UTC
Jean-Marc, I appreciate your taking the time to read my post, but I do hope you'll quote my arguments completely -- the fragment you pulled really mischaracterizes the point I was making.
Posted by Tom Lee on Friday, August 5 2016 12:44am UTC
Nice to hear from you Jean-Marc, thanks for your thoughtful comments! I stand by my statement about the relative ease of handling technical matters in a project like OSM. I don’t wish to trivialize the many deep contributions of OSM’s developer community, but it’s evident to me that the desire to write code and set up servers is more plentiful in open-source projects than a willingness to manage volunteers or communicate plans. Your comment on the fragility of developer motivation does not match my experience: technical contributions in OSM are valorized while organizational ones are not. OSM suffers from an imbalance against soft skills and good management. Like you, I also believe that automation vs. craft is a strawman argument. I’ve said elsewhere in this thread that I’m disappointed it’s being used to misconstrue my point, which is to identify communities of OSM contributors distinct from the traditional craft-mapper consensus. Some of craft’s rough characteristics include rejection of remote mapping, preference for individualism, and rejection of “bureaucratic” or “political” process. Robot and crisis mappers have their own approaches, and preference for automation may be the least of them. A distinguishing characteristic may be a comfort with and desire for professionalization or even bureaucracy, if you agree with Jeffrey L. Pressman’s definition from his 1973 book Implementation (http://mike.teczno.com/notes/books/implementation.html, page 133): “The costs of bureaucracy may emerge in a different light when they are viewed as part of the price paid for predictability of agreement over time among diverse participants.” I hope for a future OSM that centralizes the power to welcome newcomers and the stewardship of the license. It will require a professional, political, strategic approach instead of the stone soup one. I don’t know if existing contributors feel so strongly about retaining control; we may find that they feel more strongly about adapting to future circumstances and turning those non-cooperative end-users into community members. I am glad you took the time to write, and I appreciated the chance to respond.
Posted by Michal Migurski on Friday, August 5 2016 7:13am UTC
Curious: How do 'robot' & 'crisis' mappers tagging the opening times of my local butcher or know the the bicycle track's surface is gravel? To check, are you http://www.openstreetmap.org/user/Michal%20Migurski ? No edits recorded in six years. if it's you, where are your public records of edits? Thanks Dave F.
Posted by Dave F. on Saturday, August 6 2016 11:34pm UTC
Hi Dave, I’m https://www.openstreetmap.org/user/migurski in OSM. To answer the other part of your question: many crisis mappers operate locally, like the four people in the photo above (I borrowed it from HOT's website header). They just happen to do so in the context of an organization like the Red Cross or HOT, with goals that may or may not be met by tagging business hours and bike lanes.
Posted by Michal Migurski on Sunday, August 7 2016 7:31am UTC
Well you've all lost me. Everything you have said has gone way over my head. As the mother of a UK OSM mapper (armchair and craft, I think) I was immediately on the defensive. I have no idea about any of the technicalites but I do know that the infectious enthusiasm of those you seem so against is what has made me a passionate champion of OSM. On top of that despite my technical limitations I was eager to take what were walking papers (now field papers) on holiday with me to "map" a village that had no house numbers, just house names. You see, not living in the US we do NOT have North & South X Street and house numbers that go from 1 to several thousand. on a perfect grid. Every chance I get I extol the virtiues of OSM and why people/companies should use/provide open data. It is the hard footwork of all those beer-swigging "surveyors" that got OSM off to such a good start. It is always easy for Johnny-come-Latelys to criticise how things have been done in the past. Play nicely children!!!!!!
Posted by Ruth on Friday, September 23 2016 8:19pm UTC
Thanks for your comment Ruth, and for your involvement in OSM! Interestingly, I don't think there's been much come-lately participation in this thread. I'm pleasantly surprised at how many long-time project participants have been motivated to respond.
Posted by Michal Migurski on Saturday, September 24 2016 6:57am UTC
I personally like your post; you have shared good insights and experiences. Keep it up.
Posted by affordable websites pakistan on Thursday, October 13 2016 7:10am UTC
This post is truly astonishing one! I was pleased to peruse this, all that much valuable.
Posted by Auditing Case Solutions on Thursday, October 13 2016 12:11pm UTC
Sorry, no new comments on old posts.