Tag Archives: ArcGIS

Matters relating to the ArcGIS platform from ESRI Inc.

Importing GeoJSON data in ArcGIS Javascript maps

One GIS issue we’re often faced with here at Cranfield University is interfacing data and systems that weren’t originally intended to work together seamlessly. That usually means loading data from legacy, or open source formats into proprietary systems, or vice versa. The challenge, on this occasion, was taking a GeoJSON feed of points and layering them onto a web map built with the ArcGIS JavaScript API. Some open source web mapping platforms, such as Leaflet, offer a simple library function for importing a GeoJSON feed as a vector layer for your map. No such luck in this case.

The solution is reasonably simple and essentially involves building a JSON object to look something like an ArcGIS FeatureService. In fact the FeautureLayer contructor can take a JSON object as a parameter instead of an online FeatureService, we just need to be aware of the exact data format that it’s expecting and programatically build that object up from the GeoJSON (or whatever other data) that we have.

First we create an empty feature collection:

var featureCollection = {
  "layerDefinition": null,
  "featureSet": {
    "features": [],
    "geometryType": "esriGeometryPoint"
  }
};

Then give the feature collection a layer definition:

featureCollection.layerDefinition = {
  "geometryType": "esriGeometryPoint",
  "objectIdField": "ObjectID",
  "drawingInfo": {
    "renderer": {
      "type": "simple",
      "symbol": {
        "type" : "esriSMS",
	"style" : "esriSMSCircle",
	"color" : [ 67, 100, 255, 70 ],
	"size" : 7
      }
    }
  },
  "fields": [{
      "name": "ObjectID",
      "alias": "ObjectID",
      "type": "esriFieldTypeOID"
    }, {
      "name": "some_other_field",
      "alias": "some_other_field",
      "type": "esriFieldTypeString"
    }
  ]
};

At this point our featureCollection object is in the correct format to be passed in to a FeatureLayer:

featureLayer = new FeatureLayer(featureCollection, {
  id: 'myFeatureLayer',
  mode: FeatureLayer.MODE_SNAPSHOT
});	

The Feature can now be added to the map. Note that the layer doesn’t actually contain any data at this stage, so wait for the layer to finish being added then begin the task of loading in the external data and adding points to our FeatureLayer. The lat/long coordinates of each point, i, that we’re after are stored within the GeoJSON object as
response.features[i].geometry.coordinates[0] and response.features[i].geometry.coordinates[1]

function requestData() {
  var requestHandle = esriRequest({
    url: "data/sample.json",
    callbackParamName: "jsoncallback"
  });
  requestHandle.then(requestSucceeded, requestFailed);
}

function requestSucceeded(response, io) {
  //loop through the items and add to the feature layer
  var features = [];
  array.forEach(response.features, function(item) {
    var attr = {};
    //pull in any additional attributes if required
    attr["some_other_field"] = item.properties.<some_chosen_field>;
		
    var geometry = new Point(item.geometry.coordinates[0], item.geometry.coordinates[1]);
    		
    var graphic = new Graphic(geometry);
    graphic.setAttributes(attr);
    features.push(graphic);
  });

  featureLayer.applyEdits(features, null, null);
}

function requestFailed(error) {
  console.log('failed');
}

Here is the resulting map:

GeoJSON data in ArcGIS Javascript map

GeoJSON data in ArcGIS Javascript map

Note that the code above requires the modules listed below and also assumes you already have your web map established and ready to accept layers.

"esri/map",
"esri/layers/FeatureLayer",
"esri/request",
"esri/geometry/Point",
"esri/graphic",
"dojo/on",
"dojo/_base/array",
"dojo/domReady!"

Extracting legend RGB values from ArcGIS Server using JavaScript

Most technical issues that developers face these days are relatively easily overcome with a bit of web research. It’s rare that you come across something that someone else hasn’t already solved and blogged about online. This week, however, we had a problem that didn’t appear to have a readily available solution online, hopefully this blog post might help someone else that is searching for a similar solution.

Whilst putting together a web mapping application using the ESRI JavaScript API, we needed to be able to pull out the red, green and blue (RGB) components of the legend colours from a map layer. Some research revealed a few solutions in Visual Basic that could be run against ArcGIS Desktop, but we were after something we could run via the web, ideally from JavaScript. The ESRI API offers the esri.dijit.Legend class, which does a good job of pulling all the correct data from your map layer on ArcGIS Server and rendering a full legend on your page. However, it only offers startup(), refresh() and destroy() methods, with no way of picking the legend apart or interrogating its individual components. We figured that, in order to render the legend in HTML, ArcGIS Server must be making the information we wanted available to this class somehow.

The solution lies in the fact that ArcGIS Server makes all of this information available as part of one large layer description, in JSON format. This can be accessed over the web from your ArcGIS Server using the following URL:
http://<arcgis_server_name>/arcgis/rest/services/<folder>/<service>/MapServer/<layer>?f=json
which produces something like this

{"currentVersion":10.1,"id":0,"name":"CF1_ISISSoilMap","type":"Feature
 Layer","description":"","definitionExpression":"",
"geometryType":"esriGeometryPolygon","copyrightText":"",
"parentLayer":null,"subLayers":[],"minScale":0,"maxScale":0,
"drawingInfo":{"renderer":{"type":"uniqueValue","field1":
"Association_Symbol","field2":null,
"field3":null,"fieldDelimiter":", ","defaultSymbol":null,
"defaultLabel":null,"uniqueValueInfos":[{"symbol":{"type":"esriSFS"
...

At first this isn’t easily readable. Changing the URL parameter from f=json to f=pjson formats things a little better on the page, which helps:
http://<arcgis_server_name>/arcgis/rest/services/<folder>/<service>/MapServer/<layer>?f=pjson

{
 "currentVersion": 10.1,
 "id": 0,
 "name": "CF1_ISISSoilMap",
 "type": "Feature Layer",
 "description": "",
 "definitionExpression": "",
 "geometryType": "esriGeometryPolygon",
 "copyrightText": "",
 "parentLayer": null,
 "subLayers": [],
 "minScale": 0,
 "maxScale": 0,
 "drawingInfo": {
  "renderer": {
   "type": "uniqueValue",
   "field1": "Association_Symbol",
   "field2": null,
   "field3": null,
   "fieldDelimiter": ", ",
   "defaultSymbol": null,
   "defaultLabel": null,
   "uniqueValueInfos": [
    {
     "symbol": {
...

This might be fine for small amounts of JSON data, but still wasn’t that easy to navigate for the 3500 lines of text that was being output for the layer in question.

There are a number of excellent desktop and web based tools for making JSON data easier to read, a Google search for “JSON Viewer” will bring up a wide selection. One particular favourite of ours is http://jsonviewer.stack.hu/. Pasting the layer description data in, we were able to drill down into the various objects until we found exactly what we were looking for; the legend RGB values.

JSON visualiser

JSON visualiser

The section we’re interested in is:
JSON.drawingInfo.renderer.uniqueValueInfos
which is an array of each of the legend items. Each object in the array contains information on the legend label, style, colour and more. The red, green and blue component values that we were after are stored as the following, respectively:
JSON.drawingInfo.renderer.uniqueValueInfos[i].symbol.color[0]
JSON.drawingInfo.renderer.uniqueValueInfos[i].symbol.color[1]
JSON.drawingInfo.renderer.uniqueValueInfos[i].symbol.color[2]

with JSON.drawingInfo.renderer.uniqueValueInfos[i].symbol.color[3] holding an opacity value. All are integers from 0-255. In the examples above, i refers to the index of a particular legend item.

Other useful legend data include:
JSON.drawingInfo.renderer.uniqueValueInfos[i].label
JSON.drawingInfo.renderer.uniqueValueInfos[i].description

You could use the data above in a live JavaScript application to interrogate ArcGIS Server on the fly. Note that you should use the f=json parameter if you do this, for performance reasons. However, in this particular instance, we wanted to extract these values into a CSV file for storage in a database elsewhere. There was one small stumbling block that needed to be overcome and that was that not all legend items contained simple block colour, some were patterned or used repeated images instead of a plain colour fill. Luckily, the JSON data helped us filter these out by providing the following:
JSON.drawingInfo.renderer.uniqueValueInfos[i].symbol.type
For block colour legend entries, this variable is set to “esriSFS”. In this particular case, we were able to ignore any entries that weren’t equal to esriSFS.

Here is the JavaScript function for producing the required CSV. Note that this example makes use of the native JSON.parse() function which doesn’t work in some older versions of IE. For full browser compatibility, the DOJO and JQuery libraries offer suitable cross-browser alternatives.

function getRGB(){
  var XMLRequest = new XMLHttpRequest();
  var JSONURL = "<location of your JSON data>";
  XMLRequest.open("GET",JSONURL,false);
  XMLRequest.send();
  var retrievedJSON = XMLRequest.responseText;
  var retrievedObj;
  retrievedObj = JSON.parse(retrievedJSON);

  var myHTML = "Label,Red,Green,Blue<br />\n";
  for (var i=0;i<retrievedObj.drawingInfo.renderer.uniqueValueInfos.length;i++){
    myHTML += retrievedObj.drawingInfo.renderer.uniqueValueInfos[i].label;
    if (retrievedObj.drawingInfo.renderer.uniqueValueInfos[i].symbol.type == "esriSFS")
    {
      myHTML += "," + retrievedObj.drawingInfo.renderer.uniqueValueInfos[i].symbol.color[0];
      myHTML += "," + retrievedObj.drawingInfo.renderer.uniqueValueInfos[i].symbol.color[1];
      myHTML += "," + retrievedObj.drawingInfo.renderer.uniqueValueInfos[i].symbol.color[2];
    }
    myHTML += "<br />\n";
  }
  document.getElementById("RGBValues").innerHTML = myHTML;
}

With the following <div> required further down in the <body> of the page:

<div id="RGBValues">
  RGB Values to go here - something went wrong if you can see this.
</div>

The output looked something like this, which was copied and pasted into Excel for tweaking and then imported into our database.

CSV data output to browser

CSV data output to browser

ArcGIS Utilities MXDDoctor and DocDefragmenter

mxd_doctorA number of users of ArcGIS here at Cranfield University have been recently experiencing similar issues, whereby for no obvious reasons the ArcGIS ‘mxd’ project file suddenly starts to balloon in size. A recent example was a simple project file that became 240Mb in size.

Another side effect of this, and in fact the issue that drew this to our attention, was that GIS projects could start to take 5 or 10 minutes to save. In effect we believe this was due to the size of the project file.

Since this is clearly affecting a number of our users, and apparently we are not alone (see http://forums.arcgis.com/threads/60696-Very-large-MXD-file-sizes-in-10.1), it was investigated. Amongst the solutions discovered we were able to find out about, and so report here, the powerful utilities produced by ESRI entitled MXDDoctor and DocFragmenter (link).

The MXDDoctor helps analyse and fix damaged and potentially corrupted mxd project files. By contrast the DocDefragmenter can realign the data held within an mxd file. The good news is that if you have ArcGIS, you also have these utilities. They are stored in the tools folder of your installation (e.g. C:\Program Files (x86)\ArcGIS\Desktop10.1\Tools). Make sure you have a backup of the MXD file first, close ArcGIS, and then give them a try. For the particular mxd we investigated, the original 240Mb filesize was cut dramatically to 415Kb by the MXDDoctor, and to 159Mb by the DocDefragmenter – clearly one should select and use the tool best suited to the specific circumstance.

As an epilogue to the issue of long project file save times, it also seems that a potential cause of such issues may lie in the way that ArcGIS saves and stores geoprocessing tasks that are, as yet, not run. In ArcCatalog or ArcGIS, following the menu Geoprocessing -> Results shows the ‘Results’ dialogue. Here, along with the options ‘Current Session’ and ‘Shared’, you may also see ‘Previous Session’. Here, under the ‘Not Run’ option, you may find a substantial number of jobs. You may find you can right mouse click this category and delete all the jobs. Doing so seemed to improve the performance as well as using MXDDoctor.