Category Archives: Insights for ArcGIS

Matters relating to the Insights platform for ArcGIS Online from ESRI Inc.

ESRI Insights for ArcGIS – 2

ArcGIS Enterprise: Installing ArcGIS Portal 10.5 Beta

Introduction

In the next part of our blog, the GIS team here at Cranfield University, are setting up the newly launched ESRI ArcGIS Insights app. We are documenting the process of installing and testing this software, adding some helpful commentary on the way that should hopefully help others tread the same path!

To get ESRI Insights for ArcGIS to work, there are a number of pre-requisites, which we will be installing step by step. These include:

  • ArcGIS Server 10.5Beta
  • Portal for ArcGIS
  • ArcGIS Web Adaptor
  • ArcGIS Data Store
  • An instance of MS SQLServer Database
  • A JDBC 4.0 Compliant driver

Having installed ArcGIS Server 10.5Beta successfully, we now need to install the Beta edition of ArcGIS 10.5 Portal. Portal for ArcGIS allows you to share maps, applications, and other geographic information with others, with the shared content being delivered through a website – which can then be customised as required.

ArcGIS Portal 10.5Beta

To get going with Portal, we copied over the Portal tarfile from the early adopter site ‘Portal_for_ArcGIS_Linux_105_beta1.tar.gz’, which contains the installation files for the Portal installation. This tar/gz file contains a folder called ‘Documentation’ – within which is a web/html set of instructions. We found it useful to extract these files off to a separate computer for consultation as the process unfolded.

From the earlier ArcGIS 10.5 Beta Server installation, we had already downloaded to our installation folder (our home directory on the test server) the sample provisioning authorisation file provided ‘ArcGISforServerAdvancedEnterprise_Server_105alpha.prvc’. We had edited the header for this file with our details, but apart from that we left the codes alone.

The next step was to unpack the installation media, thus:
~> gunzip Portal_for_ArcGIS_Linux_105_beta1.tar.gz
~> tar –xvf Portal_for_ArcGIS_Linux_105_beta1.tar

This creates a folder ‘PortalForArcGIS’ with all the installation media in it ready to go. From here, we ran the setup programme, thus:
~> cd PortalForArcGIS
~> ./Setup -l yes -m console

Note the use of the console mode installation

This seemed to run OK, but we noticed a warning flagged up.

========================================================================
Portal for ArcGIS 10.5 Diagnostic Tool
Hostname: SERVER NAME
========================================================================
DIAG000: Check for installation as root [PASSED]
DIAG001: Check for 64-bit architecture [PASSED]
DIAG002: Check OS version [PASSED]
DIAG003: Check hostname for invalid characters [PASSED]
DIAG005: Check system limits [PASSED]
DIAG004: Check installed packages [WARNING]
DIAG016: Check Portal for ArcGIS ports [PASSED]
DIAG024: Check localhost resolution [PASSED]
------------------------------------------------------------------------
There were 0 failure(s) and 1 warning(s) found:
WARNINGS:
------------------------------------------------------------------------
*** DIAG004: The following required packages were not found:
dos2unix

‘dos2unix’ huh? OK, so we will clearly need that too. The old PC/Unix text file end of line issue – this is nothing to do with ESRI’s software. Dos2Unix is just a useful utility to help move text files between the various end of line formats.

We installed the dos2unix tool by compiling the source code and installing it manually and testing it was working OK – but we continued to see the same warning message from Portal. Puzzling – then we read the warning again – it says ‘package’ not found. OK it is looking at the package manager inventory to see what is installed. We re-installed dos2unix with the package manager ‘yum’.

~> yum install dos2unix
Note that yum is the redhat equivalent of ‘apt-get install’
That all worked fine and the Portal check passed OK. The installation of Portal now continued. The terms and conditions are shown for review, and the default locations (home/arcgis) – we accepted these.

Portal Authorisation

As part of the installation, we provided the explicit path to the provisioning file made available as part of the beta download ‘PortalforArcGISServer_105alpha.prvc’. However, this was not accepted, and so we needed to try again using the script authorisation tool ‘authorizeSoftware’ provided in the installation. However, remembering we also had this issue when installing Server, and how we fixed that, we edited the provisioning file header with our own details, saving to a new file ‘PortalforArcGISServer_105alpha_edited.prvc’.

Note that using ‘authorizeSoftware -f’ allows a later application of a provisioning file to an installation, thus:
~/arcgis/portal/tools> ./authorizeSoftware -f /home/gisadmin/PortalforArcGISServer_105alpha_edited.prvc

~/arcgis/portal/tools> ./authorizeSoftware -s
--------------------------------------------------------------------------
Starting the ArcGIS Software Authorization Wizard

Run this script with -h for additional information.
--------------------------------------------------------------------------
Product Ver ECP# Expires
-------------------------------------------------
portal_1000 101 ecp906762680 30-jan-2017

[shows all software licenced]

So far so good, the software is all installed and licenced. Now time to fire up Portal for the first time:

We entered the URL of the server:
https://localhost:7443/arcgis/home
(substitute ‘localhost’ for the fully qualified URL of your server)

At first this address did not work. We realised that this was again as the firewall needed updating for this port number (7443), as it was blocking it by default. A quick few edits in ‘lokkit‘ later and we are up and running.

~> sudo lokkit --port= 7443:tcp --update

ArcGIS Portal

As a new install, we selected ‘Create new Portal’, then completed the form presented.

ArcGIS Portal

Web Adapter

ESRI say ‘ArcGIS Web Adaptor is a required component of Portal for ArcGIS which allows you to integrate your portal with your existing web server and your organization’s security mechanisms’, noting that ‘the ArcGIS Web Adaptor (Java Platform) on Linux allows you to integrate your existing Java-based web server with ArcGIS Server and Portal for ArcGIS.’ So we need to install Web Adapter too.

The gz/tar file for Web Adapter was downloaded and unpacked as with the earlier archive distribution files.

~> gunzip Web_Adaptor_Java_Linux_105_beta1.tar.gz
~> tar -xvf ./Web_Adaptor_Java_Linux_105_beta1.tar

This creates a folder ‘WebAdaptor’ with all the installation media in it ready to go. From here, we ran the setup programme, thus:
~> cd WebAdaptor
~> ./Setup -l yes -m silent
[ArcGIS 10.5 Web Adaptor (Java Platform) Installation Details]
UI Mode..................silent
Agreed to Esri License...yes
Installation Directory.../home/USER/arcgis/webadaptor10.5
Starting installation of ArcGIS 10.5 Web Adaptor (Java Platform)...
...ArcGIS 10.5 Web Adaptor (Java Platform) installation is complete.

Note the ‘-m’ silent mode setting used in the setup.

So at the end of this blog, we now have ArcGIS Server, ArcGIS Portal and the WebAdapter all up and running.

Lastly, we can make sure all the ports we need are opened correctly through the firewall.
~> sudo service iptables status

In the next blog, we will continue installing the other pre-requisite software tools for ESRI Insights for ArcGIS.

Thanks for reading!

ESRI Insights for ArcGIS – 1

ArcGIS Enterprise: Installing ArcGIS Server 10.5 Beta

Introduction

Keenly anticipated here at Cranfield University, is the newly launched ESRI Insights for ArcGIS app, part of the new ArcGIS Enterprise suite, which, amongst other things, can be deployed to explore the use of Hadoop/HDFS technologies with geospatial data – offering powerful spatial analytics capabilities to this data.

So what is Insights for ArcGIS? Well ESRI have advised us that “Insights for ArcGIS is an app that you access through ArcGIS Enterprise that allows you to perform exploratory and iterative data analysis. With a minimal drag-and-drop interface, you can answer questions with data from ArcGIS services, Excel spreadsheets, and data warehouses.” This sounds great – we are also interested in its stated ability to handle Big Data databases, offering for all these sources easy access to the most widely used GIS analysis tools. Insights for ArcGIS is designed to enable easy analysis of data, revealing inherent patterns to gain situational awareness, as well as providing tools to explore what-if scenarios, presented in the form of connected charts, graphs, and maps. Cranfield are grateful to ESRI (UK) for the opportunity to act as a Beta tester for this new strategic tool from ESRI, drawing on established linkages via the DREAM Centre for Doctoral Training (CDT) in Big Data and Environmental Risk Mitigation.

In a series of blog pages which we will place here on Geothread, we will document the process of installing and testing this software, adding some helpful commentary on the way that should hopefully help others tread the same path!

ESRI Early Adopter Programme

The first act in this story is joining the ESRI Early Adopter programme (https://earlyadopter.esri.com). This provides first hand access to emergent software in Beta form.

Logging into the website, all the beta edition materials are made available for Insights for ArcGIS. A key early document to look at is the Insights for ArcGIS User Guide (insights_user_guideEAP2_1-1.pdf), which outlines the stages required for installation.

Insights for ArcGIS is part of the new ArcGIS Enterprise family from ESRI. We are informed by ESRI that ArcGIS Enterprise “is the next evolution of the ArcGIS Server product line, is a mapping and analytics platform that runs on your private infrastructure. It has a flexible deployment model allowing for use completely on-premises – connected or disconnected from the open internet – on physical hardware or virtualized environments, in the cloud on Amazon Web Services (AWS) or Microsoft Azure, or any other environment that meets the basic system requirements. This flexibility also allows you to add a variety of capabilities and distribute your deployment across infrastructure that supports your business needs.” Sounds good!

The ArcGIS Enterprise product includes the following software components:

  • ArcGIS Server
  • Portal for ArcGIS
  • ArcGIS Data Store
  • ArcGIS Web Adaptor

To get Insights for ArcGIS to work, we need to install these pre-requisites, which we will be installing step by step. So we will post here a blog of our progress in installing all these bits, as follows:

  • ArcGIS Server 10.5Beta – we need 10.5 for this to work
  • Portal for ArcGIS
  • ArcGIS Web Adaptor
  • ArcGIS Data Store
  • An instance of MS SQLServer Database
  • A JDBC 4.0 Compliant driver

First things first – we need to install the Beta edition of ArcGIS 10.5 Server.

ArcGIS Server 10.5Beta

To get Insights for ArcGIS to work, we need to get ArcGIS Server 10.5 up and running. We fired up our trusty Linux server for this task. This server was already running an earlier 10.2 version of ArcGIS Server. The instructions make it clear one can upgrade – but we chose to go for a clean install by preference, so uninstalled ArcGIS Server 10.2 as a first act.

We now copied over the file from the early adopter site ‘ArcGIS_for_Server_Linux_105_beta1.tar.gz‘, which contains the installation files for the new installation. This tar/gz file contains a folder called ‘Documentation’ – within which is a web/html set of instructions. We found it useful to extract these files off to a separate computer for consultation as the process unfolded.

Next, we downloaded to our installation folder (our home directory on the test server) the sample provisioning authorisation file ‘ArcGISforServerAdvancedEnterprise_Server_105alpha.prvc‘. We edited the header for this file with our details, but apart from that left the actual codes alone. The key learning point here, was that one has to add a valid email to the header of the file. More detailed advice received from ESRI concerning this, was that:

“To authorize ArcGIS Server, use the following parameter to authorize ArcGIS Server using the provisioning file:
authorizeSoftware –f ArcGISforServerAdvancedEnterprise_Server_105alpha.prvc -e
since the provisioning file does not include an email address.”
So that is an alternative approach – editing in the email explicitly to the prvc file worked OK for us.

The next step was to unpack the installation media, thus:
~> gunzip ArcGIS_for_Server_Linux_105_beta1.tar.gz
~> tar –xvf ArcGIS_for_Server_Linux_105_beta1.tar

This creates a folder ‘ArcGISServer’ with all the installation media in it ready to go. From here, we ran the setup programme, thus:
~> cd ArcGISServer
~> ./Setup -l yes

Running ‘Setup -help’ shows all the options available. This starts of the console process of installing ArcGIS Server. You go though various pages about the installation destination (we accepted the defaults for all these choices), and you view the terms and conditions of the Beta software.

At the end of this process, the installation asks for the full path to the provisioning licence authorisation file. This was provided and the installation ran on. At the end of this, the prompt says to press enter to exit – clearly this act starts the server up as, until you get the prompt back, the server will not run.

The installation is placed by default in a folder ‘/home/arcgis/server’, (home as in the home folder of the installing user). Operationally, we might put the files somewhere more conventional (e.g. /opt), but this is fine for our testing purposes. Within the folder is a ‘tools’ folder with some useful utilities. The following were useful:

~> cd ~/arcgis/server/tools
~/arcgis/server/tools> ./serverinfo
Server: 10.50.0.6318
JRE: 1.8.0_65
Tomcat: 7.0.64.0
Geronimo: 2.2.2
Wine: wine-1.8-rc3-985-g4f7221d

~/arcgis/server/tools> ./patchnotification/patchnotification
===============================================================================
ArcGIS for Server and Extensions Patch Notification
===============================================================================
Installed components:
Component Version
ArcGIS Server 10.5
===============================================================================
Available Updates:
ArcGIS Server
(no updates available)
===============================================================================
Installed Patches:
(none)
===============================================================================
To browse a full list of Esri patches and service packs, visit the Esri Support site:
http://support.esri.com/en/downloads/patches-servicepacks/

~/arcgis/server/tools> ./authorizeSoftware -s
[shows all software licenced]

Note authorizeSoftware -f would allow later application of a provisioning file to an installation.

So far so good, software installed and licenced. Now time to fire up the interface:
We entered the URL of server for the first time:

http://SERVER.NAME.HERE:6080/arcgis/manager

All worked well. This was the first time Server was started, so we were asked if we wanted to create a New site, or join another – we selected create a new site:
ArcGIS Server

Next, we are asked to create a username and password for the Server manager:
ArcGIS Server

Next, we specify the root server folder and the configuration store location:
ArcGIS Server
A summary is shown:
ArcGIS Server
And now we waited whilst the installtion finished (actually quite a long wait – but we are patient!)
ArcGIS Server
Finally it completes and we can log in for the first time:
ArcGIS Server

There then followed problems for us! It had seemed like a ‘good idea’ to stop and restart the server (utilities in the folder ~/arcgis/server). However, this was not a good idea! We tried to log back into the Server manager URL, thus:

http://SERVER.NAME.HERE:6080/arcgis/manager

Port 6080 is the http communication port. ArcGIS Server immediately tries to switch to HTTPS on port 6443, e.g.:

https://SERVER.NAME.HERE:6443/arcgis/manager

This didn’t work and we lost access to the server. It wasn’t obvious what had happened here at first, as our old version of ArcGIS Server 10.2 had worked fine. After investigation, it transpired the new https port of 6443 was not the same as the https port used in the earlier installation. The firewall was blocking the new port – quickly remedied. However, even then we still had a problem connecting. The URL was trying to connect to an apparently absent server. Fortunately, Server has some diagnostic tools.

~/arcgis/server/tools/> ./serverdiag/serverdiag
========================================================================
ArcGIS Server 10.5 Diagnostic Tool
Hostname: SERVER NAME
========================================================================
DIAG000: Check for installation as root [PASSED]
DIAG001: Check for 64-bit architecture [PASSED]
DIAG002: Check OS version [PASSED]
DIAG003: Check hostname for invalid characters [PASSED]
DIAG024: Check /etc/hosts for hostname entry [PASSED]
DIAG004: Check installed packages [PASSED]
DIAG005: Check system limits [PASSED]
DIAG008: Check HTTP port [PASSED]
DIAG009: Check HTTPS port [PASSED]
DIAG010: Check Xvfb ports [PASSED]
DIAG020: Check hostname IP address mismatches [PASSED]
DIAG026: Check processes for ArcGIS core services [PASSED]
DIAG020: Check hostname IP address mismatches [WARNING]
DIAG026: Check processes for ArcGIS core services [PASSED]
------------------------------------------------------------------------
There were 0 failure(s) and 1 warning(s) found:

ESRI have a good page explaining the diagnostic warnings here:
http://server.arcgis.com/en/server/latest/administer/linux/checking-server-diagnostics-using-the-diagnostics-tool.htm

We quickly realised there were inconsistencies in the server hosts file (nothing to do with ESRI), again quickly remedied. Finally the system started up and worked. We logged onto the Manager page:

ArcGIS Server

We noted the ‘Certificate error’ – the software actually provides its own default certificate just to get it all going – we can fix that later, there is lots of help online to do this – this is a test installation in any case:
ArcGIS Server

And now finally we see the main manager screen again:
ArcGIS Server

And then for the first time we can see data being served out of the ArcGIS Server 10.5 software – fantastic!
ArcGIS Server

In the next blog, we will start installing the other pre-requisite software tools for ESRI Insights for ArcGIS, starting with Portal for ArcGIS .

Thanks for reading!