Exocortex

April 16, 2010

ESRI’s Java Web ADF – more information

Filed under: ESRI — Soumya @ 9:05 am

I had written about ESRI’s Java Web ADF a long time back when I first started this blog. In that entry, the description was pretty much a rehash of the material from ESRI’s web site. Since then I have had some more experience with this API and after months and months of procrastination I finally took it upon myself to write it up.

I must add this caveat though: what I will write here is my understanding of the system and not based on the excellent (sarcasm intended) documentation that you generally get from Java Web ADF resource center. So, as is always the case in these situations, feedback is essential (and definitely well appreciated).

Let me start with a graphic:


[As usual this graphic was created using Steve Hanov's online drawing tool and edited using the Pixlr online photo editor.]

I agree the graphic is not very professional and actually pretty cartoonish but I think it is good enough to give a mental picture of the discussion that follows.

On the right end of the above graphic, we can see the ArcGIS server. Although it might seem from the graphic that the SOC is the only component of the ArcGIS server, it is not entirely true. Another component of the server ecosystem is the SOM. I have simplified the diagram a bit and have not shown the SOM or the physical machine(s) that host those components. These facts are not relevant to this discussion.

The goal of any application that integrates ESRI’s ArcGIS Server is to offload geospatial computation to the server side. (Just to be clear, in this discussion the web application (normally hosted by a web application container like Tomcat) is the “client” that requests geospatial computations to the ArcGIS Server that is the “server”. In this discussion we are not going to talk about the DHTLM-JavaScript-CSS based user interface client that resides on a (human) user’s browser.) These geospatial computations are encapsulated by combinations of various ArcObjects.

Since the web application resides in a different process space than the ArcGIS Server (be it on the same machine or on a different machine as shown in the above graphic) how can we get hold of the ArcObjects? Well, ESRI exposes ArcObjects using Microsoft’s DCOM technology. So if you are familiar with DCOM programming (which I am not, so I am not going to dwell on it too much) then you get hold of the DCOM-based ArcObject proxies and invoke the relevant methods directly.

In my case, my web application is a Java based one. One way to make the Java code to invoke methods on ArcGIS hosted ArcObjects is by the use of SOAP. A method invocation is encapsulated in a SOAP message and then transmitted down the wire to the ArcGIS Server which converts the SOAP message to a call on an ArcObject and finally sends the response back to the caller, again using SOAP. We could easily have used DCOM’s in-built serialization protocol if that is supported in Java.

The ADF is made up of 2 primary packages: com.esri.adf.web.data and com.esri.adf.web.faces. The latter package is focused more on the front-end components that make up the web application and is not really the topic of discussion here. The former package can be thought of as containing all the “domain” objects that make up a geospatial application. These domain objects are quite intelligent in the sense that they are aware of where to get data and computation services from and how to manage their own lifecycles. These “intelligent domain” objects come preprogrammed with servicing the most standard geospatial computations. Essentially, these objects are preprogrammed with all the logic necessary to invoke SOAP requests to the ArcGIS Server and marshall and unmarshall data.

In the typical development scenario, the geospatial part of the web application consists of a combination of one or more objects from the com.esri.adf.web.data package. Classes like WebContext, WebMap, WebToc, WebQuery etc all have preprogrammed logic that knows how to interact with the ArcGIS Server to service standard operations like panning a map, zooming in and out, spatial queries etc.

Sometimes, the typical usage may not be enough. During that time one might want to manipulate the ArcObject proxies directly. These come in the package com.esri.arcgisws. Just to be clear, my guess is that the objects in the com.esri.adf.web.data package use the proxies in the com.esri.arcgisws package.

Hopefully the explanation above will help one understand how all the different packages are weaved together in the ESRI Java Web ADF ecosystem.

Powered by ScribeFire.

July 29, 2009

PostGIS installation

Filed under: PostGIS — Soumya @ 8:46 pm

To test out PostGIS I decided to install it on my laptop. It should at least fit there :) . PostGIS is an extension of PostgreSQL, so you need to install PostgreSQL first. You can find the download of PostgreSQL for your platform here. I use a Mac and it turned out that there is no straightforward package installation for the Mac. (Actually there is but it is maintained by a vendor of PostgreSQL and PostGIS services.) The other alternative is to build both PostgreSQL (and then PostGIS) from source. Even though I was not thrilled with the approach I soldiered on. Turns out, it isn’t as bad an experience as I thought it would be.

On a Mac, there is a build engine available called Fink that provides a tool that downloads source code, applies patches, resolves dependencies, configures and builds software. Obviously, you need to install Fink first on your Mac. For Fink to work, you need to have the right compilers installed also. It turns out that on a Mac, these compilers and whatnots come in a separate downloadable package called Xcode. You need to create a free account and jump through some links once you are logged in before you can download and install Xcode. Once you find it, the process of installation is straightforward.

With Xcode installed, I downloaded and installed Fink. Figuring out how to use Fink to download the PostgreSQL database (and then the PostGIS extensions later on) was also not difficult. I just followed the instructions on the Fink site. The command was: $fink install postgre for the PostgreSQL database. (I will get to the PostGIS part a little bit later.) Once you issue this command, it took quite a while for Fink to go through the entire process of installing the entire database. There was tonnes of output as can be expected from a full-blown build process. But the whole process was fairly smooth.

Finally the time came to install the PostGIS extensions. The extensions are actually a bunch of data types and functions that reside as a schema in the PostgreSQL database. However, these need to be built also and so Fink needs to be used once again. This time the command was: $fink install postgis83 (the 83 represents the 8.3 version of the PostgreSQL database with which it is compatible). Again there will be a huge amount of output and activity. The end result is a file called lwpostgis.sql that will be created in the directory /sw/share/doc/postgis83 on your Macintosh HD. For this file to run, you need to create a database called ‘postgis’ in your PostgreSQL database. The documentation for the database explains how to create the database.

In a nutshell, you first need to define the environment variable PGDATA to a location where the data files of PostgreSQL will be stored and then run the script ‘initdb’. Next the stored procedure language support needs to be installed on the database ‘template1′ by running the command $createlang plpgsql template1. Then you can create the database ‘postgis’ by running the command $createdb postgis. Since this is based of the ‘template1′ database, ‘postgis’ will have the stored procedure support already built-in. (All the steps mentioned in this paragraph are very nicely explained in the PostgreSQL database documentation.)

With the ‘postgis’ database created, you can now run the lwpostgis.sql script to create the data types and the spatial functions in ‘postgis’. The command is $psql -f /sw/share/doc/postgis83/lwpostgis.sql -d postgis.

June 18, 2009

Starting out with PostGIS

Filed under: GIS, GeoServer, PostGIS — Tags: , — Soumya @ 11:07 am

For some work that I am doing outside my regular employment, I decided to check PostGIS out. This blog is supposed to be a journal of my experience. As you can understand, I am just now starting out with PostGIS (and also PostgreSQL), hence certain comments will change over time. I hope to provide enough cross-references to point out how my understanding evolves over time.

First, some background information on what led me to PostGIS. At this time, I am trying to establish a web-based GIS consulting business. (More on the business experiences in a separate blog, there is not much to write yet :) .) We (I have a partner) decided to have our own GIS setup in-house. However, most of the commercial packages that we are familiar with are too expensive and problematic to set up by a small establishment like ours. At about this time, a friend of mine who is also in this business (of GIS) suggested open source. We decided to give it a shot and hence we stumbled across PostGIS (actually, we came to know about it through GeoServer, uDIG, but again that is another story). [We looked at the spatial support of MySQL also, but from our research we found out that it is not very mature. We did not verify it ourselves, after reading that MySQL spatial support is immature and PostgreSQL + PostGIS is mature we decided to check out the latter first. Probably not very scientific, but in the interest of time....]

We are still in the evaluation stage. But before we go too far…how do you evaluate something like PostGIS? I am sure there are some “methodologies” for doing things like these and I am sure all these “methodologies” are fighting over each other on what the “correct methodology” is but again in the interest of time we established our own ground rules and boundaries. Those are:

  1. The spatial database should have licensing requirements that are consistent with our financial and legal goals — essentially free to use without any licensing restrictions.
  2. The database must be easy to install and use and should follow standards as much as possible.
  3. The database project must have a good user base as established by active user forums etc.
  4. The database must support common spatial operations like geometry relationships, geometry queries etc.
  5. Performance metrics should match some numbers that we had established.

The above evaluation criteria, as listed, kind of gloss-over the actual numbers and parameters. It might not be too important about what our boundaries are as different people may have different thresholds. I believe that the criteria themselves are a good guide as to how to evaluate PostGIS (or any alternative along the same lines).

Criteria 1 was the easiest to find out. PostGIS is distributed with a GNU GPL as you can see right on the home page. Since we do not anticipate making changes in the PostGIS implementation itself this option works out for us. Criteria 2 is the topic of a different post, so I am not going into too many details here. Suffice to say, I did not face too many problems installing the database on my Mac. To satisfy criteria 3 we found that there is good documentation for PostGIS which goes over many things quite well. As the core of this database is based on standards, a lot of understanding can be obtained from the specifications themselves. Typical database related tasks can be found at the PostgreSQL documentation. The documentation has helped us figure out most of our problems although it must be made clear that the kinds of problems that we have faced so far are very “beginner-level” problems. There is an email list that seems to have a pretty good archive. A few companies also provide support to PostGIS related issues but it comes for a price.

As mentioned before, PostGIS implements the OGC Simple Feature Access specifications (and actually extends it). These standards define a standard feature definition and ways to interface with such features. OGC standards are adopted by all major players in the GIS space and so far there has been no major hiccups or break aways (at least publicly). This means that in the forseeable future this standards body will be supported by these players. This in turn means a certain level of stability of the standards. This satisfied our criteria 4. In some of our initial tests performance metrics were pretty good. However, there were some situations where the numbers went through the roof. Obviously this is not a very descriptive way of talking about metrics and I intend to cover that in a different post. For now, the numbers were not such that we had to abandon PostGIS. (Note to self: need to learn more optimization tricks.) This introductory post was more of a checklist that we had assembled for our evaluation and as we have seen it turns out that PostGIS scores pretty well. You might have caught a certain level of apprehensive tone in the way the post is written and that is intentional. Having worked in the software industry for a few years I have learnt to be cautious. This has no reflection on the product itself. Our decision is made. At this stage we are going with PostGIS. If we hit a road block in the future we will reevaluate things again.

February 16, 2009

Which JAR files are required for an application?

Filed under: ESRI, GIS — Tags: , , — Soumya @ 12:42 pm

Every time I create a new Java Web ADF application a lot of JAR files get added in the WEB-INF/lib directory. It is quite obvious that not all of these libraries are needed for the project and I always wanted to clean that part up. Finally I got a chance and found out that the following JAR files are not needed if you are writing an application that uses an ArcGIS server as the data source:

  1. arcgis_webcontrols_wms
  2. axis-schema
  3. arcgis_ejb_stubs
  4. arcgis_jcs
  5. arcgis_webcontrols_arcweb
  6. arcgis_webcontrols_ejb
  7. arcgis_webcontrols_arcims
  8. axis-ant
  9. mlibwrapper_jai
  10. arcgis_arcweb_stubs
  11. arcims_jconnect

Our application utilizes most of the mapping capabilities but does not include any sort of editing. I do not know yet if some of the above JAR files will be needed if we include such capabilities.

January 27, 2009

About resources and proxies to ArcObjects

Filed under: ESRI, GIS — Tags: , , — Soumya @ 12:49 pm
The com.esri.adf.web.data.GISResource interface provides access to a proxy preconfigured with the connection details of a service on the SOM. The interface also provides access to the functionalities that are applicable for that resource.

GISResource instances are defined as managed beans in the faces-config.xml. The resources are then tied up with the WebContext of the application.

One example of a GISResource implementation is the AGSMapResource.This resource provides a proxy called MapServerPort that proxies the mapping service on the SOM. Behind the scenes the service on the SOM interacts with the MapServer ArcObject on the SOC. Hence, using the MapServerPort one can send specific commands to the ArcObjects back-end (through the service on the SOM), commands that are applicable to a map. Some examples are querying map data, manipulation of map image etc.

Another example of a GISResource implementation is the AGSGeocodeResource. This resource provides a proxy called GeocodeServerPort that proxies the GeocodeServer ArcObject on the SOC.

Example of a GISResource instance declared as a managed bean:
<managed-bean> <managed-bean-name>agsws0</managed-bean-name> <managed-bean-class> com.esri.adf.web.ags.data.AGSMapResource </managed-bean-class> <managed-bean-scope>none</managed-bean-scope> <managed-property> <property-name>endPointURL</property-name> <value> http:/ip:port/arcgis/services/serviceGroup/serviceName/MapServer </value> </managed-property> <managed-property> <property-name>user</property-name> <value>#{agswsUser1}</value> </managed-property> <managed-property> <property-name>alias</property-name> <value>Some Alias</value> </managed-property> <managed-property> <property-name>functionalities</property-name> <map-entries> <map-entry> <key>map</key> <value>#{agsMap}</value> </map-entry> <map-entry> <key>tile</key> <value>#{agsTile}</value> </map-entry> <map-entry> <key>toc</key> <value>#{agsToc}</value> </map-entry> <map-entry> <key>query</key> <value>#{agsQuery}</value> </map-entry> </map-entries> </managed-property> </managed-bean>
(Anything above in #{} are defined elsewhere and is not shown here.)
Example of the above map resource being tied up to the WebContext:

<managed-bean>
<managed-bean-name>mapContext</managed-bean-name>
<managed-bean-class>
com.esri.adf.web.data.WebContext
</managed-bean-class>
<managed-bean-scope>session</managed-bean-scope>
<managed-property>
<property-name>webSession</property-name>
<value>#{esriWebSession}</value>
</managed-property>
<managed-property>
<property-name>attributes</property-name>
<map-entries>
<map-entry>
<key>map</key>
<value>#{map}</value>
</map-entry>
<map-entry>
<key>overview</key>
<value>#{overview}</value>
</map-entry>
<map-entry>
<key>toc</key>
<value>#{toc}</value>
</map-entry>
<map-entry>
<key>graphics</key>
<value>#{graphics}</value>
</map-entry>
<map-entry>
<key>query</key>
<value>#{query}</value>
</map-entry>
<map-entry>
<key>history</key>
<value>#{extentHistory}</value>
</map-entry>
<map-entry>
<key>geocode</key>
<value>#{geocode}</value>
</map-entry>
<map-entry>
<key>results</key>
<value>#{results}</value>
</map-entry>
<map-entry>
<key>scaleBar</key>
<value>#{scaleBar}</value>
</map-entry>
</map-entries>
</managed-property>
<managed-property>
<property-name>resources</property-name>
<map-entries>
<map-entry>
<key>agsws0</key>
<value>#{agsws0}</value>
</map-entry>
</map-entries>
</managed-property>
</managed-bean>

(Again, anything above in #{} are defined elsewhere and is not shown here.)

Example Java code that retrieves the resource:
WebContext webContext = webMap.getWebContext();

Map resources = webContext.getResources();
AGSMapResource agsMapResource = (AGSMapResource) resources.get(”agswso”);
Example Java code that sends a query to the back-end ArcObject:
URL url = agsMapResource.getEndPointURL();
MapServerBindingStub mapserver = new MapServerBindingStub(new URL(url), null);
MapServerInfo mapinfo = mapserver.getServerInfo(mapserver.getDefaultMapName());
MapDescription mapdesc = mapinfo.getDefaultMapDescription();
RecordSet recordSet = mapserver.queryFeatureData(mapdesc.getName(), layerId, query);
In the above the layerId variable is an int that specifies which specific layer needs to be queried and query is an instance of a QueryFilter that specifies the details of the query that is to be run.

January 23, 2009

General description of ESRI’s Java Web ADF

Filed under: ESRI, GIS — Tags: , — Soumya @ 10:06 pm

The Java Web ADF, part of the ArcGIS Server ADF, is a library of objects that assist in the development of GIS web applications. The library consists of a bunch of objects which are organized in a MVC framework. The web controls define standard interactions with maps and also provide hook ups to backend ArcObjects. ESRI decided to build the Web ADF based on JSF and hence a knowledge of JSF is necessary for developing applications using the Java Web ADF. The Java Web ADF is made up primarily of 3 groups of objects:

  • View: Objects in this layer consist of preprogrammed visual controls. These are JSF components that can be plugged into a JSP using JSF tags. The backing beans behind these controls serve as controllers too.
  • Model 1: Objects in this layer are beans that provide the data layer for the objects in the view layer. The data objects provide support for the data on the web. These objects also act as a business logic layer for certain standard GIS functionality. Such functionality is provided by interacting with the server (ArcGIS server) objects.
  • Model 2: Objects in this layer complement the objects in the Model 1 layer by providing purely GIS data and functionality. The objects in this layer are provided with intelligence of how to pull data from appropriate resources.

The view objects deal with presentation and user interaction. To deal with user interactions, the Java Web ADF contains the following categories of objects:

  • Commands: These require a 1-step interaction from users. On the web page, once the user clicks on a button or some other control something happens on the server side. The server acts on information available at the time of the clicking. Examples can be “Zoom to full extent”, “Information at current point” etc. Implementation-wise, such constructs are implemented as JSF actions or action listeners depending on whether the interaction results in a navigation change or not.
  • Tools: These require at least a 2-step interaction from users. This means that the interaction is initiated by the user clicking a control and then continues the interaction by performing some more actions. An example can be “Zoom to rectangle” which is initiated by the user clicking on a Zoom In or Zoom Out button and then continuing by setting a rectangle. After the user interaction is completed a request is sent back to the server side where a server side component handles the request.
  • Tasks: While commands and tools handle the more traditional and more visual of the map controls, tasks (and the associated task framework) are supposed to handle the more non-traditional interactions with geospatial data. The result of running these tasks are generally result sets but these results can impact the visual aspects also. Examples can be customized queries that result in the highlighting of features (from the result) in the map.

General description of ESRI’s ArcGIS Server

Filed under: ESRI, GIS — Tags: , , — Soumya @ 10:03 pm

Briefly, ArcGIS Server is a collection of server-side components (ArcObjects) providing GIS functionality and a mechanism to interact with these ArcObjects. ArcGIS Server is organized in a system of servers.

The system of servers hosting the ArcObjects consist of a Server Object Manager (SOM) and one or more Server Object Containers (SOC). The SOCs contain the actual ArcObjects and perform the computations while the SOM acts as a gateway to the SOCs and performs most of the component management tasks. A client application interacts only with the SOM. In most cases, therefore, the SOM and ArcObjects can be used interchangeably.

ESRI provides APIs, called ArcGIS Server ADF, written in various languages that allows developers to interact with the SOM. These APIs contain web controls and proxies to the back-end ArcObjects. The proxies abstract out all the communication between the client application (the developers write) and the SOM. In the 9.2 version of the software, the communication between the client application and the SOM is through SOAP over HTTP.

Important URLs:
http://edndoc.esri.com/arcobjects/9.2/NET_Server_Doc/manager/administration/how_gis_svr_works.htm

http://edndoc.esri.com/arcobjects/9.2/NET_Server_Doc/developer/ArcGIS/SOAP/overview.htm

Motivations for writing about ArcGIS Server software

Filed under: ESRI, GIS — Soumya @ 9:49 pm

I am currently working with ESRI’s ArcGIS Server 9.2/9.3 (Java edition). This software is fairly new to me and so I decided to collect bits and pieces of information about the software itself and how to program with it. As other people working with this software will verify, documentation on this software is, in short, “lacking”. I am not trying to override the official documentation that ESRI provides, I am merely trying to organize my thoughts about this software. This might come in handy for my current team members, other people working with this software or even to myself a few years down the road!

Specifically, I am trying to target the following topics for now:

  1. General architecture of the ArcGIS Server (primarily 9.2, currently evaluating the new REST style architecture of 9.3 so that might be covered also).
  2. Java Web ADF (9.2).
  3. Flex API (9.3).
  4. JavaScript API (this is optional for me right now).

Code samples may or may not be present. I will try to provide some.

Comments are welcome. I understand that I am fairly new to this and so I may not be “getting it correct” all the time. Part of the reason for writing this blog is to make sure that my understanding is consistent with others experienced with this software.

Powered by WordPress