Exocortex

April 22, 2010

IIS, Tomcat, SSL and now JSSE

Filed under: Uncategorized — Soumya @ 3:32 pm

This one was interesting….

Recall that I am working on a project where I had to set up a web application on Apache Tomcat that users can connect to through IIS (an through a secure channel).

In the application, there is a requirement to open up a URL connection to another secure site (essentially a URL starting with https). I got this weird exception message: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.

As usual, a simple Google search led to an article that talks about this very problem. Turns out that the Java Secure Socket Extension of the Java runtime in which Tomcat runs (and hence the web application) cannot connect to the external secure site as the runtime has not been told that the external site is trusted. The fix for this is to store the certificate from this external site in a place which the runtime can access. So the next time around the URL is being accessed, the runtime will allow it.

Following the instructions in the above article I got to generate a certificate store that then needed to be put in a place where the runtime can see. There was a small hiccup here. I thought that the <JDK_HOME>\jre\lib\security is where the certificate store (a file called jssecacerts) needs to reside. Actually it was <JRE_HOME>\lib\security which obviously makes more sense.

Now the world is back to normal…..

P.S: The link to the Sun Java documentation about security certificate stores and all is broken. Here is the correct link and here is the link to the specific section on customizing certificate stores.

Powered by ScribeFire.

April 16, 2010

ESRI’s Java Web ADF – more information

Filed under: ESRI — Soumya @ 9:05 am

I had written about ESRI’s Java Web ADF a long time back when I first started this blog. In that entry, the description was pretty much a rehash of the material from ESRI’s web site. Since then I have had some more experience with this API and after months and months of procrastination I finally took it upon myself to write it up.

I must add this caveat though: what I will write here is my understanding of the system and not based on the excellent (sarcasm intended) documentation that you generally get from Java Web ADF resource center. So, as is always the case in these situations, feedback is essential (and definitely well appreciated).

Let me start with a graphic:


[As usual this graphic was created using Steve Hanov's online drawing tool and edited using the Pixlr online photo editor.]

I agree the graphic is not very professional and actually pretty cartoonish but I think it is good enough to give a mental picture of the discussion that follows.

On the right end of the above graphic, we can see the ArcGIS server. Although it might seem from the graphic that the SOC is the only component of the ArcGIS server, it is not entirely true. Another component of the server ecosystem is the SOM. I have simplified the diagram a bit and have not shown the SOM or the physical machine(s) that host those components. These facts are not relevant to this discussion.

The goal of any application that integrates ESRI’s ArcGIS Server is to offload geospatial computation to the server side. (Just to be clear, in this discussion the web application (normally hosted by a web application container like Tomcat) is the “client” that requests geospatial computations to the ArcGIS Server that is the “server”. In this discussion we are not going to talk about the DHTLM-JavaScript-CSS based user interface client that resides on a (human) user’s browser.) These geospatial computations are encapsulated by combinations of various ArcObjects.

Since the web application resides in a different process space than the ArcGIS Server (be it on the same machine or on a different machine as shown in the above graphic) how can we get hold of the ArcObjects? Well, ESRI exposes ArcObjects using Microsoft’s DCOM technology. So if you are familiar with DCOM programming (which I am not, so I am not going to dwell on it too much) then you get hold of the DCOM-based ArcObject proxies and invoke the relevant methods directly.

In my case, my web application is a Java based one. One way to make the Java code to invoke methods on ArcGIS hosted ArcObjects is by the use of SOAP. A method invocation is encapsulated in a SOAP message and then transmitted down the wire to the ArcGIS Server which converts the SOAP message to a call on an ArcObject and finally sends the response back to the caller, again using SOAP. We could easily have used DCOM’s in-built serialization protocol if that is supported in Java.

The ADF is made up of 2 primary packages: com.esri.adf.web.data and com.esri.adf.web.faces. The latter package is focused more on the front-end components that make up the web application and is not really the topic of discussion here. The former package can be thought of as containing all the “domain” objects that make up a geospatial application. These domain objects are quite intelligent in the sense that they are aware of where to get data and computation services from and how to manage their own lifecycles. These “intelligent domain” objects come preprogrammed with servicing the most standard geospatial computations. Essentially, these objects are preprogrammed with all the logic necessary to invoke SOAP requests to the ArcGIS Server and marshall and unmarshall data.

In the typical development scenario, the geospatial part of the web application consists of a combination of one or more objects from the com.esri.adf.web.data package. Classes like WebContext, WebMap, WebToc, WebQuery etc all have preprogrammed logic that knows how to interact with the ArcGIS Server to service standard operations like panning a map, zooming in and out, spatial queries etc.

Sometimes, the typical usage may not be enough. During that time one might want to manipulate the ArcObject proxies directly. These come in the package com.esri.arcgisws. Just to be clear, my guess is that the objects in the com.esri.adf.web.data package use the proxies in the com.esri.arcgisws package.

Hopefully the explanation above will help one understand how all the different packages are weaved together in the ESRI Java Web ADF ecosystem.

Powered by ScribeFire.

February 12, 2010

IIS, Tomcat and SSL

Filed under: .NET — Soumya @ 8:32 pm

In keeping with my adventures with Microsoft technologies, my latest task required me to venture into IIS-territory. Specifically, I had to connect Tomcat with IIS, so that web applications hosted by Tomcat are routed through IIS. On top of that the communication between the browser and IIS needs to be secure. (The communication between IIS and Tomcat happens within the network, so can be insecure.)

This has been done to death by many folks all around the world, so I really have not a lot to add to it. I am going to keep this short and refer primarily to web sites that I found extremely useful. (Thanks to them also, btw.)

First, a simple graphic to show what I am trying to achieve, just in case things are still not entirely clear:

[Image created using this awesome tool by Steve Hanov at this site and then cropped using another awesome tool called Pixlr that is available at the site here.]

As can be seen, the browser needs to connect to IIS through a secure channel. Internally, IIS communicates with Tomcat using the AJP 1.3 protocol. And just to make things interesting there is a firewall sitting on the machine where Tomcat is running.

IIS and Tomcat can be connected together by an ISAPI filter called Tomcat Connector. This is the only Apache-approved way to connect the two. The filter comes in the form of a DLL from Apache. Instructions on how to do this can be obtained from this site. The instructions are pretty good. However, I still have a few comments to add:

  1. Make sure that you download the appropriate DLL for your particular machine architecture. If you have a 64-bit machine you need to download the 64-bit version, and that too, for the right architecture (IA64/AMD).
  2. In the IIS Admin link above, it is not very clear where to attach the ISAPI filter. It would seem from the text that you need to attach the filter to the ‘Default Web Site’ or the site that will host the jakarta virtual directory. Conceptually, that seems to make sense, but for some reason that did not work out. I added the filter to the ‘Web Sites’ item under IIS. I believe this means that the filter applies to all web sites. It is not a problem for me as, as I will have only one site anyways.
  3. Make sure that the directory that contains the ISAPI filter, the properties files and where logs are to be written have the right permissions for the appropriate operations for the IUSR_<machineName> and IWAM_<machineName> users. I gave both those users Read & Execute, List Folder Contents, Read, Write permissions for both those and also propagated the permissions down the directory structure (only for the directory containing the filter components). This probably should be made more robust by figuring out who needs exactly what.
  4. The symptoms of the filter not loading properly are:
    • Obviously, things don’t work!
    • The arrow beside the ISAPI filter will be red and pointed downwards (should be green and pointing upwards).
    • The ISAPI filter does not generate any log files in the directory specified.
    • IIS does not generate any log files under C:\WINDOWS\system32\LogFiles\W3SVC1 or wherever the IIS log file directory is set to be.

Tomcat has an in-built AJP Connector that listens on port 8009. In the firewall, make sure there is an exception that allows traffic through port 8009.

To test the above setup replace the usual URL to the web application, http://<machine_name>:8080/<Context>, with
http://<machine_name>/<Context>.

Once this is working, we need to set up SSL for the web site hosted by IIS. The idea is to establish a certificate, signed by a well-known certificate authority like VeriSign or Thawte and turn on the SSL requirement for the web site. The process is fairly simple and is again explained well here. The only thing that I want to point out is that for development purposes, obtaining a properly signed certificate is a lengthy and costly option. One can generate a self-signed certificate and use it for the web site.

There are many ways to create a self-signed certificate and install it for the web site in IIS but Microsoft has a very simple command line utility to do that. This tool (SelfSSL) is part of the IIS Resource Kit. This kit can be downloaded from here.

The download and installation of the Resource Kit is trivial. After the install, you can start SelfSSL by going to Start -> All Programs -> IIS Resource Kit -> Self SSL.

The command line utility has a very good help that shows up immediately when you start the program. I used the following command:

prompt> selfssl /V:365

to issue a security certificate (valid for 365 days) for the site number 1 hosted by IIS. I think the main thing to remember is the site number. In my case, I have only one site hosted by IIS, so leaving the default for option /S worked out.

This installed the certificate for my web site (that is the front to my main web app hosted by Tomcat). Going back to the IIS properties for that site (under the Directory Security tab), I now see that all buttons of the Secure Communications section are active. You can view the certificate. To turn on the requirement for secure communication with the web site click the Edit button and select the options Require Secure Channel (SSL) and Require 128-bit Encryption.

Now your web site will only respond to https:// requests and throw an exception for http:// requests. (You do not have to restart IIS after this change.)

Powered by ScribeFire.

February 10, 2010

Google’s Street View data capture issues

Filed under: Uncategorized — Soumya @ 9:51 am

These articles summarize it well:

http://www.huffingtonpost.com/2009/11/15/google-street-view-funny_n_357433.html?slidenumber=%2Bq3e80pV2EY%3D
http://www.huffingtonpost.com/2010/02/09/people-in-scuba-gear-chas_n_455787.html

February 2, 2010

32-bit DLLs on 64-bit machines

Filed under: .NET — Soumya @ 10:32 am

This is probably something very obvious to those who prefer living on the wild side (i.e .NET and the whole MS way of doing things) but this came as a little bit of a surprise to me. Shouldn’t have been surprised given the folklore and mythology surrounding development in the MS world….

Anyway, recently I downloaded some code (which came with source also, thankfully) in EXE format (yes I know it is dangerous, but it came from a reputable host (well, that is another story)). The idea was to run it on a different machine. It ran perfectly on my machine but when I tried to run the code on the other machine I got this weird error message:

Retrieving the COM class factory for component with CLSID {60434CA9-132A-11D5-B14F-00C04F79D784} failed due to the following error: 80040154.

The CLSID was different of course. After some digging around I understand that the problem is that the DLL that I had downloaded is meant for 32-bit systems but I am running them on a 64-bit machine. So the OS is unhappy….After some more digging around I found this post on the ASP.NET site. The solution is quite simple: rebuild the executable but this time explicitly tell the VS IDE to build an EXE for an x86 platform. Now take this EXE and run it on a 64-bit machine. Everything works fine. Go figure…..

Powered by ScribeFire.

ADO.NET EF and NHibernate

Filed under: .NET — Soumya @ 10:15 am

Of late, I have had the incredible fortune of working with ASP.NET MVC and developing .NET web applications. ASP.NET MVC by itself is actually not too bad. Reminds me of Grails (probably similar to Rails also, but I don’t know much about Rails). While working with anything of this nature and specially with experience in a similar stack in the Java world, there will always be a tendency to draw parallels. For example, Grails is a web application development framework that uses Spring as a container (that is dependency injection based), Hibernate as an ORM tool, Spring MVC (?) as the web framework etc; what are the counterparts in the ASP.NET framework?

I don’t want to make this post a full-blown comparison study yet. Instead, let me take it one piece at a time. Clearly, in any web application database connectivity is a critical part. Nowadays, this is done through some ORM solution. In Grails, it is handled through Hibernate (behind the scenes, GORM is the actual technology that you have to interface with as a developer). In the .NET world there are 2 options (that I know of) to do ORM – NHibernate and Microsoft’s ADO.NET Entity Framework. NHibernate is fairly easy to understand, at least conceptually, as it is similar to Hibernate. However, ADO.NET Entity Framework is somewhat different (of course, coming from the Microsoft camp). Through this post I will try to understand what the differences are.

To get the most obvious out of the way, one can use NHibernate directly. NHibernate is the .NET counterpart of Hibernate. From the looks of it, it seems similar to Hibernate, I haven’t delved into it too much to be able to know the differences. However, this much is clear — the purpose of the tool is the same as that of the Java counterpart and basic concepts are the same. Probably the only thing missing is a counterpart for annotations and hence all mappings are carried out through .hbm.xml files. Not a big deal…(actually there is something called Fluent that solves this problem quite interestingly).

After going through the help topics, this blog and this interview (ACM Queue), it seems that ADO.NET EF is Microsoft’s offering for ORM. The EF consists of an Entity Data Model (EDM), Object Services, Language Integrated Query (LINQ) and a few other miscellaneous stuff. The EDM essentially defines an object model that is created out of the “conceptual” data model rather than the “logical” data model (approach taken by most ORM tools including NHibernate). This has the advantage of separating out “logical” level details from the mappings. The “logical” level bindings are handled by the framework behind the scenes. The Object Services includes data source management, transactions, and other services. LINQ defines an abstract query algebra that can be used to query any collection. This query algebra is then implemented by various languages in the .NET world and even more interestingly have been integrated into those languages as first-class language constructs. LINQ implementations take care of another major issue, that of type matching. SQL types and the language type differences and conversions are all handled transparently by the LINQ implementations.

Hopefully, my understanding of the ADO.NET Entity Framework is correct. It might go through some more edits….

Powered by ScribeFire.

December 10, 2009

About testing

Filed under: Application Development — Soumya @ 8:55 pm

OK, this one always gets me…what are the various kinds of testing that can be performed? I am referring to software systems testing. To me, it seems that the following are the main kinds:

  1. Unit testing: This is something a developer should be doing. As part of the software development process, a developer must make sure that the code making up the granular units of the system (functions, methods, objects) work as they are supposed to. For example, if I have a method that converts a given temperature in Celsius to Fahrenheit, then that method must work irrespective of how the method is used by other units of the system. Sometimes, some units are connected to other units (big surprise). For example a service class may be depending on a DAO implementation for saving an object state in a database. So, going by the definition of what a unit test is, how can we unit test the service class? The answer is by using “mock objects”, whereby, the DAO implementation is “mocked” to produce a simulated behavior but without the baggage that a typical DAO comes with.
  2. Integration testing: This is again something a developer should be doing. However, this may not be part of his/her daily chores. It can be done once the unit tests have passed. This may be regularly done in an automated fashion by a continuous integration server. In this activity, all units must work cohesively in a real-world situation. In other words, the service, the DAO and the database must work in concert to behave in the way the coarse-grained service methods are supposed to.
  3. Functional testing: Individual units may work perfectly. The units together also behave perfectly. However, it still might not make any semantic sense, or it may not produce the effect desired in the user stories. Functional testing tries to make sure that the code is achieving the correct semantic sense. It should not be performed by developers. It can be performed by someone who is not very involved in the development process but is closer to the product owner. Nowadays, certain tools may allow this activity to be automated, so it might make sense to put this activity of a continuous integration process.
  4. Performance testing: Hopefully, this one is easy to understand. This activity makes sure that performance metrics are being met.
  5. Acceptance testing: In many definitions, acceptance testing and functional testing seem to be synonymous. However, I disagree. In my opinion, acceptance testing is based on acceptance criteria: criteria for semantic sense (developed during a backlog development), criteria for security, criteria for performance and so on. If the client (product owner or what have you), accepts the fully built software based on the criteria that have been established, then it can be said that acceptance testing has passed. Obviously, for a software system to be even considered for acceptance testing, all the other testing activities listed before must be successful. Additionally, there may be other non-software criteria that need to be fulfilled.

Although I always tend to approach system issues with my software-glasses on, in this particular case, I am taking a wider “system” look. The above definitions assume a system to be the software, database(s), file(s) etc that make up the solution. I think we can throw hardware and network in it too. All these pieces must work together for a system to be successful.

Hopefully this is a classification that will stand the test of time. Seems consistent with Wikipedia.

December 9, 2009

Multiple Profiles on Facebook

Filed under: Uncategorized — Soumya @ 8:41 pm

Hmmm…I ran into an interesting dilemma. Like the other one hundred zillion people, I have a Facebook profile too. Initially my intent was to socialize (big surprise) and started accepting friend requests from anyone and everyone. I guess that is the goal, after all it is a social network. However, my problem now is that my personal friends, my business acquaintances, my current coworkers, my past coworkers are all in my “friend” network. I realized that I maybe social but I am not very tolerant of the following facts:

  1. Too much useless “Status Updates”…
  2. My coworkers knowing about my personal friends…
  3. Too much noise in general…

I wish there was a way to categorize all my contacts. (Of course, in many cases, my past coworkers have become good friends.) I wish I could have create multiple accounts (still thinking along the lines of email…). So I looked at Facebook help and I was surprised to find that creating multiple accounts is actually against FB’s terms of use. Wow!

Seems like FB Pages is the recommended way to go. Let’s see….

October 18, 2009

Fix to the incorrect parsing of HTML

Filed under: Uncategorized — Soumya @ 7:31 pm

Seems like I found a fix for the problem due to which HTML embedded in my posts (like links etc) were being parsed incorrectly. Here is a link that discusses the issue: http://core.trac.wordpress.org/ticket/7771. Turns out that the problem lies with the fact that the XML parser on the WP server side was not parsing the XML that was coming in through the XML-RPC payload. There is a plugin that works around this problem. That plugin is here: http://josephscott.org/archives/2009/02/update-on-libxml2-issues/.
I have downloaded the plugin and installed it. Hopefully it would work. This post is a test. The proper fix is to update to the correct PHP version. I will get to it someday….

September 25, 2009

First impressions about ScribeFire

Filed under: Uncategorized — Soumya @ 4:02 pm

Liked it. Definitely the kind of tool that I wanted. However, I don’t understand one thing…and this is not specific to ScribeFire. Anytime I try to add a link through a desktop tool (or something like ScribeFire), the final rendered product seems to strip out the angle brackets. Is this something on the blog server side? Maybe a setting or something. I would imagine that the client (desktop tool or ScribeFire) would send a properly encoded contents which would mean converting the angle brackets to the right encoded values. Maybe the encoding negotiations are off, need to find out….br /br /div class=”zemanta-pixie”img class=”zemanta-pixie-img” alt=”" src=”http://img.zemanta.com/pixy.gif?x-id=d8a08d19-935d-8937-aa3b-ba5233da94e2″ //div

Older Posts »

Powered by WordPress