arrow Wednesday, 21 February 2018  
Main Menu
Home
Quality Snaps
TechBlog
PennCharts
Blog
Family
Auld Frosties
Links
Players
Players (uni)
Players (camb)
Contact Us
Pennchart Recent
pennchart radio

ESRI / GIS Technical Consultant Questions

Had to do an ESRI / GIS technical consultant interview today, and had to rustle up some questions.
Thanks to this post, which gave me a few starting points. I tried to keep the questions specific enough to require some actual knowledge but generic enough to encourage a fireside chat about the topic. The hope is that a question should elicit some sort of conversation based upon experiences etc. If you want to contribute more, send me another question @ppenn on twitter.

ESRI / GIS Technical Questions

  • What is the difference between projected coordinated system and geographic coordinate systems? When have you used either?
  • What are the limitations of Shapefile format?
  • What are the building blocks of a geodatabase?
  • What are the types of GeoDatabase? When would you use each?
  • What’s reverse Geocoding? What are the key challenges you have faced when Geocoding?
  • You want to automate a Geodatabase process (e.g. nightly compression). How do you go about that? (Geoprocessing tool)
  • How do you customize ArcMap & pros/cons of each? (commands/addins, latter can be in python/c# and “drops in”, no COM)
  • What types of domain are there?
  • Describe Versioning workflow (If two persons updating the particular row of the table in database, how does the system work?.)
  • How do you load data into ArcSDE? (bulk/simple vs object)
  • How do you create a view in ArcSDE?
  • How do you make edits to ArcSDE using native SQL? Any issues? (Multiversioned-views/VersionedViews. Speed)
  • How would you handle dealing with a more sophisticated editing model including supervisors & approval?
  • How do you communicate your geodatabase model with business users?
  • You have a desktop with a performance problem (e.g. a Search is taking 2 minutes to complete). How do you go about diagnosing it?
  • You suspect there is a bug in core ESRI/ArcFM software. What is the process you typically follow?

  • You have a sluggish Web API (SL/Flex/JSAPI) application. How do you diagnose it? (Looking for Fiddler or equivalent for diagnosis).
  • How do you reproduce an issue with ArcGIS Server? (Looking for reproducing via REST, not Web Application or Load Testing software)
  • What are the limitations with ESRI’s versioning system and how are they typically overcome? (Looking for GDBM knowledge, issues of single-threaded posting)
  • Have you designed an Enterprise GDB model? If so, what & what did you use?
  • Have you adapted an ESRI industry model? Issues / Experience?
  • Describe the main classes in an (Electric/Gas/Water) model
  • What types of Geometry have you worked with in the geodatabase? Pros & Cons?
  • You want to store average current speeds along segments of a road. How would you model that?
  • Your model includes county polygons. How do you ensure that counties do not overlap? What are the issues?
  • Your model includes connected features such as pipes or conductors. How do build the model so that you can trace upstream/downstream? What are the issues?
  • What are the issues with relating features from an external system (e.g. CIS, Billing) with features in the Geodatabase?
  • How would you model the following related objects in the geodatabase; buildings, rooms, doors, elevators?
  • Your model includes buildings with address attribution (e.g. street number/name/town/zip), but is incomplete. How do you provide an address geocoding solution which caters for that?
  • You want to display data from an aspatial table on the map. What are the options?
  • Can you give an example of when you would want to use a representation in the Geodatabase?
  • A customer reports sluggish performance during peak traffic times and suspects the hardware is underpowered. How do you investigate that (capacity planning tool)
  • When do you recommend non-versioned editing over versioned editing? Issues? (undo/redo, issues are geometric network doesn’t support)
  • When do you recommend _not_ using Citrix/RDP for editing? (3d)
  • How would you implement featureclass-specific rules in the geodatabase at the field or feature level?
  • What are the web APIs & strengths/weaknesses of each?
  • How do you debug a web API-based application?
  • How do you design a jsapi app (frameworks)?
  • How do you handle unit testing your custom code (frameworks, approach etc)? Any difficulties? (Mocking, Test Databases)
  • You want to provide an external system the ability to execute some GIS-based functionality in the Geodatabase (e.g. retrieve list of service points on a secondary overhead). What are the options for architecting that, including pros/cons?
  • You have built a fully-featured end-to-end geodatabase and front-end (e.g. ArcMap/WebAPI/Mobile) application. But it is slower than sludge. What steps do you take to address this? Where do you look for issues? How do you resolve issues if you need to take it to ESRI?
  • How do you secure ArcGIS Server e.g. for internet access? Issues?
  • Why would you want to cache services in ArcGIS Server? What issues are there?
  • What are the options for serving rasters? Issues? (Raster Catalog vs Mosaic Dataset, pre-built pyramid vs on-the-fly)
  • Your geodatabase contains data you want to display in Google Earth, what do you use to achieve that?
  • How would you go about exporting to / importing from another format e.g. AutoCAD? What are the issues? (OOTB GP tool doesn’t expose all AutoCAD attributes, FME is a stronger solution)
  • When would you use ArcObjects instead of ArcPy? What are the weaknesses of ArcObjects?
  • You want to make some edits to featureclass in ArcSDE. Which object model & objects do you need to work with?
  • When and why would you create a ServerObjectExtension?
  • You have a 40 data editors maintaining a Geodatabase. You decide to create a read-only web application using jsapi. How would you architect that? (Replication)
  • ArcFM: When do you use Field Model names vs Object Model Names? How do you define them?
  • ArcFM: What are the different types of AU?
  • ArcFM: Difference between Validation Rules and AUs — when to use each?
  • ArcFM: How do you test Feeder Manager is configured correctly (TAF/Stored Displays)
  • ArcFM: What’s Geodatabase Manager?
  • ArcFM: What’s Network Adapter?

Tour de Mobile ESRI Flex

Back at devsummit this year, I chanced upon a fantastic presentation by @mraad which demonstrated
embryonic Flex 4.5 Mobile capabilities and the ESRI Map control running within it. He published the project
as a simple map-and-query effort. I have cannibalized and extended the project with some integrated
device functionality, in an effort towards making a lightweight Tour De Mobile Flex for ESRI.

Here’s the functionality I wanted to show:-

  • Show Address on map (Reverse Geocode)
  • Measure Distance (Geoprocessing)
  • Create Points in FeatureClass (editing)
  • Locate Me on the map (use GPS)
  • Take photo and upload as attachment to amazon ec2 hosted geodatabase (use Camera)
  • Store / Retrieve layer URL in local database for dynamic layer configuration. Necessitated by ec2
    instance changing IPs every time I rebooted it.(use device database)

Let’s see you do any of the last 3 in Javascript…

Demo

Code

Here’s the code. It’s fugly, and barely at prototype quality, no patterns, consistency etc. Just pasted from various samples, basically, to show proof of concept.

If you’re going to run the code be sure to change / modify the featurelayer by clicking the “Set AGS Layer” button from the tools dialog; or in code.

Deja Vu (Write once, run anywhere)

Remember the 90s? A new technology offered developers to write one language which could be run on any
operating system — Java. In terms of developing apps, the mobile platform market is roughly at 1990 -
there is total fragmentation between OS vendors. .NET, Java and Objective-C are respectively tethered to Windows/Android/Apple.
So it seems that the time is right (as if it ever weren’t) for someone to develop a platform
for giving developers a single language to push out an app.

I have no idea if Flex is going to be the solution, but it’s not exactly going to be Objective C, is it (please)? The hello-world tutorial for Objective-C is about 50 steps. Ouch. Do I really want to learn a platform-specific language of
1980s complexity/toolsupport, or do I want a nice-and-easy-lemon-squeezey developer experience? Flex
is so easy it’s laughable. You want full-on GIS editing capability — just paste & tweak from the samples and you’re there in minutes. OK, enough religion.

Woah!

OK, so we’re not in the promised land just yet. I’m sure running Flex Mobile with ESRI on an iPad2 is fine;
but for the iPhone4 it was painful bordering on unusable at startup time. The application will
timeout before initialization is complete unless you touch the screen. Not exactly production ready but once it’s up & running performance is OK. Some more specifics:

  • Pinch Zooming. At first it looks to work as you’d expect, but on closer inspection pinching the
    screen causes it to shimmy around
  • Because the Mobile framework is spark-based, not mx-based, there are certain things that just
    don’t work when you deploy to iOS. Flashing graphics, for example.
  • InfoWindow doesn’t work so I had to throw something together for that.
  • Double-tapping doesn’t work (had to create a “Done” button for DrawTool stuff).

Relating to iOS:

  • No debugging on iOS by USB yet (exists for Android), so testing anything that required device integration was painful
  • Compiling takes several minutes & deploying is a pain (iTunes…and handing over $99 to Apple. Ugh).

Relating to Flex Mobile:

  • Flex Mobile API is designed from a screen-input perspective, with a hierarchy of screens. It’s a little
    inflexible when it comes to deviating from that pattern. With regards to a GIS environment, it’s not possible to
    swap out action buttons within the same view without reverting to skinning, which was too black-belt for me

Conclusion

Hopefully ESRI will provide a spark-based touch-sensitvie map control in future because cross-platform
development in an established language is amazingly productive. I’m equally sure that by the time phones are all dual-core
the performance problems will be a thing of the past. Using Flex gives you lots of bang for your buck in terms of rapid development, but the real gain is the fact that you can write once and deploy to Blackberry/Android/iOS. This paradigm has to be the future.

Phil Penn (@ppenn)

Reading ArcGISServer Cached Tiles from REST / C#

A modification to my print service required the ability to pull tiles from the webserver for reasons of performance when printing cached map services. Using the REST API gets really slow when images get large. Reading the tiles directly is how the Flex/SL/Javascript clients work.

The Challenge
Given an extent and a size, create a single image representing that extent from the cache without resorting to REST export. Weapons of choice: C#, using GDI+ for image manipulation and NewtonSoft for JSON deserialization. This is not a million miles from the arcscripts Map2PDF (http://arcscripts.esri.com/details.asp?dbid=16432), although this is performed entirely server-side, without access to aforementioned ESRI clients.

The algorithm

  1. Read REST Service to discover Resolution & Tiles Origin
  2. Find Overlapping Tiles
  3. Georeference the Tiles
  4. Download the Tiles
  5. Mosaic, Crop & Scale the Tiles

The ESRI Javascript API has a method called TileUtils.getCandidateTileInfo() which is responsible for the first two, so it was mostly a case of viewing the source in a browser and converting to C#.

Read REST Service to discover Resolution & Tiles Origin
By appending &f=json to the standard MapServer layer URL, the REST API returns all the tile information you need. Using Newtonsoft means you don’t have to waste time defining the structure returned. In addition to the origin, the LODs (Levels Of Detail) are what you’re after.


string url = _layerURL + "?f=json";

string jsonResponse = SendGetRequest(url);
JObject o = JObject.Parse(jsonResponse);
_serviceTileWidth = (int)o["tileInfo"]["rows"];
_serviceTileHeight = (int)o["tileInfo"]["cols"];
_dpi = (int)o["tileInfo"]["dpi"];
_originX = Convert.ToDouble(o["tileInfo"]["origin"]["x"].ToString());
_originY = Convert.ToDouble(o["tileInfo"]["origin"]["y"].ToString());

JArray lods = (JArray)o["tileInfo"]["lods"];

To get the applicable LOD for your extent I just converted the javascript (complete with cryptic names):


// Just taken ESRI's Javascript routine and plugging that in...
private JObject GetLevelOfDetail(JArray lods)
{
var wd = _sizeX;
var ht = _sizeY;
var ew = Math.Abs(xmax - xmin);
var eh = Math.Abs(ymax - ymin);
double ed = -1;
double ced;
JObject lod = null;

for (int i = 0; i < lods.Count(); i++)
{
JObject currentLOD = (JObject)lods[i];
double resolution = (double)currentLOD["resolution"];

ced = ew > eh ? Math.Abs(eh - (ht * resolution)) : Math.Abs(ew - (wd * resolution));
if (ed < 0 || ced <= ed)
{
lod = currentLOD;
ed = ced;
}
else
{
break;
}
}

return lod;
}

Find Overlapping Tiles
Now that you’ve read your metadata, you will want to find the tiles overlapping your extent.


// Gets the overlapping (candidate) tile for given coordinate.
// Call this with topleft/bottomright coordinates from your extent.
private void GetCandidateTile(double x, double y)
{
int tw = _serviceTileWidth;
int th = _serviceTileHeight;
double tmw = tw * _resolution;
double tmh = th * _resolution;

// This is the row/column required for the URL
int tr = (int)Math.Floor((_originY - y) / tmh);
int tc = (int)Math.Floor((x - _originX) / tmw);

double tmox = _originX + (tc * tmw);
double tmoy = _originY - (tr * tmh);

// This is how far from the tile origin, in pixels, our coordinates are
_offsetX = (int)Math.Floor(Math.Abs((x - tmox) * (tw / tmw)));//+mv.x
_offsetY = (int)Math.Floor(Math.Abs((y - tmoy) * (th / tmh)));//+mv.y;

// Save for later (yes, I didn't need the intermediate variables but wanted to leave source as-is)
_row = Convert.ToInt32(tr);
_column = Convert.ToInt32(tc);

_tilesOriginX = tmox;
_tilesOriginY = tmoy;
}

Georeference the Tiles
As you can see from the first tile calculation, we are also getting some offsets (in pixels) for where our extent starts in the first tile. We also need to do that for the last (lower right) tile:


private void GetLowerRightOffset(double x, double y)
{
int tr = (int)Math.Floor((_originY - y) / _serviceTileMapHeight);
int tc = (int)Math.Floor((x - _originX) / _serviceTileMapWidth);

double tmox = _originX + (tc * _serviceTileMapWidth) + _serviceTileMapWidth;
double tmoy = _originY - (tr * _serviceTileMapHeight) - _serviceTileMapHeight;

_lowerRightOffsetX = (int)Math.Floor(Math.Abs((tmox - x) * _serviceTileWidth / _serviceTileMapWidth));//+mv.x
_lowerRightOffsetY = (int)Math.Floor(Math.Abs((tmoy - y) * _serviceTileHeight / _serviceTileMapHeight));//+mv.y;

_gridRows = Math.Abs(tr - _row) + 1;
_gridColumns = Math.Abs(tc - _column) + 1;
}

Download the Tiles
Now we have gathered everything we need except for the tiles themselves. Each tile is in the format: /tile/LOD/row/column e.g. http://server.arcgisonline.com/ArcGIS/rest/services/World_Street_Map/MapServer/tile/3/300/200

For the actual downloading, I used a boss-worker threading sample from msdn, so I calculate the tiles to download first. Do this by walking along the grid.


public void CalculateTiles()
{
int mapImageX = 0;
int mapImageY = 0;

_tiles2DArray = new CandidateTile[_gridRows, _gridColumns];
// Left-to-right
for (int x = 0; x < _gridColumns; x++)
{
// Top-to-bottom
for (int y = 0; y < _gridRows; y++)
{
// URL for overlapping tile
string tileUrl = _layer.Url + "/tile/" + ((int)_lod["level"]).ToString() + "/" + (_row + y).ToString() +
"/" + (_column + x).ToString();
CandidateTile candidateTile = new CandidateTile(tileUrl, y, x, mapImageX, mapImageY);
candidateTile.SetClipExtent(0, 0, _serviceTileWidth, _serviceTileHeight);
_tiles2DArray[y, x] = candidateTile;

mapImageY += _serviceTileHeight;
}
// Reset Y, we're going left-to-right, top-to-bottom
mapImageY = 0;
// Reset map coords for x
mapImageX += _serviceTileWidth;// (clipXMax - clipXMin);
}

}

Mosaic, Crop & Scale the Tiles

Now for the fun part. Blend all the images together.


Image uncroppedTiles = new Bitmap((_gridColumns)*_serviceTileWidth, (_gridRows)*_serviceTileHeight, PixelFormat.Format24bppRgb);
Graphics g2d = Graphics.FromImage(uncroppedTiles);

foreach (CandidateTile tile in _tiles)
{
Image tileImage = GetImageByUrl(tile.URL);
g2d.DrawImage(tileImage, tile.mapImageX, tile.mapImageY, tileImage.Width, tileImage.Height);
}

This is probably a very inelegant gdi+ solution…but it works


private Image DrawCroppedTiles(Image uncroppedTiles)
{
// This routine should chop the upper left and lower right offsets off, then scale to target size

// Remove the offset so our origin is where it should be. TOCHECK: is the origin where it actually should be?
Image croppedTiles = new Bitmap(uncroppedTiles.Width - _offsetX, uncroppedTiles.Height - _offsetY, PixelFormat.Format32bppArgb);
Graphics g2d = Graphics.FromImage(croppedTiles);
// Create rectangle for displaying image.
Rectangle destRect = new Rectangle(0, 0, croppedTiles.Width, croppedTiles.Height);
Rectangle srcRect = new Rectangle(_offsetX, _offsetY, uncroppedTiles.Width - _offsetX,
uncroppedTiles.Height - _offsetY);
GraphicsUnit units = GraphicsUnit.Pixel;
g2d.DrawImage(uncroppedTiles, destRect, srcRect, units);
croppedTiles.Save(@"c:\temp\cropped.png", System.Drawing.Imaging.ImageFormat.Png);
g2d.Dispose();
Image croppedAtBothEndsImage = new Bitmap(croppedTiles.Width - _lowerRightOffsetX,
croppedTiles.Height - _lowerRightOffsetY,PixelFormat.Format32bppArgb);
g2d = Graphics.FromImage(croppedAtBothEndsImage);
g2d.DrawImageUnscaled(croppedTiles, 0, 0);
croppedAtBothEndsImage.Save(@"c:\temp\cropped_both_ends.png", System.Drawing.Imaging.ImageFormat.Png);
Image croppedClipped = new Bitmap(croppedAtBothEndsImage, _width, _height);

return croppedClipped;
}

Note that the image will get scaled (stretched) and look a little washed out if your cache doesn’t go down to the requisite scale.

Wow, that was actually easy (skype api)

Requirement
To use skype API to place a call at a future point in time, navigate through some phone options (”press one for…”) and leave a voicemail from a WAV file.

This would likely be a bat file called make_call
make_call.exe [phone_number] [wav_file] [navigation_sequence]

Background

Babies...Phones

Childcare is frequently referred to as healthcare in my household — a necessary break to recharge the adult batteries. In fact, we chose our gym based on its provision of such services. The problem is that requests to have little johnny taken care of are over-subscribed; too much demand (parents) chasing too few services (daycare). So the gym in question has a policy of only allowing parents to book 2 days ahead. And a day, in its eyes, starts at midnight. The result is that we have to make a reservation by phone at one minute past midnight. In days of old this would have been no problemo, but now it’s a non-starter to infringe on precious sleeping rights

Skype API
As it would happen, there are 3 APIs out there; Python, Java & COM. Trying to be impartial, I decided to use whichever API had the most samples. Which was Microsoft, of course :)

The Good
Although I did read something giving the API a good slaggin for speed, it seemed mighty easy to program against.
ICall call = skype.PlaceCall(callee);
That’s code even a numpty like me can believe in. The whole thing took <4 hours to get going. Va va vooooom!

The Bad
Changing audio properties for playback of the WAV required installation of an (albeit free) driver, Virtual Audio Cable. All documented here.
It was all made a little trickier by testing via Remote Desktop. Remember to click the Local Resources options tab and configure Remote Audio.

The Ugly
The only rubbish part of the whole thing is that the API doesn’t allow you to authenticate, meaning you had better check the “sign me in automatically” option to skype or you will nae get it working. It also forced me to use a scheduled task as opposed to “at”, which would have been preferable.

OK, another rubbish part is that WAV files have to conform to this spec
File: WAV PCM
Sockets: raw PCM samples
16 KHz mono, 16 bit

and the otherwise loveable Windows 7 doesn’t have a proper sound recorder anymore (grrrr…Microsoft). Their one records in _wma_ format. WTB?? Anyway, you can grab a copy of XP’s sndrec32.exe and continue to use that (albeit with dodgy warnings). But this isn’t skype’s fault, so let’s move on.

C# Code
Precious little to it beyond sleeping and getting duration on the WAV.


using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading;
using SKYPE4COMLib;

namespace SkypeTest
{
class Program
{
static void Main(string[] args)
{
try
{
string DTMF = "";
if (args.Length < 3)
{
Usage();
System.Environment.Exit(0);
}
string callee = args[0];
string wavFile = args[1];
int answerDelay = Convert.ToInt32(args[2]);
if (args.Length == 4)
{
DTMF = args[3];
}
int millisecondsToWait = GetSoundLength(wavFile);
SKYPE4COMLib.Skype skype = new Skype();

if (skype.Client.IsRunning == false)
{
skype.Client.Start(true, true);
}
skype.Attach(6, true);
ICall call = skype.PlaceCall(callee, null, null, null);

while (call.Status != TCallStatus.clsInProgress)
{
if (call.Status == TCallStatus.clsBusy ||
call.Status == TCallStatus.clsCancelled ||
call.Status == TCallStatus.clsRefused ||
call.Status == TCallStatus.clsFailed)
{
throw new ArgumentException("Call problems");
}

Thread.Sleep(500);
}
Thread.Sleep(answerDelay);
if (DTMF != "")
{
call.DTMF = DTMF;
Thread.Sleep(1500);
}
call.set_InputDevice(TCallIoDeviceType.callIoDeviceTypeFile, wavFile);
Thread.Sleep(millisecondsToWait);
Thread.Sleep(500); // half a second for luck

// Night night
if (call.Status != TCallStatus.clsFinished) call.Finish();
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
static private void Usage()
{
Console.WriteLine("Usage: SkypeTest [phone_number/user] [wav_file] [delay_seconds] [[control char]]");
Console.WriteLine("Example: SkypeTest ppenn11 #c:\temp\applause.wav 3 #");
}
static int GetWavFileLength(string wavFile)
{
FileInfo fi = new FileInfo(wavFile);
long fileSize = fi.Length;
double secondsExact = ((fileSize - 44)/(44100*(16/2))/2);
return Convert.ToInt32(Math.Ceiling(secondsExact));
}
[DllImport("winmm.dll")]
private static extern uint mciSendString(
string command,
StringBuilder returnValue,
int returnLength,
IntPtr winHandle);

public static int GetSoundLength(string fileName)
{

StringBuilder lengthBuf = new StringBuilder(32);

mciSendString(string.Format("open \"{0}\" type waveaudio alias wave", fileName), null, 0, IntPtr.Zero);
mciSendString("status wave length", lengthBuf, lengthBuf.Capacity, IntPtr.Zero);
mciSendString("close wave", null, 0, IntPtr.Zero);

int length = 0;
int.TryParse(lengthBuf.ToString(), out length);

return length;

}

}
}

Respect to some geez called the uberoverlord.

Converting Lat/Long to/from Web Mercator in C#

ESRI has a utility class for converting Lat Long from/to Web Mercator for use with bing/google maps.

They provide this functionality through a dll (ESRI.ArcGIS.Client.Bing) which requires Silverlight to be installed. Don’t have/want Silverlight on your web server, don’t have/want ArcObjects? Then nab this:

private void WebMercatorToGeographic(double mercatorX, double mercatorY, out double lat, out double lon)
{
if ((mercatorX < -20037508.3427892) || (mercatorX > 20037508.3427892))
{
throw new ArgumentException(”Point does not fall within a valid range of the mercator projection.”);
}
double x = mercatorX;
double y = mercatorY;
double num3 = x / 6378137.0;
double num4 = num3 * 57.295779513082323;
double num5 = Math.Floor((double)((num4 + 180.0) / 360.0));
double num6 = num4 - (num5 * 360.0);
double num7 = 1.5707963267948966 - (2.0 * Math.Atan(Math.Exp((-1.0 * y) / 6378137.0)));
lat = num6;
lon = num7*57.295779513082323;
}

private void GeographicToWebMercator(double lat, double lon, out double mercatorX, out double mercatorY)
{
if ((lat < -90.0) || (lon > 90.0))
{
throw new ArgumentException(”Point does not fall within a valid range of a geographic coordinate system.”);
}
double num = lat * 0.017453292519943295;
double x = 6378137.0 * num;
double a = lon * 0.017453292519943295;

mercatorX = x;
mercatorY = 3189068.5*Math.Log((1.0 + Math.Sin(a))/(1.0 - Math.Sin(a)));
}

ArcGIS Web Editing Quagmire

We have a client with a slick-looking Flex GIS client. All good, but now they want to add editing to the mix. Editing with Flex isn’t until 9.4 (why is it that every feature you ever want is in the _next_ version?), which is out of the timeframe for delivery so we’re stuck between a rock and a hard place; the stubborn rock of the ADF and the hard place of writing product where there is none in Flex/WCF/SOE.

ADF is pretty well documented. For Flex we would have to have an architecture like so:

Flex <-> WCF <-> EditingSOE <-> Nonversioned / Default Geodatabase

Flex Woes

Here are just some of my Flex editing woes :

1) We have committed to doing job workflow for approving etc (e.g. using M&M SessionManager/ESRI JTX or Workflow Foundation). This is a major headache because it implies that we will be doing versioned editing. The REST API does not give you the ability to switch versions on-the-fly. With a stateful mapserverobject accessed by the ADF you can create/change versions.
2) Client-side editing (i.e. manipulating graphics) in flex might involve some effort without the help of an esri library. [Glenn has found this
http://thunderheadxpler.blogspot.com/2008/12/editing-graphic-geometry.html]. Second biggest woe under this heading is undo-redo.
3) Tight time constraints, the ADF editing is pretty much out-of-the-box. How long will flex editing take to implement?

Elephant in the Room

And here, for completeness, is the elephant in the room (ADF):

1) Expectation set by delivering a Flex client of a responsive, AJAX-y interface without fuss. ADF is a lumbering fool.
2) Getting the existing Flex interface to seamlessly integrate with ADF.
3) Ongoing reliability concerns, issues with stateful services.

Time to get prototyping…

NUnit /noshadow shocker

Today was one of those days…when you realize the limitations of your technical capabilities. Which in my case, is knee-high to a grasshopper.

In the course of trying to inch towards the twenty-first century, Ruprict has yet again saved my bacon from a problem which I thought was intractable.
After unsuccessfully dabbling with CIFactory, we’re using Hudson for continuous integration for our Flex project. It is beautifully simple to install & configure, and seems to be unobtrusive to the developer, except when you break the build and the results are emailed to some of your worst enemies co-developers.

Anyway, I was midway through putting the WCF Rest stuff (thanksbe to redsolo) into Hudson when I started getting bizarre errors upon trying to nunit my test project:

System.IO.FileLoadException : Could not load file or assembly ‘Newtonsoft.Jso
n, Version=2.0.0.0, Culture=neutral, PublicKeyToken=null’ or one of its dependen
cies. Access is denied.

But hang on, the Newtonsoft.Json.dll is in the same folder as the test project, what are you telling me?? So, what had worked perfectly in TDD.net inside Visual Studio (i.e. running my tests) was suddenly not so in nunit-console.exe or the nunit GUI.
So after much faffing around I handed the problem off to Rupes and he found a solution. Should you get this unexpected glitch, try applying the /noshadow tag from commandline (or equivalent option in the gui).

Salvation: REST Print Service for ArcGIS Server 9.3

At version 9.3*, the non-ADF ArcGIS web clients (Flex, Javascript, Silverlight) don’t support high quality printing. By high quality I mean variable resolution (so as to support dpi greater than 96) and the ability to use the template authoring in ArcGIS Desktop for a richer, smoother cartographic experience. If you want to create a hard copy of a map in, say Flex, printing the screen is the only option out-of-the-box. If you want to create an A3-sized PDF map with legend, borders, scale bars, North Arrows etc then you are up a murky creek with nothing but a hole-ridden wooden spoon to paddle out with.

Domenico Ciavarella came up with a bravissimo solution to the problem, tailored to the ADF, which made use of a ServerObjectExtension, LayoutSOE. Basically it involved invoking the SOE to output the PageLayout from ArcObjects. I hacked that and found that it worked fine but with the following deficiencies for the REST world:-
- No support for transparency in graphics (because ArcMap doesn’t support that, doh!)
- Not REST based, you needed to have the layers ahead of time in an MXD. This was a configuration nightmare to make sure that the layers in the print service MXD were the same as the layers in each map service that the Flex client was using. And if you were consuming other people’s map services and they changed the symbology then you were SOL.
- Too slow, adding/removing layers dynamically was taking minutes of time (especially the rasters).

Anyway, as luck would have it, my homey introduced me to a clued-up chap called Vish at this year’s ESRI Developer Conference in Palm Springs and he pointed me in a radically different direction. Kind of like Skynet in Terminator 2 after they find the remnants of Arnie from T1. I was being so amateur. What would Christian Bale think?

REST to the RESCUE

REST printing with ExportMap is fast and furious. All I would need to do was:

  • Export the images from each Map Service
  • Deserialize and draw the graphics with GDI+
  • Draw a legend, using the ESRI SOAP API
  • Export Templates (PageLayout) via an ArcObjects SOE
  • Fuse the whole lot into a single image
  • Handle writing to a PDF

To continue the Bale theme, albeit in a Adam West kinda way, to the Batcave!

The Big Picture

Image


From the client perspective, all this architecture (REST service) had to do was:

  1. Provide a list of templates (e.g. Facilities Map, Transportation Map)
  2. Output the Map with the given format (e.g. PDF/PNG) from the selected template, including current layers/graphics/extent (WYSIWYG Printing)

Two methods! How hard could that be? Well, it was more than an afternoon’s work, but I’ll spare the vanilla stuff.

SOE what?

I half-inched** the aforementioned LayoutSOE to provide a user-friendly way of publishing & listing templates via the black magic of a ServerObjectExtension.
This gives the user the ability to specify templates in Catalog:
Image

To be honest, I was a little scared of the SOE thing. All this regasm/gaccing. DCOM is something I last did with VB6 a decade ago. To allay my cross-process “Unexpected Failure” fears, I made all types plain strings and arrays.

The SOE had but two functions:

  • Return Array of Templates. Done! That’s my kinda coding
  • Return PrintTemplateInfo. This was an object holding the PageLayout information e.g. map bounds, the legend bounds (if any). It was also responsible for generating the template as a PNG, for consumption by the REST service. More on that later

The second of these involved some reasonably involved ArcObjects, but I checked out everything in VBA before committing to .NET SOE. Interactively discovering ArcObjects is sooo much easier than try-compile-test in .NET. Once that PageLayout metadata was extracted, time to export the image with marginalia on it and an extent set so as to draw a scalebar. Again, the LayoutSOE helps with all that. Tante Grazie, Domenico!
Boom, baby! We’re ready to roll onto the sexy REST side. Time to let ArcObjects fade into obscurity, may it (and its retarded non-exception handling) never cross our paths again…

REST easy
The core of this architecture is a WCF-assisted REST service, which has no ties to ArcGIS Server except for connecting to the SOE. So all it needs is the Web ADF Runtime (free again at 9.3.1). The service exposes a GET service to GetTemplates, which is pretty much a passthrough to the SOE. The other service is a POST service to CreateMap, which is where the meat of the matter lies. Fasten your seatbelts, I’m going to hit the code button a few times…

Export Templates (PageLayout) via an ArcObjects SOE
The first thing that CreateMap had to do was generate the template with appropriate marginalia and scalebar. Not only was the template returned as a URL but also the extents of the map and legend within the pagelayout. From the map bounds the size was calculated (dpi * inches).

Export the images from each Map Service
CreateMap’s primary duty was to draw images from each map service, remembering the extent and the visible layers. This entailed nothing more than taking the incoming print request and converting it to a URL for executing ExportMap on the REST server. I used PNG so as not to lose quality.
So, having built a URL like “http://localhost/ArcGIS/rest/services/Basemaps/Street/MapServer/export?bbox=7659147.07493616,704961.792450929,7668305.86853904,711693.897150482&bboxSR=2913&imageSR=2913&layers=show:0,1&size=761,653&format=png24&dpi=96&f=json&transparent=false”
The extent you ask for is not the extent that you get when calling ExportMap, so we take that too. It will help with graphics transformation.

Here’s the down & dirty on getting an image from the ESRI REST service

WebRequest request = WebRequest.Create(url);
WebResponse response = request.GetResponse();
remoteStream = response.GetResponseStream();
readStream = new StreamReader(remoteStream);
System.Drawing.Image img = System.Drawing.Image.FromStream(remoteStream);

From this small acorn the might oak of REST printing will flourish…

Deserialize and draw the graphics with GDI+
Cycling through the graphics was a little daunting because it entailed diving into GDI+. But it turned out to be surprisingly straightforward, unArcObjects-esque.
The only snaggette was transforming coordinates to page coordinates, which I had to do myself. I didn’t get into GIS to sully my hands with this sort of thing, but anyway 6th grade Math(s) sufficed.
An example of drawing a square marker…


private void DrawSquare(Graphics g2d, float x, float y, float size)
{
g2d.FillRectangle(_symBrush, (float)(x - 0.5 * size), (float)(y - 0.5 * size), (float)size, (float)size);
g2d.DrawRectangle(_symPen, (float)(x - 0.5 * size), (float)(y - 0.5 * size), (float)size, (float)size);

}

Draw a legend, using the ESRI SOAP API
We had already dabbled with the Legend in Flex because the Flex API doesn’t bother drawing it for you.
So this was a case of calling that and drawing the results into a .NET image, again with GDI+.


MapServerProxy93 mapProxy = new MapServerProxy93(mapServiceURL.Replace("/rest/", "/"));

ImageType imgType = new ImageType();
imgType.ImageFormat = esriImageFormat.esriImagePNG24;
imgType.ImageReturnType = esriImageReturnType.esriImageReturnMimeData;
return mapProxy.GetLegendInfo(mapProxy.GetDefaultMapName(), null, null, imgType);

Fuse the whole lot into a single image
OK, OK, you have all of these images, how do you blend them into a single one fit for printing? Again, GDI+ to your salvation.
You grab a graphics context with the bottom image, which would be the first map service to draw (e.g. basemap aerial/street etc).

//create the graphic from the original base image
g2d = Graphics.FromImage(bottomImage);

Once you have that then Robert is your father’s brother.


static private void AddImageToGraphics(Graphics g2d, Image image, int x, int y, float imageOpacity)
{
System.Drawing.Imaging.ColorMatrix cm = new System.Drawing.Imaging.ColorMatrix();
cm.Matrix00 = cm.Matrix11 = cm.Matrix22 = cm.Matrix44 = 1;
cm.Matrix33 = imageOpacity;
System.Drawing.Imaging.ImageAttributes ia = new System.Drawing.Imaging.ImageAttributes();
ia.SetColorMatrix(cm);

g2d.DrawImage(image, new System.Drawing.Rectangle(x, y, image.Width, image.Height), 0, 0, image.Width, image.Height,
GraphicsUnit.Pixel, ia);
g2d.Save();

}

That’s it, you can keep doing that ad finitum. So, stack all your map service images, one on top of the other. Then add your graphics. Then your legend. Finally, put the template on top of it and you are done.

Handle writing to a PDF
Ah, but Imaging.Formats doesn’t include PDF. Nope, for that we’ll have to use iTextSharp — a free PDF writer for C#. It’s pretty simple to use.
Couple of lines of code to add the image to the PDF and finito.


iTextSharp.text.Image image = iTextSharp.text.Image.GetInstance(mapImage, null);
cb.AddImage(image);

Flex fool
Suffice to say that from the Flex client it was necessary to serialize the layers and the graphics to JSON. There were a few niggly parts to that, but mostly ESRI’s json serializer did the trick.

That’s about it, well, plenty more to tell but I can hear gongs in the background — I am out of time. At the end you have a blazing fast high quality cartographic map production engine for the web. As Arthur Daley once said, on ne peut pas dire plus juste que ca.


* All this will be remedied at 9.4 of ArcGIS. Why does it seem that half my gis life has been spent writing features that were just around the corner in the next version of the product?
** pinched, Cockney Rhyming slang, innit.

Testing

ipsum lorum why have i forgotten my latin?

Hello world!

Joomla! is the ideal tool for developing small to large dynamic community websites, intra company portals, corporate portals, weblogs and much more.

The MojoBlog component is a integration of WordPress within Joomla, Our aim is to develop a more semantic, flexible and powerful blogging tool for Joomla! users.

Frosties RandomSnap
kt_hood

kt_hood

top of page