Monday, 23 April 2018

Otago Central Railway [61M]: Cromwell-Alexandra 10

Last time I wrote about the process of fixing up overlap issues in the Linz aerial photography background tiles, specifically with relation to the Cromwell Gorge section of the maps, which is about half of the total distance from Cromwell to Alexandra. The work on that has comprised Steps 1 to 9 out of 10.

Step 10 is the optional, and more intensive, part of the work. Because all the retro aerial layers have been scaled to fit the 0.75 metres per pixel aerial photography layers that were used at the time. I have two choices. The harder one by a great many times is to rescale the aerial layers to fit the new tiles, although the tiles have the same number of pixels, 0.4 metre pixels are quite a bit smaller, and therefore each tile in the 0.4m resolution occupies less physical space when displayed on a map. This is all set out in the world file (.jgw) which I mentioned in a previous post, because this tells the map software the size of each pixel, and therefore how much area it needs to display the whole layer.

The other, much simpler, option, is to rescale the 0.4 m tiles to the same proportions as the 0.75 m tiles. This means each of the higher resolution tiles gets rescaled to 2560x3840 compared to the original resolution of 4800x7200. Once this has been done they can all be properly lined up and joined just as the regular tiles are. Given the size of the original tiles I have opted to merge the resized layers in side by side pairs into one layer, because it simplifies things like copying the masks. Copying a mask from one tile to another is fortunately quite straightforward, and just as it has been necessary to line up the first layer on the 0.75 metre layer underneath, the mask needs to be lined up on the original as well. So doing it this way although somewhat fiddly is probably a lot easier than realigning all the aerial photo layers that don't inherently line up, because the maps tiles do inherently line up and once I have the first ones aligned then I can just drop in the extra ones and line them up on the first ones. 

Once having got the 0.4 metre tiles all in place and the masks copied across then the next step is to export the tiles on the 0.75 m boundaries which lets us use their original world files to specify the placement and size of the layer. So the new layers will be 4800x7200 and will actually be the 0.75 m tiles with the 0.4 metre tiles overlaid on top of them, and using copies of the original masks in the new layers to ensure the retro aerial content is still visible exactly the same way.

If you look closely at the above image you can see the greenish original layer at the very top, with the brownish high resolution content most of the way down and some more of the green stuff at the bottom, that shows how the tiles are overlaid onto each other. A couple of gaps can be seen at the bottom where I have to fix some mask positioning issues but it should be OK.

Sunday, 22 April 2018

Otago Central Railway [61L]: Cromwell-Alexandra 9

So I am doing the tile merges in Gimp to fix up the tiles that are in two pieces with overlapping black borders so that I can use the latest and greatest quality Linz aerial photography of the entire line. This requires a number of different steps, these are as follows:
  1. Find two overlapping tiles in Qgis
  2. Line up the tile images in Gimp
  3. Add a mask to one of the images
  4. Use the Gimp select by colour function to select the black border on the masked image
  5. Edit the mask by filling in the selected area with black, this makes it transparent
  6. Line up the images so they overlap properly
  7. Merge the two layers so that the resulting layer is 4800x7200 pixels. It may be necessary to undo and realign if the result is slightly too tall or short.
  8. The next step after Gimping the tiles is to fix the .jgw world file for the merged layer. This involves combining values from the overlapped layers' .jgw files and probably also using the adjacent tiles' .jgw files for guidance. I have to find out just what the format of this file is to ensure I can put the right values into it.
  9. Once all the tiles are merged and aligned on the canvas I follow the usual procedure to export each merged tile to a new file. Then rename its sidecars to match.
  10. An optional final step to look at is re-rendering the historical layers over the 0.4 metre aerial imagery (they were originally done using the 0.75 metre imagery before I worked out how to fix the overlap issues with the 0.4 metre tiles). Since the different tiles don't magically line up, it will be a case of visual realignment between the 0.75 metre and 0.4 metre tiles in the same way as was used to line up the historical layers to the 0.75 metre aerials. Hopefully this will turn out to be a fairly routine step but I have left myself with an out of just putting the 0.75 metre tiles where used over the top of the 0.4 metre tiles if it gets too time consuming because fixing all the issues with the aerial photos for this section of the route has taken a lot more time and effort than originally expected.
Here is an example of what is completed so far in Gimp:

As can be seen here the overlap of the older (lighter) and newer (darker) imagery is on an angle rather than a straight line, for reasons best known to Linz. If you were to zoom in on the overlap boundary you would discover some bits of black still in there because removing all of these is not possible and I am finding even with some of the contiguous stuff I have downloaded there are small overlap gaps between adjacent tiles so the imagery is not perfectly aligned all of the time. It is, however, much better than Google Earth in this respect and I can tolerate these small errors as such.

Project Development: Aerial Photography Selection

Progress on the OCR maps has been slow this week because of other work. As far as the maps go, work on the Main South Line maps around Dunedin has been a greater priority. I had an enquiry for the maps around Dunedin Locomotive Depot and it was decided at the same time to download the full section of the 0.125 metre LDS coverage for the suburban area of Dunedin. This covers the MSL from about Owhiro at the south end, to as far north as Warrington at the north end, but inexplicably omits around 4 km in the Mihiwaka area. 

As usual after assembling together the areas covered by several different downloads totalling about 12 GB, selecting the images actually needed using Qgis has been the time consuming aspect. This time around instead of manually copying the files, which has always been a problem because some are always missed, I spent a bit of time extracting the filenames and running them through a simple PowerShell script to copy them to a new folder. Although there are several manual steps needed to get the list of filenames, with more work I hope to get to a stage of automating more of the process starting from saving the list of files as a layer definition file (*.qlr) as there is a lot of aerial photography I need for various aspects of the overall maps project and the selection of just the files needed for rail corridors will save a lot of disk space on MainPC where the files are stored for use in the maps. 

The process for the files for Dunedin is a typical example as the selection and copying process reduced this from nearly 1200 individual rasters down to about 250 saving about 10 GB of disk space on this small area. Whilst most of the areas being covered will not be at such a high resolution, it is to be appreciated that there is a great deal of aerial photography now required for the entire country and this does use an appreciable amount of disk space on the computers so anything which can be done to reduce this disk usage is very valuable.

Whilst I have spent a lot of time optimising the disk usage for this project, some of the other projects also need this work and after Otago-Southland project there is also the CWNM project for the rest of the South Island that I worked on recently with the Addington coverage. There is a large volume for the Main North Line that currently totals some 5000 individual rasters occupying 53 GB of disk space. Another example is the coverage needed for the Napier-Gisborne Line, a future aerial project example I have done some preliminary work on where I have obtained Retrolens aerials for many stations and LDS coverage for Napier-Waikokopu which is around 1000 layers gross and needs to be graded down to just the corridor which will be much less hopefully. As time goes on naturally there will be further usage around the country which will keep things happening.

I have also recently had an enquiry about the coverage of parts of Southland and I think there will not be a lot of the area covered. Once the Central line maps have been completed the other section required for Volume 12 is the Kingston Branch and it is natural I will want to look at maps of this line because I would want to proceed to assemble the complete volume for the PDF. 

The Kingston Branch started originally at Invercargill and ran to Kingston over a total length of 140 km. There is good Retrolens aerial coverage of only a few stations. It is possible I will do mosaics for Invercargill, Makarewa Junction, Winton Junction and Lumsden Junction, and as this line also has a few branches then there is the possibility of adding in Mararoa and Mossburn from the Mossburn Branch, Gore Junction from the Waimea Plains Railway, Thornbury Junction  and Tuatapere from the Tuatapere Branch and Wairio from the Wairio Branch. 

Since I have been asked about Southland in general the other areas for mosaics (subject to availability) could include Bluff, McNab Junction and Edendale Junction. These fall under the MSL which is Volume 11.

After all that, I will carry on with the Otago Central in the coming week with the Gimp editing of the coverage for Alexandra to Cromwell being the high priority so that the maps for that section can be generated, and as previously mentioned I just need to add in some detail of Alexandra itself. There are not any new aerial images presently available around Alexandra. It seems the scanning of the Otago region is very slow as they have not yet added the survey that I used of the Cromwell Gorge, although other surveys are available.