Monday, 23 April 2018

Otago Central Railway [61M]: Cromwell-Alexandra 10

Last time I wrote about the process of fixing up overlap issues in the Linz aerial photography background tiles, specifically with relation to the Cromwell Gorge section of the maps, which is about half of the total distance from Cromwell to Alexandra. The work on that has comprised Steps 1 to 9 out of 10.


Step 10 is the optional, and more intensive, part of the work. Because all the retro aerial layers have been scaled to fit the 0.75 metres per pixel aerial photography layers that were used at the time. I have two choices. The harder one by a great many times is to rescale the aerial layers to fit the new tiles, although the tiles have the same number of pixels, 0.4 metre pixels are quite a bit smaller, and therefore each tile in the 0.4m resolution occupies less physical space when displayed on a map. This is all set out in the world file (.jgw) which I mentioned in a previous post, because this tells the map software the size of each pixel, and therefore how much area it needs to display the whole layer.

The other, much simpler, option, is to rescale the 0.4 m tiles to the same proportions as the 0.75 m tiles. This means each of the higher resolution tiles gets rescaled to 2560x3840 compared to the original resolution of 4800x7200. Once this has been done they can all be properly lined up and joined just as the regular tiles are. Given the size of the original tiles I have opted to merge the resized layers in side by side pairs into one layer, because it simplifies things like copying the masks. Copying a mask from one tile to another is fortunately quite straightforward, and just as it has been necessary to line up the first layer on the 0.75 metre layer underneath, the mask needs to be lined up on the original as well. So doing it this way although somewhat fiddly is probably a lot easier than realigning all the aerial photo layers that don't inherently line up, because the maps tiles do inherently line up and once I have the first ones aligned then I can just drop in the extra ones and line them up on the first ones. 

Once having got the 0.4 metre tiles all in place and the masks copied across then the next step is to export the tiles on the 0.75 m boundaries which lets us use their original world files to specify the placement and size of the layer. So the new layers will be 4800x7200 and will actually be the 0.75 m tiles with the 0.4 metre tiles overlaid on top of them, and using copies of the original masks in the new layers to ensure the retro aerial content is still visible exactly the same way.


If you look closely at the above image you can see the greenish original layer at the very top, with the brownish high resolution content most of the way down and some more of the green stuff at the bottom, that shows how the tiles are overlaid onto each other. A couple of gaps can be seen at the bottom where I have to fix some mask positioning issues but it should be OK.


Sunday, 22 April 2018

Otago Central Railway [61L]: Cromwell-Alexandra 9

So I am doing the tile merges in Gimp to fix up the tiles that are in two pieces with overlapping black borders so that I can use the latest and greatest quality Linz aerial photography of the entire line. This requires a number of different steps, these are as follows:
  1. Find two overlapping tiles in Qgis
  2. Line up the tile images in Gimp
  3. Add a mask to one of the images
  4. Use the Gimp select by colour function to select the black border on the masked image
  5. Edit the mask by filling in the selected area with black, this makes it transparent
  6. Line up the images so they overlap properly
  7. Merge the two layers so that the resulting layer is 4800x7200 pixels. It may be necessary to undo and realign if the result is slightly too tall or short.
  8. The next step after Gimping the tiles is to fix the .jgw world file for the merged layer. This involves combining values from the overlapped layers' .jgw files and probably also using the adjacent tiles' .jgw files for guidance. I have to find out just what the format of this file is to ensure I can put the right values into it.
  9. Once all the tiles are merged and aligned on the canvas I follow the usual procedure to export each merged tile to a new file. Then rename its sidecars to match.
  10. An optional final step to look at is re-rendering the historical layers over the 0.4 metre aerial imagery (they were originally done using the 0.75 metre imagery before I worked out how to fix the overlap issues with the 0.4 metre tiles). Since the different tiles don't magically line up, it will be a case of visual realignment between the 0.75 metre and 0.4 metre tiles in the same way as was used to line up the historical layers to the 0.75 metre aerials. Hopefully this will turn out to be a fairly routine step but I have left myself with an out of just putting the 0.75 metre tiles where used over the top of the 0.4 metre tiles if it gets too time consuming because fixing all the issues with the aerial photos for this section of the route has taken a lot more time and effort than originally expected.
Here is an example of what is completed so far in Gimp:

As can be seen here the overlap of the older (lighter) and newer (darker) imagery is on an angle rather than a straight line, for reasons best known to Linz. If you were to zoom in on the overlap boundary you would discover some bits of black still in there because removing all of these is not possible and I am finding even with some of the contiguous stuff I have downloaded there are small overlap gaps between adjacent tiles so the imagery is not perfectly aligned all of the time. It is, however, much better than Google Earth in this respect and I can tolerate these small errors as such.

Project Development: Aerial Photography Selection

Progress on the OCR maps has been slow this week because of other work. As far as the maps go, work on the Main South Line maps around Dunedin has been a greater priority. I had an enquiry for the maps around Dunedin Locomotive Depot and it was decided at the same time to download the full section of the 0.125 metre LDS coverage for the suburban area of Dunedin. This covers the MSL from about Owhiro at the south end, to as far north as Warrington at the north end, but inexplicably omits around 4 km in the Mihiwaka area. 

As usual after assembling together the areas covered by several different downloads totalling about 12 GB, selecting the images actually needed using Qgis has been the time consuming aspect. This time around instead of manually copying the files, which has always been a problem because some are always missed, I spent a bit of time extracting the filenames and running them through a simple PowerShell script to copy them to a new folder. Although there are several manual steps needed to get the list of filenames, with more work I hope to get to a stage of automating more of the process starting from saving the list of files as a layer definition file (*.qlr) as there is a lot of aerial photography I need for various aspects of the overall maps project and the selection of just the files needed for rail corridors will save a lot of disk space on MainPC where the files are stored for use in the maps. 

The process for the files for Dunedin is a typical example as the selection and copying process reduced this from nearly 1200 individual rasters down to about 250 saving about 10 GB of disk space on this small area. Whilst most of the areas being covered will not be at such a high resolution, it is to be appreciated that there is a great deal of aerial photography now required for the entire country and this does use an appreciable amount of disk space on the computers so anything which can be done to reduce this disk usage is very valuable.

Whilst I have spent a lot of time optimising the disk usage for this project, some of the other projects also need this work and after Otago-Southland project there is also the CWNM project for the rest of the South Island that I worked on recently with the Addington coverage. There is a large volume for the Main North Line that currently totals some 5000 individual rasters occupying 53 GB of disk space. Another example is the coverage needed for the Napier-Gisborne Line, a future aerial project example I have done some preliminary work on where I have obtained Retrolens aerials for many stations and LDS coverage for Napier-Waikokopu which is around 1000 layers gross and needs to be graded down to just the corridor which will be much less hopefully. As time goes on naturally there will be further usage around the country which will keep things happening.

I have also recently had an enquiry about the coverage of parts of Southland and I think there will not be a lot of the area covered. Once the Central line maps have been completed the other section required for Volume 12 is the Kingston Branch and it is natural I will want to look at maps of this line because I would want to proceed to assemble the complete volume for the PDF. 

The Kingston Branch started originally at Invercargill and ran to Kingston over a total length of 140 km. There is good Retrolens aerial coverage of only a few stations. It is possible I will do mosaics for Invercargill, Makarewa Junction, Winton Junction and Lumsden Junction, and as this line also has a few branches then there is the possibility of adding in Mararoa and Mossburn from the Mossburn Branch, Gore Junction from the Waimea Plains Railway, Thornbury Junction  and Tuatapere from the Tuatapere Branch and Wairio from the Wairio Branch. 

Since I have been asked about Southland in general the other areas for mosaics (subject to availability) could include Bluff, McNab Junction and Edendale Junction. These fall under the MSL which is Volume 11.

After all that, I will carry on with the Otago Central in the coming week with the Gimp editing of the coverage for Alexandra to Cromwell being the high priority so that the maps for that section can be generated, and as previously mentioned I just need to add in some detail of Alexandra itself. There are not any new aerial images presently available around Alexandra. It seems the scanning of the Otago region is very slow as they have not yet added the survey that I used of the Cromwell Gorge, although other surveys are available.

Thursday, 12 April 2018

Otago Central Railway [61K]: Cromwell-Alexandra 8

Well this series of posts [61] is turning into a series about Cromwell-Alexandra really. There is still a surprising amount of work needed just to finish this first section of the maps as they correspond to the article series. It is being pushed ahead as fast as I can go but I have to admit it is a lot of work and has taken longer than expected. I have been surprised at the number of issues to be resolved. Fortunately it is coming together but it would not surprise me if it takes the rest of this year to complete those maps, which originally, optimistically, I had planned would all be completed with the articles.

The work still to be done is a mixture of aerial photos and map data. If I had stuck to just plain maps (diagrams as I now call them) then everything would be completed by now. Bringing in the aerial photos has added complexity and time delays, but I would not have it any other way now, because having the aerial data makes it possible to draw the track and structure layout in a yard accurately and other historical information can be viewed as well. In other words, the maps have gained immeasurably through having the aerial photo background available. The detail there means we don't need to use Google Earth as an alternative or anything to do with Google Earth at all, with all its limitations and complexities of its own. 

Anyhow, the past few days have been focused on collecting all of the aerial photos for the entire Otago Central Railway from Wingatui to Cromwell. It looks like this data set will be 2-3 GB in total, which is a far cry from the total space taken up for all the aerial photo layers I have, by only copying the images corresponding to the actual corridor back to mainpc. Right now I am working on Wingatui-Middlemarch as the aerials are broken into four groups corresponding to the article parts (Wingatui-Middlemarch and Middlemarch-Ranfurly are two distinct aerial photo sections whilst they were both covered in one article part in the NZRO series). 

This gives me an idea of how much work I need to do to get all the aerials fixed up as there are several instances where there are overlap problems. Basically there are in each case two different aerial series that overlap, usually from different years, sometimes from different resolutions, and for some reason the chosen boundary is not along the edge of the aerial tile, so extra black bits are added. If the black pieces are cut off in Gimp then I can actually align the tiles so there is no unsightly gap, and then render out custom versions of each tile to go back into the maps. I expect to start doing this in the Omakau-Cromwell section today, but I need to use the full map in Qgis to guide this, and so getting the map fully there with all the aerial photos will enable this work of editing specific tiles to get underway properly. Doing this means all the maps can use the higher resolution imagery of 0.4 metres minimum, or in some cases 0.125 metres (all of Middlemarch-Wingatui section) instead of the 0.75 metre stuff I used during some of the earlier mapping stages.

Tuesday, 10 April 2018

Otago Central Railway [61J]: Cromwell-Alexandra 7

The following are what the maps for Alexandra will now look like.

However the completion will take a few days as the following tasks have to be finished:


  • Review all the plans and data I researched for Alexandra last year.
  • There will not be a historic aerial photo for Fulton Hogan because it is a lot of work for one small area where the aerial photo is low resolution.
  • A few issues have to be fixed in Gimp with some of the aerial imagery obtained from Linz where two series overlap. For some reason the boundaries of these series are not aligned exactly north-south or east-west so their server adds black edges. The solution is to pair the images either side of this border up in Gimp and overlap them properly then re-render the two images.


Sunday, 8 April 2018

Otago Central Railway [61I]: Finishing Maps - Cromwell-Alexandra 6

A view of Alexandra in 1958:

Facilities like the works depot and the fruit loading shelter did not exist then. The APB site which they had just about taken over from the freezing works, was much less developed. There were still four houses in the precinct, and a trolley shed was located at the Cromwell end of the yard.


In 1965 the main change was in the APB area and also buildings in the lower left corner which was leased to the Electric Refrigeration Co.


By 1983 we have the Way and Works depot completed and a different building at lower left, perhaps a transport company, we have the fruit loading shelter (by then disused) and the bigger area of the APB site being used with stacked pallets clearly visible.


Unlike Ranfurly, in the post rail era, much of the Alexandra yard has been built on. There were opportunities for the local community to have sought to conserve a greater amount of the historic infrastructure, but to their later regret, none of the buildings still stand, only a couple of houses, some foundations and the main platform are to be found today.



Friday, 6 April 2018

Project Development: Site Styles

This is a very small note about map styles. I have used fills to mark out specific sites of premises which are being served by rail.

The issue is that marking the boundaries produces visual clutter on the maps and this has to be weighed against the value of marking it.

My feeling at this moment is against marking out a site's boundaries as opposed to putting a marker in the middle of the site or marking a specific building and leaving it at that. I think that is what the updated Otago Central maps will show. The maps I did of Addington Workshops didn't have a marked site boundary. I think that is what they will all look like.

I will also be limiting the number of non rail buildings I draw on sites in future. These changes collectively will reduce the amount of time I spend on drawing maps. 

Otago Central Railway [61H]: Finishing Maps - Cromwell-Alexandra 5

Having moved on to Alexandra, here is a screenshot with the 1983 aerial displayed.

The aerials for Alexandra done so far do not need major work done. All that is needed is to adjust the masks to show the oil company sidings and Vacuum Oil on the historic imagery.

The other thing is to try to find if there are any new higher quality aerials of the area where Fulton Hogan's site is, further north on Dunstan Road. The diagram of the siding was done using a fairly low quality aerial and a plan from Archives NZ. I can't release the plan on the same license as the rest of the maps without ANZ consent, so I may just use that low quality aerial if there is nothing better in Retrolens as of now.

Otago Central Railway [61G]: Finishing Maps - Cromwell-Alexandra 4

Having completed the revision of Clyde the focus will now shift to Alexandra. Once this is complete then the full set of maps for Alexandra-Cromwell can be republished on Google Photos via the NZ Rail Maps Volume 12 collection on Google Plus.

Here are three views of the second Clyde yard: 1977, 1981 and 2013.




 While here are two views of the first yard: 1962 and 2013.



Thursday, 5 April 2018

Otago Central Railway [61F]: Finishing Maps Cromwell-Alexandra 3

Last time I wrote there were some problems with the aerial photo layer I used to do the Clyde stations. This is a particular hazard with the use of the old aerial photos that are not orthorectified. It was more productive for me to create a completely new aerial photo layer project for Clyde than try to fix the existing ones but it still took most of two days, but that is just a learning curve for this type of thing, as these were some of the very early photo mosaics I have done. The net outcome after a couple of false starts is good, but there are some alignment issues in less critical places that are still visible but not relevant directly to the rail lines, because it's impossible to completely eliminate all of these. 

Here are three views of the new Clyde station and surrounds:

 1977, NZED siding, no station.
 1981, station, MOWD depot and siding
2013, as it is today.

Tuesday, 3 April 2018

Otago Central Railway [61E], Main North Line [11L]

So this is what is finalised for the fill styles.



These two of Addington show the styles for buildings and structures.

This one of the second Clyde station shows the site border styles which are dash-dot patterns. I am working on fixing up the aerial photos which turn out to be correct for 1977 (shown above) and just need some adjustment for the 1981 period, so that I can finish the maps for Clyde and move down the line to Alexandra. I don't actually use sites all that much on maps as it is, and may well look at ditching them for smaller areas as the visual clutter from the extra lines is something I want to reduce, especially at smaller scales.

Sunday, 1 April 2018

Otago Central Railway [61D]: Finishing Maps - Cromwell-Alexandra 2 / Polygonal Styles 2

Below are two samples of Cromwell and Clyde which show the type of styles I am adopting for buildings, structures and sites.


These samples show how the new styles are applied to fills such as sites, buildings and structures. This has come about because whilst I already worked out how to apply a white border to markers and path styles, making them more visible on dark or diverse backgrounds such as aerial photos, it is only just in the last couple of days that I worked out how to do this with fills.

The idea of changing from solid fills (black, grey or white) to transparent fills with a solid border, is to enable whatever is underneath that fill to be visible, which often is what we want with aerial photography whether historic or current. The solid fills blocked out too much of the detail and that was one deterrent to making more widespread use of aerial photography as a map background. In the styles you can see above, I still need to do some experimentation with different line thicknesses and dash/dot patterns to see what works best.

This means that a "rail map" by default will have the aerial photography background whether current or historic, so that is the definition to be used. A "rail diagram" will be the same map data but with a plain or faint background, usually a terrain relief layer. This high contrast option will enable more topographical data such as contours, heights, waterways, property boundaries and so on that doesn't display well against aerial photography, to be viewed, because I am not changing the styles of every piece of this data to make it visible against all backgrounds - because I want to reduce the clutter on "rail map" tiles.

As I have all but finished Addington I will be again focusing on the Central line to complete the maps which still have a lot of work needed. So far I have got from Cromwell to Clyde. The more complex stations like Clyde and Alexandra which both need a lot of tidying up will take a bit of work. Most of the other detail back to and including Ranfurly is completed. From Ranfurly down to Wingatui still needs a lot of detail and will probably take a month or three depending on what else I am working on at the same time, as I also want to push ahead more or less simultaneously with MNL, or alternating bits of both.

After playing with the styles a bit more this is the best compromise I can come up with.

Obviously we want the lines to be as undominating as possible but if they are too thin then it is too hard to tell the difference between a black line and a grey line so the thickness of the lines is the minimum as shown here and this map is another which illustrates the key reason why changing these styles has been desirable in the first place. This has been a very difficult issue to resolve over many months so I am quite pleased to have got a workable solution that now allows me to get the best possible result with the use of aerial photography backgrounds on maps. The single lines (solid or dashed) are used for buildings and structures, while double lines (solid or dashed) are used for sites.

What you can see in the image of the modern Clyde yard further up this post is noticeable misalignment in the 1977 aerial photography. The mosaics for the second Clyde station area were produced as separate Gimp projects for each generation, which is not now I would do things today as I would have put all the images into one project and just turn different layers on and off to get the required results for each rendering. This means the next stage is to do another Gimp project this time using all the aerial stuff in one project and rendering the different generations out of that one project. So work has to start on that right away as the Otago Central Railway maps have to be finished ASAP.

Main North Line [11K]: Addington 11 / Polygonal Styles 1

Some samples of the workshops site. The styles still have to be finalised before these are released on the website.



The main style that needs to be finalised here is the building outlines, because the styles have been previously designed for diagram use with a plain white background and therefore have a solid black or grey fill, now I need to decide if I just use a solid border and transparent fill as shown here. I have just worked out how to style fills with an outlined border with a white edge so the border can clearly be seen, so that is about the last style element that needs to be outlined to make sure it shows up on an aerial photography background.

Having solved this technical issue with fill styles, I can now determine that the maps will be called "rail diagrams" and "rail maps" and the former will be the plain white background, the latter will be on aerial photography. So every map will be produced with an aerial photography background because of the free reuse of the Linz aerial photography being possible. I think it is likely I will come up with a thinner border style for buildings if I decide to use a style similar to the above instead of a solid filled rectangle style. The thicker style will continue to be used as now for sites.

The below is the likely building/structure style to be used. I like the open circles for turntables.