Sunday, 18 March 2018

Otago Central Railway [61B]: Finishing Maps - Clyde 1

So while I am working on the Addington mosaic, that is a pretty slow process with a lot of downtime waiting for Gimp to resize images, the best part is with another computer doing only that and nothing else, I can carry on with maps on my main computer. And in actuality that is a dedicated VM which is where I do most of my mapping on different computers, because I can use multiple versions of the software, which I need to at the moment because of a serious bug in Qgis 3.0 affecting aerial photos. I need to use 2.18 to do most editing but I need 2.14 to do filtered layers editing because 2.18 deprecates that capability. 

So back into Central Otago and starting with Clyde and here is the 1976 aerial photography with everything else over the top of it.

Now this is a multi generational map that covers 1962, 1976, 1981 and later up to the present day. To make such maps work we have to be able to filter what is being displayed so that only the features like tracks, buildings, stations and so on for that timeframe are displayed. To achieve that I have extra fields for dates added to a lot of layers and run the same filter query on multiple layers at once and at the same time select the background aerial photography layers to be displayed and that is how we get to multi generational maps.

So Clyde is the westernmost area that is set up as multi generational, now I have to tidy all that up and push out the generations of maps for Clyde and then work my way eastward doing the same all the way along and just make sure all the stuff on the Google Plus collections is all nicely up to date and that we have maps up there for the entire line.

That will go on at the same time as Addington mosaics simply because it is an efficient use of time with the delays for making the mosaics while the computer does the hard work on those huge Gimp projects to make the aerial images for that part of Christchurch. I have almost completed Addington West now, I just have to mask off the area that I want from the Retrolens aerials and then save back the geojpegs for Qgis, probably finish that tonight. Then Addington North and Addington East to start on.

Saturday, 17 March 2018

Main North Line [11B]: Addington 2

The Addington mosaic has been a lot of fun. Working with so many large layers in one image is taxing on computer resources, and one relatively fast PC (its CPU is a Pentium G with 2 cores but no hyperthreading) with nothing else open and 16 GB of RAM is still quite slow working with what is now a 10 GB Gimp project and which keeps growing as the historic aerial layers have to be scaled up in size. Since I ended up with two large layers (scaled from around 9000 to around 15000 width) on the west side of Addington with a small slice in the middle where another layer has to be pasted in to fill the gap, I resorted as in the past to cropping the required piece out of the gap layer, otherwise it takes all day to get it scaled to the right size.

The idea of running Gimp side by side on two computers in order to get the smallest number of steps to maximise quality has been useful but the scanned images in this case are so sharp (the original scale is 1:3400) that it's been unnecessary to continue with this. At the same time it is hard to get much memory available on MainPC for this kind of thing because browsers and other stuff I do a lot of on it just seem to gobble up all the available memory. Because of this I put Qgis 2.18 onto mediapc as well and have been using that because the media player stuff doesn't use much RAM. This shows the great benefits of an OS like Linux which is designed to work with resource limited hardware, being resource efficient on good hardware that you might be using do do resource intensive tasks. 

As it turned out my strategy outlined above did not work and I have changed to using two big layers next to each other with a small slice at one end instead of in the middle, this is because the usual skewings and distortion are better seen at one end rather than in the middle of a critical area of the railway depot. However this means a lot of yesterday's work has had to be redone. This sort of outlines why only a few areas are going to get this aerial photography coverage - it takes a huge amount of time to create these mosaics especially for big areas like a major railway yard or depot. The small stations on the Otago Central line which usually only involve one or two base images and maybe the same number of aerials are child's play by comparison. In this case having a slice on the end is much simpler because it can simply be cropped off when the mosaic georeferenced copies of the original base images are saved, so the full size image can be used instead of cropping a piece of it. We will have to see if Gimp copes with all the historic images once they are being displayed and it may well prove necessary to split the project into two or three pieces.

Thursday, 15 March 2018

Main North Line [11A]: Addington, etc

Having completed sorting through aerial photos of the MNL and now having a complete set, the next task is to start doing stuff with the photos. The first thing that will happen is I will do a set of historical maps for Addington, as that is where the MNL starts. When I first started looking at Christchurch yards about three years or so ago I had Canterbury Maps coverage and I used to overlay it in Google Earth and then import the lines and shapes into Qgis. Even though those old layers still exist, I am going to redo it all from scratch and create mosaics for all the stations on top of current Linz coverage, because I can use the mosaic aerials when I publish the maps.

So for part of today I will have two computers both running Gimp to create the mosaics of Addington, which is both a MSL and a MNL station, being a junction. It is a very interesting location that like many other areas of Christchurch has seen huge change over the past 40 years. Doing two Gimps side by side is basically having one of them resizing and rotating as quickly and roughly as possible, and then the second doing the absolute minimum number of these operations to get a result that is as sharp and clear as possible, because every such operation reduces the final quality. 

I haven't forgotten the Otago Central Line but have just needed a break from it over the past few weeks but I will be pushing on with it today as well. I have to reorganise the aerial photography resources for it as well, and that will take some time to complete. But most of it is just editing the project files to find the new folder paths where the aerial photo collections have been moved onto a different server, and shouldn't actually take too long. 

The optimisation / VRT preparation script is going to be started fairly soon, maybe at the weekend, because I need to get on to the task of both making my life easier with fewer layers in VRTs, and reducing the number of images I need to keep copies of on the computer, because they are using up so much disk space. The Volume 10 resources are using 200 GB, Volume 11 we are only just starting on is 70 GB, and Volume 12 is 90 GB. However, an earlier series, Volume 7, is well optimised and only uses 3 GB. Volume 10 and 11 are expected to use a lot of space because the MNL and MSL both cover a lot of territory in Canterbury and pass through major urban areas. But I can't sustain the amount of space that some of these volumes are taking up because it fills up the disk and makes the backups too big as well. So this rationalisation improvement really has to happen, but it means more delays that will take me off mapmaking for a while.