Saturday, 24 March 2018

Main North Line [11E]: Addington 5

Well, Friday has been a whole lot more mucking around to make various adjustments to all of the Addington stuff, mainly in the area of adjusting the masking for the partial images. This has resulted in needing to re-render all the partial map tiles which is a relatively slow process. Hence the maps have not been ready in time as expected. There are still more adjustments needed where some of the tracks don't line up, so it will be several more days before mapmaking can really get underway. Here are a couple of sample images that many readers will appreciate for their historical context.

The familiar present day view of Addington with a few extra things drawn in for context.

The historical context with present day roads and tracks.

Main North Line [11F]: Addington 6

Today's not-news is of yet another hold up in mapping.

Because I made the stupendous discovery that some of the tracks aren't lined up between the old and new aerial images.

Unfortunately the nature of historical aerial imagery means that this is a risk. It's my regret that the process I used did not pick up that alignment issues were possible with the tracks in more than one or two places (the edges of the east and west ends of Addington).

Since this affects the middle part and every other part is aligned off it, this means I virtually have to start again from scratch with the middle part and then realign everything else off it.

I hope this won't take too long but the reality is that it could take another week. I will try to see if I can speed things up with two computers at once but I am not at all sure this could happen. 

Letter to the Editor of the New Zealand Railway Observer

Concerning editorial changes made to submitted articles to the New Zealand Railway Observer.

During the last three editions of the New Zealand Railway Observer (No.346-348) I have had published a series of articles entitled "The Geography of the Otago Central Railway". I enjoyed writing these articles and sought in the process of this to provide another look at this railway line and the present day relevance of it. The research behind these articles has been a part of the NZ Rail Maps project, and the maps produced by this project are a key part of the information process that has been used to produce the article series.

I regret however that I must put on record at this point my concern at certain editorial changes that were made to these articles after they were submitted for publication. These changes were made without consultation with me, and in the main are preoccupied with an obvious editorial prejudice against imperial measurements of distance such as miles. There is no legal requirement to use metric measurements of distance in the context of publications such as these, and overzealous conversion of imperial measurements results in many absurdities in written work that I have read from a variety of authors and publishers over many years. In this case the simple metrication of imperial distances fails to understand the historical differences between these measurements as they were used on the railway system of New Zealand.

The articles which I submitted used metric measurements where appropriate, and imperial measurements where appropriate. The following are the key objections I have to the metrication of the entire article:

1. It is impossible to describe a milepost as a "1.6 km post". There is no such thing as a physical post that was put into the ground alongside a railway at 1.6 km intervals. Thus, the line of my article reading for example "Between 38 and 39 miles" which has been changed into "Between 59 and 62 km" refers to actual different locations than the 38 and 39 mile pegs. I can't quite make up my mind whether "59 to 62 km" is better than the direct equivalents of 61.14 and 62.75 km which some publications would have used. Note that "59 to 62 km" in the above appear to be factually wrong. This makes about as much sense as changing the names of official publications that have "1 mile" or similar in their title.

2. The measurements of mileposts and kilometre posts were done at different times, and in the cases of many railway lines, measured actual different distances. These changes have occurred because many railway lines have been deviated or realigned. Historically when deviations or realignments have been made, the original mile or km posts do not get relocated to reflect the change in distance. Instead the engineers have historically made use of a device called a "short" or a "long" mile [1].  For many lines in NZ, "short" and "long" miles persisted until such a time as metrication in 1974 when the lines were remeasured from scratch and the kilometre and half-kilometre pegs put in. It therefore cannot be assumed that distances measured in kilometres will always correspond exactly to distances measured in miles on the same line.

3. The distances shown in the article are official distances from official NZR and its successors' publications. Chiefly these publications are the Working Timetable, and lesser publications include the chainage books which are held by Archives New Zealand. The first metric WT for the Otago Central Railway omitted distances for stations which had closed. For these stations, therefore, the only available and appropriate official distance measurement is the one from a preceding WT measured in miles and chains. Chainage books are all in miles and chains and all distances derived from them have appropriately been expressed in decimal miles. 

Researchers in this area who have made use of the New Zealand Railway and Tramway Atlas, a popular publication which includes distance measurements for the vast majority of railway stations across New Zealand, are aware of discrepancies between these measurements and those found in many of the metric era Working Timetables. We can only assume that the publishers chose to hand convert the imperial distances to metric rather than obtaining new measurements from the metric WTs. For the same reason I have not used the metric distances shown in other publications about the Otago Central Railway for station locations that are not shown in the metric WT. 

4. The maps drawn of the line are marked using the imperial mileposts obtained from chainage charts and in many cases measured from easily verifiable geographical locations shown in the same charts. Whilst there may be some minor discrepancies in actual milepost positions, these measurements offer a reasonable degree of accuracy in the placement of the mileposts on physical positions on the map. The same information is not available for most kilometre posts on the line, a number of which have been removed from the line in any case. Those metric measurements shown are mostly station locations that are in the metric WT. The measurements on the map are those which can be obtained from official railway publications according to the preceding principles and quoting a measurement that has been converted makes no sense in this context because there won't be a corresponding measurement marked on the map itself. 

Reference: [1] NZR DEO’s Office (c.1930-c.1956). Chainage O C B [Otago Central Branch] 0 Miles 00 Chains to 25 Miles 60 Chains. Archives New Zealand, R18524763.

Friday, 23 March 2018

Main North Line [11D]: Addington 4

Welcome to our Facebook Group readers (NZRailMapz) as this will be the first entirely automated blog post sent to you, courtesy of IFTTT and the Buffer service. I already use IFTTT to syndicate  my posts on the NZRM Facebook page, but now they are expected to automagically feed to the group as well.

Latest progress to report is having laid out and masked off the retros for the east side of Addington. Tomorrow (Friday) the rendering of the eight geojpegs can be carried out and that will give me everything I need for the whole of Addington. I also made some adjustments to West and North and as a result some of those geojpegs need to be re-rendered as well.

Addington was many things: several important NZR industrial sites, the junction between the MNL and MSL, a passenger station, a freight yard, and many industrial sidings. These days it is much altered but still hosts a rolling stock repair depot, the main passenger station for all long distance rail passenger services, and the altered MNL junction. All the sidings however have now disappeared.

So Addington will be a very interesting site to detail. I expect to cover all of the major sites in Christchurch this way. I did most of them previously using Canterbury Maps and Google Earth but I am doing them again because of being able to use the historic Retrolens imagery as the background for many maps.

I may as well look at other sites I can use Retrolens coverage for. One has to be circumspect because of the amount of time it takes to assemble these mosaics. Certainly the production of them for almost every station on the Otago Central Railway is unlikely to be repeated everywhere. Depending on what is available I would expect to see them for almost every station from Christchurch to Waipara and Christchurch to Rolleston. Further south or north will be more limited. Obviously Picton has been done already. I may possibly do every station on the Waiau Branch, but I am unsure about other branch lines. 

MNL going north from Christchurch will be selected from: Papanui, Belfast, Kaiapoi, Eyreton Junction, Rangiora, Ashley, Balcairn, Amberley, Waipara, Scargill, Tormore, Nonoti, Mina, Spotswood, Parnassus, Ferniehurst, Claverley, Oaro, Kaikoura, Hapuku, Clarence, Parikawa, Kekerengu, Wharanui, Mirza, Ward, Taimate, Grassmere, Blind River, Seddon, Dashwood, Vernon, Riverlands, Blenheim, Spring Creek, Tuamarina. A few smaller locations in this list are connected with track realignments.

MSL: Every station to Rolleston then selected from Burnham, Dunsandel, Bankside, Rakaia, Fairfield, Ashburton, Tinwald, Hinds, Rangitata, Orari, Temuka, Seadown, Washdyke, Timaru, Pareora, Studholme, Glenavy, Waitaki, Pukeuri, Oamaru, Waiareka, Hillgrove, Bushey, Palmerston, Merton, Puketeraki, Waitati. The whole Dunedin-Mosgiel area, and then carrying on further south, Owhiro, Allanton, Milburn, Milton, Clarksville, Stirling, Balclutha, Kuriwao, Clinton, Waipahi, McNab, Gore, Mataura, Edendale, and Invercargill.

Midland Line: Selected from Darfield, Sheffield, Springfield, Staircase, Avoca, Craigieburn, Cass, Waimak Bridge, Cora Lynn, Arthurs Pass, Otira, Jacksons, Rotomanu, Ruru, Moana, Kokiri, Stillwater etc. However Retrolens imagery is not available outside Canterbury limiting us to what can be gained from existing Linz coverage. As it happens I have already traced much of what currently exists on the West Coast lines, and also done most of Christchurch-Timaru the same way.

Wednesday, 21 March 2018

Main North Line [11C]: Addington 3

So as of last night I have finished laying out the Addington North mosaics in Gimp. Today's job is to mask off the relevant bits of the retro images, and then render the part and full images for Qgis. Whilst I have enough now to start mapping Addington, I have determined to do only two tasks simultaneously, one of which is for the Otago Central Maps and one is for the MNL maps, so I will not start mapping Addington until I have finished laying out all the maps, and at this stage there is still East to be done as this area includes the actual MNL junction. To do this post I have had to render one additional area of West which doesn't contain any railway sidings, so as to display a full view of the area.

(Our group members should hopefully see this post automatically forwarded to the group by Buffer and if so, welcome to automated posting)

West side of Addington, present day.

West side, composite view.
West side, full 1970 aerials, with modern street layout for reference. A breakdown of this map into four areas is below.
At the Matipo St end of Addington, the designated purpose of these sidings is presently unknown. They are immediately adjacent to Addington Raceway, and by 1970 were full of withdrawn stock wagons. At the north side of the main line, we can see a siding going into premises at the rear of the P&N (Patience & Nicholson) factory.
North of the mains and east of Barry Hogan Place we have these as yet unidentified premises in Princess Street with a rail siding. A stub of this siding still exists on the railways side of the fence. The rearmost building with a sawtooth roof has been demolished but the gable roofed buildings north of it still exist and today are used for warehousing. The street frontage is occupied by the modern premises of the Ministry of Education. On the east is the Princess Street electricity substation, which still exists today but doesn't have any rail sidings into it.
Immediately to the south side of the previous photo here is a view of, upper right, Addington Plant Zone at its western end, where the Way and Works Department of the Railways stored their plant and equipment. South of the tracks appears to be the siding that was used to load and unload horses for Addington Raceway. Which at the time of this picture is full of disused stock wagons.
The last photo of this western segment of Addington shows the bulk of the Plant Zone area to the north of the mainline corridor. All of this area today is office blocks and industrial premises. Whiteleigh Avenue was pushed through this area in the late 1980s following the closing and demolition of the Plant Zone site. At the south side we can see the raceway siding alongside a platform which may have been used for passenger trains at some stage, this platform still existed for many years afterwards. At lower right is the railway Signals depot which again was a casualty of the Whiteleight Avenue road extension.

Otago Central Railway [61C]: Finishing Maps - Cromwell-Alexandra 1

Well I decided that I needed to check everything starting from Cromwell so that is where I have been busy for the last few days. As it happens I had two sets of aerial photo mosaics from Cromwell, and they didn't match up, so I re-rendered both the full and part sets of retro images from the Gimp project file. A full image is displaying all of the retro aerial photo (1962 in this case) and a part image is displaying only what has been masked out of the base image (the present day coverage). So that has been going on in parallel with the Addington stuff, using a 2nd computer. And then I have got the aerials loaded into the map project to start checking them. So far I have worked my way down to the original Clyde station, using the originals to recheck detail. 

The next step is to get the Clyde data ready for rendering the multi-generational maps. This means running filters on all the data tables and checking that they display the correct details, for each generation that I have the background images available for. After that it is a case of working down the line to Alexandra and doing the same thing there, in both cases the aerial photos for these areas were done some time ago and are assumed to be correct.

After that it will be a case of updating the photo album for Alexandra-Cromwell with the maps so that it is up to date, and the albums for major and minor stations as well. From there it is a case of working down from Alexandra to Ranfurly, and from Ranfurly to Wingatui. Alexandra to Ranfurly is mostly done already, except for Auripo station which I still have to finish off, so it will be mainly a case of checking the aerial photos are all complete. Except for the stations which nearly all have retros available, the imagery will be the recent/current LINZ stuff. Ranfurly to Wingatui is the part that needs the most work, as the only stations completed are the ones that were used to illustrate the Observer article. 

That article has just been published and the magazine released. I am certainly pleased to have completed that series. Part 3 was the hardest to write, and this will be obvious in the style of writing, which is a bit more technical and less creative. I am planning to republish all three articles on this blog in due course,  probably when the maps are complete or in about a year, and will exploit the ability to include much more illustrations in a blog than in the original magazine articles. Part 3 also includes a lot of detail about private sidings at Alexandra as this information was obtained when I visited Dunedin at Christmas 2017.

Sunday, 18 March 2018

Otago Central Railway [61B]: Finishing Maps - Clyde 1

So while I am working on the Addington mosaic, that is a pretty slow process with a lot of downtime waiting for Gimp to resize images, the best part is with another computer doing only that and nothing else, I can carry on with maps on my main computer. And in actuality that is a dedicated VM which is where I do most of my mapping on different computers, because I can use multiple versions of the software, which I need to at the moment because of a serious bug in Qgis 3.0 affecting aerial photos. I need to use 2.18 to do most editing but I need 2.14 to do filtered layers editing because 2.18 deprecates that capability. 

So back into Central Otago and starting with Clyde and here is the 1976 aerial photography with everything else over the top of it.

Now this is a multi generational map that covers 1962, 1976, 1981 and later up to the present day. To make such maps work we have to be able to filter what is being displayed so that only the features like tracks, buildings, stations and so on for that timeframe are displayed. To achieve that I have extra fields for dates added to a lot of layers and run the same filter query on multiple layers at once and at the same time select the background aerial photography layers to be displayed and that is how we get to multi generational maps.

So Clyde is the westernmost area that is set up as multi generational, now I have to tidy all that up and push out the generations of maps for Clyde and then work my way eastward doing the same all the way along and just make sure all the stuff on the Google Plus collections is all nicely up to date and that we have maps up there for the entire line.

That will go on at the same time as Addington mosaics simply because it is an efficient use of time with the delays for making the mosaics while the computer does the hard work on those huge Gimp projects to make the aerial images for that part of Christchurch. I have almost completed Addington West now, I just have to mask off the area that I want from the Retrolens aerials and then save back the geojpegs for Qgis, probably finish that tonight. Then Addington North and Addington East to start on.

Saturday, 17 March 2018

Main North Line [11B]: Addington 2

The Addington mosaic has been a lot of fun. Working with so many large layers in one image is taxing on computer resources, and one relatively fast PC (its CPU is a Pentium G with 2 cores but no hyperthreading) with nothing else open and 16 GB of RAM is still quite slow working with what is now a 10 GB Gimp project and which keeps growing as the historic aerial layers have to be scaled up in size. Since I ended up with two large layers (scaled from around 9000 to around 15000 width) on the west side of Addington with a small slice in the middle where another layer has to be pasted in to fill the gap, I resorted as in the past to cropping the required piece out of the gap layer, otherwise it takes all day to get it scaled to the right size.

The idea of running Gimp side by side on two computers in order to get the smallest number of steps to maximise quality has been useful but the scanned images in this case are so sharp (the original scale is 1:3400) that it's been unnecessary to continue with this. At the same time it is hard to get much memory available on MainPC for this kind of thing because browsers and other stuff I do a lot of on it just seem to gobble up all the available memory. Because of this I put Qgis 2.18 onto mediapc as well and have been using that because the media player stuff doesn't use much RAM. This shows the great benefits of an OS like Linux which is designed to work with resource limited hardware, being resource efficient on good hardware that you might be using do do resource intensive tasks. 

As it turned out my strategy outlined above did not work and I have changed to using two big layers next to each other with a small slice at one end instead of in the middle, this is because the usual skewings and distortion are better seen at one end rather than in the middle of a critical area of the railway depot. However this means a lot of yesterday's work has had to be redone. This sort of outlines why only a few areas are going to get this aerial photography coverage - it takes a huge amount of time to create these mosaics especially for big areas like a major railway yard or depot. The small stations on the Otago Central line which usually only involve one or two base images and maybe the same number of aerials are child's play by comparison. In this case having a slice on the end is much simpler because it can simply be cropped off when the mosaic georeferenced copies of the original base images are saved, so the full size image can be used instead of cropping a piece of it. We will have to see if Gimp copes with all the historic images once they are being displayed and it may well prove necessary to split the project into two or three pieces.

Thursday, 15 March 2018

Main North Line [11A]: Addington, etc

Having completed sorting through aerial photos of the MNL and now having a complete set, the next task is to start doing stuff with the photos. The first thing that will happen is I will do a set of historical maps for Addington, as that is where the MNL starts. When I first started looking at Christchurch yards about three years or so ago I had Canterbury Maps coverage and I used to overlay it in Google Earth and then import the lines and shapes into Qgis. Even though those old layers still exist, I am going to redo it all from scratch and create mosaics for all the stations on top of current Linz coverage, because I can use the mosaic aerials when I publish the maps.

So for part of today I will have two computers both running Gimp to create the mosaics of Addington, which is both a MSL and a MNL station, being a junction. It is a very interesting location that like many other areas of Christchurch has seen huge change over the past 40 years. Doing two Gimps side by side is basically having one of them resizing and rotating as quickly and roughly as possible, and then the second doing the absolute minimum number of these operations to get a result that is as sharp and clear as possible, because every such operation reduces the final quality. 

I haven't forgotten the Otago Central Line but have just needed a break from it over the past few weeks but I will be pushing on with it today as well. I have to reorganise the aerial photography resources for it as well, and that will take some time to complete. But most of it is just editing the project files to find the new folder paths where the aerial photo collections have been moved onto a different server, and shouldn't actually take too long. 

The optimisation / VRT preparation script is going to be started fairly soon, maybe at the weekend, because I need to get on to the task of both making my life easier with fewer layers in VRTs, and reducing the number of images I need to keep copies of on the computer, because they are using up so much disk space. The Volume 10 resources are using 200 GB, Volume 11 we are only just starting on is 70 GB, and Volume 12 is 90 GB. However, an earlier series, Volume 7, is well optimised and only uses 3 GB. Volume 10 and 11 are expected to use a lot of space because the MNL and MSL both cover a lot of territory in Canterbury and pass through major urban areas. But I can't sustain the amount of space that some of these volumes are taking up because it fills up the disk and makes the backups too big as well. So this rationalisation improvement really has to happen, but it means more delays that will take me off mapmaking for a while.

Friday, 9 March 2018

Main North Line [10]: Aerial Photography

Most of the mapwork over the past few days has been with MNL aerial photography which has to be downloaded and then sorted through to pick up just the photos I want. This is a slow process because of the download limit of 3.5 GB per download from the Linz website and the number of tiles in each download being much greater than what is needed for the rail corridor. So there has been a lot of work to get full aerial coverage for this line.

The other thing that has to be done at the moment is to amalgamate aerial photos and remove the ones that are not actually used because it takes up a lot of disk space and I have had to reorganise the storage between between different computers. I am looking at writing a script to process the Qgis layer definition files that I am putting aerial photos into. I should learn how to script on Linux with Python but to speed things up, in this instance I will use Windows Powershell to process the QLR files and copy the photos listed in them to a specific folder path that the script will create. What happens after this is to use a Qgis library function to process the aerial images into something called a VRT (Virtual Raster Transform) which is supposed to make it easier to handle all the many aerial photos I am working with.

The aerial photo downloads are now more or less finished and just need a bit of extra little bits added on here and there to make them complete. I need to get on also and finish the Otago Central maps because I haven't done any work on them for the last week or two. I think I just did some mosaics of Auripo and haven't actually drawn any of the maps; nor have I worked on any of the other maps. However reorganising the resources on the computers has taken a bit of my time up lately.

I have been getting Retrolens coverage for the major deviations on the MNL as well as the current stuff and have got enough to map all of these accurately. Apart from Nonoti mentioned last time there were major ones done at Stewarts Gully, Ashley, Balcairn and Phoebe, and all of these will be addressed over further posts in the MNL series in coming weeks.

Tuesday, 6 March 2018

Main North Line [9]: Nonoti

Nonoti is a weirdly named little station in North Canterbury between Tormore and Mina. The exact distance is not recorded in the NZ Railway and Tramway Atlas, but it is between 116 and 117 km. The station closed in 1958. Around Nonoti a deviation was put in during the 1950s, apparently commissioned around 1961 (Don Spicer). 

1955 Aerial photo showing the earthworks underway.

1965: the deviation is in use and, of note, the original overhead bridge at Nonoti was replaced with another one on a slightly easier alignment and because the rail route at this area also changed. Nonoti station still had some track in place despite having shut some seven years earlier.

Complete map view.

Although drivers (e.g. Don Spicer) say the new route is much easier to work, eliminating gradients and sharp curves, the new route still requires speed reduction to 65 km/h for the curves. It is interesting that more effort was not made to ease these.

Monday, 5 March 2018

Main North Line [8]: Full aerial coverage

During the migration to Qgis 3 I have started migrating the CWNM project to this edition and it needs a bit of work to fix up various things. Alongside this I have been doing a lot to get a full set of aerial photos for every line, although particularly the Main North Line at present. 

Alongside this is a reorganisation of resources on the different computers. The ongoing issue with aerial photos has been the large amount of disk space they consume. This creates a lot of problems on mainpc as it keeps filling up the 2 TB array.

I have therefore taken a logical step to move all the aerial photos used for maps off mainpc onto serverpc. This is going to take a while to accomplish although because many of the aerial layers are not actually loaded most of the time it will not break many projects when they get moved across.

Because Qgis 3 implements a new canvas caching function that consumes a lot more resources than previous versions I am moving towards all aerial photos being loaded "on demand" in effect by implementing layer definition files saved in each source folder so that only the aerial photos currently in use actually need to be loaded. At the same time if I forget to turn on serverpc when the project is opened, reloading a qlr file will be quite easy to do.

The core custom aerial maps for each project which are quite a small number of files will probably be kept on mainpc but I will have a look into this in more detail as I progress the migration. Hopefully it will be a long time before I have to clean up any resources on serverpc.

I have changed my mind  about rushing to migrate to Qgis 3 as it still contains significant bugs. One in particular will have a major impact on aerial photography use as it gobbles up all the resources really quickly for some sort of render caching and then freezes. So back into 2.18 again. I have decided to make the most of using serverpc to run map drawing VMs and therefore am working on an optimal keyboard layout at my workstation that lets me put serverpc's keyboard directly in front of me while keeping mainpc's keyboard in the same place, therefore allowing me to use both computers simultaneously for mapwork without switching the main keyboard between computers all the time. 

Otago Central Railway [61A]: Finishing Maps

There has been quite a bit of progress with maps since the last blog post. I am just not blogging as often lately, but I will start to pick that up again. 

I have mapped the entire TGR line with all the detail available including full and half km posts and other details I can get off Youtube videos which people have done with cameras mounted to the front of locomotives. These maps will soon be published to the Google Plus collections.

The next step already mentioned is to systematically work down from Part 1 to Part 3 from Cromwell back to Middlemarch to finish all the maps throughout and update all the Google Plus collections in turn. There is quite a lot of mucking around here with particularly Alexandra and Clyde generational maps to be finished. There is also the map of Auripo in Part 3 which I think is the only station not finished therein, and various bits in Part 2. The Part 1 maps need the most work.

This has all had to be fitted around migrating to Qgis 3 which was released two weeks ago. I have spent a lot of time setting up a third computer to hold all the VMs I use to free up disk space on mainpc, and have been putting together several VMs that will be on mainpc because they will be for older versions that are archived and that need to be included in the backups, such as 2.14 which has some extra functionality not available in later editions. 3.0 is natively installed on MainPC and development builds are all on the serverpc collection of VMs.