A big PostgreSQL upgrade

I don’t write here very often about stuff I do at work, but I just finished a project that was a hassle all out of scale with the actual utility, and I wanted to document it for posterity, in the hope that the next time I need to do this, maybe I’ll have some better notes.

Introduction

A bit of backstory. I have three personal PostgreSQL database “clusters” (what PostgreSQL calls a single server, for some reason unknown to me), plus I manage two-and-a-half more at work. All of them run FreeBSD, and all of them are installed from a custom package repository (so that, among other things, we can configure them to use the right Kerberos implementation). These systems were all binary-upgraded from PostgreSQL 9.4 to 9.6 in the summer of 2020, but all are much older than that — one of the databases started out 20 years ago running PostgreSQL 7.0, and another one was at some point ported over from a 32-bit Debian server (although not by binary upgrade). We have a shared server for folks who want an easy but capable backend for web sites that’s managed by someone else, and that has about 50 GB of user data in it (including, apparently, an archive of 4chan from a decade or so ago that was used for a research project). For my personal servers, I have applications that depend on two of the database “clusters” (one on my home workstation and another on my all-purpose server), and the third “cluster” is used for development of work projects. At work, in addition to the big shared server, we have a couple of core infrastructure applications (account management and DNS/DHCP) that depend on the other “cluster” — these are separate to avoid dependency loops — and the “half” was originally supposed to be an off-site replication target for that infrastructure server, but since I never managed that, I could use it for testing the upgrade path.

Now, as I said, we were running PostgreSQL 9.6. As of December 1, that version was still supported in the FreeBSD ports collection, and so we could build packages for it, but it had just gone end-of-live as far as the PostgreSQL Global Development Project was concerned. The FreeBSD ports maintainers have a history and practice of not keeping old versions of packages that are no longer supported upstream — unlike Linux there’s no big corporate entity selling support contracts and funding continued maintenance of obsolete software packages and therefore no “megafreeze”. So it was clear that we needed to get off of 9.6 and onto something modern — preferably 14.1, the most recent release, so we don’t have to do this again for another few years. But if I got stuck anywhere in the upgrade process, I wanted it to be as recent a release train as I could possibly get onto. For that reason, I decided to step through each major release using the binary pg_upgrade process, identifying and resolving issues at a point where it would still be relatively easy to roll back if I needed to do some manual tweaking of the database contents (which as it happened did turn out to be necessary).

All but one of these databases are small enough that it would be practical to upgrade them by using a full dump and restore procedure, but of course the 50GB shared database is too big for that. I wanted to maximize my chances of finding any pitfalls before having to upgrade that database, which meant the same pg_upgrade in-place binary upgrade process for all of them. Running pg_upgrade on FreeBSD is a bit involved, because different PostgreSQL versions cannot be installed together in the same filesystem, but this part of the procedure is fairly well documented in other sources online. I have two separate package build systems, one for work and one for personal, because the work one doesn’t need to bother with time-consuming stuff like web browsers and X, whereas the personal one is what’s used by all my workstations so it has all of that. In both cases, though, package repositories are just served directly from the repository poudriere creates after every package build.

Building packages

Because poudriere builds packages in a clean environment, there is no difficulty in building a package set that includes multiple PostgreSQL releases. Where the challenge comes in, however, is those packages for which PostgreSQL (or more specifically, postgresql-client) is an upward dependency — they can only be built against one PostgreSQL version, either the default one defined in the FreeBSD ports framework, or (most relevant for my case) the one set in /usr/local/etc/poudriere.d/make.conf in the DEFAULT_VERSIONS variable. poudriere has a concept of “package sets”, packages built with different build parameters but otherwise from the same sources and build environment, which makes it easy to build the six different package repositories required for this project: we can just create a pgsql10-make.conf, pgsql11-make.conf, and so on, and then use the -z setname option to poudriere bulk to build each repository.

Now, one of the things poudriere is good at, by design, is figuring out which packages in a repository need to be rebuilt based on current configuration and package sources — so we don’t need to actually rebuild all of the packages six times. First, I added all the newer PostgreSQL packages (postgresql{10,11,12,13,14}-{client,server,contrib,docs}) to my package list, and built my regular package set with my regular make.conf and all of those versions included. (I lie: actually I’ve been doing this for quite some time.) Then, I made six copies of my repository (I could have used hard links to avoid copying, but I had the disk space available) using cp -pRP, after first checking the poudriere manual page to verify where the setname goes in the path. (For the record, it’s jail-portstree-setname.) Then I could step through each setname with poudriere bulk and only rebuild those packages which depended on postgresql-client. All I needed to do was make these additional sets available through my web server and I would be ready to go.

Because I knew this was going to be a time-consuming project, I chose to freeze my ports tree after the Thanksgiving holiday: I could either manage a regular package update cycle (which I usually do once a month) or I could do the database upgrade, not both.

The “easy” systems

For obvious reasons, I started out working on my personal machines; they all have slightly different configurations, with their own motley collections of databases, loaded extensions, and client applications. They were set up at different times and were not under any sort of configuration management, so they were all subject to some amount of configuration drift, but none support remote access, so I didn’t have to worry about synchronizing pg_hba.conf or TLS certificates — which was a concern for the work servers, which had exclusively remote clients. And since I was the only direct user of these machines, it was easy for me to make the upgrades in lockstep on all three servers, so I wouldn’t get stranded with different machines requiring different PostgreSQL releases. (That wouldn’t have been a crisis on any of those machines, but would have been a bigger issue at work where the package repository configuration is under configuration management.)

The whole process actually was pretty easy, at least at first, and worked pretty much as the various how-to articles suggest: create a directory tree where you can install the old packages, stop the old server, initdb, run pg_upgrade, start the new server, and do whatever pg_upgrade told you to do. This is not automatable! You have to actually pay attention to what pg_upgrade says, which will vary depending on database configuration, what extensions are loaded, and on the specific contents of the database cluster, in addition to which “old” and “new” PostgreSQL releases are targeted. (You must always use the pg_upgrade supplied with the new server release.) I’ll give a full rundown of the process at the end of this post.

The first showstopper issue I ran into is that PostgreSQL 12 dropped support for WITH OIDS. If you’re not familiar, in early versions of PostgreSQL (indeed, even before it was called that), every row in every table would automatically get a column called oid, which was a database-wide unique numerical identifier. There are a whole bunch of reasons why this turned out to scale poorly, but the most import of these was that the original implementation stored these identifiers in an int32 on disk, so if you had more than four billion tuples in your database, they would no longer be unique (and nothing in the implementation would enforce uniqueness, because that was too expensive). The oid type served a useful function in the internals of the database, but by PostgreSQL 9.x, actually using the oid column was deprecated, and the default was changed to create new tables WITHOUT OIDS.

It should not surprise you, if you’ve read this far, that some of these databases dated back to PostgreSQL 8, and therefore were created WITH OIDS, even if they they didn’t actually use the implicit oid column for anything. (I had to carefully check, because some of my applications actually did use them in the past, but I was able to convince myself that all of those mistakes had been fixed years ago.) None of this was an issue until I got to the step of upgrading from PostgreSQL 11 to 12 — because PostgreSQL 12 entirely dropped support for WITH OIDS tables: you can’t upgrade from 11 to 12 without first either dropping the old tables or using ALTER TABLE ... SET WITHOUT OIDS while running the old release of the server. pg_upgrade can’t patch this over for you. On more than on occasion, I got the 12.x packages installed only to have pg_upgrade fail and have to roll back to 11.x.

The first time this happened, I was almost ready to give up, but I was able to find a howto on the web with the following extremely helpful bit of SQL to find all of the WITH OIDS tables in a database:

SELECT 'ALTER TABLE "' || n.nspname || '"."' || c.relname || '" SET WITHOUT OIDS;'
FROM pg_catalog.pg_class c
LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
WHERE 1=1
  AND c.relkind = 'r'
  AND c.relhasoids = true
  AND n.nspname  'pg_catalog' 
ORDER BY n.nspname, c.relname;

(Hint: replacing the semicolon at the end with \gexec will directly execute the DDL statements returned by the query, so you don’t have to cut and paste.) Note that this procedure must be run on every database in the cluster, using the database superuser account, to get rid of any remaining OIDs.

Another important thing I ran into is that some PostgreSQL server-side extensions define new data types, and the code in those extensions must be available to the old server implementation for pg_upgrade. The easiest way to ensure this is to make sure that the old versions of all of the extension packages are installed in the same temporary filesystem as the old server packages are. In my case this was easy because I was only using extensions included in the postgresql-contrib package, which in the process above was built for every version I was stepping through.

Once I fixed the WITH OIDS issue, I completed the upgrade to 12.x and let it burn in for a day before continuing on with 13 and 14, so the whole process took about a week, but I was confident that I could do it in under four hours for the work servers if I could just deal with the OID issue.

The hard systems

I used the query above to check all of my work databases for OIDful tables, and there were quite a bunch. I was able to confirm that the ones for our internal applications were just ancient cruft. Frankly, most of the data in our shared server is also ancient cruft, so I largely did the same there, but several of the tables belonged to someone who was still an active user, and so I asked first. (He told me I could just drop the whole schema, which was convenient.) Finally, I was ready, and sent out an announcement that our internal applications would be shut down and the shared database server would be unavailable for some unknown number of hours this week. The process ended up taking about 2½ hours, most of which was spent copying 50 GB of probably-dead data.

pg_upgrade has two modes of operation: normally, it copies all of the binary row-data files from the old version to the new version, which allows you to restart the old database with the old data if necessary. There is another mode, wherein pg_upgrade hard-links the row-data files between the two versions; this is much faster, and obviously uses much less space, but at the cost of not being able to easily roll back to the old server version if something goes wrong. All of our servers use ZFS, so a rollback is less painful than a full restore from backups would be, but it’s still much better if I don’t have to exercise that option. On the big shared server, it would simply take too long (and too much space) to copy all of the data for every upgrade, but it made sense to copy the data for the upgrade from 9.6 (old-production) to 10.x, and then link for the each successive upgrades, guaranteeing that I could restart the old production database at any point in the process but not worrying so much about the intermediate steps that would be overtaken by events in short order.

Those other servers are included in our configuration management, which I had to stop during the upgrade process (otherwise it would keep on trying to revert the package repository to the production one and then reinstall the old PostgreSQL packages). This also required paying more attention to the server configuration files, since those were managed and I didn’t want to start the database server without the correct configuration or certificates (having had some painful and confusing recent experiences with this). I had to stop various network services and cron jobs on half a dozen internal servers, and call out to our postmaster to keep the mail system from trying to talk to the account database while it was down (all of these applications are held together with chewing gum and sticky tape, so if someone tried to create a mailing-list while the account database was down, the result would likely be an inconsistent state rather than a clean error). I started by copying the existing network and accounts database to the off-site server, so that I could run through the complete upgrade process on real data but on a server nobody was relying on. (I initially tried to use pg_basebackup for this, but it didn’t work, and I fell back to good old tar.) It was in running through this process that I discovered I had neglected to account for a few pieces of our managed configuration. That dealt with, I then proceeded to the production account and network database, and finally the big shared database full of junk.

The actual upgrade procedure

Note that as a consequence of having previously upgraded from PostgreSQL 9.4 to 9.6, our package builds override PG_USER and PG_UID to their old values; thus, the database superuser is called pgsql and not postgres as in the current stock FreeBSD packages. The procedure assumes that you are typing commands (and reading their output!) at a root shell.

Preparatory steps on each server

Before doing anything, check /etc/rc.conf and /usr/local/pgsql to verify that there is nothing special about the database configuration. Most importantly, check for a postgresql_data setting in rc.conf: if this is set, it will need to be changed for every step in the upgrade. Also, check for any custom configuration files in the data directory itself; these will need to be copied or merged into the new server configuration. (Because of the way our configuration management works, I could just copy these, except for two lines that needed to be appended to postgresql.conf.)

zfs create rootvg/tmp/pg_upgrade
cd /usr/src
make -s installworld DESTDIR=/tmp/pg_upgrade
mount -t devfs none /tmp/pg_upgrade/dev

Note that these steps will be different for servers running in a jail — you will probably have to do all of this from the outside. The mount of a devfs inside the destination tree is probably unnecessary; it’s not a complete chroot environment and the package setup scripts aren’t needed.

At this point, I would stop configuration management and set downtime in monitoring so the on-call person doesn’t get paged.

conffiles="cacert.pem keytab pg_hba.conf pg_ident.conf postgresql_puppet_extras.conf server.crt server.key"

This just sets a shell variable for convenience to restore the managed configuration before starting the server after each upgrade. Depending on your environment it might not be necessary; you will almost certainly have to customize this for your environment.

The following is the main loop of the upgrade procedure. You’ll repeat this for each release you need to step through, substituting the appropriate version numbers in each command.

# install the _old_ database release in the temporary install tree
pkg -r /tmp/pg_upgrade/ install -y postgresql96-server postgresql96-contrib
# Update the package repository configuration to point to the repo for the _new_ release
vi /usr/local/etc/pkg/repos/production.conf 
cd /usr/local/pgsql
service postgresql stop
# If and only if you set postgresql_data in rc.conf, update it for the new release
sysrc postgresql_data="/path/to/data/$newvers"
# Try to install the new release. This is known to fail going from 9.6 to 10.
pkg install postgresql10-server postgresql10-contrib
# If the above fails, run "pkg upgrade" and then repeat the "pkg install" command

# Create the new database control files. Will fail if the data directory already exists.
service postgresql initdb
# Check whether the upgrade can work
su pgsql -c 'pg_upgrade -b /tmp/pg_upgrade/usr/local/bin -B /usr/local/bin -d /usr/local/pgsql/data96 -D /usr/local/pgsql/data10 -c'
# The most common reason for this to fail is if the locale is misconfigured.
# You may need to set postgresql_initdb_flags in rc.conf to fix this, but you
# will have to delete the data10 directory and redo starting from the initdb.

# Same as the previous command without "-c". Use "-k" instead for link mode.</i<
su pgsql -c 'pg_upgrade -b /tmp/pg_upgrade/usr/local/bin -B /usr/local/bin -d /usr/local/pgsql/data96 -D /usr/local/pgsql/data10'

# pg_upgrade will likely have output some instructions, but we need to start
# the server first, which means fixing the configuration.
for a in ${conffiles}; do cp -pRP data96/$a data10/; done
tail -2 data96/postgresql.conf  >> data10/postgresql.conf
service postgresql start

# If pg_upgrade told you to update extensions, do that now:
su pgsql -c 'psql -f update_extensions.sql template1'
# if pg_upgrade told you to rebuild indexes, do htat now
su pgsql -c 'psql -f reindex_hash.sql template1'

# For a big database, this can be interrupted once it gets to "medium"
# (make sure to let it complete once you have gotten to the final version).
su pgsql -c ./analyze_new_cluster.sh 
# If the new version is 14.x or above, run the following command instead:
su pgsql -c 'vacuumdb --all --analyze-in-stages'

Before moving on to the next version in your upgrade path, you should probably check that the server is running properly and authenticating connections in accordance with whatever policy you’ve defined.

I followed this procedure for all of my servers, and ran into only one serious issue — of course it was on the big 50GB shared server. pg_upgrade -c fails to diagnose when the database contains a custom aggregate as described in this mailing-list thread, and the upgrade process errors out when loading the schema into 14.x. The only fix is to drop the aggregates in question (after first reinstalling and starting the 13.x server) and then recreating them the “new way” after completing the upgrade. Thankfully this was easy for me, but it might not have been.

After all that, once you’ve verified that your applications are all functioning, you are almost done: rebuild the production package set, restart configuration management, and remove the temporary install tree (there’s nothing in it that is needed any more).

Posted in FreeBSD | Tagged , | Comments Off on A big PostgreSQL upgrade

The Turnpike Extension is too wide

For two decades, my homeward commute (when I’m driving, and these days when I’m also not working from home) has been the same: head south on Mass. Ave. to Newbury St., take the on-ramp formerly known as “exit 21”, and then the Mass. Pike west to Natick. This on-ramp has always been dangerous, with very limited visibility and no merge zone; I’ve narrowly avoided crashes innumerable times by either slamming on the brakes or flooring the accelerator. I didn’t avoid a crash once, many years ago, and got rear-ended by someone roaring down the ramp behind me as I stopped for heavy traffic. This two-mile stretch of highway is four lanes in each direction, with no shoulder; you can see the area I’m talking about on this map (from Google Maps):

This design made a modicum of sense in the original configuration of the Turnpike Extension, from 1963 until the introduction of physically separated E-ZPass lanes in the 2000s: with a mainline barrier toll at the Allston interchange as well as exit and entrance tolls there, westbound traffic was frequently slowed or stopped as far back as Comm. Ave. if not all the way to Copley Square. But when MassDOT implemented all-electronic tolling on the Turnpike in the 2010s, the Allston interchange was reconfigured to have three full-speed through lanes in both directions — which eliminated the bottleneck at Allston. To be clear, this change didn’t increase the capacity of the Turnpike; it simply meant that traffic was not stopped in Allston in addition to being stopped at the Copley (eastbound) and Newton Corner (westbound) lane drops at times of congestion. MassDOT is now engaged in a process that will hopefully replace the outmoded and wasteful Allston interchange, which was designed the way it was to support a connection to the never-built I-695 Inner Belt freeway.

The Prudential Center, one of the first major freeway air-rights projects in the US, was built in conjunction with the Turnpike in 1961–63. The John Hancock Tower and Copley Place followed in the 1970s and 1980s, and the state has long been looking to generate even more revenue from its valuable Back Bay real estate. Since late 2020, two air-rights projects have been under construction between Mass. Ave. and Beacon Street — with a third one under contract but yet to begin — and as a result there has been a lane restriction in both directions between Copley and Beacon St. to allow the construction crews to safely install the buildings’ foundations. After much hue and cry over how much of a traffic impact this would have, the end result has been … nothing. (Granted that traffic has been reduced somewhat as a result of the pandemic, but not so much as that.) With the bottlenecks through Allston and Copley already being limited to three lanes, the traffic capacity simply isn’t limited by the work zone — at least not any more than it is limited by the unsafe merges that were always there. Eastbound traffic still backs up at the Copley lane drop, and westbound traffic doesn’t back up until Market St. or even closer to Newton Corner. On the westbound side, there simply isn’t that much traffic entering at Copley Square or Mass. Ave. and exiting at Allston — that’s simply not a route that makes sense for most of the trips that could conceivably use it — so the traffic that takes exit 127 westbound is traffic that came from I-93 or points east, and without a toll barrier on the exit ramp, that traffic can queue along the whole length of the ramp without backing onto the mainline Turnpike.

This aerial photo shows the air-rights parcels (under-construction parcels shaded in purple, future parcel in green):

(I may be misremembering the parcel numbers, and didn’t bother to look them up, so they’re not labeled on the map.)

One of the most controversial aspects of the project to replace the Allston interchange (which is also supposed to include a bus/train station and significant new construction on the Harvard University-owned former railyard) has been the area called “the throat”, where MassDOT is trying to thread a two-track railway and twelve lanes of freeway (eight lanes of the Turnpike, four of Soldiers Field Road) in a very narrow stretch of land between the Boston University campus and the Charles River, without impacting traffic on the MBTA Worcester commuter rail line or on the Turnpike or on Soldiers Field Road during construction, and without any permanent structures in the Charles, and without unduly limiting use of the Dr. Paul Dudley White path, which parallels Soldiers Field Road. That section is highlighted in the map below:

Community advocates have largely recognized this as a fool’s errand, and contrary to the city’s and state’s climate goals to boot. Fixing the interchange and removing the viaduct through the throat, and of course building the new train station, are all recognized as positives, but even with a Federal Highway Administration waiver for limited shoulder widths, it has proved impossible to squeeze all of these roadways into the space allowed without either elevating one over the other or building into the Charles. Of course, the “impossibility” is a sham: it’s only “impossible” because both MassDOT (through the effort of former MassDOT secretary Stephanie Pollack) and the Department of Cars and Roads Conservation and Recreation have refused to countenance any reduction in freeway capacity. It is quite clear from the results of the current work zone that the Turnpike is two lanes too wide between Allston and Copley, and it would be a tremendous boon for safety and the environment if it was reduced to six lanes with full shoulders and safe merges at the entrance ramps. Likewise, although I do not have as much direct knowledge, the bottlenecks on Soldiers Field Road are all downstream, on Storrow Drive, especially at the Bowker Interchange and at Leverett Circle: Soldiers Field Road could stand to be just two lanes wide in this stretch.

These width reductions (24 feet on the Turnpike, 20 feet on Soldiers Field Road) would have a very limited impact on traffic, if DOT and DCR stopped their obstructionism, and the result would be a much cheaper, more constructable Allston project with bigger buffers between the freeway traffic and park users.

Posted in Transportation | Tagged , | Comments Off on The Turnpike Extension is too wide

A busy week for transportation legislation

It’s Memorial Day weekend, and in Massachusetts that means two things: the state legislature is debating the budget and the state’s draft Capital Investment Plan has been published for comment. In Washington, the Biden Administration has published its official budget request for Congress (which will go right in the trash; Congress writes its own budget), but the administration’s so-called “American Jobs Plan” is being debated (and torn to shreds) by Congress, and in addition, the five-year surface transportation authorization — which was given a one-year extension last year because of the pandemic — is also being debated.

As a result, the transportation funding situation for the coming year is even less clear than it usually is. The US Department of Transportation released a glossy brochure attempting to explain the President’s budget request, which called out some worthwhile initiatives but largely failed to clarify which programs were associated with which funding requests. Amtrak released its own glossy brochure, explaining to congressional delegations what sort of service enhancements it was thinking about (if only Congress would appropriate more money) — largely devoid of HSR or any particularly compelling program for capital investment or frequent service.

I pointed out on Twitter a few weeks ago that Joe Biden is both the first and last president of his generation — the Silent Generation. He took the oath of office as a United States Senator just a few days after I was born; his political formation is quite different from both the G.I. Generation who preceded him in birth and the Baby Boom Generation who preceded him to the presidency. This is reflected in his speechmaking, in his non-confrontational approach to governing, and especially in his approach to policy: he is going to defer to Congress, and while he will privately jawbone Manchin and Sinema, ultimately he is going to sign whatever Congress passes, and he isn’t going to make a big deal of it if some of his major legislative initiatives founder in the Senate.

While I do not doubt the sincerity of Biden’s support for the proposals his administration has put forward, I believe he values the appearance of bipartisanship more than he values livable cities or indeed an inhabitable planet, and fundamentally, when the autoists in Congress finish chewing up and spitting out his infrastructure proposals, turning them into more of the same polluter-subsidizing climate arson, he will acquiesce without protest. His only “red line”, so far as I can perceive, is his refusal to increase taxes on polluters — who by and large make less than $150,000 a year and drive a light-duty truck. The rich are simply not numerous enough for any taxes aimed at changing their behavior to have any meaningful effect on the existential crisis of our time. We need more and heavier sticks, like a carbon tax, like a national VAT, that would actually be paid by the vast majority of people (myself included), in order to incentivize a meaningful amount of behavioral change.

OK, enough about the situation in Washington — what’s going on on Beacon Hill?

This past week, the Senate unanimously passed its version of the state budget on Thursday. I believe there is one more procedural vote to come, this Tuesday, and then it will go back to the House, which will take another procedural vote, and then both houses will appoint members of a conference committee to hammer out the numerous small differences between the chambers. (Of course, the conferees know who they are already, even though they haven’t been formally appointed, and through this whole process, the staff of both chambers has been tracking the points of disagreement so they know what they need to hammer out.) This whole process might take a week or two, then both chambers will vote again to accept the conference report (by a veto-proof majority) and then deliver the final engrossed text to the governor’s desk, at which point he will veto a bunch of items, and then both chambers will have a day-long vote-a-thon to override most or all of the individual vetoes. Hopefully they can get this all done by the end of June, at which point the legislature will have to take up the FY21 close-out supplemental budget (to reconciles the budget that they passed back in January with what the state actually spent).

The budget as passed by the Senate does not include any revenues from the Biden administration’s “American Rescue Plan”, for the simple reason that the Baker administration doesn’t yet have guidance from Washington on how the state is allowed to use the money. This is also an issue for the Capital Investment Plan, which I’ll discuss next. (The MBTA budget assumes the availability of ARP funds to cover operating expenses, but is not itself subject to legislative approval.) It does include a significant drawdown of the state’s “Rainy Day Fund”, which will presumably be reversed in a supplemental budget once the federal guidance is received on eligible expenses. (ARP funds are generally available for allocation through the end of calendar 2023, but cannot be used to reduce state tax rates, fund pension obligations, or various other things state legislatures might want to do, so each federal agency charged with disbursing ARP money has to go through a rulemaking or similar procedure to issue official guidance about which expenses are or are not eligible and what the state must certify in order to access the funds — the previous Coronavirus relief bills, “CARES” and “CRRSSA”, had similar administrative complications but subtly different requirements.)

Without debate, the Senate adopted an amendment by Sen. Joe Boncore, chair of the Transportation committee, which restructures and increases the fee charged for TNC (i.e., Uber and Lyft) rides, adds additional reporting measures, and creates a trust fund into which the state portion of the fee revenue is paid. Most significantly, the Boncore amendment creates a “public transit access fee”, an additional twenty-cent charge for trips that both begin and end within “the fourteen cities and towns”, and paid into a segregated account to support a low-income fare program to be established by the MBTA. (Which are “the fourteen cities and towns”? They are the communities in the original, inner-core MBTA district, which still receives the majority of bus and rapid transit service.) The amendment passed 37–2 on a non-recorded standing vote, but it remains to be seen whether this language will survive the conference committee; if it does, expect the governor to veto it. (It’s likely that the legislature will override the veto if it gets that far.) Unlike with the bond bill back in January, the legislature will remain in session after the budget is passed, so there is no possibility of a pocket veto.

I should note here that neither House nor Senate budget includes provisions for an MBTA board of directors, to replace the current five-member, unpaid Fiscal and Management Control Board, which expires at the end of June. The governor’s budget as filed included such language, but it was dropped from the budget by House Ways & Means, and it was also left out of the Senate budget. (Senate Minority Leader Bruce Tarr proposed an amendment to restore the governor’s language, but it was “bundled” with several other amendments and rejected on a voice vote.) There are companion House and Senate bills in the Transportation Committee which would establish a new board, but thus far, with only five weeks left of the FMCB’s legal existence, the committee has not chosen to advance either one. The bills are H.3542 by Rep. Meschino and S.2266 by Sen. Boncore; the language in both looked the same on spot-checking but I did not do a line-by-line comparison. In addition to expanding the board to seven members, S.2266 would provide for some local representation, by giving the existing municipally-appointed Advisory Board the right to appoint one MBTA board member, and would authorize an annual stipend of $12,000 for each board member except the Secretary of Transportation. While this is not the exact structure I would prefer, time is of the essence and I would like to see one of these bills reported out of committee within days to allow for a smooth transition from the old board to the new.

Finally, on to the state’s Capital Investment Plan (CIP), which the MassDOT board and the FMCB voted to release for public comment last Monday, six weeks before the deadline for it to be adopted and generally much too late for any public comments to make a significant impact. As with last year, significant uncertainties related to the aftermath of the pandemic and the availability of federal support have been used to justify a one-year “maintenance of effort” CIP rather than the five-year CIP the law requires. These uncertainties do not get the state out of its federally-mandated five-year State Transportation Improvement Plan, through which all federal grant programs flow, so we still have some idea of what might be funded in the out years simply because it has to be programmed in the STIP; the Boston Region MPO will meet on Thursday to endorse its FY22-26 TIP, which is the largest regional contribution to the STIP, and this will then flow through to the state CIP, which both the MBTA and MassDOT boards must formally adopt at their joint meeting at the end of June. The state and the federal government operate on different fiscal years (the state’s is July to June and the feds use November to October), which means the exact alignment of the two plans depends on the exact scheduling: some SFY22 projects are funded with FFY21 obligations.

One thing we do know about the Rescue Plan is that it includes an additional $175 million of funding to recipients of Federal Transit Administration capital improvement grants in fiscal year 2019 which have not yet entered revenue service. Up to 40% of this could go to the Green Line Extension — except that the GLX project is running under budget and may not need any more money. The text of the act makes the distribution of funds non-discretionary, but the agency will have to determine what Congress intended by this provision in the case of funds to be distributed being in excess of obligations. The MBTA and FTA are in discussions to see how the GLX money could be reallocated — the original funding agreement includes a clawback provision for Cambridge and Somerville if their local contributions turn out not to be required, but if the surplus comes in beyond that amount, after all contractor claims are resolved, the MBTA would like to use the money for other priorities. This should be resolved by the time the boards vote on the CIP in four weeks, so it’s likely that there will be some transfers of these funds in the final CIP that aren’t shown in the draft.

Having said all of that, and in the knowledge that my comments will have no meaningful effect on the process, I still chose to email the state with my comments, which will probably get a formal reply from the staff some time in September. Here is what I said, lightly edited for presentation here:


I will begin my comments with some process issues.

  • While the “accessible” PDF version of the draft is definitely more accessible than the “story map” (which has undocumented requirements for computer hardware and is difficult to navigate or resize), it is still lacking in some basic information, such as the actual location (at least city or town) of projects in the MBTA section of the plan. Many of the “project descriptions” are quite cryptic, even for someone who regularly attends/watches the board meetings, and need a more complete explanation. That said, the breakdown of programmed expenditures in the last four columns of Appendix A is an appropriate and helpful way to present the status of a project in the absence of an itemized plan for the out years.
  • To repeat my comments from previous years, the schedule for comment is far too rushed. Anyone who has followed this process over time knows that all of the important decisions have been made by the staff already, sometimes months ago, and as a result there is almost zero chance that public comment will result in any changes to the draft before the boards vote to adopt it in a few weeks. The capital planning process needs to be open and transparent, and that starts with publishing the universe of projects and assigned priorities well before the end of the fiscal year so that members of the public can develop reasoned arguments about which should be advanced or delayed.
  • While I am sympathetic to the desire to do a short-term capital plan given the uncertainties over whether Congress will pass an infrastructure bill, it is unfortunate that the draft CIP only shows one year, and does not show projects in the out years that might have an opportunity to be accelerated if additional funding is made available. This is important information and citizens deserve to have at least some details so that we can make a case to our representatives and before the various boards. Many agency priorities have changed and numerous projects have been accelerated, so it is not possible to refer back to the FY20-24 CIP for information about FY23 and FY24 projects.

I have one comment regarding Highway Division programs: I am disappointed that the Weston Route 30 reconstruction and bike/ped safety improvements project was not programmed by the Boston Region MPO and said so in comments on the draft TIP. Should additional funding become available before the end of FY22 I urge that consideration be given to programming this important project.

My remaining comments are all regarding the MBTA section of the draft.

  • Should the Green Line Extension come in under budget (as suggested at last Monday’s board meeting), and should FTA allow the MBTA to reprogram the FFGA funds, I strongly support funding the Red/Blue Connector and/or advancing the bus facility modernization program by replacing Arborway garage.
  • However, I remain unalterably opposed to the destruction of the North Cambridge trolleybus network as currently proposed by MBTA staff. Trolleybuses are inherently more efficient than battery buses, do not require supplemental diesel heaters, and are already zero-emissions vehicles; North Cambridge has been a trolleybus carhouse since it was converted from streetcars in the 1950s (when most Mattapan carlines were dieselized, contributing to today’s environmental injustice in that neighborhood). The Transportation Bond Bill specifically authorized “transit-supportive infrastructure” program funds to be used for trolleybus infrastructure, including electrification, and the MBTA should be making plans to expand North Cambridge trolleybus service to other nearby bus routes such as the 68, 69, and 77, by extending the trolley wire and/or acquiring battery-trolleybuses with in-motion charging.
  • The continued progress on making commuter rail stations fully accessible is laudable. However, I continue to be concerned that upgrading stations to full high-level platforms is being approached solely as an accessibility issue, and thus being advanced piecemeal, rather than as a significant constraint on operations, staffing rationalization, and competitive rolling stock procurement — as was obliquely pointed out by Alistair Sawers in his presentation before the boards last Monday. While I strongly support completion of the current platform accessibility projects (all of them on the Worcester Line), future investments in platform upgrades need to be done more strategically.
  • In particular, given the response of the FMCB to Mr. Sawers’ presentation — specifically, endorsing the idea of proceeding with a traditional procurement for EMU rolling stock — construction of high-level platforms on the remaining Providence and Fairmount Line stations needs to be prioritized, packaged as a single unit of design to control costs, and put out to bid ASAP, preferably in FY22, to ensure that these lines will be able to use standard rolling stock purchased in a competitive marketplace rather than bespoke trains with nonstandard multi-level boarding. Platform upgrades on other lines should be prioritized on a line-by-line basis, so that remaining diesel lines can be converted to remote door operation and the reprocurement of the operating contract can go to bid without the burden of unnecessary assistant conductors.
  • The placeholder commuter-rail project labeled “future fleet” should obviously be reprogrammed as an explicit EMU procurement. The General Court has made it quite clear that it is the policy of this Commonwealth to electrify the commuter rail network, using overhead catenary electrification and EMU rolling stock, and has authorized nearly a billion dollars in bond issuance over the next decade to put it into practice. It is time for the MBTA and MassDOT to get in line.

Project-specific comments:

  • P0170: station design for full access to both platforms should be advanced.
  • P0261: the description says “3rd track feasibility study” but other MBTA documents and presentations have implied that the third track was actually going to be progressed to design and eventual construction. Please clarify.
  • P0650 and others: since coach availability has been an issue recently, even during the period of pandemic-reduced schedules, I support continued lifetime extension and overhauls of legacy rolling stock to keep this equipment running while electrification is being pursued at the greatest practical speed.
  • P0863: strongly support construction of a south-side maintenance facility, but caution that the design needs to be able to accommodate articulated EMUs which are several times longer than legacy coaches, so as not to constrain rolling stock procurement.
  • P1009: what FTA compliance actions are these? For a $57mn program this needs to be spelled out explicitly.
  • P1011: GLX hasn’t even finished constructing the maintenance facility, and you’re already looking to spend $12m to modify it?
Posted in Transportation | Tagged , | 1 Comment

Weekend excursion: Stations of the B&M New Hampshire Main Line/MBTA Lowell Line

As I’ve neared the end of this series of posts, I’ve gotten a bit better at procrastinating, so most of the photos this post is based on (see the associated photo gallery) were taken a month ago now, and I’m drawing a lot on unreliable memory and aerial photos (and a bit of Wikipedia) to bring this together. It’s an interesting time to be writing about this line, for a number of reasons I’ll try to articulate.

As the title suggests, today’s Lowell Line was historically the Boston & Maine’s New Hampshire Main Line, with passenger service north through Nashua, Manchester, and Concord into the White Mountains and through Vermont to Montreal. In Chelmsford, north of Lowell, the line connects with the Stony Brook Railroad and becomes part of Pan Am’s freight main line from Western Massachusetts to Maine. There have been discussions on and off about re-extending commuter rail service to Nashua (where the historic B&M station apparently still stands) and even Manchester, but the discussions have always foundered on New Hampshire’s refusal to subsidize anything other than private automobiles. Recently, Amtrak released a map of possible service extensions which included service as far as Concord — Amtrak, unlike the MBTA, has both the legal right to operate on any railroad and a mandate to provide interstate service, and already operates Downeaster service along the line as far as Woburn.

As I described in more detail in my survey of the Western Route, the B&M planned in the 1950s to run all longer-distance services north of Boston via the NHML, with trains to Haverhill and Portland using the “Wildcat” Branch in Wilmington to access the northern part of the Western Route. This made a good amount of sense (and still does, which is why the Downeaster does so) because the NHML is the highest capacity line on the ex-B&M network, and the second-highest-capacity on the entire MBTA system: it’s the only North Side line with no single-track bottlenecks and no drawbridges other than at North Station; even slow diesel trains can maintain decent speeds because the stations are few in number and fairly widely spaced. All of the stations have platforms for both tracks, allowing bidirectional service without scheduling difficulties, although with the exception of the recently constructed Anderson RTC/Woburn station, they are all low-platform (all except West Medford and closed-for-demolition Winchester Center have mini-highs).

Which then brings me to the saga of Winchester Center, one of the two stations that got me started on this series of travels back in March. Winchester was scheduled for accessibility upgrades, with final design nearly complete and construction supposed to be put to bid in the second half of this year, when regular inspections early this year revealed safety issues with the old station’s platforms. Rather than perform emergency repairs, the MBTA chose to simply demolish the old station early, while commuter-rail ridership was low due to the pandemic, and remove the demolition from the scope of the reconstruction contract, reducing the cost and allowing construction to proceed more quickly. While I did not make it to Winchester Center in time to see the old platforms, I did get pictures of the demolition work in progress. (Not literally in progress, though, because I made my visit on Easter Sunday when no work was taking place.)

So with all that out of the way, let’s go station-by-station. With the historic stops in East Cambridge, Somerville, and Medford Hillside all long gone, the first stop on the modern Lowell Line is at West Medford. The station is located next to the West Medford post office (in fact the inbound shelter looks to be attached to the side of the building) and it is inaccessible, with only low-level platforms on both tracks. A few years ago, the MBTA’s system-wide accessibility program rated West Medford one of the highest priority stations to receive full accessibility upgrades, but I haven’t seen anything to indicate that this has been advanced in the capital program since then, not even as far as a 10% design. In the 2018 passenger counts, about 600 people a day used West Medford — which is pretty good for a commuter rail station but only the fourth-busiest suburban station on this line. Much of the station’s popularity can be explained by its assignment to the inner-core fare zone, zone 1A, so travel to North Station costs only as much as a subway fare and is much faster than taking the bus to Sullivan and then transferring to the Orange Line into town. (It will be interesting to see how the popularity of this stop changes when the Green Line Extension opens, since it will operate much more frequently and offer bus connections closer to West Medford.)

The route runs on a viaduct through much of Winchester; Wedgemere station is located near the south end of that viaduct, where the railroad crosses the Aberjona River at the north end of Upper Mystic Lake. In the middle of a wealthy residential neighborhood and without practical bus connections, Wedgemere gets a surprising amount of traffic compared to its 120-stall town-owned parking lot, about 300 riders in 2018. With the closure of Winchester Center station, only four tenths of a mile to the north, Wedgemere is currently the only station in the town of Winchester, but with much more development within walking distance, Winchester Center had about 50% more traffic.

North of Winchester Center, a long-abandoned branch once led to downtown Woburn, with the main line running through a largely industrial area on the east edge of the town, before crossing under Route 128 into a truck-oriented wasteland of industrial parks. At the Route 128 overpass, Mishawum station was formerly the primary station serving Woburn, located between two toxic-waste cleanup sites, “Wells G & H” and “Industri-plex”. It used to be accessible, and was upgraded with a ramp system on the inbound platform and mini-highs on both platforms before being abandoned in favor of a new station half a mile deeper into auto-dominated industrial-park hell. The former parking lot, shared with the Woburn Logan Express, has turned into a bank office building and a Dave and Buster’s. The station still stands, and still seems to be receiving some maintenance, but at some point in the last decade, the mini-high platforms were partially demolished to reuse the folding steel platform edge at another station. As a result, Mishawum is the only MBTA station to have been accessible, and then made inaccessible. As late as 2018, long after the new station was opened, Mishawum was (apparently illegally?) still being served as a flag stop by a handful of trains a day; the 2018 traffic counts (32 passengers a day) are the most recent mention of any kind I can find of it. The town of Woburn apparently wants to see service maintained at Mishawum, because as we shall see, its replacement is even farther from where any humans can be found without a steel exoskeleton, whereas there is a residential neighborhood not too far southwest of Mishawum. But it’s no longer shown on public schedules, and with the MBTA’s slow diesel trains it’s really too close to the new station to even be a flag stop. Even with electrification, a new station at Montvale Ave. or Salem St. would have a much larger catchment of Woburn residents and result in a more appropriate interstation.

The new station in question is of course Anderson Regional Transportation Center, which is an enormous ocean of parking, nearly 2,000 spaces, accessible only via an unwalkable car sewer with a direct exit off I-93, connected to a combination bus stop and train station, and owned and operated by Massport. Of course, it hardly matters that it’s unwalkable, because in the middle of this toxic waste site (the Woburn Industri-plex Superfund site) there’s nothing you’d want to walk to or from. For train facilities, the station has two overhead pedestrian bridges, one connecting the high-level center platform to the second floor of the station building, and the other, at the far northern end of the platform, connecting to the northwest edge of one of the enormous parking lots. In addition to the MBTA commuter trains, the Downeaster stops here, and presumably if the proposed Amtrak service to Concord ever gets off the ground, it would as well (and probably Lowell, too). When I visited, the parking lots were barren, and Logan Express bus service had been suspended due to the pandemic. Despite the horrible location, the station definitely got plenty of use, with more than 1,200 passengers a day in 2018. (One wonders how many of those passengers are actually driving down I-93 from New Hampshire.)

The next station north, Wilmington, is where the Wildcat Branch diverges to the north as the main line heads north-northwest. The turnout is located just north of the outbound platform, resulting in offset platforms. The single-track Wildcat only connects to the outbound track, but a universal crossover south of the station allows access to both tracks; passenger service using the Wildcat does not currently make a stop a Wilmington, so it matters little that the branch only serves one platform. There is a 200-space MBTA-owned parking lot on the east side of the tracks, but this is far too small to account for the average daily ridership of 575; there is also an apartment complex, “Metro at Wilmington Station”, at the south end of the inbound low-level platform.

There’s a long interstation, about 6 miles, between Wilmington and North Billerica, but the line runs through wooded, low-density areas nearly the entire length. Just south of North Billerica is the B&M’s former maintenance yard, now an industrial park called Iron Horse Park, a 553-acre Superfund toxic waste site, including numerous landfills and former waste lagoons, which are contaminated with a variety of solvents, heavy metals, asbestos, and pesticides. Iron Horse Park is in its 37th year of EPA-supervised cleanup, partially funded by the MBTA, which made the mistake of acquiring 150 acres of the property in the 1970s as it began the process of taking over the B&M’s commuter rail operations. The MBTA’s new backup rail operations center is being constructed in a less restricted part of the park.

I actually went to North Billerica station first, before heading down to Iron Horse Park. It’s another two-track station with low-level platforms and mini-highs, made slightly more interesting by its 19th-century station building (although it’s been extensively renovated, to the point that I had figured it was new-old-style rather than Actually Old when I visited). The station has two surface parking lots, operated the Lowell RTA, with 540 spaces between them, and is also served by two LRTA bus routes, helping to explain its over 900 daily riders in the 2018 statistics. As the sun was setting, I did not make it all the way to Lowell on my initial trip, but returned a week later as part of a wrap-up trip that also included stops in Worcester, Lawrence, Rowley, and Newburyport.

At Lowell, I found LRTA’s exceedingly expensive and aggressively human-enforced parking garage, located over the rail line and next to LRTA’s central bus hub. Google Maps initially wanted to take me into the west garage entrance, which I found blocked with Jersey barriers, and when I found the entrance that was nominally open, I found that it was (unlike every other RTA garage) not equipped with automatic ticketing and payment systems, and the human who was supposed to sell me a ticket was not in their booth. I moved on, not wanting to spend $8 to park for 15 minutes, stopping to take a few quick pictures of the bus hub and the commuter-rail platform, but was chased away by an LRTA employee in a pickup truck. The platform here is a low-level center platform, between the westernmost pair of tracks, with a half-length high-level platform accessed from the 700-space garage, which is built across the tracks. The line quickly narrows to two tracks north of the station before crossing the Pawtucket Canal, and narrows to a single track at the wye with the Stony Brook. There is no layover facility on the Lowell Line, so trains entering and leaving service must do so at Boston Engine Terminal in Somerville; the lack of such a facility is one of the major constraints on increasing service on the line (because there is little room to store additional trainsets that would be required). In 2018. more than 1,500 people a day used the station.

That concludes the March–April run of MBTA station “weekend excursions”, but the project as a whole is far from complete: in addition to the new stations currently under construction (six stations of South Coast Rail, to open 2024; New Chelsea, opening later this year; New Natick Center and Central Falls–Pawtucket, opening next year) there still remain all of the stations that I avoided because they were in crowded areas and there’s still a pandemic on: Boston Landing, Lansdowne, Back Bay, Ruggles, Forest Hills, South Station, JFK/UMass, Quincy Center, Braintree, North Station, Malden Center, and Porter, plus the rest of the Fairmount Line and three stations in Rhode Island. In addition, Mansfield station, which I last saw while it was still under construction, fully opened in 2019. I’ll be fully vaccinated in a few days, and weather permitting, I still have plenty to do and see.

Posted in Transportation | Tagged , , , , | Comments Off on Weekend excursion: Stations of the B&M New Hampshire Main Line/MBTA Lowell Line

Automatic generation and validation of train schedules

Passenger railroads throughout the world have mechanisms to generate timetables for both service and capital planning purposes. The way I’ve done this in the past is with the Mk. 1 eyeball: you come up with a schedule, maybe draw some stringline diagrams to determine minimum separations, and then shift around the run times to ensure that there are enough resources at each crossing to allow for the desired schedule. Sometimes, of course, this doesn’t work, and it’s painstaking and laborious, and nearly impossible to answer questions like “What’s the minimum investment (in tracks and switches, or in increasing speeds) to allow for the schedule we want?” Actual railroads don’t do it this way, of course — their networks are much more complex, and they have more constraints than are necessarily obvious from aerial photography. They use software to validate their timetables, and in many cases will use linear-programming libraries to find the schedule that maximizes equipment utilization, minimizes capital investment, or optimizes some other criterion.

Last week, the MBTA Fitchburg Line schedules were announced for the spring rating. The Fitchburg Line has been under construction for the entire month of April, so with the majority of the line being bustituted, the MBTA and its contractors chose not to publish the spring schedule at the same time as the other lines. A co-worker of mine lives in West Concord; before COVID-19 regularly took the Fitchburg for her daily commute, and we had an email exchange about how service could be improved over the one-train-per-hour schedule that has been introduced. This line is mostly double-tracked, except for short single-track segments in Fitchburg and Waltham which constrain the schedules that can be operated. I noted to my colleague that I didn’t know much about SAT solvers; I thought it obvious (at least to a computer scientist) that this question could be represented as a satisfiability problem, for which there are lots of known techniques and libraries to perform the computation. (The general boolean satisfiability problem for three or more variables, called “3SAT”, is known to be computationally intractable; someone who found an way to solve it efficiently would instantly win all of the major prizes in computer science and operations research. In the mean time, there’s been a lot of research into making solvers faster even given though there is no categorically efficient algorithm.)

This question was bugging me over the course of the week, so I did a Google search to see if I could come up with some plausible code that might work. I first got distracted into looking at a constraint solver called “kiwi” (which is a reimplementation of an academic solver called “Cassowary” designed for use in responsive user-interface implementations), but found that it was too limited to even figure cycle times, which was my starting point (not even looking at single-track constraints like the Fitchburg or the Old Colony). I went back to Google and added some keywords to suggest that I was interested in how railroads use solvers, and got some different results.

For whatever reason, I landed on Y. Isobe, et al., “Automatic Generation of Train Timetables from Mesoscopic Railway Models by SMT-Solver“, published in IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences in 2019, for which the first author maintains a web page (probably what I first saw in the Google results, rather than the paper itself). The research was a collaboration between a government institute, AIST, and the East Japan Railway Company (better known as “JR-East”). “SMT” is an acronym for “Satisfiability Modulo Theories“, a generalization of the boolean satisfiability problem (plain old “SAT”) to include richer data types, such as sets and arrays, along with integers and real numbers, and relations such as set-inclusion and numerical inequalities. This sounded like not only the exact sort of thing to be looking at, but quite possibly someone had already done the (not inconsiderable) work of proving that it worked on a model of an actual railroad.

After skimming the paper the first time, I realized that I did not understand their railroad infrastructure model, or perhaps I do not understand how JR-East designs railroad infrastructure. I figured I’d work through the examples first, but immediately ran into snags (after first installing the prerequisites). It took an entire evening to figure out that the code was written for an older release of the Z3 SMT solver, and the current version of Z3 has a slightly different output format that needed adjustments in the parser. The code for the paper is written in OCaml, a language I do not know, so I had to figure out enough of the debugger to figure out where the parse was failing, and then learn enough OCaml syntax to figure out what the failing code was looking for and how to make it look for something different.

The examples, as it turns out, didn’t help. Once I fixed the parser, I could run the examples through the solver and it would generate the expected output, looking very much like the solutions shown in the paper. (I should maybe look more closely at the code, because it does a nice job of generating stringlines directly to PDF or SVG using OCaml bindings for the cairo graphics library, and maybe I could steal that even if I can’t make the solver work.) The difficulty was not with the solver itself, but trying to wrap my head around the way it models railroad infrastructure — what the authors call a “mesoscopic model”. (The term isn’t theirs, it’s cited to an earlier paper that I haven’t read.)

In this paper’s version of a “mesoscopic model”, there are two kinds of fixed objects: “links”, which are just given arbitrary unique numbers, and “structures”, which can be stations or interstations. There’s the first loose cobble to trip you up: you’d think a “link” would correspond to the track between stations — i.e., an interstation — but no, that’s a “structure”. Rereading section 2 of the paper doesn’t really clarify matters, but staring at the examples some more makes it clear that a “link” is more like the interlocking at a station throat. The structure model assumes that every track can connect to every other track at such a junction. My first attempt to model the Worcester Line — chosen because it’s the most familiar to me and has capital construction projects that I could model the effects of — foundered on this issue: while there are many universal crossovers on the newer part of the line, not every station has one, no station except Framingham has one at both ends, and when I looked at how train routings were specified, it’s a sequence of “links”, not stations, so I would have to explicitly specify which track each service pattern would use, which is one of the things I had expected the solver to figure out for me.

After looking again at the major worked example in the paper, which deals with sequencing local and Shinkansen trains on the single-track Tazawako Line, I figured that I should start with a single-track MBTA line, and perhaps things would become clearer. Even there I had trouble, because it seemed like, in the example, the trains could only pass each other at stations, and JR had built nearly every station with two or three platform tracks. (In fact, it was only just now, as I am writing this, that I looked at the Wikipedia article and saw that the paper’s authors had modeled two passing sidings on the line as stations! The solver doesn’t distinguish between the two types of “structure”, but the presentation in the paper and in the solver’s graphical output does. And oh, by the way, all of the comments in the example data are written in Japanese, which I can neither read nor display.) I figured the shortest and least complex of those would be the Greenbush Line, so I spent an evening panning through Google Maps using the “measure distance” tool to figure out how long the interstations were and how much of each was single- vs. double-tracked.

That got me into a more fundamental issue, which I freely admit to just fudging as a means of playing with the tool to see if I could get anything interesting out of it. All of the intervals given in the structure model are in units of time, not distance. This makes sense for stations, but when you want to apply it to tracks, you run into the issue that the time is going to depend on the ultimate routing: a train that approaches an interlocking with a green signal is going pass through it faster than a train that has a restrictive signal, and there’s nothing in the model that allows you to tell the solver that. In fact, unless forced to look for a better schedule, the solver would often come up with bizarre delays for no obvious reason, because Z3 isn’t an optimizer — it finds a solution that satisfies the constraints, but not necessarily the best one. The higher-level timetable solver offers some manual knobs that allow you to force this, in particular a max_time parameter, but unfortunately it’s global. The basic assumption is that not only do you already know the route, but you have already modeled the rolling stock’s performance on the route and you know exactly how much time it is expected to take for all phases of the trip. (And probably also that you’re using equipment which has decent acceleration and braking performance in the first place, i.e., EMUs.)

So rather than try to simulate an HSP46 hauling four bilevel coaches, without specific knowledge of track speed limits and minimum separations, I just reverse-engineered times from the published schedules. There is an object in the model which represents a single run of a train, and you can override the times specified in the structure part of the model if you have trains with different performance characteristics or stopping patterns, but I did not explore that aspect — if I did anything more with this solver, it would probably involve creating a higher-level language to describe lines and trains which could then output the enormously redundant input language of the timetable solver. (And at that point, I’d probably be close to ripping apart all of the OCaml bits and interfacing Z3 to my train acceleration model directly. I don’t think I’ll get there, because that’s a full-time job.)

It was then that I learned, rather to my surprise, that the published schedules don’t work. Clearly they must, in reality, since the schedule is what Keolis actually operates, but the solver couldn’t satisfy the schedule as I had reverse-engineered it. Unfortunately, the solver can’t tell you what’s wrong: it just outputs “unsatisfied” and you have to hack away at the model description until you get something that works enough to generate a schedule, and then look at the graphical output to see where delays are being inserted to account for errors in the model. I did finally get my Greenbush Line model to work, although it didn’t end up looking entirely like the MBTA schedule it was based on.

Sometimes it’s not an error in the model; there are some significant limitations in the solver as well. The most important of these is that it can’t handle turning trains, whether on the platform (for a mid-route short turn) or at a stub-end terminal. I was able to make a model of the Fitchburg Line sort of work, by including Westminster Layover as a “station” and making all trains run through to Westminster, where it doesn’t matter if it’s an hour to the next run, rather than turning on the platform at Wachusett, but when I tried to model short turns at Littleton, there was simply no way to tell the solver “no, this specific train has ten minutes to turn around and must immediately head back whence it came, it can’t just sit there blocking the platform while three other trains go by”. I am certain that this requirement (and round trips in general) can be implemented in a solver like this, by adding additional constraints, but again, modifying the solver logic is a job for a professional (and someone who actually knows both OCaml and SMT solvers). The problem is equally significant at the city terminals: the model wants to have all trains arrive at platform 1 and depart from platform 2, and that’s a physical impossibility — just not one that you can encode in the model without explicitly assigning tracks to trains and manually generating separate “inbound 1”, “inbound 2”, etc. — which again is something I wasn’t interested in doing by hand.

One thing that the solver can do is work with “periodic” schedules; i.e., those that are repeated the exact same way throughout the day at a specified interval. It can’t figure the period for you, but if you give it a period, it will add in the modular arithmetic in the SMT problem definition to ensure that multiple trips of the same set of trains don’t conflict with each other. This makes it easy to figure out how “clockface-compatible” a particular service model is: if it is still satisfiable with period = 3600, you can run each train hourly. If it works with period = 1800, you can run two per hour, and so on. (If it only works with period = 4500 then unfortunately you’ve got a service that repeats every 75 minutes, ugh.) This inspired me to look more closely at the Dorchester bottleneck on the Old Colony lines. All trains pass through Quincy Center, and nearly all trains stop at both Quincy Center and JFK/UMass (which was never intended to have a station, but it just barely turned out to be possible to squeeze one in east of the Red Line tracks). I could take my Greenbush model, delete everything south of Quincy Center, and just see directly what the capacity of that bottleneck was.

Stringline diagram showing maximal Old Colony service

This stringline shows a solution for service every 12 minutes between Quincy Center and South Station. Note that the actual trip takes 20 minutes (1200 seconds) in one direction but only 18 minutes the other way.

By fiddling with the period parameter, I was able to get five trains per hour (12-minute headways, which is to say, a 720-second repetition period) to work with the existing model based on slow diesel trains. I then took Alon Levy’s simulation of trip times with a modern EMU instead of antiquated diesel trains, and made an interesting discovery: although modern equipment cuts the travel time in half (to 10 minutes from 20), it doesn’t help with the bottleneck: a repetition period of 600 seconds (10-minute headways, 6 trains per hour) doesn’t work. But, if you could somehow double the Dorchester single-track, then look what’s possible:

Stringline showing 10 trains per hour with Dorchester double-tracking

This stringline shows what happens if you increase frequency to 10 trains per hour by twinning the single-track south of JFK/UMass station.

That (extremely expensive) intervention doubles the capacity of the line, opening the prospect for frequent service as far as Brockton and South Weymouth. At ten trains per hour, you could have service every 15 minutes to Brockton and every half hour to Kingston and Greenbush (assuming other bottlenecks along those lines were relieved — I haven’t checked that the schedule would work because I haven’t actually encoded the other Old Colony branches). This does get into another issue with the railway solver: the period parameter should really be an attribute of the train, not global, because you’d like to jointly solve multiple services with different frequencies that all share a common bottleneck without laboriously open-coding the whole pattern of repetition. (Ideally, you’d like to be able to solve the entire network as a single model!)

Anyway, for those who are interested, you can see my version of the original “RW-Solver” code on my GitHub fork, which includes some of the kluged model files I’ve been playing with.

Posted in Computing, Transportation | Tagged , , , | Comments Off on Automatic generation and validation of train schedules

It’s budget time again (already?)

Last week, the House Committee on Ways and Means reported out its version of the state budget for fiscal year 2022, H.4000, and the Mass. House of Representatives gave it second reading. It’s supposed to be taken up again a week from Monday, and of course there’s a huge pile of amendments (nearly 1,200 proposed). I’m not sure if the deadline for filing amendments has passed already — this is announced on the House floor but it’s not on the General Court’s public “budget process” web site. I dug through the budget to see what’s in it this year, and was surprised to find very little of transportation interest; I guess everyone is already worn out from the current year’s budget and the transportation bond bill, both of which passed in the waning hours of the last General Court a little more than four months ago.

I already went through this in a Twitter thread but wanted to set out the story a bit more cohesively here.

The basic transportation provisions of the budget are little changed from last year: the Massachusetts Transportation Trust Fund gets $351 million (down from $401m in the Governor’s request), the Regional Transit Authorities get $94 million (up from $90.5m in the Governor’s request), and the MBTA gets $127 million. The MBTA appropriation, called “additional contract assistance”, is in addition to the $1.2 billion in dedicated sales-tax revenue that the MBTA receives automatically, and in recent years, before COVID-19, has been used to fund capital projects and the salaries of MBTA employees working on them. Because this grant is unrestricted, it could be used for both operating and capital expenses, and has been used in the current fiscal year to help plug the hole in the operating budget caused by the 90% drop-off in commuter-rail ridership. The $127 million appropriation is unchanged from last year. The Ways & Means budget splits the RTA appropriation, with most going to formula funding but a small fraction (about 4%) going to a discretionary incentive grand program administered by MassDOT Rail and Transit.

The budget bill has the usual collection of “outside sections” (non-appropriations language), of which the only notable item is section 9, which provides for transfer of development rights in local zoning codes. The Governor’s budget included outside sections related to data collection by TNCs (Uber and Lyft) and establishing a permanent MBTA board, but these were not included in the House Ways & Means text. (Both provisions could still be reported out as a separate bill.)

With that, we go on to the amendments that have been filed. There were only 10 transit-related amendments that I caught in my scan through the titles as submitted. (Members have the opportunity to amend or withdraw their amendments, but they’re not supposed to change the subject matter, and the House leadership will select the ones they don’t like and “include” them in a “consolidated amendment” that simply omits the language, ensuring they they don’t even get a vote.)

49
Funds a first-/last-mile transportation program in Maynard
183
Funds a “Quiet Zone” study for the Franklin Line (would allow trains to not blow horns at grade crossings if study findings are implemented)
445
Funds renovation of Stoughton Depot
566
Funds part of the Town of Ayer’s Depot Square project
756
Gives priority in RTA funding to those agencies whose current operating assistance accounts for less than half of operating costs, cosponsored by a large number of representatives
757
Strikes the Ways & Means budget’s funding for discretionary RTA performance grants and redirects the appropriation towards formula funding
787
Funds a noise mitigation study for Boston Engine Terminal and the Grand Junction railroad in East Cambridge and Somerville
881
Reinstates the language from the Governor’s budget for a permanent MBTA board
951
Renames Ball Square station on the new Medford branch of the Green Line “Ball Square/South Medford”
1142
Funds a noise abatement study for the Wildcat Branch

I am pretty indifferent to most of these proposed amendments, but I have Opinions about the Governor’s proposed MBTA Board structure:

  • I think the size of the board, seven members, is good, although my preference is for the Secretary not to have a vote.
  • I think the composition of the board is problematic.
  • I think the term limits and the tying of certain members’ terms (other than the Transportation Secretary) to the Governor’s term is bad.
  • I think there should be more local representation.
  • I think there should be more expertise and a requirement for some international experience.
  • I do not think the board should be able to act by written consent in the absence of an emergency.
  • I think the members of the board should be paid for the substantial time commitment involved.

Here is my suggested alternative:

  1. The board shall consist of six voting members, serving staggered three-year terms, subject to reappointment:
    • Four members appointed by the Governor, who shall not be otherwise employed by the Commonwealth
    • One member appointed by the Mayor of Boston
    • One member appointed by the MBTA Advisory Board
  2. Of the members appointed by the Governor:
    • at least one shall be knowledgeable in public authority finance or auditing;
    • at least one shall be knowledgeable in transit operations;
    • at least one shall be knowledgeable in transportation engineering or heavy civil construction;
    • at least one shall be a native speaker of a language other than English;
    • at least one shall have lived as an adult outside the United States for at least twelve months; and
    • at least two shall be riders.
  3. The Secretary of Transportation shall be an ex officio member but shall have a vote only to break ties and to establish quorum.
  4. The board shall elect its own chair and vice-chair.
  5. When first constituted, the board shall draw lots to determine the length of each member’s initial term.
  6. The board shall have standing committees on safety, finance and audit, capital programs, and such other matters as the board determines necessary or convenient.
  7. The board may act by written consent of a quorum of members if it determines that an exigency exists requiring immediate action, and shall publish notice of such exigency expeditiously.
  8. Members of the board, other than the Secretary, shall be entitled to a stipend of $500 for each board meeting they attend.
  9. Chairs of board committees, other than the Secretary, shall be entitled to a stipend of $250 for each day on which they preside over a committee meeting, except days when the full board meets.
  10. The board shall hold a public meeting and take comment at least 18 times per year.
  11. The board shall report, at least annually, on the authority’s financial position, progress on capital programs, greenhouse gas emissions, and safety record.
Posted in Law & Society, Transportation | Tagged , | Comments Off on It’s budget time again (already?)

Weekend excursion: Stations of the Old Colony Railroad (MBTA Kingston and Middleboro Lines)

Of all the passenger routes bringing commuters from Boston’s suburbs into the city, none were hit quite so hard by the automobile age as those of the former Old Colony Railroad. The construction of the Route 3 and Route 24 freeways in the 1950s saw the bottom drop out of travel demand on the parallel railroads, and the Old Colony’s owner, the New Haven Railroad, ended passenger service in the late 50s; shortly thereafter, the Old Colony bridge over the Neponset River in Dorchester burned down. The New Haven and its successors, Penn Central and Conrail, were able to continue to service their remaining freight customers south of Quincy, running local trains north from a freight yard in Middleboro [I will be using the simplified spelling throughout this post], and back south along the lines to Plymouth and Greenbush. So the situation remained for nearly four decades. In the mean time, the Commonwealth acquired nearly all of the rail lines and abandoned rights of way in Eastern Massachusetts from the railroads, converting some to trails and preserving others as working railroads through rights agreements with various operators (including Conrail’s successor-in-interest, CSX, which retained operating rights on the line from Middleboro to Braintree).

In the 1990s, as a part of mitigation for the Central Artery/Tunnel Project, the state agreed to start up a new commuter-rail service along the three former Old Colony branches that had operated to South Station four decades previously. I’ve already discussed the Greenbush Line, which due to a controversial and unnecessary tunnel through Hingham opened some years later. The other two lines, the former Old Colony main line through Brockton and Middleboro and the former line to Plymouth, were completely reconstructed with all-new stations. The old tail end of the Plymouth line to downtown Plymouth was entirely abandoned, with a new terminal at Cordage Park in North Plymouth, and the state built over a mile of brand-new track, including a tunnel under the interchange of routes 3 and 3A, to serve an enormous park-and-ride in Kingston next to the former town dump. Both lines reopened in 1997, and with Greenbush remain to this day the only fully ADA-compliant, 100% accessible lines in the MBTA mainline rail network. Because both lines shared common design and construction and opened at the same time, and I toured both on the same weekend, I am going to treat them as a single unit. The photo galleries, however, are separate: Middleboro and Kingston/Plymouth.

One issue I should bring up first: the Red Line extension to Braintree was built in the 1970s largely on top of the Old Colony’s trunk line through Dorchester, where it paralleled the existing Red Line tracks and the Southeast Expressway. As a result, a two-mile stretch of the new Old Colony trunk, from roughly Freeport St. to Southampton St. Yard, is limited to a single track. As this single-track section is located right outside the terminal limits of South Station, it represents a serious limit on the capacity of all three Old Colony branches, so much so that various proposals to increase frequency on the Middleboro line invariably involve a forced transfer at either Quincy Center or Braintree for passengers on the other lines. (The Greenbush Line branches off between Quincy Center and Braintree, just to complicate matters, although it is a full wye junction so Greenbush trains could turn south to Braintree and run through to the Kingston line.) This is a more restrictive bottleneck than any of the other single-track sections on the revived Old Colony, since it is shared by all three branches.

One other issue I might as well address here: the T calls the terminal station on the Middleboro Line “Middleborough/Lakeville”, and sometimes calls the whole line that as well. The current terminal is actually in Lakeville, but this station is to be replaced by a new station along the main line as a part of the South Coast Rail project, as all Middleboro trains will be extended to either New Bedford or Fall River in 2024.

Let’s start with some other features common to all of the stations on the Kingston/Plymouth and Middleboro lines:

  • Stations are generally located outside the central business districts of the towns they serve; with one exception, the few Old Colony depots that still stand did not get passenger service restored.
  • As a consequence of the previous, the interstation (distance between stations) is much longer than on legacy commuter lines, which were built to serve pedestrian traffic in town centers.
  • All stations have enormous parking lots, ranging from a few hundred spaces up to the thousand-space mega-lot at Kingston.
  • Since these were completely new-build stations, they were required to fully comply with the Americans with Disabilities Act, and thus have full-length high-level platforms, and all the assigned rolling stock is equipped with remote door release. Some still require bridge plates for safe wheelchair boarding.
  • The ends of the platforms are located far enough from grade crossings that trains serving the stations do not foul the crossing.
  • While the right-of-way for both lines is at least two tracks wide throughout, both lines are predominantly single-track with modest passing sidings for timed meets. Even where a double-track section exists adjacent to station limits, the MBTA largely chose to economize on station construction by single-tracking through stations. (There are three exceptions: Halifax and Montello, which have two side platforms, and Brockton, which has a center platform.)
  • Most stations have some landscaping elements, typically raised planter beds with granite curbs for edging. Since I visited in April, there were few plantings visible other than leafless trees.

A particular concern that I noted on the Kingston/Plymouth line, and not the Middleboro line, was poor condition of the platforms, especially tactile edge treatments but also spalling concrete in other areas of the platform. The Middleboro line platforms seemed much better maintained, which is hard to account for given that they are the same age.

When I did this excursion, I started on Saturday in South Weymouth and made my way down to Plymouth, then (having an hour or so of daylight remaining) hopped over to Lakeville and headed up the Middleboro line to Bridgewater. At that point the sun was starting to get quite low in the sky, so I headed home and resumed on Sunday afternoon at Campello, the next station north of Bridgewater and one of three stations serving Brockton. (I finished off that day on the north side, to be published next week.)

Now a few notes on individual stations. We’ll start with South Weymouth, which is the station on the Kingston/Plymouth Line that’s probably the best placed to grow its residential walkshed, thanks to the redevelopment of the former Naval Air Station South Weymouth, and there are several large multifamily projects either open or under construction within a short walk of the station. This station embodies nearly all of the common design features I noted above. The new Abington station is about two miles south of the historic town center; the old North Abington depot (on Railroad Ave.) still stands and is now a restaurant — Regional Rail would allow locations like this, that are accessible to walk-up passengers, to be restored as infill stations. (I didn’t know that the old station still existed when I was doing the trip or I would have stopped there on my way.) The line is double-tracked through most of Abington, but narrows to a single-track north of Abington station. A former branch line from North Abington to Hanover has been abandoned and converted to a trail.

Whitman station has an interesting archaeological exhibit, where the Old Colony’s former roundhouse turntable was excavated, but is otherwise unremarkable. Hanson station was built next to the old South Hanson depot, which still stands (closed and unused, so far as I can tell, not even a tenant); the town of Hanson doesn’t really have much of a center and there’s nothing of significance near the rail line at any point, so I guess it’s as good a station location as any other. A passing siding begins near the Halifax town line and continues through Halifax station, the only double-track station on the entire line.

South of Halifax, the historic line continues to North Plymouth, while a newly built branch runs another mile to the thousand-space parking lot at the former Kingston town dump. Unfortunately, because some idiot signed off on not making the junction a full wye, it is impossible for a train to serve both Plymouth station and Kingston parking lot without two time-consuming train reversals, adding as much as 30 minutes to the schedule — to which the MBTA’s response was to send all peak and half of the off-peak trains only to Kingston, practically stranding the residents of the large new-build apartment complex at Cordage Park next to the station, and in response to disappointing ridership caused by the poor service, Plymouth station has now been closed, possibly for good. This makes me unbelievably angry, because clearly Plymouth is an actual origin and destination, and the Kingston transfer station/solar array/wind farm/layover yard/golf course/car dealership is not. While GATRA operates some buses that serve both Kingston station and downtown Plymouth, it’s hard to overestimate how inconvenient the station location actually is — all because they valued car storage over serving an actual place.

Moving over to the Middleboro line, I stopped first at the soon-to-be-closed Middleboro/Lakeville station (which is entirely in Lakeville): another huge parking lot — not quite as huge as Kingston’s — next to a single platform. The station sits on the Cape Main Line, the only rail route to and from Cape Cod, and I imagine must cause some operational issues for traffic to and from the Cape. (This may be why the Cape Flyer runs such a limited schedule, when there is surely enough weekday traffic in the summer to operate seven days.) The Old Colony Main Line, Cape Main, and Middleboro Secondary have a full wye junction at Pilgrim Junction in Middleboro; the original Middleboro station was located north of the wye on the Main Line (it no longer stands) and that is where the new station will be as well. The infield of the wye is also where the overnight layover yard for this line is located, although that too will become redundant when South Coast Rail opens, as both New Bedford and Fall River will have their own layover facilities.

North of Middleboro is Bridgewater station, and I’m not really sure what to say about it. The station is located on the campus of Bridgewater State University, and isn’t especially convenient to get to if you aren’t already at the school. I’m sure it’s a great boon for students and staff alike, although the 2018 passenger counts show the reverse-commuting ridership to be negligible, and most passengers are riding into the city during peak periods. (It’s possible that there is less park-and-ride mode share on weekends, when residential students would be more free to travel; the data don’t exist to show one way or the other.) Brockton Area Transit Authority (BAT) operates public shuttles in Bridgewater for the university.

I returned the following day to start out at the southernmost of three stations in Brockton, Campello — named after a former “village” and postal district in the city, which itself was formerly known as North Bridgewater. There’s not really anything interesting to say about it, except perhaps that it is located next to one of three recycling companies in Brockton that CSX still serves with freight trains over the line.

A mile or so north of Campello is the beginning of the four-track Brockton Viaduct, although it has long been limited to just two tracks. The old downtown Brockton depot was located on the viaduct, but was demolished in a fit of “urban renewal” after passenger service ended in the 1950s and was replaced by a rather unattractive central police station. BAT’s central bus hub, the “BAT Centre”, is located across commercial street from the old depot, and was designed to mimic the materials and style of historic Massachusetts railroad depots (some others by the same designer as the former Brockton station still stand to be used as references). Somewhat later, BAT built a parking garage next to the bus station, and this accounts for most of the parking for the train station as well. (There are accessible spaces next to the police station.) Because the viaduct was built for a four-track main line, there was plenty of room for a 20-foot-wide high-level center platform, although the police station still looms over riders. Access to the platform is only from the Commercial St. (east) side of the tracks; a single ungated pedestrian grade crossing over the northbound track is the only egress from the platform.

The viaduct ends north of the Ashland St. overpass, and the two-track line continues north at grade (with overpasses for the cross streets) to Montello, another former village and postal district. Montello station is a two-track side-platform station with parking lots on both sides and a single pedestrian grade crossing at the north end of the platform. It’s also the only station in Brockton served by MBTA buses in addition to BAT; the #230 Quincy Center–Montello bus, which primarily serves Randolph and Holbrook, lays over here. Two BAT routes serve Montello station, but stop outside the Spark St. entrance on the east side of the station rather than entering.

The final station (at least until we get back to Braintree) is Randolph/Holbrook, which was literally built on the border of those towns. The rail line at this point is entirely in Holbrook, as is the platform and the larger of the two parking lots, but Randolph has its “own” parking lot. There is a pedestrian grade crossing at the south end of the single platform, and I had the opportunity to watch a southbound train pass while I was there. The double-track from Braintree Junction ends literally at the north limits of the station; there is clearly enough land for a second platform and a second track, so the T must have decided to economize by not building a two-platform station.

Posted in Transportation | Tagged , , , , , | Comments Off on Weekend excursion: Stations of the Old Colony Railroad (MBTA Kingston and Middleboro Lines)

Weekend excursion: Stations of the B&M Western Route/MBTA Haverhill Line

It’s been nearly two weeks since I took this trip on a windy Saturday in March, and since that time I’ve seen nearly all of the remaining stations in the MBTA system, so if I’m a little hazy on the details, please forgive me. Meanwhile, enjoy the companion photos by opening the link in another browser tab.

As I was describing in my survey of the parallel Eastern Route last week, the Boston & Maine Railroad’s original route from Boston to Maine went via the industrial cities of the Merrimack Valley — specifically, Lawrence and Haverhill — before heading north into Exeter, Durham, and ultimately Dover, New Hampshire, before crossing the Piscataqua River into Maine. When the B&M acquired the Eastern, the original main line became known as the Western Route, but there was enough local traffic to keep both lines in service. Over time, traffic dwindled, and the B&M had a substantial retrenchment. In the 1950s, most passenger service to the Merrimack Valley and points north was rerouted via the B&M’s New Hampshire Main Line and the “Wildcat” branch through Wilmington. Transportation officials in Boston were looking at the likely end of commuter rail service, as more freeways were built to siphon traffic away from the railroads, and the Western Route was one of their prime targets, since it had a three-track right-of-way through the inner suburbs, making it an attractive alignment for a surface relocation of the old Charlestown Elevated. Plans were drawn up in the 1960s to extend the newly renamed Orange Line through a new tunnel under the Charles all the way up to Reading, with an express track to speed service through Malden, Medford, and Charlestown for riders from more distant suburbs.

When service on the “Haymarket–North” extension opened in the mid-1970s, there was insufficient support to extend the route all the way to Reading, and so the commuter-rail service has remained, largely unchanged, ever since. Even before the Orange Line came, decades of disinvestment had seen the second track lifted along much of the route, resulting in severe limitations on frequency and quality of service to this day. Two tracks remain from Melrose to Wakefield and from just beyond Lawrence to Haverhill, but this only suffices to operate service every 45 minutes — and the northern end must still be shared with slow freight and the Amtrak Downeaster. The Downeaster, like the old B&M intercity service it replicates, runs via the Wildcat Branch, as do a couple of Haverhill commuter trains a day. In the new Spring, 2021, schedule, 23 inbound trains per day serve Reading every 45 minutes, with 11 of those originating in Haverhill, so passengers on the outer end of the line get 90-minute headways. Two inbound trips and one outbound trip run express via the Wildcat Branch, saving about 11 minutes. This does mean that North Wilmington only gets one inbound peak-period trip, but that’s actually not much of a loss — in the pre-COVID schedule, North Wilmington didn’t get any peak-period service.

The new schedules raise an interesting question for many of the municipalities along the line: the parking regulations for many town-owned station lots currently only require payment in the morning. With the T now operating trains every 45 minutes all day long, on the theory that this will attract “non-traditional” passengers, will there be pressure for the towns to extend enforcement later in the day — or, in the alternative, will the lots go back to filling up at 8 AM, restricting use of the off-peak trains to only walk-up riders?

As on previous trips, I did not photograph any of the commuter-rail stations that are also rapid transit stations, so as to avoid other people to the extent possible, so let’s get started with Wyoming Hill, the first station north of the Orange Line terminus at Oak Grove. Like many of the stations on this line, there is municipally-owned parking, in this case a small lot just west of the station. It’s located in a reasonably dense neighborhood, with a mix of retail, dining, apartments, and small-lot single-family residences, and only two short blocks from Melrose’s Main St. It’s also only half a mile from the next station north, Melrose/Cedar Park — much closer than would normally be considered appropriate, but not uncommon for an old commuter line. That said, Wyoming Hill gets about 20% more ridership (or did, in 2018), owing to its somewhat denser surrounding development. The line is still double-tracked here, but both tracks have only inaccessible low-level platforms. (The single-track merge point is just north of Oak Grove station, a mile south; you can just barely see the signal controlling the interlocking from the south end of the platform.) It would be a relatively easy station to make accessible, after removing both the standard MBTA platform canopies and the town-owned enclosed shelter at the north end of the inbound platform, but from the ridership it’s not hard to see why this hasn’t been prioritized.

Melrose/Cedar Park has a bit more parking, but in a less dense neighborhood, and the daily ridership roughly tracks with the parking. Like Wyoming Hill, it’s a two-track station with only inaccessible low-level platforms, and an even lower priority for platform improvements. Melrose Highlands is by far the most popular station in Melrose, with significantly more multifamily and commercial real estate nearby. Melrose Highlands does have mini-high platforms, and would seem to be a good candidate for full high-level platforms (and construction would be relatively simple given the lack of pedestrian grade crossings).

Moving north into Wakefield is the first odd duck on a line full of odd ducks. Greenwood station would appear to serve a bank branch and a small retail district on one side, and a small single-family subdivision on the other, but its 80 passengers a day are mostly using on-street parking which the town has reserved for this purpose north of the business district. Unsurprisingly for a station with such minimal ridership, Greenwood has offset low-level platforms, a pedestrian grade crossing, and no concessions to accessibility. (There is, at least, a bus transfer, although given the cost of commuter rail tickets, many bus riders will take the more affordable transfer to the Orange Line at Malden Center.)

The other station serving Wakefield is a real contrast, although it too is inaccessible. Located west of Wakefield’s main business district, Wakefield station has restaurants on the platform, a classic B&M station building, and is surrounded by low-rise mixed-use development and a church. A nearby industrial zone, once served by the rail line, is being transformed into a higher-density residential neighborhood. In 2018, Wakefield was the second-busiest station on the line, after only Reading, so it’s a real disappointment that it isn’t accessible — it clearly should be, and the outbound platform would be easy to fix. Unfortunately, the buildings abutting the inbound platform are too close, and would have to be relocated or demolished to meet current standards, which is presumably why the MBTA has chosen not to do anything about the problem.

Next stop north from Wakefield is the first “WTF?” moment of this tour: Reading, the busiest station on the entire line. Reading is a two-platform, single-track station! And not in the obvious way, with platforms on both sides of the single track — no, at Reading, the second platform serves the nonexistent second track that was lifted some time before 1969. Yet the MBTA continues to maintain it! It’s obvious that restoring the second track would be key to making the station fully accessible; the current platform (serving what should be the inbound track) is split in half by the old station building (now leased out to a cafe and a mortgage broker), which sits too close to the track to allow for sufficient clearance. There is presently a mini-high on the existing platform, so despite the high passenger volume, improving this station is not a high priority for the T. (With the new schedule turning half of all trains at Reading, dwell times are less of an issue, since the train has to reverse anyway, but restoring the double-track and constructing a full-length high-level platform on the restored eastern rail would still be an operational improvement.) Like Wakefield, Reading station is on the west side of the town’s CBD.

The “WTF?” moments just keep on coming as we try to find the next station, North Wilmington. A limited-service station, North Wilmington sees one peak-period trip each way, but early morning, midday, and late evening trains all stop here, if only we can find where “here” is. After several tries, I eventually found the small, unmarked, town-owned parking lot, separated from the “station” by a vacant lot. What passes for a station here is single-track, of course, and inaccessible, of course — there’s barely even a platform, and the only amenity is a bus shelter. (OK, so that’s still slightly more than Plimptonville ever had.) The prudent thing to do with this station would be to close it, and route all trains on the outer parts of the line via the Wildcat Branch, allowing for more frequent service on all three resulting branches.

Heading north, the double-track resumes with the junction of the Wildcat Branch, only to end across the street from Ballardvale, the next station. In keeping with the B&M’s plan of running all Lawrence service via the New Hampshire Main Line, the Wildcat Branch is the through route at Ballardvale, and it’s the Western Route that ends ignominiously in a bumper block just south of the grade crossing. Ballardvale is the third “WTF?” station on this line, and the oddest of the bunch: the original platform was clearly built to serve the now-disconnected northbound Western Route track, and when that track was lifted, they simply dumped several yards of hot-mix in the trackbed to extend the platform. (Why oh why did they not keep the track that already had a platform?) But it gets worse: 300 feet beyond the end of the new platform, there is a mini-high — yes, this is an “accessible” station — and the mini-high was obviously built to serve the missing Western Route track. That means that it was built that way by the MBTA some time after 1990! Since the track that actually exists is 10½ feet away from the mini-high, the T added a wooden high-level platform extension to the mini-high. But it’s still 300 feet north of the rest of the platform! So either the train serves the regular platform, inches forward 300 feet in a cloud of diesel smoke, and serves the mini-high, or else it just serves the mini-high and half the cars in the train don’t platform at all. Who signed off on this design? Ballardvale is located in a fairly low-density neighborhood, with largely single-family housing and conservation land, although it was once important enough to have its own post office; about 200 people a day used the station in 2018, many of them using the 120 paid parking spaces at the station.

The double-track resumes yet again just north of Ballardvale, but the second track is inaccessible from the south for another mile. It hardly matters for passenger service, though: the second track is freight-only, and cannot platform at either Andover, the next station north, or Lawrence.

Meanwhile, what about Andover? It’s a busy station, with nearly 400 passengers a day in 2018, far more than its 150-space parking lot alone could support. There’s a good amount of development in the vicinity, both multifamily and commercial, and Andover’s downtown business district is a short walk away to the southeast. The station is also served by multiple MVRTA bus routes. As noted above, the second track does not platform at Andover, and the platform for the first track is low-level, with a mini-high at the north end. Upgrading this platform would be a challenge due to commercial abutters.

Our next stop is Lawrence. I did not go to see the old Lawrence station, which was a low-level center-platform station just east of Parker St., and which has apparently been abandoned in place. The new, fully accessible Lawrence station is a single side platform connected to MVRTA’s McGovern Transportation Center, a 400-stall parking garage on Merrimack St. (The station is located across Merrimack St. from the Wood Worsted Mill, once the world’s largest, which has been redeveloped into a variety of residential and commercial uses under the name “Riverwalk”.) An empty trackway separates the passenger track from the freight tracks, providing room for either another through track or an island platform, should future passenger schedules warrant it. (A short, temporary, low-level platform remains on the opposite side of the railbed, attached to an old industrial building; it’s accessed via a pedestrian grade crossing at the east end of the high-level platform but not accessible and not currently served by any passenger trains unless there’s a switch malfunction.) A short distance east of the platform, the passenger siding merges with the main line once again, and has a crossover to access the outbound track, so the final two stations operate as normal two-track stations.

Bradford station doesn’t have any obvious reason to exist — it’s not even half a mile from downtown Haverhill and the end of the line, and there’s not much nearby in the way of either origins or destinations. But there was a station here already (a historic B&M station building, a clone of the one at Swampscott, still abuts the line, although it’s now a cafe), and there was plenty of land available, so it made more sense as an end-of-line layover facility to minimize deadheading — and of course there’s a huge parking lot. MVRTA’s Bradford bus garage is located across the street from the layover facility. The station itself is a pair of disappointing low-level platforms with mini-highs (why could they not build a high-level center platform here, there was plenty of room?) and pedestrians cross both the main line and the yard leads at grade. A walkway north of the outbound mini-high connects to a nearby residential neighborhood off Laurel Ave. on the east side of the tracks (the west side is the Merrimack River). The ocean of parking here is only about half-used: there are more than 300 spaces for a station that saw 170 passengers per day in 2018. Just beyond the station to the north, the remnants of a branch line diverge to the east, running through Groveland and Newbury to connect with the Eastern Route at Newburyport; this line, abandoned decades ago, has largely been rail-trailed and is also an electrical transmission path for much of its length.

The final station on our tour is also the terminus for MBTA service on the Western Route, although Amtrak Downeaster service continues through New Hampshire to Maine, and Pan Am freight service continues through Maine to interchange with various Canadian railways. The city of Haverhill extends all the way to the New Hampshire state line, but there’s no obviously useful place for a third Haverhill station, and New Hampshire refuses to subsidize rail service, so there’s nothing north of downtown Haverhill that the MBTA might serve. (Amtrak’s next stop is Exeter.) MVRTA recently built a 315-stall garage and pedestrian overpass to supplement the MBTA’s 150-space surface parking lot, but the station as a whole is a bit of a disappointment, and you’d think with all the residential development nearby in downtown Haverhill this station could manage more than a paltry 290 boardings. Of course, it has low-level platforms with mini-highs, and multiple pedestrian grade crossings make it more challenging to upgrade, especially since the overpass to the garage connects to the middle of the outbound platform. (MVRTA charges $4 a day, whereas the MBTA’s surface lot is only $2 a day, so you’d have to imagine the RTA is losing a lot of money on its brand-new garage right now, with pandemic-suppressed ridership.) On the current schedule, the whole trip from Haverhill to Boston is scheduled to take 55 minutes for the once-daily Wildcat Branch express, or 66 minutes on the local train via North Wilmington, which is likely still faster than driving during rush hour — but a fully modernized electric service could make the Wildcat route in just 38 minutes, a substantial savings.

Three more lines to go: Plymouth/Kingston, Middleborough, and Lowell!

Posted in Transportation | Tagged , , , | Comments Off on Weekend excursion: Stations of the B&M Western Route/MBTA Haverhill Line

Weekend excursion: Stations of the Eastern Route/MBTA Newburyport Line

Attention conservation notice: another 3,000 words about an MBTA commuter rail line and its stations, which could easily have been 5,000 words if I hadn’t gotten tired of writing and skimped on the exposition towards the end. Already more photos to edit and text to write from subsequent travels.

Once upon a time, there was a railroad called the Eastern Railroad. Its main line ran from Boston to Salem, where it intersected numerous other railroads, and was not long thereafter extended across the Danvers River to Beverly, Newburyport, and ultimately Portsmouth, N.H. Another branch connected the main line to the fishing ports of Gloucester and Rockport on Cape Ann.

Then the Eastern came under control of the Boston & Maine. The B&M already had a route to New Hampshire and ultimately Portland, Maine, by way of Durham and Dover, N.H., and Sanford, Maine. For many years, there was enough traffic to support two parallel mainline railroads, and the B&M’s original route via Dover became known, quite logically, as the Western Route, while the Eastern Railroad’s old mainline remained the Eastern Route. The Eastern served important industrial and port cities like Chelsea, Lynn, Salem, Beverly, and Newburyport, while the Western served two of the mill cities of the Merrimack Valley, Lawrence and Haverhill, before heading up into New Hampshire a few miles inland of the Eastern. The Western proved to have more staying power as a freight route; when passenger service on the Eastern was discontinued north of Newburyport, there were not enough freight customers to keep the Eastern Route in service, and the line north of the Merrimack River was abandoned. Parts of it are now a rail-trail, and elsewhere the land was simply sold for development.

Two Sundays ago, I went to have a look at the stations of the Eastern Route. I didn’t make it all the way to Newburyport — the daylight ran out when I was in Ipswich — but I did go to the end of the Rockport Branch. The photographs I took can be viewed on my SmugMug gallery, and you might want to middle-click that link to open another tab so it’s easier to switch back and forth between my narrative and the photos.

A few overall comments on the line. This line is one of three identified by the MBTA board in its December, 2019, resolution on rail transformation for priority to receive frequent, all-day, electrified service. (TransitMatters recently released a report on this, which I have deliberately avoided reading before writing this piece in order to form my own impressions.) The board resolution identified an “Environmental Justice Corridor” from Boston to Lynn, but because of the physical constraints of the line, it would almost certainly extend at least to Beverly Depot, where there is room to turn trains without fouling the main line (and in fact some trains already turn there). This section of the whole line is relatively up-to-standard, with only two partially-accessible stations — Swampscott and Beverly Depot, both of which have historic structures on the platform that make full accessibility an engineering challenge — not counting Chelsea which is already being replaced. Beyond Salem, all of the stations I visited have only partial accessibility, if any. (I expect Rowley and Newburyport, which I did not see, to have been built recently enough to be fully accessible, but wasn’t able to verify this in person — when I manage to get out there, I will update the photo gallery with pictures of those stations.)

Overall, the condition of the stations reinforces my impression that the MBTA is pursuing a policy of deliberate neglect of commuter-rail stations, allowing them to deteriorate until they are closed for emergency repairs, in preference to actual ongoing maintenance which might trigger an obligation to make the stations fully accessible under ADA and state regulations. Nearly every station I visited had spalling concrete, rusting steel, splintered timbers, and buckled asphalt. Some of them were actually in quite disturbing condition, even if the structural engineers aver that they are still sound.

Three of the stations have huge downtown parking garages, and it’s no surprise that two of them (Salem and Beverly Depot) are the two busiest on the entire line according to the 2018 manual ridership counts. The third huge parking garage, at Lynn Central Square, is reported to be significantly underutilized, and the ridership counts would seem to bear that out, with less than a quarter of Beverly’s and a fifth of Salem’s — this is largely due to the fact that Lynn has frequent, affordable bus service, whereas the train is infrequent, extremely expensive ($7.00 vs. $2.40 for the bus with a transfer to the subway) and in pre-COVID times arrived in Lynn packed to the gills, with no room for additional passengers to board. The MBTA has experimented with discounted fares for Lynn and Chelsea in the past, but nothing short of full fare equalization with comparable bus-subway journeys is likely to fix this.

Now on to the tour. We start in Chelsea, at Mystic Mall (known better to many as “the Chelsea Market Basket”). There isn’t a train station here, but there will be soon: the MBTA is relocating the existing Chelsea station at 6th & Arlington, which was already partially demolished to make way for the SL3, a quarter-mile to the west, where the current terminus of the SL3 is already located. The new station will of course be fully accessible, but riders who were previously within walking distance of Bellingham Square will now have to transfer to the bus. On the other hand, it makes the retail, service, and hospitality businesses clustered around Everett Ave. significantly more accessible by train. Overall, the station move is probably a wash; either location favors some people and disfavors others. The station relocation project was bundled with a bunch of other grade-crossing and signal improvements that are supposed to reduce the impact of the train on traffic in Chelsea.

Our next stop is that existing Chelsea station, which is right next to the SL3’s “Bellingham Square” stop, although it’s still about a quarter-mile walk from the station to the actual Bellingham Square. There’s not a lot of employment in this area, but there is a good amount of housing, including public housing; the area is cut off by the viaduct of the Northeast Expressway (US 1) from the auto-oriented business district along Everett Ave. Chelsea station originally had a platform for both tracks, but when the SL3 came through, on the former right-of-way of the Grand Junction Railroad which ran parallel to the Eastern here, the outbound platform was demolished and a lot of hot-mix was dumped between the tracks as a replacement. (This was legal because the old station was inaccessible already and the new station was in the capital program.) In the 2018 counts, an average of just over 200 passengers a day used Chelsea station (as contrasted with 11,000 passengers a day on the 111 bus).

There’s a long gap between Chelsea and River Works, the next station; the state legislature included funding in the Transportation Bond Bill to construct a new station in Revere, at the former Wonderland dog track (after which the terminus of the Blue Line is also named). River Works is a private station for employees of GE Aircraft Engines in Lynn; there is no public access so I didn’t go see it. A developer has made a deal with GE to redevelop part of the site which includes an easement for public access to the station; I don’t expect to see the station before the new development is built (and probably built a new station as well).

That brings us to the aforementioned Lynn Central Square. Lynn, as they say, used to be an important place, but it has fallen on hard times, and many of its residents cannot afford $7 each way to take the train into Boston — $9.40 if they need to take the subway to their actual destination, because these are people who also cannot afford the $232 monthly pass that would include subway rides. What the powers that be thought they were accomplishing by erecting a thousand-stall parking garage in the middle of a working-class city like Lynn is beyond me. But they did, and then of course they failed to maintain it properly, and it shows.

Lynn station itself is interesting: it’s built on an old viaduct, remnant of an abandoned plan by the Eastern to widen its mainline to four tracks and eliminate grade crossings for faster service, which was largely obviated by the purchase of the Eastern by the B&M. Only the section in Lynn was ever built, and the line’s capacity through Salem remains limited to this day by the single-track tunnel, so it matters little that the line has been reduced to just two tracks through Lynn now. The extra-wide viaduct did make it possible for the station to be upgraded to a full-high center-platform configuration, although the inbound and outbound sides are oddly offset for reasons which are unclear to me. At least that means there are no significant freight clearance problems that would prevent building full platforms everywhere else on the line: freight trains must access the line from the south, since the route to Portsmouth has been abandoned.

The two-track viaduct continues north from Lynn into Swampscott, where there’s a cute little station building in a classic B&M style. I only saw the inbound side, where there is a large parking lot; the station building is on the outbound side, along with another lot. The two lots together hold only 127 stalls, which is far fewer than the reported 825 passengers a day this station served in 2018, so the majority of those passengers must be walk-ups or walking transfers from the buses on nearby Essex St. I was at the station while an outbound train to Newburyport made a stop, and even with limited service on a Sunday afternoon there were some passengers both boarding and alighting.

It’s another long interstation between Swampscott and Salem. An infill station at South Salem has been proposed, which would serve more people and destinations, such as Salem State University, but would also help to buffer traffic through the single-track Salem downtown tunnel, one of the oldest cut-and-cover railroad tunnels in the country. Before the the old Danvers River drawbridge burned down, the previous Salem station was located in the open cut south of the tunnel; after the fire, a temporary station was constructed north of the tunnel portal, and the previous station was abandoned as being too difficult to make accessible. (Remnants of the old single-platform station can be seen in aerial photos, including an abandoned stairway down from Mill St., but I did not make any attempt to go there.)

The Salem tunnel is unusual for another reason, which is that part of a wye was located inside it. The Danvers Branch diverged from the main line inside the tunnel, and it was on this wye lead that the temporary station (now the city-owned “Crescent” surface parking lot) was located. These tracks were lifted after construction of the new station and replacement drawbridge, so the Danvers Branch today is accessible only from Beverly, if it has any remaining traffic at all. Both portals remain today, and the abandoned branch is used to store sand and salt for treating icy surfaces.

The modern Salem station has a 728-stall parking garage, and is by far the busiest station on the line, nearly twice as busy as Beverly Depot, the next-busiest. There is a full-length high-level platform immediately north of the tunnel portal, an indoor waiting room, and a secure bike storage cage. The garage and full-high platform were built in the 2010s, opening in 2014, but the platform was relocated to the current location as soon as the drawbridge was replaced and service north to Ipswich and Rockport resumed. Because the station is so new, everything about it is in excellent condition, which is just about the only station I saw on the entire line about which one could say that.

On the other side of the Danvers River is the city of Beverly. Other than Boston, no city or town in Massachusetts has more commuter-tail stations than Beverly, even after one of them has had its service suspended, leaving four active stations: Beverly Depot, North Beverly, Montserrat, and Beverly Farms. (Needham also has four; Newton and Brockton have three each, as I believe does Melrose.) The latter two stations are located on the Rockport Branch, and are currently bustituted while the Annisquam River drawbridge in Gloucester is replaced; rail service is planned to return by the end of July. In the 2018 counts, Beverly Depot served more passengers than all other Beverly stations combined, by a factor of two, despite its 500-space off-site parking garage.

Downtown Beverly is pretty impressive for an old industrial city (the old local industry was shoe manufacturing, as well as shoe-factory equipment); Beverly Depot is located on the western edge of the downtown, but it’s easily walkable from a substantial old mixed commercial-residential neighborhood, and CATA, the local RTA, operates a low-fare circulator bus that serves the station (although not every train). The station itself has been converted into a restaurant but it maintains its historic features including a large wooden-roofed canopy on the inbound platform. A private lot on the inbound side augments the MBTA-owned garage and the city’s short-term meter parking. North of the station, there’s a universal crossover and a layover siding, allowing trains to short-turn, although the new two-trains-per-hour schedule only calls for two short-turn trains a day. I saw a southbound train serve the station while I was on the platform.

I followed the Rockport Branch from downtown Beverly all the way out to the end. During the Gloucester drawbridge replacement project, the MBTA had originally planned to maintain single-track service to Cape Ann, but after construction began, it was determined that the existing bridge was unsafe. As it was the early days of the COVID-19 pandemic and passenger volumes were very light, the T decided to accelerate the demolition of the drawbridge rather than making temporary repairs to a structure that was in the process of being replaced. The change in schedule also meant that the Positive Train Control installation east of the bridge did not need to be completed before the federally imposed December, 2020, deadline (since the bridge was out of service and the rest of the line was thus disconnected from the rail network). In addition, numerous other maintenance-of-way activities, including tie and ballast replacement, were rescheduled to take advantage of the full shutdown of rail service. During weekdays, service remains active as far east as West Gloucester — the easternmost stop west of the drawbridge — but on weekends, the entire branch is bustituted, so all of the stations I visited were deserted of both passengers and trains (although there were walkers and bikers).

Of particular note along the Rockport Branch is Prides Crossing station, to which service is now suspended. While the 140-year-old station building now houses a confectioner’s shop, there are only three commuter parking spaces, and only a handful of local residents used the station, which was a flag stop on a small number of weekday peak trains. The station never got enough use to even have a proper inbound platform; instead, it’s got the MBTA-standard “dump some asphalt between the tracks” walkway for the inbound passengers to board, assuming the engineer stops the train in exactly the right place. Beverly Farms station is less than a mile away, and has nearly an order of magnitude more daily passengers, so hopefully this service suspension really is permanent.

I don’t have a lot to say about the other stations along the branch; they are all quite similar, with low-level platforms and mini-highs. West Gloucester is weird because the inbound platform is much shorter than the outbound, for no obvious reason; it’s currently being used as a laydown area for the drawbridge construction as well as the first stop for weekday inbound trains for the duration of the bustitution. (Manchester is the last stop for weekday outbound trains; I’m not sure why outbound trains are unable to serve West Gloucester.) The line is single-tracked from the Annisquam River all the way to Rockport; although the drawbridge was double-tracked and will be replaced with two separate single-track bascules, the tracks merge just on the east side of the draw in any case, so I’m not sure what is gained.

Returning to the main line, there are five remaining stations: North Beverly, Hamilton/Wenham, Ipswich, Rowley, and Newburyport. The Merrimack River bridge north of Newburyport is no longer in service, and the line beyond it has been rail-trailed, so Newburyport is the end of the line for the foreseeable future.

North Beverly is our first stop, and it’s an odd location, wedged in behind a Route 1A strip mall. It has the usual low-level platforms with pedestrian grade crossings, and the mini-highs are in poor condition. There seems to be no topographical barrier to building full-length high platforms here, other than the need for crossing the tracks, which could easily be accommodated by adding a grade crossing and ramps just north of the platforms, which could connect to the back of the Stop & Shop supermarket north of the strip mall — there is already a vehicular grade crossing, Dodge St., at the south limits of the station. North Beverly is about 2½ miles north of Beverly Depot, which is an entirely reasonable interstation for the population density.

Hamilton/Wenham continues the theme of low-level platforms with mini-highs, although there is only one platform since the line is single-tracked north of North Beverly. The towns of Hamilton and Wenham share quite a few things, including a library and a school district; the station is also split between the two towns, being built right on the town line.

Ipswich was for much of the MBTA’s existence the northern terminus of passenger service on the line, and there are likely still outdated system maps showing it as the “Ipswich/Rockport Line” floating around. Newburyport and Rowley did have service under the B&M, and at times under the MBTA, but disputes over subsidies and service levels made the operation an on-again-off-again proposition. Because Ipswich was the terminus for so many years, there is a small layover yard just south of Ipswich station, where trains were formerly idled overnight waiting for the next morning’s service to resume. As at Hamilton/Wenham, the station itself is a single low-level platform and a large surface parking lot on the west side of the single track, with a mini-high platform at the north end.

Posted in Transportation | Tagged , , , | Comments Off on Weekend excursion: Stations of the Eastern Route/MBTA Newburyport Line

Weekend excursion: Stations of the Franklin Line

It’s getting on nine days since I actually did most of this travel, and in the mean time I’ve also seen stations on three other lines (although one of them has historic connections to this one), so I’ll at least try to be a bit brief. (So, um, only 3,000 words.) As always, you can follow along with my photo gallery on SmugMug.

The Franklin Line is a disappointingly underutilized transportation facility. Originally constructed as a mainline railroad, one of several competing for business between Boston and New York, like the others it came under the control of the New York, New Haven and Hartford system. For the New Haven, the line became the Midland Route, providing an alternative to the heavily-used Shore Line (today’s Northeast Corridor) into Connecticut. Nearly all of today’s Franklin Line was originally double- or even triple-tracked, but in the years of railroad decline, culminating in the New Haven’s insolvency and inclusion in the government rescue program that created Conrail, the second track was lifted and overpasses were removed to reduce maintenance costs, resulting in a largely single-tracked line with some passing sidings. In its current configuration, the Franklin Line is unable to support frequent service. In the past few years, the MBTA and its rail operations contractor Keolis began a program to restore the double track to large segments of the line, a few miles at a time, with the stated goal of improving headways to as short as 35 minutes, but this project was put on hold during 2020.

It’s a huge disappointment because over the past decade and a half, more than a thousand housing units have been added within walking distance of the Franklin Line, and likewise tens of thousands of square feet of retail and office space. A more frequent service, with improved, more accessible stations and faster trip times, would be a real boon to the people who chose to move to new apartment buildings in Franklin, Walpole, Norwood, and Dedham. (Even Norfolk is getting into the act, albeit with a small detached-single-family condo development rather than a 200-unit apartment complex.) The MBTA and Keolis should be commended for the new April, 2021, schedule on the line, which will provide 15 hourly trains from South Station to at least Walpole, with most continuing on to Franklin, as well as additional peak-period service. The T really should resume the double-track program and take whatever additional measures are necessary to crank the headways down to 30 minutes rather than 35 — some of which probably involves improving stations to eliminate bottlenecks at inadequate and inaccessible platforms. Closing Plimptonville station, as has been done on a temporary basis, is a step in that direction, but needs to be made permanent, however much I’d hate to strand all 12 people who used the one round-trip a day that stopped there, and is probably a requirement for finishing the restoration of the double track there. In a Regional Rail future where the whole trip from Boston to Forge Park takes only 42 minutes rather than 67 minutes as currently timetabled, it’s conceivable that the long-talked-about restoration of passenger service to Milford might finally make sense.

I started my trip in Franklin at Forge Park/495 station, the current outer end of the line. Unlike most recent MBTA extensions, there’s no overnight layover facility here; constrained layover capacity has apparently been an issue for the MBTA’s Railroad Operations department for many years, going back at least as far as the original extension of the line (which was paid for by National Development, the developer of the nearby industrial park, which also built Dedham Corporate Center/128 station). Like all newer commuter-rail stations, it’s an ocean of parking surrounding a small station building with full-length platforms, although it’s old enough to have those be low-level platforms on both sides of the track, with a single mini-high on the same side as the station house. (The two huge parking lots are on opposite sides of the track, and several pedestrian grade crossings permit crossing the track in case a passenger should forget in which ocean of parking their SUV happens to be sitting.) Just west of the station, the former West Central St. vehicular overpass has been rebuilt as a pedestrian bridge; automatic signal territory ends immediately west of the overpass.

Franklin station, sometimes called “Franklin/Dean” after the local four-year college, is a real frustration. It’s nestled into Franklin’s cute little downtown, within walking distance of Dean College, shops and services, and a new 200-unit apartment complex — but it’s wholly inaccessible. The 2018 manual passenger counts, which were conducted before the apartment complex opened, showed a daily traffic of 630 passengers each way (including a handful who went the short distance between Forge Park and downtown Franklin). Yet, this station was single-tracked in recent memory, as the platform makes painfully obvious: it was clearly extended by removing the outbound track and dumping several yards of hot-mix onto the trackway where that second track used to be. This does mean that it would be ludicrously easy to make the station fully accessible, because the current platform is well away from the station building, and so all the more frustrating that the MBTA has chosen not to do so, given the fact that it serves a significant ridership and an educational institution. Immediately south of the station, the line divides: the old freight mainline was not acquired by the state when purchasing the line from CSX due to environmental cleanup liabilities the state was unwilling to assume after performing due diligence, which held up the sale for a decade, and a single-track branch line to Milford serves the current commuter-rail service. (The remaining freight customers on the branch are served by the Grafton & Upton out of Westborough; the mainline is abandoned about two miles south of the station and has become the Southern New England Trunkline Trail.)

Just north of Franklin station is “FRANK” interlocking and the MBTA’s very limited two-track layover facility, built on a formerly three-track section of the right of way; phase 2 of the double-track project involves extending the double-track from the north end of the layover facility to Norfolk, about 3½ miles. (The theory behind this is that, once the trains have left the layover for the day, the layover tracks can be used for through-running trains.) This is the phase of the project that began in early 2020 and was paused for the pandemic; it’s unknown when work will resume. (The third phase of the project, which was in design when work was halted, will restore the second track between Norwood Central and Walpole, likely with a few gaps, as I’ll note below.)

At Norfolk, we can actually see lots of evidence of the double-track project having gotten started with grubbing and grading on the right-of-way, although no track placement. It’s not clear whether the plans included construction of a proper second platform at the station, although the trackbed is certainly wide enough to support it. The parking situation at Norfolk is a little odd, with most of the parking a very long walk south of the station and up a long set of stairs — I didn’t see the main lot during my trip and was only able to figure out what the long walkway was for by examining the aerial photographs closely. (And at least for now don’t expect to learn much about the double-track project from the aerials, nor likewise about the various recently built apartment buildings — the public aerials are a couple of years out of date.) At the north end of the station is a grade crossing over Rockwood Road, and the completed double-track from phase 1 of the project runs from just north of there to the “WALPOLE WEST” interlocking, at the south end of the former passing siding south of Walpole station.

Walpole itself is a bit of a mess — a historic mess, but a mess nonetheless. The station was built at the diamond crossing of what are now the Franklin Line and the CSX Framingham Secondary, and CSX has a small marshalling yard on the Framingham side of the station to switch trains for industrial customers on the Framingham Secondary and on the Northeast Corridor, some of which are served via the northern Franklin Line from Readville due to access restrictions on the NEC. While the crossing was built as a full diamond, at least one of the wye legs has been lifted, and in a 2010 report, it was reported that the MBTA was operating game-days-only Foxboro special-event service using a backup move because the switch from the southbound Franklin line to Foxboro and Mansfield was not reliable. (This must have been fixed in preparation for the pilot all-day service to Foxboro, because when that service was introduced it was called out specifically as not serving Walpole station.) The station was built on the diamond crossing proper, to serve passengers on both lines, which came under the same Old Colony umbrella in the late 19th century, but passenger service on what is now the Framingham Secondary ended before World War II and is no longer a particular concern. For operational simplicity, Alon Levy suggests moving the station northeast, past Elm Street and closer to downtown Walpole, which would allow for construction of full-length platforms and would not be a significantly more difficult walk for most passengers; this short section of the line is already double-tracked.

Next stop, Plimptonville! Or actually not, since the single daily round-trip that called at the ten-foot asphalt “platform” has been discontinued — hopefully for good. As I mentioned above, in 2018, this sorry excuse for a station, with gravel parking lot the size of two SUVs, served a dozen passengers, which most have been mostly people who lived within walking distance. With the station out of the way, there’s nothing preventing busting up the platform to restore the second track, and once that’s done it can never be reopened because this station can never be made accessible in its present location. Wikipedia says the current platform is ten feet long, which means it doesn’t take much of a braking error on the part of the engineer to entirely miss the mark.

Windsor Gardens, like Plimptonville, is problematic for future service on a double-tracked line. Also like Plimptonville, it serves only local residents — it’s practically a private station, with no formal public access except by train. It was the last stations built before the MBTA took ownership of the line, and serves only the residents of the apartment complex formerly known as Windsor Gardens. (Now called “The Commons at Windsor Gardens” because landlords can’t help but change the names of their properties when they change hands.) Despite the somewhat limited service, more than 250 people used this station in the 2018 counts, and it’s definitely one that is worth keeping, both for the MBTA and as a valuable and exclusive amenity for the landlord. The challenge is how — if it is even possible — to extend the double-track through this section, as the platform (again, like Plimptonville) sits on top of the old southbound trackway, in a fairly narrow section of the right of way. Even making the station accessible is likely to be challenging, although it’s a challenge that absolutely should be taken up — but who is going to pay for it? It seems likely that, in the short term at least, this will remain a single-track station, with a neckdown between two otherwise double-track segments.

Norwood Central brings us back to the “ocean of parking” that’s such a theme of the MBTA Commuter Rail network — although at least there is more high-density residential construction immediately adjacent to the line. The closest large employer, Norwood Hospital, has been closed since a flood in June, 2020, and is not expected to reopen before the end of 2021. At least there’s a solar canopy over part of the parking lot. South of Norwood Central, a couple of apparently active (but very poor condition) freight sidings persist, one of which appears to still be getting freight deliveries, which may help to explain why such a busy station (more than a thousand passengers a day in 2018) has only mini-high platforms; constructing full-length high platforms should be a priority, but the station has the usual problem of requiring expensive vertical circulation structures to allow for full closure of the pedestrian grade crossings (which are incompatible with high-level platforms).

There’s no sign of a depot at Norwood Depot, nor even a junction, but there is more recently-built housing and another very large parking lot. A light-industrial building at the far north end of the platform does show signs of having had boxcar doors at one time, although no other evidence of a historic freight siding remains. The footfall at Norwood Depot is surprisingly low compared to its sibling to the south, under 300 a day in 2018, which is presumably a consequence of its smaller (but still substantial) sea of surrounding parking. Perhaps if the MBTA sold some of that parking to another new residential development, there might be more traffic here. There are very long low-level platforms with mini-highs at the south end, although the mini-highs seem somewhat longer than at many other stations, and surprisingly, have a wood deck rather than concrete. The inbound platform is width-constrained, which will make construction of any vertical circulation difficult; in the mean time, the station limps along with limited pedestrian grade crossings.

Islington is one of two stations in Westwood — the other being the far larger and more accessible Route 128/University Ave. station. Although it’s wedged between Routes 1 and 1A, the station is in a single-family-residential neighborhood, has relatively limited parking, and is likely convenient only to its immediate neighbors. (The parking lot is also in poor condition.) Nonetheless, it did manage to attract more than a hundred daily riders in the 2018 counts despite being skipped by a number of the AM peak trains. (I tried to count the number of different service patterns operated on this line in the 2018 passenger data, and gave up when I got to ten.) Islington is also very close to Dedham Corporate Center, on the other side of Route 128 (the freeway, not the station), which has much more parking, denser neighboring residential development, and easier access, so it’s not surprising that it gets relatively low ridership compared to its nearest “competitors”.

Speaking of which, a sudden snow squall interrupted my tour of the line. When I got out of the car at Dedham Corporate, I found the weather to be unconducive for photography, and went home (after taking a few pictures of the swirling snow in the parking lot). I went back the following Saturday afternoon to finish off the Franklin Line with the two remaining stations. Dedham Corporate of course has the usual ocean of parking, accessed from the East St. exit off Route 128, but on the other side of the tracks, two large chain apartment complexes (an “Avalon” and a “Jefferson”) front the station from across Rustcraft Road. Dedham Corporate Center, the industrial park, is located on the former site of the Rust Craft Greeting Card Company’s factory, once the world’s largest, and had its own station to serve employees; the station was closed due to low traffic in the early MBTA era, and the new station was built in the late 1980s by the developers of the industrial park. The apartment complexes came much later, and the Legacy Place lifestyle center, which is on the other side of the apartment buildings, even later still. When the station was originally built, there was no access from the Rustcraft Road side, and in fact a chain-link fence still blocks access to the platforms, but there is now a “kiss and ride” (signed as such!) and a small opening in the fence about 400 feet south of the drop-off area. (So you can drop someone off at the drop-off but they still have to walk most of the way down the platform before they can actually get into the station.) As at other stations of this age, there are full-length low-level platforms with no tactile warning, with a pair of mini-high platforms at the south end of the station, and pedestrian grade crossings make full-length high platforms difficult to implement, although there is at least plenty of room in the right-of-way to add ramps or elevators at either end of the platforms.

The final stop on our tour of the Franklin Line is Endicott. It’s a very small station, inaccessible, in a low-density residential neighborhood with parking restricted to Dedham residents. Between the parking (which is small but not trivial) and the local residents, this station seems to attract a decent amount of patronage — 250 passengers a day in 2018. It’s also used as a cut-through by the local neighborhood: I saw a family on bicycles use the ramp and grade crossing while I was there taking pictures. Yes, even though the station is not accessible as a station, it still has a ramp to get from street level up to the platform, and even some HP/V restricted parking. What the town of Dedham expects one to do once a wheelchair user gets up onto the platform is unclear, but at least they made the effort. Like with other stations on this line, vertical circulation is the biggest challenge for making the station fully accessible, especially in the middle of a residential neighborhood as it is — neighbors would undoubtedly show up and filibuster any public meeting that proposed making the station properly accessible.

That’s the end of this tour. The next stop on the line is Readville, which is … a thing. I haven’t revisited Readville since my 2019 tour of the low-platform Providence Line stations, and nearly all Franklin Line trips join the Southwest Corridor at Readville, while the historic Midland Route follows the Dorchester Branch, today’s Fairmount Line, into South Station. I did stop by Fairmount station itself, but haven’t seen any of the newly constructed Fairmount Line stations and will delay that until after the pandemic when I can more comfortably do so by transit.

Posted in Transportation | Tagged , , | Comments Off on Weekend excursion: Stations of the Franklin Line