Importing to Piwigo

Piwigo is pretty good!

I mean, I mostly use Flickr at the moment, because it is quick, easy to recommend to people, and allows photos to be added to Trove. But I’d rather host things myself. Far easier for backups, and so nice to know that if the software doesn’t do a thing then there’s a possibility of modifying it.

To bulk import into Piwigo one must first rsync all photos into the galleries/ directory. Then, rename them all to not have any unwanted characters (such as spaces or accented characters). To do this, first have a look at the files that will fail:

find -regex '.*[^a-zA-Z0-9\-_\.].*'

(The regex is determined by $conf['sync_chars_regex'] in include/config_default.inc.php which defaults to ^[a-zA-Z0-9-_.]+$.)

Then you can rename the offending files (replace unwanted characters with underscores) by extending the above command with an exec option:

find -regex '.*[^a-zA-Z0-9\-\._].*' -exec rename -v -n "s/[^a-zA-Z0-9\-\._\/]/_/g" {} \;

(I previously used a more complicated for-loop for this, that didn’t handle directories.)

Once this command is showing what you expect, remove the -n (“no action”) switch and run it for real. Note also that the second regex includes the forward slash, to not replace directory separators. And don’t worry about it overwriting files whose normalized names match; rename will complain if that happens (unless you pass the --force option).

Once all the names are normalized, use the built-in synchronization feature to update Piwigo’s database.

At this point, all photos should be visible in your albums, but there is one last step to take before all is done, for maximum Piwigo-grooviness. This is to use the Virtualize plugin to turn all of these ‘physical’ photos into ‘virtual’ ones (so they can be added to multiple albums etc.). This plugin comes with a warning to ensure that your database is backed up etc. but personally I’ve used it dozens of times on quite large sets of files and never had any trouble. It seems that even if it runs out of memory and crashes halfway, it doesn’t leave anything in an unstable state (of course, you shouldn’t take my word for it…).

Manually upgrading Piwigo

There’s a new version of Piwigo out, and so I must upgrade. However, I’ve got things installed so that the web server doesn’t have write-access to the application files (as a security measure), and so I can’t use the built-in automatic upgrader.

I decided to switch to using Git to update the files, to make future upgrades much easier and without having to make anything writable by the server (even for some short amount of time).

First lock the site, via Tools > Maintenance -> Lock gallery, then get the new code:

$ git clone https://github.com/Piwigo/Piwigo.git photos.samwilson.id.au
$ cd photos.samwilson.id.au
$ git checkout 2.8.3

Copy the following files:

/upload (this is a symlink on my system)
/local/config/database.inc.php
/local/config/config.inc.php

The following directories must be writable by the web server: /_data and /upload (including /upload/buffer; I was getting an “error during buffer directory creation” error).

Then browse to /upgrade.php to run any required database changes.

I’ve installed these plugins using Git as well: Piwigo-BatchDownloader, Flickr2Piwigo, and piwigo-openstreetmap. The OSM plugin also requires /osmmap.php to be created with the following (the plugin would have created it if it was allowed):

<?php
define( 'PHPWG_ROOT_PATH', './' );
include_once( PHPWG_ROOT_PATH . 'plugins/piwigo-openstreetmap/osmmap.php' );
?>

That’s about. Maybe these notes will help me remember next time.

Piwigo rocks

I have been using Piwigo for a couple of years (photos.samwilson.id.au), and have been really happy with it. The ability to work with large numbers of photos (uploading lots, and bulk-editing) is what made it a pleasure to use to start with; these are usually the initial tasks one does with this photo-gallery software, and they’re usually where systems are not at their best. Now I’ve got a few thousand photos in it, I’ve gotten the hang of a reasonable workflow, and Piwigo has mostly receeded to the background and just carries on working without issue. I’ve added my albums’ URLs to all sorts of places, including in printed archival descriptions, and feel pretty committed to sticking with Piwigo.

So it was nice to recieve a newsletter from the Piwigo development team, talking about their recent shift of the codebase to GitHub, a new Java desktop synchronisation client, and other things. If one doesn’t actively haunt the forums, it’s hard to remember that Piwigo is still a going concern — but I’m very glad that it is!

Open source software is great, I love using it and contributing to it. But sometimes it goes away. :( Of course, that happens to proprietary apps too, but with FOSS failures I feel sad, because it feels like I’ve personally failed the project (I should’ve been more involved). It’s one of the reasons it’s good to pay for free software. I’m glad Piwigo makes money from their piwigo.com service (well, I assume that’s what keeps the lights on).

Anyway, all I wanted to say was: thanks for Piwigo.