SeaPIG Wiki Administration
There's not much to administering a wiki, but what there is goes here.
See AdminGroup for the list of wiki administrators.
TODO
- Add more textchas
- Generate stats for wiki (track bandwidth)
Macros/actions not migrated: BlogEdit action, OutsideLinks, PageSignature, RecentBlogs macros
Comments
Go to WikiAdmin/UserComments and add your comments--they'll show up here.
Changelog
2008/03/05 Removed most spam pages --BradeyHonsinger
Removed most spam pages following steps below. List of deleted pages is here:
Send mail to the list if you see something I missed or shouldn't have deleted.
- Remove empty directories (created when antispam prevented saving page or user cancelled) [3382 pages]:
# find /home/seapig/seapig.org/wiki/data/pages -maxdepth 1 -type d -empty -print0 | \ xargs --null rm -rf
- Remove pages with Asian characters in the title [164 pages]:
# ls | grep '(e[0-9]' | xargs rm -rf
- Remove pages with no upper-case characters in the title (can't be wiki words) [77 pages]:
# ls | grep -v [A-Z] | xargs rm -rf
- Remove pages with '+' in the title [51 pages]:
# ls | grep '(2b)' | xargs rm -rf
- Go through remaining pages by hand, and delete spam (and other unused) pages [16 pages]
2008/03/04 Upgraded to Moin 1.6.1 --BradeyHonsinger
Really wanted "textcha" capabilities to stop spam. Ubuntu doesn't have packages for 1.6.1 yet, so uninstalled 1.5.7 from aptitude and installed 1.6.1 from source.
Note that markup has changed in 1.6–see HelpOnLinking (especially the “If you used Moin before...” section at the bottom), HelpOnMacros
- Remove/reinstall:
# aptitude remove python-moinmoin moinmoin-common # cd /usr/local/src # wget http://static.moinmo.in/files/moin-1.6.1.tar.gz # tar xvzf moin-1.6.1.tar.gz # cd moin-1.6.1/ # python setup.py install --record=install.log
- Fix pre-migration problems:
Update /home/seapig/seapig.org/wiki/Wikiconfig.py to from MoinMoin.config.multiconfig import DefaultConfig, as instructed by migration script (must be done before migration script will run)
Create /home/seapig/seapig.org/wiki/data/meta after a Google search on "KeyError: 'data_format_revision'" (apparently didn't get created before as part of migration process, and must be done before migration script will run).
# sudo -u www-data cat > /home/seapig/seapig.org/wiki/data/meta data_format_revision: 01050700 ^D
Update from MoinMoin.util.antispam import SecurityPolicy to from MoinMoin.security.antispam import SecurityPolicy (apparently that changed at some point too, and must be fixed before migration script will run)
- Migrate:
- Run migration script once to start migration
# sudo -u www-data moin --config-dir=/home/seapig/seapig.org/wiki --wiki-url=seapig.org migration data
Copy /home/seapig/seapig.org/wiki/data/rename1.txt to rename2.txt as instructed by migration script
- Re-run migration script
Copy over new moin.cgi, update path to wikiconfig.py in it
- Run migration script once to start migration
Enable textchas in wikiconfig.py:
# members of this don't get textchas #textchas_disabled_group = u"AdminGroup" textchas = { 'en': { u"What city is Seapig in?": ur"seattle", u"What state is Seapig in?": ur"(washington|wash|wa)", u"What is your favorite color?": ur"(red|green|blue|yellow|purple|orange|white|black|arrgh)", u"Who is the BFDL?": ur"(guido|guido van rossum|gvr)" # ... }, # you can add more languages if you like }
2008/03/04 Fixed up config on new server --BradeyHonsinger
- Problem: list and wiki backups not working (didn't notice until now...)
Moved /etc/cron.weekly/seapig-{list,wiki}-backup scripts from old server (and fixed up wiki backup script to get files from /home/seapig/seapig.org instead of /var/www/seapig.org).
Installed /usr/bin/mail so that backups scripts would work (aptitude install mailx).
Problem: list archive page not updating (http://lists.seapig.org/pipermail/seattle-python/)
Following this message:
# cd /var/lib/mailman # cp \ archives/private/seattle-python.mbox/seattle-python.mbox \ archives/private/seattle-python.mbox/seattle-python.mbox.bak # bin/cleanarch \ < archives/private/seattle-python.mbox/seattle-python.mbox.bak \ > archives/private/seattle-python.mbox/seattle-python.mbox # bin/arch --wipe seattle-python # rm archives/private/seattle-python.mbox/seattle-python.mbox.bak
For some reason the links on the archive page are no longer to the gzipped files, but at least they're there now. Will need to watch and make sure it's still updated as it should be. Didn't find anything wrong with the files, permissions, or locations (although running cleanarch did clear up some glitches with messages that were misfiled).
2007/12/23 Moved list to new server --BradeyHonsinger
Installed and configured according to https://help.ubuntu.com/community/Mailman.
Apache 2 configuration: Didn't follow steps--started from existing /etc/apache2/sites-available/lists.seapig.org.
Moved non-Mailman files to /home/seapig/lists.seapig.org/www:
# mkdir /home/seapig/lists.seapig.org # cp -R /mnt/606/var/www/lists.seapig.org/www /home/seapig/lists.seapig.org/ # chown www-data.www-data lists.seapig.org/
Edited DocumentRoot in /etc/apache2/sites-available/lists.seapig.org to point to /home/seapig/lists.seapig.org/www.
- Postfix configuration: as specified
- Copied list files over:
# cp -R /mnt/606/var/lib/mailman/archives/private/seattle-python.mbox /var/lib/mailman/archives/private/ # cp -R /mnt/606/var/lib/mailman/archives/private/seattle-python /var/lib/mailman/archives/private/ # cp -R /mnt/606/var/lib/mailman/lists/seattle-python /var/lib/mailman/lists/ # /usr/lib/mailman/bin/check_perms -f
Couldn't easily put them in /home/seapig/lists.seapig.org--research for next time.
Confirmed that list was functional--logged in via Mailman web page to view settings, than posted test message to list.
2007/12/22 Moved server to Ubuntu 7.10 --MikeOrr and BrianDorsey
Migrated to Moin 1.5.7.
2007/03/08 Updated Moin after vulnerability warning, switched to prefork MPM --BradeyHonsinger
Upgraded to python-moinmoin 1.5.2-1ubuntu2. See http://www.ubuntu.com/usn/usn-423-1.
# apt-get update ... # apt-get install python-moinmoin Reading package lists... Done Building dependency tree... Done The following packages will be upgraded: python-moinmoin 1 upgraded, 0 newly installed, 0 to remove and 15 not upgraded. Need to get 69.4kB of archives. After unpacking 0B of additional disk space will be used. Get:1 http://security.ubuntu.com dapper-security/main python-moinmoin 1.5.2-1ubuntu2.2 [69.4kB] Fetched 69.4kB in 0s (96.2kB/s) (Reading database ... 24211 files and directories currently installed.) Preparing to replace python-moinmoin 1.5.2-1ubuntu2 (using .../python-moinmoin_1.5.2-1ubuntu2.2_all.deb) ... Unpacking replacement python-moinmoin ... Setting up python-moinmoin (1.5.2-1ubuntu2.2) ... # dpkg-query -s python-moinmoin Package: python-moinmoin Status: install ok installed Priority: optional Section: python Installed-Size: 108 Maintainer: Jonas Smedegaard <dr@jones.dk> Architecture: all Source: moin Version: 1.5.2-1ubuntu2.2 ...
Moved from worker MPM to prefork MPM, in an attempt to avoid invoking OOM killer on linode box.
# apt-get install apache2-mpm-prefork Reading package lists... Done Building dependency tree... Done The following packages will be REMOVED: apache2-mpm-worker The following NEW packages will be installed: apache2-mpm-prefork 0 upgraded, 1 newly installed, 1 to remove and 15 not upgraded. Need to get 198kB of archives. After unpacking 12.3kB disk space will be freed. Do you want to continue [Y/n]? Get:1 http://security.ubuntu.com dapper-security/main apache2-mpm-prefork 2.0.55-4ubuntu2.1 [198kB] Fetched 198kB in 0s (205kB/s) dpkg: apache2-mpm-worker: dependency problems, but removing anyway as you request: apache2 depends on apache2-mpm-worker (= 2.0.55-4ubuntu2.1) | apache2-mpm-prefork (= 2.0.55-4ubuntu2.1) | apache2-mpm-perchild (= 2.0.55-4ubuntu2.1); however: Package apache2-mpm-worker is to be removed. Package apache2-mpm-prefork is not installed. Package apache2-mpm-perchild is not installed. mailman depends on apache | httpd; however: Package apache is not installed. Package httpd is not installed. Package apache2-mpm-worker which provides httpd is to be removed. (Reading database ... 24210 files and directories currently installed.) Removing apache2-mpm-worker ... * Stopping apache 2.0 web server... [ ok ] Selecting previously deselected package apache2-mpm-prefork. (Reading database ... 24206 files and directories currently installed.) Unpacking apache2-mpm-prefork (from .../apache2-mpm-prefork_2.0.55-4ubuntu2.1_i386.deb) ... Setting up apache2-mpm-prefork (2.0.55-4ubuntu2.1) ... * Starting apache 2.0 web server... [ ok ]
2007/01/30 Moved to new server
Moved wiki to MikeOrr's server (aka sluggo.scrapping.cc). Upgraded to Moin 1.5.2 in the process (no need to run migration scripts--data format stayed the same). Need to re-setup backup job at some point. Should probably re-evaluate wikiconfig.py to see if there are new features we want to take advantage of, too.
2006/03/21 Setup wiki backup
Finally created a backup procedure for the wiki--a tarball is created weekly and placed in an FTP-accessible directory on the webserver. Email is sent to admins to notify them that it's there. Didn't want to email it out like the list backup, since it's ~7 MB.
2005/02/08 Setup anti-spam, AdminGroup
Enabled Moin spam filter (see AntiSpamGlobalSolution), setup ACLs and AdminGroup to enable access to LocalBadContent.
2005/02/06 DNS cutover
DNS now points to new wiki, spam starts
2005/02/02 Moved to Construx server, upgraded to Moin 1.3.3
Migration steps:
- Tar up the old site and move to new server
Remove old, unchanged SystemPages (now in underlay dir)
- Migrate to new format using migration steps
Gory details:
Removing system pages:
SeaPig current version is 1.2.3, but pages look like 1.0 (pre-1.0?).
- Download moin-1.0.tar.gz from sf.net
- Compare files, look for unchanged:
$ diff -r --brief --report-identical-files moin-1.0/wiki/data/text/ data/text/ | \ grep identical | awk '{print $4}' | awk 'BEGIN { FS="/" } { print $(NF)}' AbandonedPages Aktuelle_c4nderungen AufgegebeneSeiten ...
Save list to unchanged-pages
Kill both data/text/ files and data/pages/ dirs in list, and data/backup/ files:
$ cd data/text $ cat ../../unchanged-pages | xargs rm $ cd ../../data/pages $ cat ../../unchanged-pages | xargs rm -r $ cd ../../data/backup $ cat ../../unchanged-pages | xargs -I '{}' find . -name "{}.*" | xargs rm
Migrating to new format
- Grabbed moin-1.3.3.tar.gz, unpacked, and started migration:
$ cp -R data moin-1.3.3/MoinMoin/scripts/migration/ $ cd moin-1.3.3/MoinMoin/scripts/migration/ $ ./12_to_13_mig1.py ... $ ./12_to_13_mig2.py ... ... $ ./12_to_13_mig4.py data.pre-mig4/pages/ZopeZen/backup/1035510141000000 -> data/pages/ZopeZen/backup/1035510141000000 Traceback (most recent call last): File "./12_to_13_mig4.py", line 148, in ? convert_eventlog(opj(origdir, 'event.log'), opj('data', 'event-log')) File "./12_to_13_mig4.py", line 63, in convert_eventlog data[0] = str(convert_ts(float(data[0]))) # we want usecs ValueError: empty string for float()
Error on mig4 step with data.pre-mig4/pages/ZopeZen/backup/1035510141000000; killed file and restarted:
$ rm data.pre-mig4/pages/ZopeZen/backup/1035510141000000 $ rm -r data $ mv data.pre-mig4 data $ ./12_to_13_mig4.py
Noted that original wiki pages/ is corrupt:
$ ls data.pre-mig1/pages/ ... very_20interesting very_5f20interesting very_5f5f20interesting very_5f5f5f20interesting very_5f5f5f5f20interesting very_5f5f5f5f5f20interesting very_5f5f5f5f5f5f20interesting
The actual wiki page is very_20interesting; the others are not valid. There are a lot of these--there are ~300 real wiki pages and ~1200 pages dirs. Some of these are deleted pages; I'm just going to purge them now (see below).
- Fix:
$ cd data.pre-mig1 $ cd text $ \ls > ../../real-pages $ cd ../pages $ \ls > ../../pages-dirs $ cd ../.. $ cat real-pages pages-dirs | sort | uniq -u ... feed very_5f20interesting very_5f5f20interesting very_5f5f5f20interesting very_5f5f5f5f20interesting very_5f5f5f5f5f20interesting very_5f5f5f5f5f5f20interesting $ cat real-pages pages-dirs | sort | uniq -u > corrupt-pages $ cd data.pre-mig1/pages $ cat ../../corrupt-pages | xargs rm -r
- Now, start over with fixed data dir
$ rm -r data data.pre-mig2 data.pre-mig3 data.pre-mig4 $ mv data.pre-mig1 data $ ./12_to_13_mig1.py | tee data-mig1.log | grep -i error ...
- Still have error w/ZopeZen; kill backup w/error
- This isn't a problem w/ZopeZen, it's a problem with event_log (see error)
- Our event.log is (almost) empty--just a newline. Replace with empty file.
- That did it for step 4, now on to step 5...
- Appears to have worked!
Cleanup
Remove old/out-of-date plugins/macros: rm -r data/plugins/* (see TODO)
- Remove intermap.txt (doesn't look like anything new)