basic ikiwiki on ubuntu 22.10
It's a sad fact of computing in 2022 that you can't just "sit down and write", given requirements something like the following:
- documents in sensible, specified format amenable to external version control, backups, and other manipulation
- easily searchable
- works on standalone computer -- if it's in the cloud, then it's a service, not an item of computing you can do yourself (must)
- presentable, or even looks good, both on screen and in print
- editor works by default in prose (paragraphs) mode, and never makes the user worry about line-wrapping
- documents by default, including in editor (which may be same as viewer), are presented at sensible of abstraction, such as titles, sections and paragraphs, not as pretend sheets of paper
- outline view (optional)
- visual peace (no flashing cursor or other distractions) by default
- beautiful by default, with custom styling possible
- decent hyperlinking
Off the top of my head, the closest an application comes to an acceptable experience is Apple's "Notes" app, which ships with iOS. It doesn't meet all of the above, but it gives a fairly nice experience.
There are perhaps four approaches taken to the problem of self-authored documents.
The most common, including for me kinda, is the cloud, which is out of scope here. Leader seems to be Google Docs. Google Docs shows its roots in the C++ Word Processor world, being clunky, sluggish, based on pictures of pieces of paper, font and formatting-centric, ugly, and failing to provide a proper abstraction of a document. I heard a couple of years ago that there's a new feature, where you can choose not to have the "document" presented as a pretend series of pieces of paper. Major breakthrough, dipshits!
Another popular approach is the "pile of MS Word documents". We are still in the C++ Word Processor world here, but we are at least in scope. The formats are widely supported by different word processors, including free ones like libreoffice, formerly openoffice, abiword, the KDE one, and so on. You can stick with Word '97 for tried and tested binary format, or choose docx for the reasonably-fully-specified, reasonably-fully-supported, but complicated and complificated docx format. Avoid all the idiot formats from the period when each crapplication thought they had to invent their own format, such as openoffice's one (odt), KDE's one, and so on.
I wouldn't be surprised if these piles tend to end up searchable these days on Windows's and MacOS's built-in search, but I don't know. They are not bigly amenable to version control because they are either binary or so xml-tastic that they may as well be binary, but I guess you could track snapshot saves.
Print output tends to be around adequate.
Editing documents at a correct level of abstraction, rather than at the formatting level, requires getting to know one of the applications fairly well.
I'm almost selling myself on this option. For me the blockers are the sheer unpleasantness of using any of the applications concerned, and the lack of searchability on free OSs.
A typical example is the pervasive problem of irrelevant file-rescue on application startup. The applications should be designed so that instances are independent. The editor / viewer in your work area for "getting the greenhouse fixed" (perhaps this is a virtual desktop) should not know or care about the editor / viewer in your work area for "submit latest aml homework to bank". Instead, everything is coupled into a singleton instance per user session (roughly). The result, coming back to the example, is that you get in from the garden, try and open libreoffice on "getting the greenhouse fixed", and, instead of getting to do what you wanted to do, you are taken on an ordeal starting out with "latest aml homework was not saved properly", giving you three options which all sound bad and scary, with each option only leading to contradictions, further "choices", more windows, increasingly serious admonitions, until you are annoyed enough to give up and use the notebook in your pocket. If you press "cancel" or "no" enough times, you may get through, but they trick you with double-negatives, so you're probably stuck. If and when you do get through, you are using the ugly, clunky C++ thing with hundreds of stupid formatting buttons, as though adding each one gave a "feature" and was therefore good. And probably a flashing cursor, ugly squiggles under words, and the rest of it.
As for searchability, I've just tried on Ubuntu 22.10, and the built-in desktop search (click on "Activities" in top left and use search box) worked on the filename but not the content. Nowhere near good enough.
Another approach is the "pile of text files". Here the author gives themselves an empty directory, a text editor, and the freedom to make up their own conventions, adhoc mini-formats even, within this. And to use grep to search, tex or a wordprocessor to print, and other more or less unixy tooling. This has worked well in several situations, but it's not going to come close to meeting the "just write" requirements I outlined above.
This brings me to the fourth and final approach for today, which is to use (a particular kind of) wiki software.
Wikis tend to use minimal, or at least small, text formats. These formats are of varying degrees of specifiedness. Markdown has sadly gone down the route of "dialects", which means it's dead -- it's adhoc and not specified. Someone could freeze one of the markdown "dialects", give it its own name, check it's well specified, promise to never change it, work out how to prevent "dialects" of that (trademarks, general threats, etc) and then there'd be something specified. Other formats include asciidoc, restructured-text, and the wikipedia one. They are amenable to version control, as it currently exists, because text.
Searchability tends to be built-in, via the web interface.
Wikis tend to be designed for use primarily via the web, but some allow gracefully-degraded use as a pile of files. That means that for local web-interface use you'll have to do some sysadmin, and it's going to involve unpleasantness like web servers, system services, interposing gethostbyname via /etc/hosts.
There aren't exactly any good ones, but the one I've made use of over the years is ikiwiki from Joey Hess. It's full of warts, which I might come back to, but without further ado, here's a way to get it working on Ubuntu 22.10. Not a nice way, just.. a way.
It's set up as and by a user but with root access. The user here is called "tj". In Ubuntu you are not allowed to have home directories anywhere other than "/home", or all crapplications, which they call "snaps", stop working. I'm not shitting you.
The assumption is that you're not overly concerned with other users. It's not really a multi-user host.
We are starting from an empty wiki and not using version control. Choice of nginx for web server is arbitrary.
Procedure
install packages
# apt install ikiwiki xapian xapian-omega nginx
disable idiotic auto-run of nginx
# systemctl stop nginx
# systemctl disable nginx
choose and note some names / addresses
You'll have to keep track of where these crop up in the config.
- local user: I chose "tj"
- wiki name: I chose "smallwiki"
- local service name: I chose "smallwiki.local"
- local ip and port to provide service on: I chose 127.0.0.11 and port 1025 (if you choose below 1024, you will run into "privileged port" problem, requires different config)
create service directory in user's home directory
Can change name "smallwiki" in:
tj~$ mkdir -p srv/smallwiki
ikiwiki setup file
This is gnarly. In ~/srv/smallwiki/ik.setup:
use constant SVD => "/home/tj/srv/smallwiki";
use IkiWiki::Setup::Standard {
wikiname => "smallwiki",
# it demanded url
url => "http://smallwiki.local:1025",
cgiurl => "http://smallwiki.local:1025/cgi/ik.cgi",
usedirs => 0,
srcdir => SVD . "/small.in",
destdir => SVD . "/small.out",
add_plugins => [ qw(
goodstuff httpauth search
rawhtml brokenlinks typography inline
) ],
# disable_plugins => [qw(passwordauth)],
discussion => 0,
wrappers => [
{
cgi => 1,
wrapper => SVD . "/ik.cgi",
wrappermode => "00755",
},
],
};
empty / minimal in, out dirs
tj:~/srv/smallwiki$ mkdir small.in small.out
tj:~/srv/smallwiki$ echo hello wiki > small.in/index.mdwn
initial rendering run, plus make the cgi wrapper (really)
tj:~/srv/smallwiki$ ikiwiki --setup ik.setup
Look in small.out -- there should be tons of boilerplate garbage.
set fastcgjiwrap running as user
tj:~$ mkdir srv/fcgiwrap
Make /tj/home/srv/fcgiwrap.run:
#!/bin/sh
export HOME=/home/tj
cd $HOME
# This seems to be wrong, given this from fcgiwrap(8):
# By default fcgiwrap expects a listen socket to be passed on file
# descriptor 0, matching the FastCGI convention. The recommended
# way to deploy fcgi‐wrap is to run it under a process manager
# that takes care of opening the socket. However, for simple
# configurations and one-off tests this option may be used.
# which is unclear to me. What exactly is passed on FD 0 and how? Is
# it talking about actual posix FD passing?
# But since the top result for "running fcgiwrap and sockets" is
# total gibberish, and this is better than that, can keep it for now.
SOCK=srv/fcgiwrap/fcgiwrap.socket
rm -f $SOCK
exec fcgiwrap -s unix:$SOCK
As root, make /etc/systemd/system/fcgiwrap.tj.service with content:
[Unit]
Description=fcgiwrap
[Service]
User=tj
ExecStart=/home/tj/srv/fcgiwrap/fcgiwrap.run
[Install]
WantedBy=multi-user.target
and make it executable:
tj:~$ chmod +x srv/fcgiwrap/fcgiwrap.run
start the service:
# systemctl enable fcgiwrap.tj
# systemctl start fcgiwrap.tj
and check:
# systemctl status fcgiwrap.tj
# ls -l /home/tj/srv/fcgiwrap
(expect to see a socket file)
set name and addr
add to /etc/hosts:
127.0.0.11 smallwiki.local
nginx config
create logs dir:
write nginx.conf:
In ~/srv/smallwiki/nginx.conf :
daemon off;
# barfs w/o this, defaults to /run/nginx.pid
pid logs/nginx.pid;
# Damned thing makes you write this, has no working default:
events {
use epoll;
worker_connections 128;
}
error_log logs/error.log info;
http {
server_tokens off;
include /etc/nginx/mime.types;
charset utf-8;
access_log logs/access.log combined;
server {
server_name wiki.local;
listen 127.0.0.11:1025;
location / {
root small.out;
}
location ~ ^/cgi {
root /home/tj/srv/smallwiki;
rewrite ^/cgi/(.*) /$1 break;
include /etc/nginx/fastcgi_params;
fastcgi_param REMOTE_USER webserver;
fastcgi_pass unix:/home/tj/srv/fcgiwrap/fcgiwrap.socket;
# otherwise it doesn't know where to look. Must be better way.
fastcgi_param SCRIPT_FILENAME /home/tj/srv/smallwiki/$fastcgi_script_name;
}
}
}
Put Systemd unit file in /etc/systemd/system/smallwiki.tj.service:
[Unit]
Description=smallwiki
[Service]
User=tj
ExecStart=/home/tj/srv/smallwiki/smallwiki.run
[Install]
WantedBy=multi-user.target
and the script just referenced to actually run it (can't give args in an ExecStart!), so /home/tj/srv/smallwiki/smallwiki.run :
#!/bin/sh
export HOME=/home/tj
export SVD=$HOME/srv/smallwiki
# requries:
# - logs dir
cd $SVD
exec nginx -p . -c nginx.conf
make it executable :
$ chmod +x ~/srv/smallwiki/smallwiki.run
Activate and run:
# systemctl enable smallwiki.tj
# systemctl start smallwiki.tj
check running:
# systemctl status smallwiki.tj
browse to it
Go to http://smallwiki.local:1025/ and be amazed if it works, debug if not.
try a search via the search box.
This tests the cgi and search functionality, and is less likely to fail than an edit.
try an edit
This often fails by going to a login choice page (openid and 2 other choices) instead of treating us as logged in as "webuser" via REMOTE_USER because we've given httpauth as a plugin in ik.setup.
I haven't tracked down what causes this. Somehow ikiwiki, in its opaque form as a generated binary executable, decides to ignore our request for the web server to do the authn, and instead goes down the track of using its hairball built-in authn system. And then, just as you've decided you will never find it, and why not just replace ikiwiki with something less bizarre it can't be that hard, the problem just goes away, and there you are editing.
commentary on procedure
* package installation of the one thing you want should get a working setup, you shouldn't have to know magical package combos
* while writing this on blogger, getting repeated issue: set to subheading, write subheading, go to write para, ITS STILL IN SUBHEADING MODE. So put in para mode. Later, prepend text to para, ITS BACK IN SUBHEADING MODE. FUCK OFF.
* sometimes selecting "default font" doesn't change selection, so formatting is broken. When "default font" doesn't work, have to choose a specific one, but I don't know which one default font is, so I choose Helvetica, and it looks different. It's like Google is trying to give me an anti-example in "just sit down and write" editing.
* nginx doesn't support plain cgi. There must be a nicer way of mapping a URL to a CGI program than what we've got here, even if it "works".
* running fcgiwrap is nasty because you (the wrapper script) have to remove the socket file each time, and its manual is unclear on what it thinks is the "proper" way to handle this unix domain socket file's lifecycle.
* you will always see "could not open error log file" as first message from nginx, it's always been there as one of its quirks
* the issue with deferring to web server's authn and REMOTE_USER may have to do with browser state, but I did try with a fresh browser
The Bigger Picture
- per-user process-supervision support. cron's @reboot doesn't cut it. user should be able (subject to system policy) to install and control their own services, to be run as them under process supervision
- owned tcp ports. See "The Persistent Idiocy of Privileged Ports on Unix".
- local communication over unix domain sockets instead of localhost 127/8 tcp sockets. Web server that serves over such a socket. Browser that can connect to such a socket (perhaps when it is launched it's limited to just that socket, to avoid cross-domain issues, and this becomes a type of app)
- web server with sensible defaults, that can be launched with around one parameter, and work
- web server that can run cgi progs directly without having to set up fastcgi daemons (yes, apache httpd of course,can)
- web server packaging that doesn't start an instance of it just because it got installed
- wiki software that operates as normal cgi prog, instead of generated "wrapper"
- working wiki setup by naming at most one package, instead of having to know arcane combinations of packages
- nice text-ish document format(s), might already exist as asciidoc / restructured text / a named freeze of a markdown "dialect"
Would be enough to keep a small team, with suitable taste, busy.
Comments
Post a Comment