Introduction

A TiddlyWiki configured to work collaboratively through git. I did it for my personal use case, and it is probably not the correct way to do it, but feel free to try it or to take some parts of it as an inspiration. The goal is to have a button inside the Wiki that you can click to commit and push your changes and which take care of conflicts.

TiddlyGit no longer uses the Bob plugin (TW5-Bob). Bob is not maintained anymore, and its file watcher could delete a tiddler while you were typing in a form (without going through the trashbin). TiddlyGit now uses the TiddlyWiki node.js server (--listen), with a small plugin of its own (see the notes below). If your wiki is still in the old Wikis/<name>/ layout used with Bob, see Coming from the old layout to convert it.

Quick guide

• installs TiddlyGit
• test everything is working with test.sh
• start the server with the run.sh
• navigate to http://127.0.0.1:7070/
• press the "github" button - this will commit your changes, merge the changes of the other contributors and push them
• the page is blocked during the synchronization (so that nothing you type is mixed up with the changes merged by git), in every tab and browser connected to the server (within a few seconds for the others, and the server refuses their changes meanwhile, they are sent again after), an "Unblock anyway" button lets you get it back if it takes too long (if the synchronization has not started yet, because the browser cannot send its changes to the server, it is cancelled)
• a notification tells you when the synchronization is done, the merged changes appear in the browser without reloading the page

Notes

The server is the TiddlyWiki node.js server (--listen), with the tiddlygit plugin (see tiddlygit-plugin/) which:

• loads the tiddler files created, modified or deleted on disk (by a text editor, git...), they reach the browser within 5 seconds,
• runs synchronize.sh when the synchronization button is pressed. The server doesn't write any file while the script runs, and reloads the files changed by git when it ends.
• saves the drafts (tiddlers being edited) in wiki/tiddlers/drafts/, ignored by git.
• renames a tiddler stored in two files (a file copied by hand, or the same tiddler created on both sides of a git merge): the one of the second file gets a title prefixed by Duplicate - , and is saved in a new file. Keep one of them and delete the other.
• in the browser, keeps what you are typing when the server version of the tiddler changes at the same time (the browser sends its changes at most once per second): your version wins.

If there is any problem during the synchronization, the notification says so, and the output of git is in the $:/temp/tiddlygit/synchronize tiddler.

Don't hesitate to create an issue if you have a question or if there is any problem.

update.sh also installs some plugins I use: Relink, Edit Buttons by telmiger, and Shiraz, Trashbin and Commander by kookma.

Merge and Conflicts

The tiddler files (.tid and .json) are merged field by field by tiddly-merge.py:

• a field changed on one side only takes the value of that side, even when both sides changed other fields,
• when both sides changed the text, it is merged line by line,
• when both sides changed the tags (or the list field), they are merged tag by tag: the tags removed by one side are removed, the tags added by both sides are kept (in your order, the other ones are inserted after the tag preceding them),
• conflicts on these unimportant fields are ignored (your value is kept): "modified", "created", "nouvelle-tache", "fields-to-show", "date-start", "date-start-temp", "days-count", "days-count-temp", "show-day-record".

Then, any tiddler with remaining conflicts will be duplicated (one for each version), with added link at the bottom in each one referencing the other one, they will also be tagged with the GitConflict tag. You can then simply remove/edit these tiddlers before committing/pushing again with the GitHub button.

TiddlyWiki saves a tiddler as .json when one of its fields has leading or trailing spaces. The tiddlygit plugin then keeps it as .json, and synchronize.sh merges a tiddler saved as .json on one side and as .tid on the other.

synchronize.sh also resolves the conflicts git cannot give to the merge driver (tiddly-merge.py --resolve-conflicts):

• a tiddler deleted on one side and modified (or renamed) on the other is kept, tagged GitConflict, with a note saying who deleted it,
• a tiddler renamed differently on both sides is a conflict on its title: your title is kept, and the other one is a new tiddler tagged GitConflict,
• a binary tiddler (like an image) changed on both sides keeps your version, the other one is saved as a new tiddler tagged GitConflict.

When a conflict is left anyway (on a file that is not a tiddler, like tiddlywiki.info), the merge is cancelled: your wiki stays as it was, and the synchronization fails with the list of the conflicting files, to merge by hand.

The other .json files are merged structurally: objects key by key, arrays element by element and strings line by line. When this is not possible (conflict, or formatting that cannot be kept), they are merged line by line like git does by default.

Some particular tiddlers are also completely ignored, see the .gitignore file.

Dependencies

• git
• nodejs
• npm
• python3
• libnotify

On Archlinux

sudo pacman -S git nodejs npm python libnotify

On Ubuntu

sudo apt install git-all nodejs npm python3 libnotify-bin notify-osd

Test

To test that everything will work correctly on your computer, run the command ./test.sh. It tests the version committed in your repository (not your uncommitted changes), in a copy in test_tmp/: your repository and its remote are not changed. It should end with these messages:

✅ Merge driver generated the following files:
wiki/tiddlers/test-3f2a9c1e.tid
Deleting /home/dionisos/projets/programmation/tiddlygit/test_tmp
✅ Test completed successfully
🧹 Cleaning up...

The test server listens on port 7171, so test.sh can run while your own server is running.

If the test doesn't complete successfully, please verify the dependencies, and then create an issue.

The server and the tiddlygit plugin are tested with node --test tests/. The end-to-end test in a real browser (tests/firefox.test.mjs) needs Firefox, it is skipped when Firefox is not installed.

Installation

Preferred way:

git clone https://github.com/dionisos2/tiddlygit.git Then modify tiddlygit/.git/config to change the remote link.

or

Fork https://github.com/dionisos2/tiddlygit on GitHub Then git clone your_fork

Then

cd tiddlygit
installation.sh

Update

Once, add TiddlyGit as the upstream remote:

git remote add upstream https://github.com/dionisos2/tiddlygit.git
git remote set-url --push upstream no_push

The second command disables pushing to TiddlyGit, to avoid pushing your wiki to it by mistake (./upgrade.sh also does it). If you cloned TiddlyGit directly, make sure origin no longer points to TiddlyGit (git remote -v) before synchronizing: the GitHub button pushes to origin.

Then, to update, commit (or synchronize) your changes and run:

./upgrade.sh

It asks before stopping the server (and does nothing if you refuse), merges upstream/master, then runs update.sh to install the new version of the plugins. Restart the server afterwards. If the merge fails, resolve the conflicts, commit, then run ./update.sh and restart the server.

On the other computers using the same wiki, the update arrives with the next synchronization, but the plugins are not versioned: run ./kill_tiddlygit.sh, ./update.sh, then restart the server.

The plugins TiddlyGit no longer uses (like TW5-Bob) are removed by update.sh and by the synchronization (see remove_old_submodules.sh). Their directory is kept, but ignored by git, when it has changes or a repository of its own.

If git refuses to merge unrelated histories

If your repository was created from a copy of TiddlyGit instead of a clone or a fork, it doesn't share its history with TiddlyGit, and git merge upstream/master fails with fatal: refusing to merge unrelated histories. Join the histories once, after committing your changes and stopping the server:

git fetch upstream
git show upstream/master:join_upstream.sh > /tmp/join_upstream.sh
bash /tmp/join_upstream.sh upstream/master
git diff --cached HEAD
git commit
./update.sh

join_upstream.sh:

• converts your wiki to the wiki/ folder if needed (see below) and commits it,
• merges TiddlyGit, keeping your wiki but taking the TiddlyGit files (scripts, plugin, merge driver...) from upstream: if you modified some of them, check git diff --cached HEAD before committing,
• removes the files TiddlyGit deleted (like the Bob plugin), unless you modified them,
• merges .gitignore, and adds the tiddlers TiddlyGit needs if you don't have them (SynchronizeButton, SynchronizeButtonImage, GitConflict).

The merge is left uncommitted so that you can review it. Later updates are normal merges (git merge upstream/master).

Coming from the old Wikis/<name>/ layout (Bob plugin)

The wiki is now in the wiki/ folder (wiki/tiddlywiki.info, wiki/tiddlers/), like a basic node.js TiddlyWiki. To convert a wiki in the old layout, stop the server (pkill -f "tiddlywiki.js Wikis/"), commit your changes, then:

git fetch upstream
git -c merge.directoryRenames=true merge upstream/master
./convert_wiki_folder.sh
git commit -m "Move the wiki to wiki/"
./update.sh

merge.directoryRenames=true lets git move the tiddlers you created to wiki/tiddlers/ during the merge. convert_wiki_folder.sh then:

• moves the remaining files, including the local files ignored by git (PersonalConfiguration.tid, $:/config/...),
• keeps your tiddlywiki.info,
• removes what belonged to Bob (settings/, $:/Bob/EditingTiddlers...),
• makes your SynchronizeButton use the server instead of Bob (your tags and other changes are kept), and lists the other tiddlers using Bob's websocket messages,
• updates .gitignore.

If you have several wikis in Wikis/, give the one to convert: ./convert_wiki_folder.sh Wikis/<name>. If a file exists in both folders with a different content, the script stops without changing anything and lists these files. The git history of a tiddler is still available with git log --follow wiki/tiddlers/<file>.

Usage

Only having to enter your credential one time

If you use SSH to connect to your repo (preferred way)

eval "$(ssh-agent -s)" && ssh-add ~/.ssh/id_rsa

If you use your login and password to connect to your repo

git config --global credential.helper store

This will create a ~/.git-credentials where the password is in clear (https://git-scm.com/docs/gitcredentials).

If you don't want your password to be stored in plain text, you can use this instead:

git config credential.helper 'cache --timeout=3600'

Starting the server

Then start the server with run.sh and open http://localhost:7070/ in a web browser. (you can change the port with ./run.sh port=8080)

While you are working on your wiki, you can click the button with a GitHub icon to synchronize your work with the remote repository.