In addition to reading a lot of books, I read a lot of news. I love KOreader, and like to read using that tool as much as I can. I've experimented with a lot of tools: Pocket, Wallabag, Calibre, Calibre2OPDS, COPS and many more. But none of them provided the simple, seamless integration of my reading list with KOreader that I desired. So I pieced together my own using some really nice open source tools.
The tools:
Syncthing
I use
Syncthing to sync my books between my computers and my devices running KOreader. I also use it to sync a lot of other things between devices. Syncthing is open source, peer-to-peer (no server required) sync software available for a wide variety of platforms. Even if you aren't interested in the "Read It Later" solution I describe in this post you should consider using Syncthing to sync your device(s) running KOreader. There is an Android app,
instructions for Kindle Touch and just this week, thanks to
tshering,
a simple installer for Kobos running KSM. It should be fairly easy to put Syncthing on other e-reader devices. Even if you can't or don't want to install Syncthing on your device, you can
use Syncthing for a very easy USB sync solution.
Five Filters
Five Filters offers a variety of content-related tools that may be of interest. The one I use most heavily (and the one used in my "Read It Later" solution) is called "
Push to Kindle". Don't worry, despite the name a Kindle is not required. If you submit the URL of web page to this tool, "Push to Kindle" creates a nicely formatted .epub, .mobi or .pdf which can be emailed to your Kindle device (hence the "Push to Kindle" name) or downloaded to your computer. (Note that if you prefer to run "Push to Kindle" on your own server, an open source release is coming soon.
Pandoc (optional)
Pandoc is an open source document converter. It is a very powerful (albeit complicated) tool. I use it as a backup to the Five Filters downloads, since for some unknown reason images are stripped from the Five Filters epubs. For some of the websites I follow, the images are very important (eg, financial charts) so I use Pandoc to generate epubs for them. The downside is that the outputted epubs are not nearly as pretty as their Five Filters counterparts. I'm sure that this could be fixed with stylesheets etc but I have not looked into this. Again, using Pandoc is entirely optional. It is available for
a wide variety of platforms. If you need to install from source (this won't apply to most people), I recommend
creating a "relocatable binary".
A Simple Script
Here is a simple script I wrote to use these tools together:
Code:
#!/bin/bash
# a simple script to download an epub version of a given web page from http://fivefilters.org/kindle-it/
# or (optionally) generate an epub version of the given web page using Pandoc (http://pandoc.org/)
# change the next line to the absolute output path where you would like the epub to be saved inlcuding the trailing '/'
savepath="$HOME/Documents/"
# OPTIONAL: the absolute path to the list of domains for which you want epubs with images (less pretty output)
# Use one fully qualified domain name (https://en.wikipedia.org/wiki/Fully_qualified_domain_name) per line.
# Pandoc must be installed to use this feature.
pandoclist="$HOME/.config/pandoclist"
now=$(date +"%s") # store the current time
url=$1 # store the input URL
furl=${url#*://} # remove the 'http://' or 'https://' from the input URL
domain=$( echo "$furl" |cut -d/: -f1 ) # get the domain for checking against Pandoc list
# the next line contains the options to pass to Five Filters
durl='http://fivefilters.org/kindle-it/send.php?context=download&format=epub&url='
durl+=$furl # construct the full URL of the epub request URL
oname=$(basename $url) # save the last part of the URL, which we will use to name the epub
oname="${oname%.*}" # remove the file extension (eg .html)
oname+=-"$now" # add a timestamp to prevent overwriting of files with same name
oname+='.epub' # add the .epub file extension to the output name
opath=$savepath$oname # define the absolute path to the output file
if grep -Fxq $domain $pandoclist # check for match in the Pandoc list
then
pandoc -r html $url -t epub -o $opath # generate the epub and store it in the specified directory
else
wget -b -q $durl -O $opath # download the epub and store it in the specified directory
fi
Putting It All Together- Install Syncthing on a computer. Optionally, also install Pandoc on the same computer.
- [Install Syncthing on your KOreader device(s), or set up your Koreader device(s) for simple USB sync.
- Configure the folders to be synced between your computer(s) and your KOreader devices(s). See http://docs.syncthing.net/intro/getting-started.html. I use one folder ("Books", with subfolders) for my books, and another folder ("News") for epubs gathered with the above tools.
- Put my simple script on your computer and make sure it is executable. Make sure you edit it to set where the epubs should be saved (this should be the same as one of your synced folders), and optionally, the location of your list of websites for which Pandoc should be used instead of Five Filters.
- Now test it. From the command line, in folder where your script is:
./<name of script> <URL of web page>
- Assuming it is working as you like it, set up your browser, RSS aggregator etc to pass a URL to the script with a simple keyboard shortcut. This is left as an exercise for the reader.
Now, at the press of a couple of buttons on your computer, any URL you desire will be turned into an epub and automatically send to your KOreader device(s).
Enjoy! Suggested improvements or alternative approaches welcome.