Open source tech

From low tech network

(Difference between revisions)

Latest revision as of 21:44, 11 August 2015

tech

Most of this page is about open source software. Many of the packages listed are fast and light on system resources but can take time to learn how to use.

linux

Debian
Arch Linux rolling release. with many options on installation
DistroWatch.com information about the different linux distributions.
UNetbootin Software to create bootable usb linux system. Runs also on windows and mac.
The Linux Documentation Project

window manager stuff

Openbox very basic window manager - you can install additional features yourself, making it very customisable but not working well straight out of the box. You will probably need to install tools for power mangement, networking, managing displays, etc.
urukrama's Openbox keybindings
awesome tiling window manager - meaning windows are arranged with the keyboard into tiles. Good for if you dont like to use the mouse and also good for making the most of display space.
Comparison of Tiling Window Managers - ArchWiki

shell

terminator terminal emulator with many features, including tabs and split windows

Some useful keys

Ctrl+Shift+O Split terminals Horizontally.
Ctrl+Shift+E Split terminals Vertically.
Ctrl+Shift+Right/Left/Up/Down Move parent dragbar.
Alt+Right/Left/Up/Down Move to different terminals

tmux terminal multiplexer - for mouse free control of terminal windows
commandlinefu.com
zsh the zsh shell. an alternative to bash with some interesting features. some examples of using zsh

with zsh, instead of cd you can use z with part of the name of the path you want to move to, favouring recently and frequently used paths. another useful one is being ables to type ls **/*.txt to find all txt files deeper in the directory structure.

zsh-lovers man page
pretzo a configuration framework for zsh
30 handy bash aliases

web

vimperator vi style control of firefox/iceweasel web browser. So you can control the web browser without using the mouse.
Surfraw Shell Users' Revolutionary Front Rage Against the Web - open websites from the commandline

You can alias sr="surfraw" and do sr wikipedia chard to jump to wikipedia article about chard. See the list of elvi (website search scripts for surfraw).

tools, toys, reference

conky system monitor
yadr yet another dotfile repo. a bundle of configurations for vim, zsh, ruby and git. The idea is that rather than spending a lot of time configuring software, a collabourative effort is made to bring together sane configurations and plugins etc.
aap makes it easy to download, build and install software
calmar.ws tips and info on linux,vim, vimperator, awesome, other stuff
refcards.com free quick reference cards
fcron.free.fr apparently good for running cron jobs on a system not always switched on.
OpenVZ container based vitualisation for linux.

vim text editor

Learning vi or vim takes a while. But if you edit a lot of text files it is worth learning. The keybindings in vim are used in a lot of other programs, and vim is installed on a lot of different systems which makes it a very useful thing to know, even if you just know the very basics.

file management

ranger terminal based file manager with vi style keybindings and many features including 'rifle' file opener, previewing, tabs, bookmarks, tagging. ranger is not a two pane file manager like mc or vifm, but uses 'miller columns' showing the directory hierarchy (similar to mac os 'finder'). The centre column is active, the left column the higher level directory, and the right column shows either a deeper directory or file information or preview.
mc Midnight Commander - 2 pane terminal based file manager (norton commander style)
vifm vi file manager. Two pane and with vi style keybindings
xfe X File Explorer - light file manager for x-windows. Can have 2 pane and directory tree. Some norton commmander style keybindings.
gnome commander File manager for gnome, norton commander style.
gentoo Light, fast 2 pane file manager
SpaceFM multi-panel tabbed file and desktop manager
Baobab gnome disk usage analyser - visual representation of disk usage - great for when you are trying to free up space
ncdu - NCurses Disk Usage, a simple, fast program for analysing disk usage.
sshfs mount a remote filesystem via ssh

email

mutt - email program. Its not for everyone, it takes a lot of configuring but once you have got it right it can work well and save time.
Steve losh's mutt guide
mu (maildir-utils) program for searching email stored in maildir format.

To index your emails, mu index --maildir=mymail You can do this for several mail directories, and you may want to set it up as a cron job to periodically index your new emails. mu find cabbage will show emails about cabbage. You can also bind mu find to a key in mutt, and have the results stored as a temporary maildir, which can accessed in mutt: (from mutt wiki)

# Mu
# first macro does the search according to user query, second macro jumps to result list
macro index <F8> "<shell-escape>rm -rf ~/.mu/results; mu-find -o l -l ~/.mu/results " "mu-find"              
macro index <F9> "<change-folder-readonly>~/.mu/results\n" "display mu-find results"

notmuch program for searching email
mutt sidebar patch for mutt with folders listed in a sidebar
getting up and running with a sane mutt setup
realprogrammers.com mutt guide
How I tamed my mutt – cmdln.org (a sysadmin blog)
fefe.da Mutt FAQ
Mailman GNU Mailing list manager

non-commercial email providers

These email providers are non-commercial groups which take security and privacy seriously. Please add more

graphics

gimp gui graphics editor (like photoshop)
inkscape gui vector graphics editor (like corel draw)
feh a fast and light image viewer

---using feh---
left - previous image
right - next image
up/down - zoom
/ - zoom to fit window 
d - toggle display filenames
Ctrl + delete - delete file
m - menu

imagemagick powerfull tools for converting, editing and composing images. Good for batch image processing.
graphicsmagick a fork of imagemagick - apparently more efficient.
G'MIC GREYC's Magic for Image Computing - an image processing framework. Commandline tools, Gimp plugin and C++ library.

document and book viewers

MuPDF - a fast, lightweight pdf viewer with some vi style keybindings.

 f = toggle fullscreen mode
 W, H = zoom page to exactly fit width or height of window
 <, > = skip back or forwards 10 pages

apvlv - another pdf viewer, vi style.
calibre - ebook library manager
FBReader reader for epub etc

electronic books

gen.lib.rus.ec library genesis. Huge book collection. Particularly a lot of science type stuff. Based in russia, with many mirror sites, it is also possible to set up your own mirror or offline repository, the SQL database dump can be downloaded. Some ISPs are beginning to block library genesis, which means you need to find a mirror which is not blocked, or access it through TOR or a VPN.
Project Gutenberg One of the oldest free ebook collections on the net. 46,000 titles.
internet archive also hosts many books.
http://clockwise3rldkgu.onion/ clockwise libraries - a good ebook library online available through TOR.
resrc.io list of free programming language resources
DIY Book Scanning a community devoted to DIY/low cost book scanning hardware and software
Book scanning wikipedia article
openlibrary.org ebooks

backup/archiving

rsync - a powerful backup/file copying tool. rsync only copies differences in the files, to minimise the amount of data being transfered. This means it responds well to transfers which are interupted and restarted. Commonly used options:

 rsync dir1/ dir2  =note the slash to put contents of dir1 into dir2 (often what u want)
 -a  =archive mode- recursive, preserve times, permissions etc (often what u want)
 -v  =verbose
 --progress  =show progress
 -n  =dry run (dont actually do it, just a practice to see what would happen)
 --delete  =delete files in the destination that were not in the source
 -u  (update) skip files that are newer on the reciever.  this can be run in both directions to syncronise

Grsync - rsync GUI interface frontend for Linux
unison file syncroniser. Unlike rsync, unison ensures both filesystems have the newest version of a file. tutorial on setting up unison

diff -qr source destination     # q = brief r=recursive 
comm -3 <(ls ~/dir-new/) <(ls ~/dir)        # compare two directories

Find duplicate files:

find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

Find and delete duplicate files:

find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d |  xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate | cut -f3-100 -d ' ' | tr '\n.' '\t.' | sed 's/\t\t/\n/g' | cut -f2-100 | tr '\t' '\n' | perl -i -pe 's/([ (){}-])/\\$1/g' | perl -i -pe 's/'\/\\'\/g' | xargs -pr rm -v

network attached storage (NAS)

nas-central.org wiki about hacking NAS devices.

booklet printing on linux

some tools, for example, for reordering pages of an a4 documents to make an a5 booklet or zine.

pdf2ps                                              # convert a pdf document to postscript      
psbook in.ps out.ps                                 # reorder the pages to make a book,  
psnup -s1 -2 book.ps booklet.ps                     # shrink to 2 pages on each A4 side
ps2pdf in.ps out.pdf                                # convert it back to a pdf

If you dont have a duplex printer probably you want to print the even pages, put them in the paper drawer (the right way around! very confusing!) then print the odd pages

lpr -P myprinter -o page-set=even -#1 infile.pdf    # print the even pages  
lpr -P myprinter -o page-set=odd -#1 infile.pdf     # print the odd pages

encryption

cryptoparty handbook
gpg Gnu Privacy Guard. Tool for encrypting files and email.

Most simple usage - gpg encryption of a single file:

 gpg -c file

This prompts for a password and gives file.gpg. To decypt:

 gpg -d file.gpg > file

Disk encryption article on ArchWiki a good explanation of disk encryption and a comparison of the methods.
truecrypt

tools for those who have little or no connection to internet

kiwix wikipedia offline reader - read wikipedia when not connected to internet. The english wikipedia (text only, no images, user pages or discussion pages) compresses to about 8gb! You will need to download this data in 'zim' format, available as a torrent. list of wikipedias
openstreetmap collaborative mapping project. it is also possible to download openstreetmap (even the whole planet) for offline viewing using a software called navit (which i still havent succussfully used), other software projects are Gosmore, Emerillon and Marble. It seems that rendering map tiles from a downloaded map takes a lot of computing and rendering software tends to be designed for servers. I have not found downloadable pre-rendered tiles for easy, fast offline viewing. Any ideas? I would be great to have quickly accessible offline maps.
OfflineIMAP software for syncronising imap mailboxes. It is useful for unreliable or slow internet connections or when you want to read email offline.
hitchwiki.org can also be downloaded to view offline hitchwiki while offline. The database is downloaded in XML format which can be viewed with various kinds of dictionary viewing software.

offline dictionary for linux

goldendict which can work with freedict dictionaries. To add, for example, scottish gaelic to german dictionary (on debian):

apt-get install dict-freedict-gla-deu

for a list of all dictionaries (on debian):

apt-cache search dict-freedict

Ding A very good German-English dictionary lookup program

filesharing

torrent411 good torrent tracker (french) requires registration and has advertising but some good stuff on there. 'Télécharger'='Download'
rutracker.org good torrent tracker (russian) also requires registration.
zeropaid filesharing news and forum
myananamouse.net torrent site specialising in english ebooks and audiobooks. it is a private site with a complicated regestration process and many rules, but there is a lot of books on there.
tracker2.postman.i2p i2p torrent tracker (you must be running i2p to access)
imule p2p sharing software for i2p (also an i2p eepsite)
eiskaltdc++ and linuxDC++ are linux clients for the Direct Connect filesharing protocol. uhub is hub software (server).
soulseek qt music sharing network client software. nicotine is a soulseek client written in python

audio software

hydrogen audio forums about audio software etc
rockbox open source firmware for music players, eg. ipod. Very useful if the original firmware wont play certain formats, or has restrictions on copying music off the player. Can run on Sansa clip.
mpd music player daemon. music playing software designed to be run over a network. If there is a network attached computer connected to an amplifier and running mpd, other computers on the network can control the player by using mpd client software (of which there are many to choose from, i use ncmpcpp). Can run on raspberry pi. Client and daemon can also both be running on the same computer. The database is very fast to update and search.
cmus curses based music player software (linux, console based). with playlists, play queue, filtering, vi style search and command mode, dynamic keybinding.
moc another, simpler, curses based player. moc has two panels, with a similar look to midnight commander. On the left panel is a file browser and the right is a playlist.
randomplay even simpler console player. plays random tracks.
DeaDBeeF GUI music player. Lightweight, extendable with plugins, and has tabbed playlists and a design editor similar to foobar2000 (a very good windows music player).
beets music collection organising tool. useful for tagging and organising a large collection.
gmusicbrowser jukebox software designed for a large music collection
Linux multimedia studio (LMMS) free music production software, with VST compatibility.
sox command line sound tool. can be used as simple synthesiser, and to convert audio formats.
Mumble voice chat software. Mumble was originally developed for chat whilst gaming and is now also popular for conferencing etc.
audacity free audio editor and recorder.
Mixxx Open source DJ software. For monitoring (listening to the next track to be played on the headphones) it is best to have a soundcard with multiple outputs. Many soundcards have surround sound, meaning you can use the 'front' channels for output and the 'rear' channels for monitoring. A cheaper way to do it would be to use two soundcards, for example your computer's on-board card for main output and a cheap USB card for monitoring. An even cheaper way might be to use the left channel for ouput and the right channel for monitoring. Mixxx also works with various types of midi hardware, so you can control pitch, cueing, equalisers and other stuff with real twiddley knobs and sliders.
xwax: Digital vinyl on Linux Open source software for DJing using time-coded vinyl (like Serato scratch). To control the playback of a digital track using a record player. xwax can work together with Mixxx.
tweakheadz has guides on making music with software and hardware

mps-youtube terminal based youtube player and downloader. very good for finding and downloading music fast, without the clutter of the youtube site.
ccmixter.org online community for musicians, hosting a lot of creative commons lisenced audio.
soulseek qt music sharing network. nicotine is a soulseek client written in python

to take 30 random mp3s and put them in a directory (i use this to randomly choose some of my music to put onto my mp3 player):

find -name "*.mp3" | sort --random-sort | head -n 30 | xargs -n 1 -i^Cd '\n' cp {} /media/mp3player/randomtunes

audio hardware

diyaudio.com forums about diy audio hardware projects
tweakheadz has guides on making music with software and hardware
vintagesynth.com online resource for vintage synthesisers and other audio hardware
Standuino DIY open source audio hardware
speakerplans.com designs for speaker enclosures
horn response - speaker enclosure design software. can use with abaca? audio abacus

usenet

tin text-based usenet newsreader software
eternal september Free text only usenet provider (registration required)
aioe free text only usenet provider (no registration required)

other

Many of these links need to be (or already have been) put elsewhere on the wiki

mighty ohm some open source hardware designs and a soldering guide in many languages
Desoldering - wikipedia article for learning how to recuperate electronic components.
Desoldering wikibook
circuit circle / SCHRÄGERUNDE DIY electronics and circuit bending -workshops, concerts, installations, performances
Toner refill - wikipedia article on refilling photocopier laser printer toner

freemusicarchive.org
internet archive archive of internet sites, also many media files hosted here, you can put media files here too.
Use google translate from the command line with some extra features.
Google translate awk program very similar
open frameworks c++ library for creative projects
RONJA free technology project for optical point-to-point data links (using visible light). The design has a reported 1.4km range and 10Mbps full duplex data rate.
modulatedlight.org information and experiments using light for data links, mostly using high power LEDs rather than lasers, as they work better over long distances.
ttfa.net/12volt
A Tao of regular expressions notes on using regular expressions (for vi, grep, etc)
mattmahoney on data compression much info on data compression. PAQ8 text compression, while computationally expensive, could be very useful for low-bandwidth radio data networks.
pbx
x25, a kind of packet switched network
skype like conferencing system software project proposal. see also ZRTP and SRTP
sdf.org sdf public access unix system
retroshare secure, decentralised communication and filesharing platform
nmap network ping scan tool
wikidata - project to put wikipedia information into a semantic format, linking topics across languages and categories.