Nicolas314

All my geeky stuff ends up here. Mostly Unix-related

Camels

leave a comment »

camels

I read somewhere in a math history book that numbers were actually invented to count camels. Someone wanted to send over a herd of camels to be sold on a market on the other side of the desert and they did not trust the camel escort. How would the receiving party know if some camels had not been stolen on the way? So they used a fairly simple principle: line up your camels, put one pebble in front of each. Gather the pebbles, put them in a small jar, burn the cork, hand it over to the escort.

On the receiving end, break the jar, put one pebble in front of each camel. You will know immediately if camels are missing.

This apparenly went on for a while, until someone figured out that instead of lining up pebbles and camels you could shorten the process by writing signs on the jar to indicate how many pebbles were inside. On the receiving end you just had to look at the signs and compare to what you saw. In case of doubt, break the jar and line up pebbles and camels. And then it was just a matter of time until somebody noticed you don’t need the pebbles and the jar. Just cook a clay tablet in an oven with a text indicating how many camels you are sending.

I have no idea if this story is true or not, but I like the way it stresses the breakthroughs that have happened. Going from a bijection pebbles/camels to a bijection in camels/signs was brilliant. I expect the first attempts were likely to just draw plain strokes on the jar, as many as there were camels in the herd. The next breakthrough was simplifying a whole bunch of strokes into a single sign, e.g. using a hand to signify the number 5. And the last one was to realize that the jar and pebbles were unneeded.

Another shift that amazes me to this day is how money actually works. The first currency tokens had actual value, they were made of metal you could melt and use if you so wanted. When the first bank notes were introduced, they switched from actual value to a potential: the note said that you could obtain real metal if you were to exchange that note in a bank.

We now live in a world where I can pay my lunch by waving a piece of plastic over a radio-equipped terminal connected to a bank. My plastic contains numbers that cannot be found on any other credit card, which are used to authenticate me. Now my bank makes a promise to pay my meal to the restaurant’s bank. No metal or paper changes hands.

Since a few years, things are shifting again. Instead of waving a credit card containing my unique account identification numbers, I can now use a mobile phone that contains a series of numbers that are only valid for myself, my account, for today, and for limited amounts. This is what they call tokenization and the reason it is booming is that it is a lot simpler to store temporary tokens with limited value than long-term banking credentials with unlimited powers. Security needs not be that high, though you still need to be able to authenticate account owners in a very secure way, but there are plenty of ways to achieve that.

Among the strongest methods we know today to authenticate someone, the most popular relies on the fact that you cannot split a big number into a multiplication of primes. If you tried with a gigantic computer, it would require more heat to power than is available in the universe.

We have come a long way since camel-counting.

Advertisements

Written by nicolas314

Wednesday 1 November 2017 at 11:37 pm

Posted in fun

Tagged with , ,

I am not your daughter

leave a comment »

sleepless

You called me quite late. Some time during the middle of the night, and for whatever reason I had left my phone on. I picked it up and was greeted by your anxious voice:
– Isabelle, is that you?
Part of my brain was still actively dreaming, but the part that was emerging found the idea preposterous. Do I sound like an Isabelle? I croaked:
– No Madam, this is not Isabelle.
– Oh come on Isabelle, it’s Mum. Stop playing fool with me, I recognize your voice, please..
Even half awoken, the tension in your voice was definitely noticeable. You wanted to talk to your daughter and nothing would stop you.
– Madam, I can assure you I am not your daughter. In fact I am a man and my name is Nicolas.
– Bullshit! Isabelle, talk to me!
Your old lady’s tone left me no choice but to obey, so I gave up and decided to play along.
– Alright Mum, you got me. What’s up?
You seemed surprised. Apparently Isabelle does not give up so easily when playing this game. But the sudden joy of being able to talk to your daughter was so great that you could not help it. You started talking about your neighbours at the retirement home, how the nurses were treating you, and had many complaints about the food and such. I listened very carefully at first and quickly dozed off, we were half way during the night after all.

You called again a bit later, and again, and again. We spent our night like this: you finally talking to your daughter, and me sleeping through 10-minute intervals. Finally one of your nurses must have found out you were secretly phoning at night and you stopped calling.

You never called again. I hope you found the right number for Isabelle and she takes good care of you.

Written by nicolas314

Monday 2 October 2017 at 9:42 am

Posted in Uncategorized

Long live NAT!

leave a comment »

ipv6-no-thanksHome networking can be a lot of fun: setting up a name service, a guest network, or traffic rules, leads to an endless joy of discovering new RFCs or creativity in the very active field of artistic configuration file syntax.

I thought I had seen everything until I tried to set up IPv6 connectivity for my home network. Little did I know that this would eat up so many of my precious free evenings. The following writeup is here to remind me never to try that kind of shit ever again, and as a warning to future generations who might want to dig into this kind of topic. Life is short, there are many better things to do than attempt to set up a new addressing scheme for your home network. Long live the NAT king!

The Start

It all began when I noticed that my ISP provided me with a unique (native!) IPv6 prefix to use on my home network. Something like:

2001:1234:5678:9abc::/56

Since I was not familiar with IPv6 addresses, it took me a while to find out that the first 64 bits of a 128-bit IPv6 address designate a network, and the last 64 are reserved to differentiate hosts on that network. My provider handing me a /56 means I have 64-56 = 8 bits to play with, i.e. I can instantiate 256 home networks, each having up to 2^64 = 18,446,744,073,709,551,616 hosts. Overshot a bit, maybe.

So where do I start? Do I have to install specific software? Where? Do I need to buy specific hardware? How many services are needed? And thereby started my long painful descent into the horrific world of IPv6. Toss and loose 1d20 sanity points immediately.

My ISP unfortunately did not provide any help as to what I am supposed to do with the IPv6 thingie they gave me. No single help page, very few discussions on their forums, and all exchanges I had with customer service were completely useless. Best I could find were discussions between customers of an ISP in the US that provides a similar setup. That is thin.

Say you received a /56 prefix from your ISP. If that prefix ever changes e.g. because you switched to a new ISP, you want things to work automagically because that is the way things currently work with IPv4: changing my public IPv4 address does not change anything to my home network.

In order to do that, IPv6 suggests that home networks use two sets of addresses: the public ones derived from the ISP-provided /56, and another private address space based on something else called a ULA (Unique Local Addresses). You get to choose your own ULA on your home network(s), preferrably based on a good random number generator, but nothing prevents you from taking something like fc00:caca:caca:caca:caca::/48. If anybody else on the Internet picks the same network prefix you will get into trouble when trying to get intimate with each other, e.g. by establishing a VPN between both worlds. We had exactly the same problem when trying to join two sites using IPv4 NAT’d 10.0.0.0/8 subnets, so this is not really a regression. Fun fact: if you have no ULA in France you can always say “Il manque ULA sur mon réseau”.

How do you get to choose this ULA? If you happen to have a single router on your home network it should just be a matter of digging through the router IPv6 setup until you find it. But most home networks are now running multiple routers that are all unaware of each other, and all convinced they are masters of the universe. You will most certainly end up with several ULAs. Some of your devices will get several addresses and you will have to understand your own network topology to know which address to use to access them. Prepare for glorious hours of debugging, which is particularly great when facing addresses that are mostly made of bloody random bits.

Why several routers on the same home network? Simply because you may be running several DSL connections, or maybe you have a VPN started somewhere away from your edge router, or maybe you connected your smartphone and it offers another potential exit to the Internet. You also get a virtual router when you start virtual machines on a desktop.

To make things simpler, every network interface on your machines will also generate a local address that is only valid for its closest neighbours, called a link-local address. Unfortunately you won’t go far with that one as it is not supposed to cross boundaries. Think of it as a 127.0.0.1 that extends to the other side of the cable but not further.

Ok so we have now several adresses for each machine on the network.  Figuring out which one should be used (incoming or outgoing) is just an unspecified, incredible mess. The link-local address can only be used on very specific physical links, the ULA address cannot be routed to the Internet, and the public addresses you have may change at any moment, e.g.  through your smartphone sharing a 4G access.

At that point we have just determined that your printer currently identified as ‘printer’ also known as 192.168.1.20 in IPv4 will now be accessible as:

– fe80:bffa:3d5f:5f8d:b4cf:1749:b01c:5b2f for machines directly connected to it through an Ethernet cable
– fc00:c465:3b76:b34d:38f7:da19:2586:1cbd for machines living on the same internal network.
– 2001:61af:ff44:b148:4fc3:0097:f35d:c806 for machines on the internet when reached through a first ISP, and another public address for each available ISP connection.

Oh joy.

Of course normal human beings are not meant to remember this kind of random shit. For this kind of thing you have DNS.

DNS you said? What DNS?

There are really two ways machines can obtain an IPv6 address: SLAAC and DHCPv6. SLAAC means Stateless Address Auto Configuration, whereby a machine obtains a prefix and derives its own IP address from it, e.g. based on its own MAC address. Cool, right? You do not have to assign individual addresses in static DHCP leases, every machine does it on its own. But then: how do you know which address was self-assigned by your very smart printer?

There are dedicated neighbour-discovery protocols for that, but they are mainly designed to make sure that addresses are locally unique and routers know where to find them. This is only taking care of establishing a link, there is nothing dedicated to associating a name to a self-assigned IP address. And if there was, how would you know who to believe? If two machines on the local network claimed to be ‘joe’, what should happen?

To be fair, there are solutions like Bonjour, also known as zeroconf, but they are unlikely to work on lightweight or old devices. Shoot again.

Back to square one: if you want to reach your own machines using human-usable names you need to run DHCPv6, a protocol that was designed to compensate for such things. And there you go: back to static leases, addresses assigned by a router, attached to a name, and you end up doing exactly the same kind of shit you used to do with IPv4 local networks, except this time the addresses are much easier to screw up.

Even worse: if the self-assigned IPv6 addresses are not related to MAC addresses, it means every single host on your local network will have generated its own random address, forcing you to manually harvest them from all devices. But you know how to do that on your connected toaster, right?

What’s in it for the average home network user? Pretty much nothing. The fact that every single one of your home devices has a potentially reachable address on the intertubes is downright scary. Internet service is for internet servers, not for sensors and other IoT bullshit. First thing you will want to do is bullet-proof your firewall to make sure nobody but you can access your printer from the Internet, and hope things are Ok with your IoT shit.

The story did not just end up with me reading thousands of pages on the Internet and a couple of paper books. I hacked every single computer in my house to run IPv6, starting with the routers under OpenWRT, LEDE, FreeBSD, OpenBSD, pfSense, OPNSense, and later moving on to all client OS machines: OSX, Linux, Android, *BSD, and even some Windows boxes, blimey.  I instantiated dedicated DHCP and DNS servers, configured static addresses, automatic ones, bridges and NATs and firewall rules and what-have-you, and I ended up with some machines working under IPv6 only, some under IPv4 only, some that could use both stacks, and some (a lot) that were just unreachable no matter what. Yeah, I also crashed my Internet access several times. Omelet and eggs.

Let me try to put it this way: some of my home machines are servers, e.g. a NAS or a printer. I want to be able to print on ‘printer’ or mount a share on ‘NAS’ without having to remember random 128-bit numbers. Silly me. Since I want to use names I have to assign addresses myself from a router running DHCPv6. Neither NAS nor printer need to be available to the public. So what did I gain compared to a local IPv4 network? Hmm… Address management is not fun with 32 bits, imagine with 128.

Or maybe I am just old-fashioned, trying to manually assign names to my home machines. This might be an idea for a new product: a router that would automatically identify hosts on the home network and show them on a single web interface, allowing you to assign names and forget about addressing altogether. Might get in trouble when you have several identical devices but I’m sure there would be a way. If such a product exists I have not seen it yet.

On the other hand, if I want to browse the Interwebs in v6, I found out that mounting a SOCKS proxy on a remote cloud box works perfectly well. No need to configure anything, just ssh -D and the IPv6 world is mine to browse.

Summing it all up

Address assignment is not easier than IPv4. Still requires a dedicated DHCP and DNS server, only more complicated to configure. You are facing the tedious task of gathering self-assigned IPv6 addresses from all hosts and copying them onto your DHCPv6 server, hoping the self-assignment method won’t change soon.

Routing is now different, but not easier. New constraints are imposed on knowing which interface to bind to when reaching out to the Internet.

Firewalling the whole thing with a mix of IPv4 and IPv6 might tear you a new one. I can already lock myself out of a router with human-readable firewall rules, I cannot imagine doing the same thing with batshit-crazy addresses and feel safe.

You know what? I will stick to glorious NAT’ing until this mess is sorted out. Good news is that there are many bright people currently working on the topic. All I hope is they eventually come up with something that you and me can use without having to read through a million pages of RFCs, compile obscure daemons, or purchase new boxes as if I did not have enough of them.

Talking about RFCs, this one is trying to gather very sensible requirements about home networks:

https://tools.ietf.org/html/rfc7368

If you have 20 minutes to spare, you should watch this talk:
https://www.youtube.com/watch?v=wQdfWUsG4uI

If you really insist on switching your home network to IPv6, I would recommend reading this rant first:

IPv6 at home (published 2012, still relevant):
http://www.kloepfer.org/ipv6-homenet.html

And to get an idea about how messy it is to get IPv6 configured on Linux:

IPv6 Set up an IPv6 LAN with Linux
https://www.jumpingbean.co.za/blogs/mark/set-up-ipv6-lan-with-linux

In its current state I can only dismiss the current IPv6 definition for home networks as very incomplete and unworkable for non-professionals.  Let’s hope RFC 7368 will be handled by qualified, creative, and pragmatic people.

Til then, there is no place like 127.0.0.1

Written by nicolas314

Tuesday 28 February 2017 at 11:41 pm

My own little farm

with 2 comments

zotac_ci323_03Virtualization is fun! Virtual Machines are nothing new, we have all been using VirtualBox, qemu, or VMWare at some point to try out new stuff, bring up the odd Windows instance to run annoying software, or whatever. At work we use thousands of VMs for millions of things. The hardware price tag is pretty hefty though: if you want to start a reasonable number of VMs on the same racked server you need very large amounts of RAM and disk space, placing it beyond reach in terms of price for home usage.

Not any more! Prices are dropping for heavy machinery faster than the time it takes to look up prices on Amazon. I found this little gem from Zotac and purchased one for a mere 180 euros from a French site:

Zotac CI323

The little beast sports a quad-core CPU, two Realtek NICs, and a whole bunch of USB ports (including two USB3). Add on top of that an extension card for WiFi and Bluetooth. Perfect choice to build a home router in a VM and leave space for other VM instances. You need to add RAM and disk, the box comes empty. I scavenged 8GB RAM and an SSD disk from a previous build and off we go.

It has been a while since I last had a look at virtualization solutions.  Took me several days to look them up individually and find out what they offer. All the solutions I tried are described below.

Option 1: run VirtualBox on a desktop

Install a convenient desktop like Mint or Ubuntu, run VirtualBox on top.  Unfortunately not a very good option as the VMs would not be as close to the metal as I would want. Dismissed.

Option 2: run Linux containers

Containers are neat but they are Linux only. I would like to run BSD and maybe Windows VMs too on the same hardware, so dismissed.

Option 3: Run a bare metal hypervisor

The main options I could find are:

  • VMWare: run VMWare OS as hypervisor, run any OS on top.
  • bhyve (pronounced like beehive), the FreeBSD hypervisor
  • Proxmox
  • KVM: use virtualization routines offered in the Linux kernel. This can be started from any Linux distro and conveniently run pretty much any OS.
  • Xen: use a Xen kernel as bare-metal hypervisor, run any OS on top.

VMWare ESXi was my first choice but had to be quickly dismissed: my box NICs are Realtek and VMWare dropped support for those a few versions back.  Annoying. There are convoluted HOWTOs explaining how to hack the install ISO to add missing drivers and stuff but I do not want to play that game. The whole setup would probably be broken in the following release so no thanks.

I installed FreeBSD 11 and tried out bhyve. Installing FreeBSD on this particular hardware was a real chore: for some reason the integrated SD card reader has driver issues and booting the machine took up to 10 minutes because of a nasty timeout spitting out kernel traces. I finally succeeded in disabling the driver on boot by adding stuff to device.hints after hours of googling and tests. To be fair, I have always faced issues with hardware support on FreeBSD, but to be completely fair: these are the only issues I ever faced. The OS is so polished and professional it is a real pleasure to use. Other parts of the box were immediately recognized and activated: Realtek NICs and the WiFi+Bluetooth (Intel) board.

Anyway: bhyve is relatively easy to learn, documentation is good enough, and it should run any BSD or Linux-based VM without any effort. Running Windows or OSX VMs would probably not be a good idea though. I have not tried but it seems a bit daring. If bhyve offered an easy-to-use GUI I might have stuck with it, but I finally dismissed it because it is still too young compared to other existing solutions.

KVM: the idea would be to install a very small Linux instance and use it to manage VMs on top with KVM. I tried several:

Ubuntu desktop is far too heavy for a “very small Linux instance”. I cannot believe a simple desktop is using so much RAM and CPU. I tried to manually remove stuff after a default installation and broke the machine most completely after having erased ‘evolution’. Forget it.

Ubuntu server is fine enough without GUI, but I would like to have a minimal X11 environment to run VM management software. Unfortunately, as soon as you start adding GUI stuff to an Ubuntu server you start piling up gigs of desktop software you do not want. I could probably figure it out but did not have the patience to do it.

Arch Linux is a royal pain to install. Manjaro (a fairly straight Arch derivative) gets you to a fully configured machine in a matter of minutes.  Problem is: I do want stability on my VM farm and a rolling release is probably not the best choice. Dismissed.

Minimal Debian install worked great. All hardware perfectly supported. And then I tried some KVM tutorials, messed up a bit further with Xen tutorials, and ended up with a completely borked machine. Don’t ask me what went wrong, I just got frustrated of randomly killing processes and rebooting the hardware. There are certainly good HOWTOs out there explaning how to transform a base Debian install into a Xen/KVM server but I did not find them. Dismissed.

Alpine Linux to run KVM: did not try, but seems like a possible option.

I tried Proxmox but the default ISO does not install, it crashes miserably after a few minutes of timeout. I have no idea what is going on, but I dismissed Proxmox at that point and came back to it later. Read on.

At that point I was left with Xen as bare metal hypervisor. I focused on Xen Server, a free Citrix project. The OS is based on CentOS 7 with a modified kernel and a GUI on top.

The XenServer install procedure is rather straightforward. Answer a few questions and let it roll. On the next reboot you get an ncurses-based interface on the console that allows you to achieve the bare minimum: configure the host, start/stop VMs, that kind of stuff. You can also do the same through ssh (ssh in then use xconsole).

Beyond that you need to find a Windows desktop because the only management solution they offer is a heavy Windows client. You get a very decent management interface that looks a lot like the VMWare Sphere client, from which you can control pretty much everything. The fact that it only runs on Windows is a major pain but to be honest: you only use it to configure new VMs. Once they are started you access them through ssh, vnc, or rdesktop, so no need to maintain a live Windows machine just for that.

In less than two hours I managed to install on XenServer:

  • A minimal Alpine Linux running nginx
  • An OPNSense instance
  • A pfSense instance
  • A Windows 8.1 desktop
  • A FreeBSD 11.0 VM, no X11

I still felt like something was missing though: XenServer would not recognize my WiFi/Bluetooth board. It would have been cool to dedicate a VM to make a stand-alone access point, so I kept trying more stuff.

Among all the options I tried, the only one that had all my hardware covered without hitch was Debian. Proxmox is based on Debian jessie, so if I succeed in installing it there should be a way to make things work. Let’s try again. I started from Debian and installed Proxmox on top. The guide I used is here:

https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Jessie

This works and happens to be quite smooth.

NB: I managed to completely destroy my setup when I decided to change the host IP address without telling Proxmox first. Rebooting the machine does not help, it goes into an endless loop, fails to reconfigure the network, and dies in horrible pain. I took the shortest path and re-installed from scratch. Good advice: DO NOT CHANGE THE PROXMOX HOST IP ADDRESS.

Proxmox is now working beautifully well. The advantages over XenServer for me are multiple:

  • LXC + KVM support: Proxmox supports LXC containers and KVM Virtual Machines in approximately the same way. Of course, containers are much lighter to install, start up, shut down, or backup.
  • Proxmox is completely open-source. XenServer probably has proprietary parts somewhere, though I did not investigate more than that.
  • Proxmox offers a pure Web interface: no need for a heavy Windows client.  You can also open a VNC console on any virtual machine directly from your browser, which is incredibly convenient.
  • Based on Debian, Proxmox identified and supports all my hardware.

Just for fun, I created a local WiFi access point based on alpine Linux by instantiating an LXC container, assigning the wlan0 interface to it, and booting the right daemons.

The next VMs I created are:

  • An alpine Linux desktop under LXC
  • Various alpine Linux boxes under LXC to run simple services
  • An Ubuntu desktop (under KVM)
  • A Windows 8 desktop (under KVM)
  • A MacOS Sierra desktop 
  • pfSense and OPNSense as KVM appliances, to evaluate them
  • An OpenBSD box to play with pf in command-line mode
  • A FreeBSD11 box

All these virtual goodies run on the same hardware as I write these lines.

My next task will be to select a solution to use as a home virtual firewall appliance. Meanwhile I am just having fun popping up and down virtual machines as my mood goes.

Completely useless but tons of fun!

Written by nicolas314

Tuesday 8 November 2016 at 3:43 pm

OpenWRT on EdgeRouter Lite

leave a comment »

erlite-3-900x500Installing OpenWRT on EdgeRouter Lite

This installation procedure does not require any extra hardware beyond a Phillips screwdriver to open the router box. I believe it is completely reversible and (hopefully) does not void your warranty.

Objective: replace EdgeOS on your EdgeRouter Lite by a recent version of
OpenWRT picked from the LEDE project.

You need:

  • A powerful Linux box, preferrably a multi-proc 64-bit machine with tons of RAM. This will only be used just once to compile OpenWRT.
  • An extra USB thumb drive. Its dimensions should be short: less than a few centimeters, otherwise it won’t fit inside the box.

Ok now off to build an OpenWRT image:

. Log onto your Linux box

. Download the latest sources from LEDE project:

 git clone https://github.com/lede-project/source
 cd source

. Prepare the tree for compilation for ERLITE:
You will have to select a number of options to build an image tailored to
the EdgeRouter Lite. I could put here a ready-made config file but as these
things tend to evolve quickly, it would probably be obsolete in a matter of
days. So bear with me: start the configuration with

make menuconfig

. Target System: Cavium Networks Octeon
. Target Profile: Ubiquiti EdgeRouter Lite

. Target Images: make sure ‘ext4’ is selected, then Select that line to
open up a menu for ext4 configuration. Change the number of inodes to
60,000 instead of the default 6,000. Also select GZip images, and finally modify the root filesystem partition size to something more comfortable,
say 500 MB. This space will be taken off your USB stick so if you have
more space you can increase that to whatever you have. With 500 MB you
should have enough space to put all the packages you need.
. Global build settings: select Select all kernel module packages by
default.

Beyond that take your pick for packages you want included by default in
your image. My selection is:

  • Base system: base-files, block-mount, busybox, ca-bundle, ca-certificates, dnsmasq, dropbear, firewall, fstools, jsonfilter, lede-keyring, libc, libgcc,
    libpthread, librt, libstdcpp, mtd, netifd, opkg, procd, rpcd, sqm-scripts, sqm-scripts-extra, swconfig, ubox, ubus, ubusd, uci, usign
  • Administration: sudo
  • Development: ar, binutils, gcc, gdb, make, objdump
  • Kernel modules: everything should already be selected as module. You want
    to change some of these to be compiled into the kernel otherwise it will
    fail to find the ext4 root on USB:

    • Filesystems: select kmod-fs-ext4, kmod-fs-msdos
    • USB Support: kmod-usb-core, kmod-usb-storage, kmod-usb-storage-extras
  • Languages: select whatever programming languages you want to see in a
    default install. I usually make sure at least Lua and Python are selected.
  • LuCI: make sure LuCI is selected. Take your pick for applications you
    want to install. I usually select luci-app-openvpn, luci-app-commands,
    luci-app-firewall.
  • Network: if you want your router to act as an OpenVPN client or server,
    make sure it is selected under VPN. Pick either openvpn-openssl or openssl-polarssl.
  • Utilities: bash, bc, file, grep, gzip, less, lsof, openssl-util, strace, tar, tmux, usbutils

Feel free to select more packages but each additional one will take extra
compilation time.

. Type ‘make’ and let the magic go on.
. When finished, the result is stored as:

bin/targets/octeon/generic/lede-octen-erlite-ext4-sysupgrade.tar

This file contains everything we need to build a bootable USB drive for
the EdgeRouter Lite. This file should be 500 MB large since you
selected that size above for your root filesystem, but it is mostly made
of zeroes so if you use bzip2 you should be able to reduce its size to a
more manageable 50-60 MB, which is more convenient if you need to toss it
around the network.

. Put the sysupgrade.tar file onto a local Linux machine and extract it:

tar xvf lede-octen-erlite-ext4-sysupgrade.tar

. The directory contents are:

sysupgrade-erlite/
sysupgrade-erlite/kernel
sysupgrade-erlite/root
sysupgrade-erlite/CONTROL

. Now insert your USB thumb drive into a local Linux machine and prepare
the filesystem. We need a first (small) FAT32 partition to hold the kernel,
a second 500 MB partition to hold the root:

fdisk /dev/sdX # Where X is the letter assigned to your USB drive
New partition: 1, 32 MB in size, type c (WIN95 FAT32 LBA)
New partition: 2, 500 MB in size, type Linux (default)
Optional: New partition: 3, the rest of your drive, type Linux (default)
Make the first partition bootable (a).
Type 'w' to save your changes.

Create the FAT32 filesystem with:
mkfs.vfat /dev/sdX1

. The default uboot configuration on the EdgeRouter Lite wants a file
called ‘vmlinux.64’ in the first (DOS) partition, so let’s do just that:

mount /dev/sdX1 /mnt # Mount the DOS partition
cp sysupgrade-erlite/kernel /mnt/vmlinux.64
umount /mnt

. Dump the root filesystem contents onto the second partition:

dd if=sysupgrade-lite/root of=/dev/sdX2 bs=1M

. If you have a third partition, create a new filesystem on it with:

mkfs.ext4 /dev/sdX3

. You are done with the USB drive!
. Open the EdgeRouter Lite. There are three small screws to remove on the
back. The box slides open if you push gently.

. Remove the existing USB stick inserted in the reader on the motherboard.
Be gentle: you need to insist a bit to take it off but it is not stuck.

. Insert the USB drive you prepared. Close the box, put the screws back,
and boot the router.

. If you connect a PC to the central NIC (labeled eth1) you should receive
an address on 192.168.1.0/24 from which you can ssh to 192.168.1.1 or open
a browser to http://192.168.1.1

. Welcome to OpenWRT/LEDE! Set a root password and you should be done.

The first things you probably want to do:

  • Change interface names to associate eth0 to WAN, and bridge eth1 and eth2 to
    LAN.
  • Edit the configuration to mount the third partition on the USB drive on /home. This is cool to add non-root users and give them a real flash storage.
  • Run ‘opkg update’, install missing packages.
  • Install OpenVPN configurations and test them.
  • Add ssh keys for root login in /etc/dropbear/authorized_keys
  • Install dotfiles to feel at home

Problems I have seen and their solutions:

The LEDE build is not so robust, sometimes it fails in parallel mode
because some dependencies seem to be compiled too late. If you get
compilation errors, using ‘make -j1’ should solve all issues. On a powerful
server with tons of RAM you need 2-3 hours to compile the whole set,
depending on how many packages you selected.

The version of LEDE you just compiled will quickly be out of sync with the
official package repositories. As soon as the kernel is changed in the LEDE project HEAD, all kmod packages from the LEDE repositories will refuse to install with opkg. This is the reason why you had to select “Build all kernel modules” in menuconfig: all kernel modules are already part of the image you created. This problem should go away once LEDE has released its first stable version.

I had most trouble with the ext4 filesystem definition: my first attempt
generated an ext4 of 50 MB which is far too small. After increasing that
size to 1GB, I still ran into “not enough disk space” errors and figured
out the number of inodes was too low (6,000). If you install a lot of
packages you need more inodes. Both points are addressed in the above
procedure. I also tried with an insanely high number (600k inodes) and the
resulting filesystem cannot be mounted.

Filesystem size is indicated in bytes, but fdisk counts in MB, based on a
power of 2. This yields a small discrepancy between the 500MiB filesystem
you generated with the build and the 500MB you reserved in the partition
table.

Once up and running, my router quickly ran into starvation problems. One
machine on the network could use the whole bandwidth and cut off every
other machine. I installed QoS packages: sqm-scripts, sqm-scripts-extra,
and luci-app-sqm, configured the queue to a fair scheduler, and got rid of
starvation issues. For some reason I could not get the pre-compiled
versions of these packages to work, I had to re-install them from official
repositories.

I wanted to add a Samba server to be able to use the rest of the USB drive
as a shared space but it is not a good idea. Samba takes ages to compile
and the daemon uses too many resources for such a small piece of hardware.

If you want to add ssh keys for the root user, remember the default ssh
server is dropbear, not openssh. dropbear expects root ssh keys to be
stored in /etc/dropbear/authorized_keys. You can also add root ssh keys
through LuCI.

The default shell for root is /bin/sh. You can change it to /bin/bash after
installing it and modifying the root entry in /etc/passwd.

Enjoy your fancy new router!

 

Written by nicolas314

Sunday 16 October 2016 at 4:58 pm

Samba+FreeBSD+OSX Finder

leave a comment »

Tried to run Samba44 on FreeBSD to share files on the local network with OSX machines. Took me a while but I finally figured out that in order to get rid of the infamous ERROR -36 from the Mac Finder, you have to disable sendfile with:

use sendfile = no

As a matter of fact, let me post a complete working example of smb4.conf that works between my FreeBSD-hosted Samba and my Mac (Yosemite):

[global]
# Following line is useful on Linux, not on FreeBSD apparently
# socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=131072 SO_SNDBUF=131072
realm=*
max protocol=SMB3
winbind enum groups=yes
winbind enum users=yes
large readwrite = yes
max xmit=32768
min receivefile size=2048
use sendfile = no
aio read size = 2048
aio write size = 2048
write cache size = 1024000
read raw = yes
write raw = yes
getwd cache = yes
unix charset = UTF-8
oplocks = yes
workgroup = HOME
server string = PO
security = user
map to guest = Bad User
log level = 5
log file = /var/log/samba4/samba.log
max log size = 50
interfaces = 192.168.0.0/16
hosts allow = 127.0.0.1 192.168.0.0/16
hosts deny  = 0.0.0.0/0
#dns proxy = no
# I do not want Samba to serve printers, thanks
printing = bsd
printcap name = /dev/null
guest ok = yes
guest account = nobody
load printers = no
printing = bsd
printcap name = /dev/null
disable spoolss = yes
ea support = no

# Share the directory in /data/share as 'share'
[share]
path = /data/share
public = yes
guest ok = yes
writable = yes
browsable = yes
create mask = 0777
directory mask = 0777
force user = nobody
follow symlinks = yes

In addition, OSX does not support guest access to an SMB server. The simplest solution I found was to set up a user called ‘nobody’ without password:

smbpasswd nobody
-> Press ENTER twice to set no password

And then access your share by pointing your Mac Finder to smb://nobody@HOST/share and click Connect.

Note to self: never ever install a Samba server after 7pm if you want a full night sleep.

 

Written by nicolas314

Wednesday 12 October 2016 at 12:01 am

EdgeRouter Lite

with 3 comments

erlite-3-900x500

My endless search for the ideal home router made me buy a piece of hardware called EdgeRouter Lite by Ubiquiti. The price point is sweet (around $100), making it a damn expensive home router or a damn cheap professional one. For that price you get:

  • A Cavium Octeon processor: 500MHz, two cores, rated 1000 bogomips, MIPS64 architecture, big-endian.
  • Half a gig of RAM
  • Three GBit NICs
  • No wireless
  • No fan, no noise
  • OS completely contained on an easily accessed USB stick on the motherboard, so essentially as much drive space as you want.

The last point is the most important: by just removing three small Phillips screws you can unplug the original USB thumb drive and replace it with your own, equipped with your favourite operating system. If everything fails you can always switch back to your previous state, put the screws back and call it a day. That should not void your warranty but I am no lawyer.

The provided operating system is called EdgeOS, based on Vyatta, itself based on Debian. It seems Vyatta development is now halted and Ubiquiti is now steering EdgeOS alone. I used EdgeOS on that router for about six months and have to admit being rather satisfied. The router is sold as the fastest switching home appliance on the market, claiming 1 million packets per second. In order to reach that kind of speed with a (dual-core) 500MHz processor on three GBit NICs you need additional specialized hardware that is only available through proprietary drivers provided with EdgeOS. So be it.

I have a beef with proprietary router firmware though: each vendor seems to feel obliged to invent their own management language. Cisco, Mikrotik, Ubiquiti, you name it. Everything is meant to be controlled from the command line, which is great, but instead of navigating through a familiar Unix environment you need to learn half a million new (proprietary) commands, their syntax, side effects, and how to commit, save, or restore configurations.  This is a royal pain in the butt and I have no desire to go get some training to configure a home appliance.

To be fair, open source versions have had the same issue for years, though some made a huge effort to provide good web-based GUIs for configuration and avoid having to invent a new configuration language altogether. Tomato and DD-WRT have really pushed things forward to reach a decent level of user-friendliness. You only need to know about networking and do not have to worry about learning yet another obscure syntax to handle those.

Too bad: both projects seem to be pretty much abandoned today. DD-WRT has not seen a stable release in almost a decade and Tomato still courageously lives on, maintained by a handful of dedicated devs working from home. The communities for Tomato and DD-WRT are dwindling fast in favour of OpenWRT.

OpenWRT is the most advanced open source router project today. It is well designed, based on a single syntax for configuration files, and supports pretty much every piece of router hardware under the sun. The project was recently forked by its own developers into the LEDE project, which is now the version I am following as closely as possible.

Back to the EdgeRouter Lite: what’s not to love?

Beyond the proprietary software and syntax, EdgeOS offers a web-based GUI that looks fancy and neat but covers only a very, very limited portion of what can be achieved through a command-line interface. This is very frustrating. I love command lines as your next geek but don’t force me to learn a syntax I will use nowhere else just to achieve mundane stuff.

After six months of customizing my home router to my own needs, I had gathered scripts lying around e.g. to extract a list of known MACs or some stats.  And when I updated EdgeOS to another minor version, everything fell apart.  That irked me to no end, pushing me once more into the arms of an open source alternative.

Support for alternative firmware for this router is not obvious to find.  OpenWRT has an incomplete wiki page about it. A couple of guys have succeeded in installing FreeBSD but I did not feel up to the task. Debian supports big-endian MIPS64 machines, and a project called DebWRT offers support for this router, merging both Debian and OpenWRT in a single solution. This is cool but I am a bit terrified about using a straight Linux distro to build a router. If all I have to handle iptables is a bash shell and miles of manual pages, this is not going to work, I hate the iptables syntax with a true passion. The unique config file format used by OpenWRT is a real blessing, there is no way I am going back to one config file format per daemon.

So I started from scratch, built my very own version of a LEDE instance, including all the software I want to run on this box. The process is error-prone and it took me several evenings to get straight. In order not to lose information, I will be detailing everything I did in a next post, hoping it could be useful for someone else.

The net result is a pure LEDE box that has been running without hiccups for a few days now. Configuring routes, VPN, DHCP, DNS is a walk in the park thanks to user-friendly OpenWRT. All my scripts are working again, I can handle backups myself, and I even installed dedicated web and Samba servers. Next step will be to install an ad-blocking name server.

I am certainly losing in terms of performance but I won’t see the difference. Without proprietary drivers, hardware acceleration is gone.  This should not be an issue considering my home GBit network is currently handled by a separate switch and my Internet connection is limited to a mere 20MBit/s, magnitudes below what the router needs to provide. The day I get a GBit Internet connection at home, I will always have a choice to switch back to EdgeOS with just one unplug/plug of a USB key. Or maybe someone will have reverse-engineered the proprietary drivers by then?

There is one alternative I have been looking deep into: using pfSense or OPNsense to build my own firewall. The approach sounds good. I believe the BSD family is technically a lot better than anything Linux-based. This is particularly true in terms of network security software.

Trouble is: pfSense/OPNsense is extremely greedy. You can build a 15 euro router with OpenWRT but you need PC-sized gear to run pfSense, including at least 1 GB of memory and a lot more than mere megabytes of storage (OpenWRT fits in just 4 megs). The cost of a pfSense appliance can easily run in 400-500 euros, which does not make any sense from a budget point of view.  Most people going down that road recommend re-purposing an old PC for the task, but I have absolutely no intention of storing a hungry 300W loud old PC box next to my 20Mbit/s DSL modem, this would be insane.

There lies the whole beauty of this exercise: find the cheapest, least power-hungry, and most efficient way to set up a home routing solution that is easy and fun to configure, flexible enough, and secure. I stopped building my own PCs years ago and cover that need now by building small appliances from scratch, compiling the whole OS myself.

Tinkering is fun!

Written by nicolas314

Wednesday 5 October 2016 at 10:03 pm