EdgeRouter – Last resort recovery of failed EdgeOS device


This article describes a method to completely re-install the EdgeOS platform on the EdgeRouter. If the EdgeRouter becomes unresponsive (e.g., web server will not load), you can use this as a last action method to rescue the device.

warning_25x25.png Note: The information in this tutorial was provided by user (find original post here), and it is not supported by Ubiquiti. Use under your own risk. This tutorial is for ER-Lite and ER-PoE only.

Table of Contents

  1. Intro
  2. Steps
  3. Related Articles


Back to Top

Technically speaking, this is the EdgeOS kernel recompiled and stripped down using embedded initramfs with recovery tools. You connect to your router via serial, boot it via TFTP and get into a minimal rescue system.

The following scripts for automated recovery procedures are included:

  1. emrk-factory-reset — removes the config.boot, optionally making a backup copy inside the config dir so you can access it easily after reboot.
  2. emrk-remove-user-data — removes any data from writable dirs, getting the system back to clean EdgeOS (all config data, user installed packages on so on are removed). It also removes copies of user data left after resets, as pressing reset button doesn’t really remove the data but renames the old writable dir to some random name and creates a new one. Useful if you are going to give away your ERL to an untrusted third party.
  3. emrk-reinstall — downloads release tarball from URL and reinstalls EdgeOS from scratch.

Links to download scripts: http://packages.vyos.net/tools/emrk/

warning_25x25.png Note: emrk-remove-user-data and emrk-reinstall are highly disruptive! The former will remove everything but the stock EdgeOS image. The latter will remove and re-create all the partitions so all your data, if you still have any, they will be lost and irrecoverable.

Note: After trying the basic reset methods, use this recovery method only if there is no hope to recover your installation. Use at your own risk!



Back to Top

1. Download the image: http://packages.vyos.net/tools/emrk/0.9c/ (MD5 sum: 3545d69a8912fd967c7301031c47960e)

2. Install a TFTP server. On UNIX systems some is always available from ports/repos, Windows users may use this one: http://tftpd32.jounin.net/ Copy the image to your TFTP directory.

It’s better to set up a DHCP server. On ISC DHCPD, specify

option tftp-server-name "x.x.x.x"; # Your TFTP server address next-server x.x.x.x; # Your TFTP server address option bootfile-name "emrk-0.9c.bin";
filename "emrk-0.9c.bin

3. Connect to your router via serial. Power on or reboot and before EdgeOS kernel loads, press any key (I usually press any key repeatadly since it starts booting). This will get you to the bootloader console where you can boot your router manually.

4. Connect router eth0 to your network.

5. If you use a DHCP server, issue the following command in U-Boot CLI:

dhcp;tftpboot;bootoctlinux $loadaddr

 If you don’t have a DHCP server, you may configure network manually:

set ipaddr x.x.x.x (where x.x.x.x is desired router IP address)  set netmask (or whatever mask you want)  set serverip y.y.y.y (where y.y.y.y is your TFTP server address)  set bootfile emrk-0.9c.bin  ##Optionally set gatewayip z.z.z.z (where z.z.z.z is your default gateway)


bootoctlinux $loadaddr

6. If everything goes right, after bootoctlinux command you will see the kernel booting and will be asked if you want to continue after a scary disclaimer. Answer “yes” (case insensitive) or “no”, than asked to configure network. You may choose DHCP or configure it statically.

7. To reinstall EdgeOS from scratch, you will need to download the tarball and put it on your HTTP/FTP/SSH server. 

8. Here is an example of session with reinstall (kernel boot messages omitted).

Loading EMRK 0.9a Mounting filesystems Bringing up eth0  Checking boot partition Boot partition looks intact Attempting to mount boot partition Boot partition successfully mounted Looking for kernel file Found a kernel Checking kernel MD5 sum file Found kernel MD5 sum file Checking kernel MD5 sum Kernel MD5 sum is correct  Checking root partition Root partition looks intact Attempting to mount root partition kjournald starting.  Commit interval 5 seconds EXT3 FS on sda2, internal journal EXT3-fs: mounted filesystem with writeback data mode. Root partition successfully mounted Looking for system image file Found a system image file Checking system image MD5 sum file Found system image MD5 sum file Checking system image MD5 sum System image MD5 sum is correct  ********************************************** Welcome to EdgeMAX Rescue Kit!  This tool is distributed under the terms of GNU General Public License and other licenses  Brought to you by SO3 Group  WARNING: This tool is not developed, officially supported or endorsed by Ubiquiti Networks!  Using it may lead to destroying your router configuration or operating system  Ubiquiti Networks support will not help you with using it or fixing consequences of using it.  This tool itself is distributed without any warranty and authors are not liable for any damage it may cause  By using this tool you agree you are doing it at your own risk and understand what you are doing  *********************************************  Enter 'Yes' to proceed, 'No' to reboot yes or no: yes  Do you want to configure network via DHCP? yes or no: yes udhcpc (v1.17.1) started Sending discover... Sending select for Lease of obtained, lease time 86400 /usr/share/udhcpc/default.script: Resetting default routes route: SIOCDELRT: No such process /usr/share/udhcpc/default.script: Adding DNS  EMRK provides some scripts for automated recovery procedures:  emrk-factory-reset -- reset config to factory default emrk-remove-user-data -- remove all the user data including     config and everything emrk-reinstall -- reinstall EdgeOS from scratch     (wipes any user data too)  Enter 'reboot' to reboot your router   BusyBox v1.17.1 (Debian 1:1.17.1-8) built-in shell (ash) Enter 'help' for a list of built-in commands.  /bin/ash: can't access tty; job control turned off EMRK> EMRK>emrk-reinstall  WARNING: This script will reinstall EdgeOS from scratch If you have any usable data on your router storage, it will be irrecoverably destroyed! Do you want to continue? yes or no: yes Unmounting boot partition Unmounting root partition Re-creating partition table Creating boot partition Formatting boot partition mkfs.vfat 3.0.9 (31 Jan 2010) Creating root partition Formatting root partition Mounting boot parition Mounting root partition kjournald starting.  Commit interval 5 seconds EXT3 FS on sda2, internal journal EXT3-fs: mounted filesystem with writeback data mode. Enter EdgeOS image url: http://dl.ubnt.com/firmwares/edgemax/v1.6.0/ER-e100.v1.6.0.4716006.tar   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                  Dload  Upload   Total   Spent    Left  Speed 100 68.3M  100 68.3M    0     0  8319k      0  0:00:08  0:00:08 --:--:-- 6318k Unpacking EdgeOS release image Verifying EdgeOS kernel Copying EdgeOS kernel to boot partition Verifying EdgeOS system image Copying EdgeOS system image to root partition Creating EdgeOS writable data directory Cleaning up Installation finished Please reboot your router

If you want to do things manually, there are busybox with symlinks to its applets (cat/grep/rm/cp/mv/tar/etc., ping, traceroute, nc etc.), curl, parted, and mkfs.ext3/mkfs.vfat.

I used it on my on ERL when I replaced the USB stick with a 8GB one yesterday. Hope you find it useful, any bug reports or suggestions are welcome.

Here are the scripts it uses: http://packages.vyos.net/tools/emrk 

Related Articles

Back to Top

How to Establish a Connection Using SSH   

Leave a Reply

Your email address will not be published. Required fields are marked *