Juniper SRX Stuck in loader prompt

January 19, 2022

Insert a USB on a working SRX, then copy the partition

Insert the USB on the broken SRX and boot from USB

loader> nextboot
Platform: srx-trident
eUSB
usb
loader> nextboot usb
Setting next boot dev usb
loader> reboot
Resetting…

To do so, use the command below: request system snapshot media internal slice alternate
The slice seems to be a hidden command; therefore, you would have to type it in manually.

Recovering the Junos image in primary partition

When you spot that a primary partition has failed you should try to recover it as soon as possible as you are left with only one root partition. The recovery of the primary partition can be done easily by taking a snapshot of the root file system in the secondary partition and copying it to the primary partition.

The following command takes a snapshot of the currently active partition (secondary partition) and copies it to the alternate partition (primary partition).

root@SRX345>request system snapshot slice alternate
Formatting alternate root (//dev//da0s1a)...
Copying '//dev//da0s2a' to '//dev//da0s1a' .. (this may take a few minutes)
The following filesystems were archived: //

Run the following command to verify that you have a valid backup image.

root@SRX> show system software backup
Backup JUNOS package information:
File name: //cf//packages//junos-15.1X49-D150.2-domestic
File size: 254838138

You can also use the show system storage partitions command to check both partitions.

https://kb.juniper.net/InfoCenter/index?page=content&id=KB20554

Working output – shorten version

loader> ?
Available commands:
watchdog enable or disable kernel watchdog
bcachestat get disk block cache stats
autoboot boot automatically after a delay
boot boot a file or loaded kernel
lsdev list all devices
nextboot set next boot device
more show contents of a file
read read input from the terminal
echo echo arguments
unset unset a variable
set set a variable
show show variable(s)
? list commands
help detailed help
install install JunOS
include read commands from a file
ls list files
lsmod list loaded modules
unload unload all modules
load load a kernel or module
reboot reboot the system
heap show heap usage
save save U-Boot environment
export export variables to U-Boot environment
loader> boot
can’t load ‘/kernel’
can’t load ‘/kernel.old’
no bootable kernel
loader> nextboot
Platform: srx-trident
eUSB
usb
loader> nextboot usb
Setting next boot dev usb
loader> reboot
Resetting…

SPI stage 1 bootloader (Build time: Apr 26 2020 – 21:42:44)

U-Boot 2013.07-JNPR-3.9 (Build time: Apr 26 2020 – 21:42:45)

Octeon unique ID: 040000708015f31e0245
…..
…
PCIe: Port 2 not in PCIe mode, skipping
Net: octrgmii0
Node 0 Interface 4 has 1 ports (AGL)
Boot Media: eUSB usb
Found TPM SLB9660 TT 1.2 by Infineon
TPM initialized
USB1: Starting the controller
USB XHCI 1.00
scanning bus 1 for devices… 2 USB Device(s) found
USB0: Starting the controller
USB XHCI 1.00
scanning bus 0 for devices… 2 USB Device(s) found
scanning usb for storage devices… 2 Storage Device(s) found
Type the command ‘usb start’ to scan for USB storage devices.

Press SPACE to stop autoboot: 0
SF: Detected SF with page size 256 Bytes, erase size 64 KiB, total 8 MiB
SF: 1048512 bytes Read: OK
SF: 1048576 bytes Read: OK

Starting application …

SF: Detected SF with page size 256 Bytes, erase size 64 KiB, total 8 MiB
[0]Booting from usb slice 1
Consoles: U-Boot console
Found compatible API, ver. 3.9

FreeBSD/MIPS U-Boot bootstrap loader, Revision 2.10
(slt-builder@svl-junos-pool87.juniper.net, Sun Mar 4 10:30:52 PST 2018)
Memory: 4096MB
[0]Booting from usb slice 1
/boot/init.4th loaded.
Loading /boot/defaults/loader.conf
/kernel data=0x126bb74+0x1c04e4 syms=[0x4+0xba2c0+0x4+0x11d559]

Hit [Enter] to boot immediately, or space bar for command prompt.
Booting [/kernel]…
Kernel entry at 0x801000c0 …
init regular console
Primary ICache: Sets 16 Size 128 Asso 39
Primary DCache: Sets 8 Size 128 Asso 32
Secondary DCache: Sets 1024 Size 128 Asso 4

…

Timecounter “mips” frequency 1200000000 Hz quality 0
da1 at umass-sim1 bus 1 target 0 lun 0
da1: Removable Direct Access SCSI-4 device
da1: 40.000MB/s transfers
da1: 7680MB (15728640 512 byte sectors: 255H 63S/T 979C)
da0 at umass-sim0 bus 0 target 0 lun 0
da0: Fixed Direct Access SCSI-4 device
da0: 40.000MB/s transfers
da0: 7672MB (15712256 512 byte sectors: 255H 63S/T 978C)
random: unblocking device.
hwpmc: OCTEON/4/64/0x1ff
Trying to mount root from ufs:/dev/da1s1a
MFSINIT: Initialising MFSROOT
Process-1 beginning MFSROOT initialization…
Creating MFSROOT…
/dev/md0: 20.0MB (40956 sectors) block size 16384, fragment size 2048
using 4 cylinder groups of 5.00MB, 320 blks, 640 inodes.
super-block backups (for fsck -b #) at:
32, 10272, 20512, 30752
Populating MFSROOT…
Creating symlinks…
Setting up mounts…
Continuing boot from MFSROOT…
Attaching /cf/packages/junos via /dev/mdctl…
Mounted junos package on /dev/md1…
J
Automatic reboot in progress…
Verified jboot signed by PackageProductionECP256_2020 method ECDSA256+SHA256
Verified junos signed by PackageProductionECP256_2020 method ECDSA256+SHA256
Verified junos-20.2R1.10 signed by PackageProductionECP256_2020 method ECDSA256+ SHA256
Checking integrity of BSD labels:
s1: Passed
s2: Passed
s3: Passed
s4: Passed
** /dev/bo0s3e
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 94741 free (21 frags, 11840 blocks, 0.0% fragmentation)
** /dev/bo0s3f
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 1746346 free (386 frags, 218245 blocks, 0.0% fragmentation)
Checking integrity of licenses:
Checking integrity of configuration:
rescue.conf.gz: Passed

LPC bus driver
lpcbus0 on cpld0
tpm0: on lpcbus0
tpm: IFX SLB 9660 TT 1.2 rev 0x10
Loading configuration …
.
..
Additional routing options:kern.module_path: /boot//kernel;/boot/modules -> /boo t/modules;/modules/ifpfe_drv;/modules;
kld netpfe drv: ifpfed_dialer pvid_cryptosoft0: on motherboard
IPsec: Initialized Security Association Processing.
db kld ipsecs
.
Doing additional network setup:.
Starting final network daemons:.
setting ldconfig path: /usr/lib /opt/lib
starting standard daemons: cron.
root@SEDG-ABC-SRX01% alization:.
root@SEDG-ABC-SRX01% ization:.
root@SEDG-ABC-SRX01% s:set cores for group access
root@SEDG-ABC-SRX01%

root@SEDG-ABC-SRX01% clear
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 98068 free (28 frags, 12255 blocks, 0.0% fragmentation)
chassis.ko loaded Loading JUNOS chassis module
chassis_init_hw_chassis_startup_time: chassis startup time 0.000000
Thu Jan 13 17:16:25 GMT 2022

SEDG-ABC-SRX01 (ttyu0)

— JUNOS 20.2R1.10 built 2020-06-25 13:55:10 UTC

— NOTICE: System is running on alternate media device (/dev/da1s1a).

root@SEDG-ABC-SRX01%
root@SEDG-ABC-SRX01%
root@SEDG-ABC-SRX01% cli
{secondary:node0}
root@SEDG-ABC-SRX01> show system snapshot media internal

node0:

Information for snapshot on internal (/dev/da0s1a) (backup)
Creation date: Aug 27 17:03:31 2020
JUNOS version on snapshot:
junos : 20.2R1.10

node1:

root@SEDG-ABC-SRX01> … media internal slice alternate

node0:

error: Snapshot to alternate slice cannot be performed as internal is not the boot media

node1:

Formatting alternate root (/dev/da0s2a)…
Copying ‘/dev/da0s1a’ to ‘/dev/da0s2a’ .. (this may take a few minutes)
The following filesystems were ABChived: /

{secondary:node0}
root@SEDG-ABC-SRX01>

{secondary:node0}
root@SEDG-ABC-SRX01> show system snapshot media internal

node0:

Information for snapshot on internal (/dev/da0s1a) (backup)
Creation date: Aug 27 17:03:31 2020
JUNOS version on snapshot:
junos : 20.2R1.10

node1:

{secondary:node0}
root@SEDG-ABC-SRX01>

{secondary:node0}
root@SEDG-ABC-SRX01> show chassis cluster status
Monitor Failure codes:
CS Cold Sync monitoring FL Fabric Connection monitoring
GR GRES monitoring HW Hardware monitoring
IF Interface monitoring IP IP monitoring
LB Loopback monitoring MB Mbuf monitoring
NH Nexthop monitoring NP NPC monitoring
SP SPU monitoring SM Schedule monitoring
CF Config Sync monitoring RE Relinquish monitoring
IS IRQ storm

Cluster ID: 1
Node Priority Status Preempt Manual Monitor-failures

Redundancy group: 0 , Failover count: 0
node0 100 secondary no no None
node1 50 primary no no None

Redundancy group: 1 , Failover count: 0
node0 100 secondary no no None
node1 50 primary no no None

{secondary:node0}
root@SEDG-ABC-SRX01> show system snapshot media al?
No valid completions
{secondary:node0}
root@SEDG-ABC-SRX01> show system snapshot media ?
Possible completions:
internal Show snapshot information from internal flash
usb Show snapshot information from device connected to USB port

{secondary:node0}
root@SEDG-ABC-SRX01> show system snapshot media internal

node0:

Information for snapshot on internal (/dev/da0s1a) (backup)
Creation date: Aug 27 17:03:31 2020
JUNOS version on snapshot:
junos : 20.2R1.10

node1:

Information for snapshot on internal (/dev/da0s1a) (primary)
Creation date: Nov 27 18:07:49 2021
JUNOS version on snapshot:
junos : 20.2R1.10
Information for snapshot on internal (/dev/da0s2a) (backup)
Creation date: Jan 13 17:27:02 2022
JUNOS version on snapshot:
junos : 20.2R1.10
root@SEDG-ABC-SRX01% unmount /altroot
unmount: Command not found.
root@SEDG-ABC-SRX01% unmount /altroot
unmount: Command not found.
root@SEDG-ABC-SRX01% exitcal, noatime, read-only)
logouton /dev (devfs, local, multilabel)
root@SEDG-ABC-SRX01% exitlocal, noatime)
exitpackages on /junos/cf/packages (nullfs, local, noatime)
devfs on /junos/cf/dev (devfs, local, noatime, multilabel)
{secondary:node0}s (cd9660, local, noatime, read-only, verified)
root@SEDG-ABC-SRX01> lfs, local, noatime)
devfs on /junos/dev/ (devfs, local, noatime, noexec, read-only, multilabel)
{secondary:node0}junos/cf/packages1 (nullfs, local, noatime)
root@SEDG-ABC-SRX01> fs, local, noatime)
/dev/bo0s3e on /config (ufs, local, noatime)
{secondary:node0}f/var (ufs, local, noatime)
root@SEDG-ABC-SRX01> , asynchronous, local, noatime)
/cf/var/jail on /jail/var (nullfs, local, noatime)
{secondary:node0}t-api on /web-api/var (nullfs, local, noatime)
root@SEDG-ABC-SRX01> var/log (nullfs, local, noatime)
devfs on /jail/dev (devfs, local, noatime, noexec, read-only, multilabel)
{secondary:node0}/mfs (ufs, asynchronous, local, noatime)
root@SEDG-ABC-SRX01>
root@SEDG-ABC-SRX01% su –
{secondary:node0}
root@SEDG-ABC-SRX01>

{secondary:node0}
root@SEDG-ABC-SRX01>

{secondary:node0}
root@SEDG-ABC-SRX01> request system snapshot slice alternate

node0:

Formatting alternate root (/dev/da1s2a)…
Copying ‘/dev/da1s1a’ to ‘/dev/da1s2a’ .. (this may take a few minutes)
The following filesystems were ABChived: /

node1:

Formatting alternate root (/dev/da0s2a)…
Copying ‘/dev/da0s1a’ to ‘/dev/da0s2a’ .. (this may take a few minutes)
The following filesystems were ABChived: /

{secondary:node0}
root@SEDG-ABC-SRX01> show system storage partitions

node0:

Boot Media: usb (da1)
Active Partition: da1s1a
Backup Partition: da1s2a
Currently booted from: active (da1s1a)

Partitions information:
Partition Size Mountpoint
s1a 579M /
s2a 587M altroot
s3e 185M /config
s3f 5.0G /var
s4a 324M recovery
s4b
s4e 15M

node1:

Boot Media: internal (da0)
Active Partition: da0s1a
Backup Partition: da0s2a
Currently booted from: active (da0s1a)

Partitions information:
Partition Size Mountpoint
s1a 2.4G /
s2a 2.4G altroot
s3e 185M /config
s3f 2.1G /var
s4a 224M recovery
s4e 15M

{secondary:node0}
root@SEDG-ABC-SRX01> request system reboot media internal
Reboot the system ? yes,no yes

Shutdown NOW!

[pid 5208]

{secondary:node0}
root@SEDG-ABC-SRX01>
*** FINAL System shutdown message from root@SEDG-ABC-SRX01 ***

System going down IMMEDIATELY

Jan 13 17:59:32 init: interface-control (PID 2371) terminate signal 15 sent
JWaiting (max 60 seconds) for system process vnlru_mem' to stop...done Waiting (max 60 seconds) for system processvnlru’ to stop…done
Waiting (max 60 seconds) for system process bufdaemon' to stop...done Waiting (max 60 seconds) for system processsyncer’ to stop…
Syncing disks, vnodes remaining…0 0 0 done

syncing disks… Syncing disks, buffers remaining… 2 2
Final sync complete
Uptime: 48m19s
Rebooting…

Starting application …

SF: Detected SF with page size 256 Bytes, erase size 64 KiB, total 8 MiB
[0]Booting from usb slice 1
Consoles: U-Boot console
Found compatible API, ver. 3.9

Continuing boot from MFSROOT…
Attaching /cf/packages/junos via /dev/mdctl…
Mounted junos package on /dev/md1…
J
Automatic reboot in progress…
Verified jboot signed by PackageProductionECP256_2020 method ECDSA256+SHA256
Verified junos signed by PackageProductionECP256_2020 method ECDSA256+SHA256
Verified junos-20.2R1.10 signed by PackageProductionECP256_2020 method ECDSA256+SHA256
Checking integrity of BSD labels:
s1: Passed
s2: Passed
s3: Passed
s4: Passed
** /dev/bo0s3e
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 94741 free (21 frags, 11840 blocks, 0.0% fragmentation)
** /dev/bo0s3f
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 1746310 free (462 frags, 218231 blocks, 0.0% fragmentation)
Checking integrity of licenses:
Checking integrity of configuration:
rescue.conf.gz: Passed

Creating JAIL MFS partition…
JAIL MFS partition created
Boot media /dev/da1 has dual root support
** /dev/da1s2a
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 98068 free (76 frags, 12249 blocks, 0.0% fragmentation)
chassis.ko loaded Loading JUNOS chassis module
chassis_init_hw_chassis_startup_time: chassis startup time 0.000000
Thu Jan 13 18:04:35 GMT 2022

SEDG-ABC-SRX01 (ttyu0)

Tagged with:

Categorised as: Uncategorized

Memorise