'sha256sum Illegal instruction on musl amd64'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       busybox
Subject:    sha256sum Illegal instruction on musl amd64
From:       ben.busybox () backplane ! be
Date:       2023-02-28 22:17:19
Message-ID: 8A57ED25-B681-4D2C-A8D8-67FD3F8C7955 () backplane ! be
[Download RAW message or body]

Hi,

I'm having an intermittent issue with "BusyBox v1.36.0 (2023-01-03 22:49:12 UTC)" \
(the one from the Docker image busybox:musl) when running on amd64 GitHub actions \
runner VMs (azure).

When I use sha256sum it is getting terminated with SIGILL, Illegal instruction. The \
issue is hard to reproduce but I have a GitHub actions CI/CD job that I can re-run \
repeatedly (no changes to code, environment, data input, etc) that will occasionally \
have the issue. I managed to capture a core dump.

/ # file core-sha256sum.10.1677605036 
core-sha256sum.10.1677605036: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), \
SVR4-style, from 'sha256sum -w -s -c -', real uid: 0, effective uid: 0, real gid: 0, \
effective gid: 0, execfn: '/bin/sha256sum', platform: 'x86_64'

# gdb /bin/sha256sum core-sha256sum.10.1677605036
Reading symbols from /bin/sha256sum...
(No debugging symbols found in /bin/sha256sum)
[New LWP 10]
Core was generated by `sha256sum -w -s -c -'.
Program terminated with signal SIGILL, Illegal instruction.
#0  0x0000000000401161 in ?? ()

If I use "layout asm" it shows this:
> 0×401161 or (%rax),%al

Here's the result of an strace:

2023-02-28T21:24:48.2816253Z execve("/usr/bin/sha256sum", ["sha256sum", "-w", "-s", \
"-c", "-"], 0x7ffe64982460 /* 5 vars */) = 0 2023-02-28T21:24:48.2816600Z \
arch_prctl(ARCH_SET_FS, 0x51c258)       = 0 2023-02-28T21:24:48.2816865Z \
set_tid_address(0x51cbd8)               = 15 2023-02-28T21:24:48.2818368Z getuid()    \
= 0 2023-02-28T21:24:48.2818764Z brk(NULL)                               = 0x1803000
2023-02-28T21:24:48.2819202Z brk(0x1805000)                          = 0x1805000
2023-02-28T21:24:48.2821626Z mmap(0x1803000, 4096, PROT_NONE, \
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x1803000 2023-02-28T21:24:48.2822306Z \
mmap(NULL, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = \
0x7f54496bc000 2023-02-28T21:24:48.2822893Z read(0, \
"e4d5808efbd4239a2f496b6055ac15b2"..., 1024) = 154 2023-02-28T21:24:48.2823486Z \
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = \
0x7f54496bb000 2023-02-28T21:24:48.2823993Z open("1.544MiB.bin", \
O_RDONLY|O_LARGEFILE) = 3 2023-02-28T21:24:48.2824623Z read(3, \
"\270\222\2W\262\203&^\1\304X\372\247H\6\261\212\220\303i0D\266tx\356\353\370\327\363\354q"..., \
4096) = 4096 2023-02-28T21:24:48.2825392Z --- SIGILL {si_signo=SIGILL, \
si_code=ILL_ILLOPN, si_addr=0x401161} --- 2023-02-28T21:24:48.2826349Z +++ killed by \
SIGILL (core dumped) +++ 2023-02-28T21:24:48.2846595Z Illegal instruction

Here's a later one with a simpler set of arguments:

2023-02-28T21:47:29.2574112Z + strace sha256sum 1.544MiB.bin 1MiB.bin
2023-02-28T21:47:29.2611500Z execve("/usr/bin/sha256sum", ["sha256sum", \
"1.544MiB.bin", "1MiB.bin"], 0x7ffc41e4baf0 /* 5 vars */) = 0 \
2023-02-28T21:47:29.2616858Z arch_prctl(ARCH_SET_FS, 0x51c258)       = 0 \
2023-02-28T21:47:29.2617173Z set_tid_address(0x51cbd8)               = 10 \
2023-02-28T21:47:29.2617428Z getuid()                                = 0 \
2023-02-28T21:47:29.2617682Z brk(NULL)                               = 0x1c4d000 \
2023-02-28T21:47:29.2623375Z brk(0x1c4f000)                          = 0x1c4f000 \
2023-02-28T21:47:29.2623919Z mmap(0x1c4d000, 4096, PROT_NONE, \
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x1c4d000 2023-02-28T21:47:29.2625076Z \
mmap(NULL, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = \
0x7f2a9de51000 2023-02-28T21:47:29.2625396Z open("1.544MiB.bin", \
O_RDONLY|O_LARGEFILE) = 3 2023-02-28T21:47:29.2625758Z read(3, \
"\270\222\2W\262\203&^\1\304X\372\247H\6\261\212\220\303i0D\266tx\356\353\370\327\363\354q"..., \
4096) = 4096 2023-02-28T21:47:29.2626698Z --- SIGILL {si_signo=SIGILL, \
si_code=ILL_ILLOPN, si_addr=0x401161} --- 2023-02-28T21:47:29.2632505Z +++ killed by \
SIGILL (core dumped) +++ 2023-02-28T21:47:29.2632762Z Illegal instruction

At the following URL the first run didn't fail but the second one did (I just hit \
rerun all jobs without changing anything else): \
https://github.com/backplane/avxtest/actions/runs/4297660126

(The name of that repo is based on my initial theory of the problem, but I don't see \
evidence that avx is involved.)

You can bring up a shell with the core dump (and binary) in question using:
docker run --rm -it --platform linux/amd64 ghcr.io/backplane/avxtest:1677620842 \
<http://ghcr.io/backplane/avxtest:1677620842>

I use alpine to get strace and earlier gdb, file, etc. but I'm working with the \
binary from the busybox:musl container.

Any idea what's going on?

Thanks,
Ben
_______________________________________________
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

[prev in list] [next in list] [prev in thread] [next in thread] 