HWAddressSanitizer support

[vo4]

Motivation

Compared to regular ASan, HWASan has a smaller overhead. The difference in practice is that HWASan'ed code is more usable, e.g. Android device compiled with HWASan can be used as a daily driver.

Example

fn main() {
    let xs = vec![0, 1, 2, 3];
    let _y = unsafe { *xs.as_ptr().offset(4) };
}

==223==ERROR: HWAddressSanitizer: tag-mismatch on address 0xefdeffff0050 at pc 0xaaaad00b3468
READ of size 4 at 0xefdeffff0050 tags: e5/00 (ptr/mem) in thread T0
    #0 0xaaaad00b3464  (/root/main+0x53464)
    #1 0xaaaad00b39b4  (/root/main+0x539b4)
    #2 0xaaaad00b3dd0  (/root/main+0x53dd0)
    #3 0xaaaad00b61dc  (/root/main+0x561dc)
    #4 0xaaaad00c0574  (/root/main+0x60574)
    #5 0xaaaad00b6290  (/root/main+0x56290)
    #6 0xaaaad00b6170  (/root/main+0x56170)
    #7 0xaaaad00b3578  (/root/main+0x53578)
    #8 0xffff81345e70  (/lib64/libc.so.6+0x20e70)
    #9 0xaaaad0096310  (/root/main+0x36310)

[0xefdeffff0040,0xefdeffff0060) is a small allocated heap chunk; size: 32 offset: 16
0xefdeffff0050 is located 0 bytes to the right of 16-byte region [0xefdeffff0040,0xefdeffff0050)
allocated here:
    #0 0xaaaad009bcdc  (/root/main+0x3bcdc)
    #1 0xaaaad00b1eb0  (/root/main+0x51eb0)
    #2 0xaaaad00b20d4  (/root/main+0x520d4)
    #3 0xaaaad00b2800  (/root/main+0x52800)
    #4 0xaaaad00b1cf4  (/root/main+0x51cf4)
    #5 0xaaaad00b33d4  (/root/main+0x533d4)
    #6 0xaaaad00b39b4  (/root/main+0x539b4)
    #7 0xaaaad00b61dc  (/root/main+0x561dc)
    #8 0xaaaad00b3578  (/root/main+0x53578)
    #9 0xaaaad0096310  (/root/main+0x36310)

Thread: T0 0xeffe00002000 stack: [0xffffc0590000,0xffffc0d90000) sz: 8388608 tls: [0xffff81521020,0xffff815217d0)
Memory tags around the buggy address (one tag corresponds to 16 bytes):
  0xfefcefffef80: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefcefffef90: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefcefffefa0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefcefffefb0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefcefffefc0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefcefffefd0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefcefffefe0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefcefffeff0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
=>0xfefceffff000: a2  a2  05  00  e5 [00] 00  00  00  00  00  00  00  00  00  00 
  0xfefceffff010: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefceffff020: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefceffff030: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefceffff040: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefceffff050: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefceffff060: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefceffff070: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefceffff080: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
Tags for short granules around the buggy address (one tag corresponds to 16 bytes):
  0xfefcefffeff0: ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  .. 
=>0xfefceffff000: ..  ..  c5  ..  .. [..] ..  ..  ..  ..  ..  ..  ..  ..  ..  .. 
  0xfefceffff010: ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  .. 
See https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#short-granules for a description of short granule tags
Registers where the failure occurred (pc 0xaaaad00b3468):
    x0  e500efdeffff0050  x1  0000000000000004  x2  0000ffffc0d8f5a0  x3  0200efff00000000
    x4  0000ffffc0d8f4c0  x5  000000000000004f  x6  00000ffffc0d8f36  x7  0000efff00000000
    x8  e500efdeffff0050  x9  0200efff00000000  x10 0000000000000000  x11 0200efff00000000
    x12 0200effe000006b0  x13 0200effe000006b0  x14 0000000000000008  x15 00000000c00000cf
    x16 0000aaaad00a0afc  x17 0000000000000003  x18 0000000000000001  x19 0000ffffc0d8f718
    x20 ba00ffffc0d8f7a0  x21 0000aaaad00962e0  x22 0000000000000000  x23 0000000000000000
    x24 0000000000000000  x25 0000000000000000  x26 0000000000000000  x27 0000000000000000
    x28 0000000000000000  x29 0000ffffc0d8f650  x30 0000aaaad00b3468

Comments/Caveats

• HWASan is only supported on arm64.
• I'm not sure if I should add a feature gate or piggyback on the existing one for sanitizers.
• HWASan requires -C target-feature=+tagged-globals. That flag should probably be set transparently to the user. Not sure how to go about that.

TODO

• Need more tests.
• Update documentation.
• Fix symbolization.
• Integrate with CI

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @oli-obk (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

[tmiasko]

Clang also enables emission of lifetime markers by default for HWASAN, should we do we same? I couldn't immediately identify how those are used, but if they are useful the code is in

rust/compiler/rustc_session/src/session.rs

Lines 1125 to 1130 in 099f27b

	pub fn emit_lifetime_markers(&self) -> bool {
	self.opts.optimize != config::OptLevel::No
	// AddressSanitizer uses lifetimes to detect use after scope bugs.
	// MemorySanitizer uses lifetimes to detect use of uninitialized stack variables.
	|| self.opts.debugging_opts.sanitizer.intersects(SanitizerSet::ADDRESS | SanitizerSet::MEMORY)
	}

[vo4]

Clang also enables emission of lifetime markers by default for HWASAN, should we do we same? I couldn't immediately identify how those are used, but if they are useful the code is in

rust/compiler/rustc_session/src/session.rs

Lines 1125 to 1130 in 099f27b

	pub fn emit_lifetime_markers(&self) -> bool {
	self.opts.optimize != config::OptLevel::No
	// AddressSanitizer uses lifetimes to detect use after scope bugs.
	// MemorySanitizer uses lifetimes to detect use of uninitialized stack variables.
	|| self.opts.debugging_opts.sanitizer.intersects(SanitizerSet::ADDRESS | SanitizerSet::MEMORY)
	}

Good spot! I didn't see the instrumentation pass or the runtime use lifetime markers, so I left it out. But if clang emits them, maybe they are actually used. I couldn't find out how/if HWASan uses them though. @pcc @eugenis do you folks know?

[pcc]

We don't currently use them, but we do plan to use them to implement use-after-scope checks in HWASan.

[oli-obk]

I have honestly no clue about the changes in this PR.

r? @tmiasko you seem to be on top of this

[tmiasko]

• The documentation for sanitizers is currently in src/doc/unstable-book/src/compiler-flags/sanitizer.md. It can be built with ./x.py doc src/doc/unstable-book/, and later found in ./build/$target/doc/unstable-book/index.html.
• The aarch64-gnu CI job should run the tests during merge, if you would like you can also enable this runner temporarily for pull request CI by including the job in pr section of src/ci/github-actions/ci.yml.

r? @nagisa

[nagisa]

Overall LGTM, r=me once nits are fixed.

[vo4]

r= @nagisa

[nagisa]

@bors r+

[bors]

📋 Looks like this PR is still in progress, ignoring approval.

Hint: Remove [WIP] from this PR's title when it is ready for review.

[nagisa]

@bors r+

[bors]

📌 Commit 9c34c14 has been approved by nagisa