Add Apple silicon slice to Mono runtime built for macOS #1309

TautvydasZilys · 2020-06-23T19:31:48Z

The Mono runtime changes were implemented & tested by Apple on Apple silicon hardware. I implemented the changes in our build scripts.

The biggest change in here is how JIT operates. On Apple silicon Macs, you're not allowed to both execute and write to write/execute pages at the same time. The way I understand it is that each thread has a state whether it can write or execute instructions in these pages. It is controlled using pthread_jit_write_protect_np function. Before the JIT can write to those pages, it needs to change the thread state to "non-protected". Before executing the JITed code, the thread state must be changed to "write-protected". This is handled using the new macros in mono/mini.h. They were implemented inline for performance reasons.

Other changes include adding ARM64 target to configure script and fixing build on the new SDK.

Please note that CI might fail - up until today I couldn't push the changes and could only run stuff locally (where it worked). I'm working on ironing out all the issues as we speak.

EDIT: CI is now green and we're producing all macOS binaries properly:

tautvydaszilys@Tautvydass-MacBook-Pro builds % file embedruntimes/osx/libMonoPosixHelper.dylib 
embedruntimes/osx/libMonoPosixHelper.dylib: Mach-O 64-bit dynamically linked shared library x86_64
tautvydaszilys@Tautvydass-MacBook-Pro builds % file embedruntimes/osx/libmonobdwgc-2.0.dylib 
embedruntimes/osx/libmonobdwgc-2.0.dylib: Mach-O 64-bit dynamically linked shared library x86_64
tautvydaszilys@Tautvydass-MacBook-Pro builds % file embedruntimes/osx/libmonosgen-2.0.dylib 
embedruntimes/osx/libmonosgen-2.0.dylib: Mach-O 64-bit dynamically linked shared library x86_64
tautvydaszilys@Tautvydass-MacBook-Pro builds % file embedruntimes/osx-arm64/libMonoPosixHelper.dylib 
embedruntimes/osx-arm64/libMonoPosixHelper.dylib: Mach-O 64-bit dynamically linked shared library arm64
tautvydaszilys@Tautvydass-MacBook-Pro builds % file embedruntimes/osx-arm64/libmonobdwgc-2.0.dylib 
embedruntimes/osx-arm64/libmonobdwgc-2.0.dylib: Mach-O 64-bit dynamically linked shared library arm64
tautvydaszilys@Tautvydass-MacBook-Pro builds % file embedruntimes/osx-arm64/libmonosgen-2.0.dylib 
embedruntimes/osx-arm64/libmonosgen-2.0.dylib: Mach-O 64-bit dynamically linked shared library arm64
tautvydaszilys@Tautvydass-MacBook-Pro builds % file monodistribution/bin/mono
monodistribution/bin/mono: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]
monodistribution/bin/mono (for architecture x86_64):	Mach-O 64-bit executable x86_64
monodistribution/bin/mono (for architecture arm64):	Mach-O 64-bit executable arm64
tautvydaszilys@Tautvydass-MacBook-Pro builds % file monodistribution/bin/pedump 
monodistribution/bin/pedump: Mach-O 64-bit executable x86_64
tautvydaszilys@Tautvydass-MacBook-Pro builds % file monodistribution/lib/libMonoPosixHelper.dylib 
monodistribution/lib/libMonoPosixHelper.dylib: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit dynamically linked shared library x86_64] [arm64:Mach-O 64-bit dynamically linked shared library arm64]
monodistribution/lib/libMonoPosixHelper.dylib (for architecture x86_64):	Mach-O 64-bit dynamically linked shared library x86_64
monodistribution/lib/libMonoPosixHelper.dylib (for architecture arm64):	Mach-O 64-bit dynamically linked shared library arm64

Signed-off-by: Tautvydas Žilys <tautvydas.zilys@gmail.com>

…undefined on darwin but was added recently which leads to unexpected behavior Signed-off-by: Tautvydas Žilys <tautvydas.zilys@gmail.com>

akoeplinger · 2020-06-23T20:18:24Z

mono/utils/mono-codeman.c

@@ -57,7 +57,7 @@ static MonoCodeManagerCallbacks code_manager_callbacks;
 #define MAX_WASTAGE 32
 #define MIN_BSIZE 32

-#ifdef __x86_64__
+#if defined(__x86_64__) && !defined(__APPLE__)


FWIW we fixed this a bit differently upstream: dotnet/runtime@cc501c2

Thanks. Looks like our fork code is quite a bit different. We'll get rid of this change when we pull upstream.

Signed-off-by: Tautvydas Žilys <tautvydas.zilys@gmail.com>

external/buildscripts/build_all_osx.pl

mono/mini/mini-arm64.h

mono/mini/mini-arm64.c

UnityAlex

lgtm

external/buildscripts/build.pl

libgc/gcj_mlc.c

joncham · 2020-06-24T18:35:32Z

mono/mini/mini-runtime.c

@@ -2804,6 +2806,7 @@ mono_jit_runtime_invoke (MonoMethod *method, void *obj, void **params, MonoObjec
 		if (!is_ok (error))
 			return NULL;
 	} else {
+		MONO_SCOPE_ENABLE_JIT_EXEC();


It seems odd this is needed in mono_jit_runtime_invoke

Without this, JITted code won't be marked executable and you'll get EXC_BAD_ACCESS when trying to jump to it.

Why don't we mark it as executable after JIT'ing instead of before executing?

Executable mode is thread local state. That means changing the mode affects it for all code that runs on current thread. If we were to switch to executable mode right after JIT, we might mess up the state of something else running on the same thread (for instance, what if somebody embeds CEF?). Therefore we scope it: just before needing certain mode, we switch to it. And then after, we switch back to whatever it was.

mono/mini/mini.h

mono/mini/unwind.c

joncham

Main concern is the presence of WRITE/EXEC macros outside of arch specific code and in places where writing seems to not occur

TautvydasZilys · 2020-06-24T18:48:03Z

Main concern is the presence of WRITE/EXEC macros outside of arch specific code and in places where writing seems to not occur

I don't really know how to abstract those macros out of generic code. Basically we need to change to "writable" mode just before writing to executable memory, and "execute mode" before jumping to it.

I will go through all of the places to make sure they're necessary, though.

TautvydasZilys requested a review from joncham June 23, 2020 19:31

TautvydasZilys and others added 6 commits June 23, 2020 12:32

Don't build macOS 32-bit binaries anymore.

107a009

Use strict/warnings in the build script to uncover bugs.

8a23ef6

Adopt correct objc call API

1a7240b

Signed-off-by: Tautvydas Žilys <tautvydas.zilys@gmail.com>

Add aarch64 macos compilation

8fc5e0e

Signed-off-by: Tautvydas Žilys <tautvydas.zilys@gmail.com>

Disable uscore check when cross-compiling

9da2c69

Signed-off-by: Tautvydas Žilys <tautvydas.zilys@gmail.com>

Do not use MONO_MMAP_32BIT on Apple platforms MMAP_32BITS used to be …

6d6f25c

…undefined on darwin but was added recently which leads to unexpected behavior Signed-off-by: Tautvydas Žilys <tautvydas.zilys@gmail.com>

TautvydasZilys force-pushed the mac-arm64 branch from 6661111 to 3ddb80f Compare June 23, 2020 19:34

akoeplinger reviewed Jun 23, 2020

View reviewed changes

lambdageek mentioned this pull request Jun 23, 2020

Add Apple Silicon support mono/mono#20014

Closed

3 tasks

GeoffroyAubey and others added 3 commits June 23, 2020 14:35

Fix restriction on RWX pages on macOS arm64

3f2278e

Signed-off-by: Tautvydas Žilys <tautvydas.zilys@gmail.com>

Use iOS abi on macOS arm64

399c21f

Signed-off-by: Tautvydas Žilys <tautvydas.zilys@gmail.com>

Support cross compiling to ARM64 in our build scripts.

bd8d105

TautvydasZilys force-pushed the mac-arm64 branch from 3ddb80f to bd8d105 Compare June 23, 2020 21:42

TautvydasZilys requested a review from UnityAlex June 23, 2020 23:34

UnityAlex reviewed Jun 24, 2020

View reviewed changes

external/buildscripts/build_all_osx.pl Show resolved Hide resolved

UnityAlex reviewed Jun 24, 2020

View reviewed changes

mono/mini/mini-arm64.h Outdated Show resolved Hide resolved

UnityAlex reviewed Jun 24, 2020

View reviewed changes

mono/mini/mini-arm64.c Show resolved Hide resolved

UnityAlex approved these changes Jun 24, 2020

View reviewed changes