Tested and Fixed — A Step Forward in Building GoodNotes Web Viewer
A Case Study on Swift & Web Assembly Performance
WebAssembly is a significant component of the GoodNotes Web project. It is powerful yet challenging to adopt as it is relatively new and lacks community resource support. The performance issue is one of the main challenges we are passionate about resolving; as in GoodNotes, we deeply care about user experiences and want to make sure GoodNotes is as smooth and slick across all platforms.
Unlike most previous performance improvements we’ve done in the past where all the solutions applied were related to the UI layer or specific hot paths, this time we decided to go one level deeper in our analysis.
In this blog post, we break down how we found a performance issue in our app for a specific toolchain (Swift WebAssembly) and fixed it — if you like performance case studies, keep reading!
Defining the Environment
Our first idea was to make a comparison using Unit Tests in order to check the exact same piece of code on different platforms and compare the execution times. In this way, we could have a slight idea about how slow it is to run Swift WebAssembly code compared to pure Swift Code.
The platforms to compare were:
- Swift Natively (running in macOS)
- WASMER (using WebAssembly binary)
- Chrome (using WebAssembly binary)
Thanks to WASMER, we can execute the WASM binary generated by the unit tests easily on our computer without using any browser. As you can imagine, using WASMER is useful just as a reference for us — in the end, we’re interested in executing it inside the browser.
Keep in mind as well, that one of the reasons why the performance is different between the platforms is because the toolchain itself is different. There are some specific libraries used in Swift WebAssembly that are different to Native Swift. A really interesting example is the libc :
- Web Assembly: https://github.com/WebAssembly/wasi-libc
- Linux Native: https://launchpad.net/ubuntu/+source/glibc
The Chosen Unit Test
We were trying to identify some expensive operations within the project that we’re just using purely Swift for (we can also execute Javascript code from Swift, so these examples aren’t valid for the purpose of this specific task). As we noticed the issue was related to the networking layer, we decided to use a profiler and find the point taking most of our execution time. After using the profiler we noticed there was a huge JSON string being parsed and taking too long to finish. Once the expensive operation was identified, we decided to use it as a reference for our unit tests. We saved the JSON we were parsing and noticed it was a huge one (5.5MB actually) and used it to run a unit test in order to measure the execution time before making any conclusions.
The test we wrote was something like this:
func testDeserialize_NotesHugeFromMemory() throws {
let notesTextJSON = URL.fetchNotesHugeJSON!
let notes = try JSONDecoder().decode(NotesData.self, from: notesTextJSON)
XCTAssert(notes.data.count > 0)
}Keep in mind the meassure functionality from XCTest is not available in Swift Web Assembly yet, so this is why we didn’t use it here.
Analyzing Results
Now that we have everything ready for running the test and checking the output, it’s time to review the results in terms of execution times on the following figures:
Outputs
- Natively & NO OPTS
Test Case ‘HybridRenderingFetchTests testDeserialize_NotesFromMemory’ passed (0.697 seconds) - Natively & OPTS
Test Case ‘HybridRenderingFetchTests testDeserialize_NotesFromMemory’ passed (0.597 seconds) - WASMER & NO OPTS
Test Case ‘HybridRenderingFetchTests.testDeserialize_NotesFromMemory’ passed (2.026 seconds) - WASMER & OPTS
Test Case ‘HybridRenderingFetchTests.testDeserialize_NotesFromMemory’ passed (1.84 seconds) - Chrome & NO OPTS
Test Case ‘HybridRenderingFetchTests.testDeserialize_NotesFromMemory’ passed (5.542 seconds) - Chrome & OPTS
Test Case ‘HybridRenderingFetchTests.testDeserialize_NotesFromMemory’ passed (4.464 seconds)
At this point, I was quite disappointed with the Swift WebAssembly performance in general.
Honestly, I was expecting numbers to be approx. 20% slower, but I never expected this huge difference; for an end-user, it means the app is about 7 times slower…
As you may have noticed, there is a difference in our execution times depending on the optimizations applied. Remember that all the compilers offer a way of setting some Optimization Flags used during compilation. Usually, when running Unit Tests, all these flags are disabled; however, we are also interested in seeing what the real end-user behaviour will be, so that’s why we executed the unit test in both scenarios.
- Testing a package without optimization flags
swift test- Testing a package with optimization flags
swift test -Xswiftc -ODon’t Give Up Yet!
When I finished measuring, I didn’t want to give up and present these numbers because I was thinking to myself, “something else must be wrong over here”, so I decided to share these numbers with the Swift WASM community to see if I could find some lights of hope there. After some conversations with the principal maintainers of the Swift WebAssembly toolchain, they agreed that these numbers were too strange. They started to investigate deeper the reason for it and quickly found a mistake: some of the libraries being shipped in the toolchain were being generated in debug mode instead of release mode. More concretely, the library that was affecting us was Foundation.
Here you have the PR where they solved the mistake:
https://github.com/swiftwasm/swift/pull/4355/files
Updated Results After Applying the Fix
Once we identified the issue and updated the toolchain, we tried to run the test again. Take a look at the numbers once more:
As you can see, execution times are now closer to the native executions — that’s amazing! If we compare before vs after just focussing on Chrome & Optimizations, we are now executing the same code almost 3 times faster. This is a massive improvement!
For me, the conclusion of this story is that sometimes we just need to speak with the appropriate person. Just by dedicating some time and sharing information we have been able to improve the performance of our app without a single line of code being changed.
Extra Ball
After the fix, all the libraries are being compiled in RELEASE mode, and the binary generated .wasm is now lighter as well:
From:
After stripping debug info the main binary size is 17.55 MBTo:
After stripping debug info the main binary size is 15.97 MBAfter compressing the WASM binary before generating the bundle, our production binary is now about 1.6 MB lighter.
This article was written by: Francisco J. Trujillo, a Senior Software Engineer focused on new technologies, open-source, clean code, and testing. He is now working as a Senior Software Engineer in the cross-platform team at GoodNotes.
