macOS bugs are causing kext failures

Back in summer of 2018, customer support at Malwarebytes started seeing people with problems activating the kernel extension (kext) in Malwarebytes for Mac. This opened a can of worms that we’re still struggling with today… as soon as we think the worms are back in the can, we start finding new ones. Unfortunately, these worms all belong to macOS, and are affecting other kexts as well.

A history of previous issues

We ultimately traced the initial problem to an issue with the /Library/StagedExtensions/ folder. This folder is used by macOS in the process of activating a kext. By default, this folder is protected by SIP, so it can only be modified by the system.

Unfortunately, we were seeing that a number of users had a StagedExtensions folder that was lacking the restricted flag. (This flag is an indication that an item is protected by SIP, and can be set via the chflags command, although only from recovery mode.) This interfered with activation of the kext, since macOS no longer considered StagedExtensions to be a secure location at which to stage the kext.

The symptom of the problem was pretty easy to spot, if you knew where to look. If you run the following command (where the -O switch shows the file flags):

ls -alO /Library/StagedExtensions

The output should show this for the StagedExtensions folder itself:

drwxr-xr-x@  4 root  wheel  restricted  128 Nov  6 13:37 .

However, on affected systems, it showed:

drwxr-xr-x@  4 root  wheel  -  128 Nov  6 13:37 .

Notice the lack of a restricted flag on the folder, indicating that it is no longer protected by SIP. Of course, this flag should only be able to be cleared by the OS itself, or by a user in recovery mode. There was no reason for users to be clearing that flag themselves, and many of the users struggled with the process of setting it again, so every indication pointed to a system bug.

This was reported to Apple on August 3, 2018 (radar FB5361175), and was fixed in macOS 10.14.1, which I confirmed, and thus closed the bug report on January 28, 2019.

Later, we started seeing the issue again in macOS 10.15 and 10.15.1, and opened a new bug report on November 6, 2019 (radar FB7430509). This bug ended up being closed by Apple for the reason “Unable to diagnose with current information,” which might have irritated me, but the problems stopped again. It must have been fixed independently of our bug report, I guess…?

The current state of kext issues

Unfortunately, the problem is back yet again. This time, however, the problem does not involve the StagedExtensions folder. This time, the issue involves the folder /private/var/db/KernelExtensionManagement/, which is a newer folder that is used in the process of activating kexts on more recent versions of macOS. I’m unsure when this folder first appeared, or exactly how it is used for kext activation compared to StagedExtensions.

Affected Malwarebytes users see an error message in the app and are unable to turn on the real-time protection (RTP) features. In many cases, these users had RTP turned on previously, but suddenly found that it was off. In other cases, these are new installs where users are unable from the start to enable RTP.

Observations of affected users showed exactly the same problem as with StagedExtensions: the KernelExtensionManagement folder had somehow had the restricted flag cleared, meaning it was no longer protected by SIP. Further investigation, via the kextutil command, revealed more information:

$ kextutil -tn /Library/StagedExtensions/Library/Application\ Support/Malwarebytes/MBAM/Kext/MB_MBAM_Protection.kext 2>&1
Error making temporary directory: 1
Memory allocation failure.
Unable to stage kext (/Library/Application Support/Malwarebytes/MBAM/Kext/MB_MBAM_Protection.kext) to secure location.

On affected systems, we’ll often see this same output for a number of other kexts as well.

We’re only seeing this on 10.14.6 and, to a lesser extent, 10.13.6. We don’t really know if the issue affects Catalina as well, as Malwarebytes no longer uses a kext on Catalina, so we don’t have any data from user reports to go on there.

According to the data we’ve seen so far, all but three cases all involve macOS 10.14.6 (18G6032), which was released on September 24. The other three cases (so far) involved older systems, released this past spring or summer: macOS 10.13.6 (17G13035), macOS 10.13.6 (17G14033), macOS 10.14.6 (18G6020).

We saw a few isolated cases months ago, but have been seeing a more significant increase in incidents, starting very roughly sometime in September. So far, there aren’t enough incidents to really make any strong conclusions, other than that it looks like another macOS bug.

This was reported to Apple on October 19, 2020 (radar FB8812838), but there has not been a response yet.

A solution to the problem

The best solution would be for Apple to solve this issue. I do believe that will happen, as we’re not the only ones seeing (and, hopefully, reporting) this issue. There’s a thread on Apple’s developer forums from other people seeing the same problem.

https://developer.apple.com/forums/thread/657789

If you are seeing this issue, or are managing machines with this issue, there is a somewhat easy fix, though it will be quite cumbersome for IT admins trying to fix problems remotely during COVID.

1 – Reboot the affected machine in recovery mode (hold command-R at startup)

2 – Enter the Terminal (from the Utilities menu)

3 – In the Terminal, enter the following command:

chflags restricted /V*/*/private/var/db/KernelExtensionManagement

4 – Reboot the machine normally

Once you have done these steps, you should be back in business.

If you’re a developer of a kext and have customers affected by this issue, please report it to Apple in as much detail as possible. The more details they have, the faster they’ll be able to find the bug and fix it.

Fear-mongering about Mac ransomware

I recently discovered an absolutely atrocious article on Mac ransomware, thanks to a friend who forwarded the link to me. The article, found on the MacUpdate blog, is a minefield of glorious wrongness and fear-mongering.

https://www.macupdate[.]com/best-picks/mac-ransomware

(URL modified to avoid increasing the ranking of the article on search engines.)

Mac ransomware reality

Before we get into a tribute to the numerous ways this article is misleading and wrong, let’s set the stage by looking at the reality of the Mac ransomware landscape.

The first real Mac ransomware – KeRanger – was spotted in March of 2016. The malware infected systems through a supply chain attack, in which a malicious copy of the Transmission app was distributed via the official Transmission website, following a hack of the site. This was, ultimately, the most successful piece of Mac ransomware, but was cut short by a 3-day delay between infection and encryption of files, during which time most people who were infected would have already been disinfected by macOS, antivirus software, or a Transmission update designed to remove the malware. Few actual cases were seen.

The next two – Findzip (aka FileCoder, aka Patcher) and MacRansom – appeared in the first half of 2017, and were flops from the start. None were ever seen in the wild in any significant numbers.

There have been some proof of concept ransomware programs that have never been weaponized for malicious purposes, and even interest in creating proof of concepts has dwindled.

To put it bluntly, all Mac ransomware – so far – has been a flop, nothing more than a minor blip in Mac malware history. There is currently no known Mac ransomware in the wild.

This is not to say it’s impossible, or that we can assume it will stay this way. However, it’s certainly not a looming threat that we need to be overly concerned about right now, especially if you take the simple precaution of implementing a good data backup strategy.

So, now that we’ve got the reality covered, let’s take a look at this laughable article from MacUpdate.

“You Can’t Ignore Mac Ransomware!”

I suppose that headlines that were more clickbait-y have been written, but it seems to me that this is pretty far up there on the clickbait rating scale.

Why can’t we ignore it? Let’s read on. Hmm, those are some scary-sounding stats.

Windows ransomware stats from MacUpdate

There’s just this one little problem: those are Windows stats. They have absolutely no bearing on the state of ransomware for macOS. The article doesn’t mention that, though, leaving the reader to infer that this information applies to the Mac.

In fact, the article actually specifically calls out that “Mac ransomware is also on the increase.” Which is hard to justify, since it went from 3 to zero and has stayed there.

Most common ransomware

The article lists a number of different pieces of Mac ransomware. But there are some problems with that list. Let’s look at each one.

FBI/MoneyPak scam (2013): This is a real thing, but was not actually ransomware. It was a scam website that tried to keep you from closing the tab/window and trick you into paying the scammers.

FileCoder (June 2014): FileCoder is also a real thing, also known as FindZip and Patcher. However, it appeared in early 2017, not 2014. There was a Windows ransomware named FileCoder that was discovered in 2014, but that’s not relevant to macOS. I’m nit-picking with the date a bit, but it’s important, as it’s a strong indicator for how much this article conflates Windows threats and Mac threats.

KeRanger (March 2016): This is also a real thing, and is for the most part described accurately. There is one vital exception, though: citing MacWorld, the article says that “KeRanger appears to be still under active development.” This is portrayed as the current state. However, that quote from the MacWorld article is in turn quoted from Palo Alto Networks’ 2016 article on KeRanger, and is not actually representative of the current state of KeRanger. There is no indication that any development has continued in the nearly 4 years since it was released.

Filezip, aka Patcher (February 2017): There are two errors here. First, this is a duplicate, as this is just another name for FileCoder. Second, the name is FindZip, not Filezip.

Ransomware-as-a-Service (RaaS): I can only assume this is a reference to MacRansom, which was RaaS ransomware. This was a real thing, available for sale online, but there’s no sign that any actual infections ever happened.

What’s the takeaway?

I could get really nit-picky about some other little, unimportant things, but that would be silly, and a little to unnecessarily picky. However, I definitely don’t like the combination of the clickbait-y way this information was presented and the inaccuracies.

Bottom line, as long as you’re backing up data to drives that are not constantly connected to your Mac, you’re always safe from ransomware. (Note the plural “drives”… you should have multiple backups, and preferably at least one should be somewhere off-site, where a disaster like fire or robbery can’t get it.) Worst case scenario, if you get infected with some future ransomware and all your data is encrypted, you just restore from the backup.

Mac malware certainly does exist. Precautions are needed for dealing with malware that doesn’t announce itself by encrypting all your files.

However, screaming about the dangers of Mac ransomware, and using inaccurate and irrelevant information to do so, is precisely the behavior that leads people to accuse security people of using scare tactics.

</rant>

Notarization of AppleScript applets

As Catalina’s release date approaches, I find myself faced with a dilemma that many others are facing. At Malwarebytes, we have a number of AppleScript applets that we can provide to customers, for troubleshooting, gathering logs, incident response, etc. Come Catalina, those customers will be unable to run these scripts without special instructions… unless we notarize them.

So, they must be notarized. The next question is, “How?” Fortunately, Rich Trouton has published an excellent document on notarizing Automator applications, which is just as applicable to AppleScript applets.

I have little to add to Rich’s instructions, but did stumble across one special case. It’s sometehing that most people probably won’t see… but if you do, it’ll bring your notarization attempt to a screeching halt.

The problem

For most independent developers, their Apple developer account will stand alone, while companies will probably have multiple developer accounts all associated with a single master account. In such cases, there’s no problem… it’s unambiguous what account is going to be used for notarization.

However, if you’re reading this, you’re probably like me, and your account is associated with more than one master account (or “provider”) that can do signing and notarization. You may not even know about that, if it was set up for you or happened so long ago that you forgot.

In such a case, on attempting to submit your app for notarization using xcrun, you will will see an error message containing the following text:

Your Apple ID account is attached to other iTunes providers. You will need to specify which provider you intend to submit content to by using the -itc_provider command.

The solution

Most likely, even if you knew your account was associated with multiple “providers,” you probably don’t know the correct short name for the desired provider to use with the -itc_provider command. Fortunately, there’s a simple way to find out. If you’re using Xcode 10 or earlier, run the following, using your developer account e-mail address and the app-specific password you have created for notarization (see Rich’s instructions):

/Applications/Xcode.app/Contents/Applications/Application\ Loader.app/Contents/itms/bin/iTMSTransporter -m provider -u you@company.com -p app-password-here

This will not work on Xcode 11, where you’ll need to do this instead:

xcrun iTMSTransporter -m provider -u you@company.com -p app-password-here

The output will be very long, and will end with a provider listing looking like this:

   - Long Name -                          - Short Name -
 1  Your Corporation                      YourCorporation
 2  Another Group                         AnotherGroup

There may be more than two, and they may be different groups at your current company, different companies, etc, and may even include some stuff you are not supposed to still have access to from a previous job (oops!).

Note the short name of the one you want to use, then run xcrun again, but add the -itc_provider like so:

xcrun altool --notarize-app --primary-bundle-id "com.apple.ScriptEditor.id.MyScript" --username "you@company.com" --password "app-password-here" -itc_provider "ProviderShortName" --file MyScript.zip

Of course, if your applet isn’t called “MyScript”, you’ll want to make some changes there, but you get the idea. 🙂